www. O S N E W S .com
News Features Interviews
BlogContact Editorials
.
The best hardware to build with Swift is not what you think
By Thom Holwerda on 2017-03-15 23:22:09

Some interesting figures from LinkedIn, who benchmark the compiling times of their Swift-based iOS application. You'd think the Mac Pro would deliver the fastest compiles, but as it turns out - that's not quite true.

As you can see, 12-core MacPro is indeed the slowest machine to build our code with Swift, and going from the default 24 jobs setting down to only 5 threads improves compilation time by 23%. Due to this, even a 2-core Mac Mini ($1,399.00) builds faster than the 12-cores Mac Pro ($6,999.00).

As Steven Troughton-Smith notes on Twitter - "People suggested that the Mac Pro is necessary because devs need more cores; maybe we just need better compilers? There's no point even theorizing about a 24-core iMac Pro if a 4-core MBP or mini will beat it at compiling."

 Email a friend - Printer friendly - Related stories
.
Read Comments: 1-10 -- 11-20 -- 21-30 -- 31-36
.
Parallel Programming
By theTSF on 2017-03-16 19:57:20
The problem we have today is Processor speed more or less peaked, we make it up by going parallel with multiple cores. However most developers develop software with a single thread in mind, so a Computer with 24 cores will perform most task as well as one with 4. Because (over simplified)
Core 1 - OS
Core 2 - IDE
Core 3 - Browser
Core 4 - Compiling

When you get a lot of cores there will just be a lot of cores being idle. Having a lot of cores makes sense for jobs that are happening on many threads working together or are taking a lot of requests at the same time.

However most programs are based on Top Down development. So one thing happens then the next. SO the advantage of multiple cores cuts down rather fast.
Permalink - Score: 3
.
RE: Parallel Programming
By Alfman on 2017-03-16 20:19:46
theTSF,

> When you get a lot of cores there will just be a lot of cores being idle. Having a lot of cores makes sense for jobs that are happening on many threads working together or are taking a lot of requests at the same time.

However most programs are based on Top Down development. So one thing happens then the next. SO the advantage of multiple cores cuts down rather fast.


I agree with your points, even though many of us acknowledge the limits of sequential programming we do it anyways. We're going to have to make some big changes to take advantage of parallelism. We keep assuming this change is coming down the pipeline, but it never arrives. Even though this is impeding progress, it might not happen... much like IPv6. (Now let's duck for cover, haha).
Permalink - Score: 2
.
RE: Parallel Programming
By tidux on 2017-03-16 22:56:09
This is part of why I enjoy my current Linux setup. Other than Firefox, most of the processes I use are small and only depend on shared network or filesystem features to work together.

* mail sync is separate from mail reading

* music playback is separate from the UI

* most things live in terminals unless they're innately tied to graphics by their nature (media viewers, GIMP, etc.)

* software building is usually done through gcc, g++, or go which all scale beautifully to the 16 threads I have available

* VMs can have a few cores and a chunk of RAM carved off for themselves without impacting the rest of the system
Permalink - Score: 3
.
RE: Try it on linux?
By unclefester on 2017-03-17 02:22:26
> They have a swift compiler for Linux, I wonder if it demonstrates the same issues. Obviously, you couldn't compile ios/mac os applications, but if its really xcode and not the OS you should be able to reproduce it with a good sample app.

It is an OSX/hardware problem according to the above link. Macs default to a power saving mode which disables cores.
Permalink - Score: 3
.
RE[3]: Comment by Alfman
By ssokolow on 2017-03-17 04:19:54
> Wow, most people have just given up trying to optimize software at that level, but you are right it does make a big difference if you pull it off.

I think a lot of us just felt the trade-off was wrong.

For the longest time, I worked in Python because I was goal-oriented and my experiences with C and C++ had taught me that "writing your own CPU-bound algorithms simply isn't worth the stress of getting them right".

When Rust came around, I got really excited about things like the ability to be much more certain I'd handled every error case thanks to monadic error-handling and the ability to encode all sorts of invariants in the type system, thanks to having a compile phase to eliminate the abstraction penalty I was used to.

That led into the focus on zero-cost abstractions, which caused me to start to re-evaluate my conclusion that "reliable + CPU-bound is simply too hard for me, so I'll take reliable and limit myself to other people's C extensions".

I was then introduced, by the Rust community, to things like "code::dive conference 2014 - Scott Meyers: Cpu Caches and Why You Care" and started to get really excited. (Come for the safety, stay for the performance)

By that point, my hopes had been raised enough that I searched around and discovered that Valgrind+KCachegrind can be used to profile CPU cache misses.

I'm getting rather psyched about trying that kind of optimization myself. (I'm also really psyched about how easy cargo-fuzz makes fuzzing Rust programs.)

Edited 2017-03-17 04:22 UTC
Permalink - Score: 2
.
RE[4]: Comment by Alfman
By Alfman on 2017-03-17 07:06:33
ssokolow,

> For the longest time, I worked in Python because I was goal-oriented and my experiences with C and C++ had taught me that "writing your own CPU-bound algorithms simply isn't worth the stress of getting them right".

Some here, I actually like many aspects of low level programming, but over the years I've just come to hate debugging and corruption in unsafe code, it's time that I'd rather spend doing something else (not to mention my many other peeves with C++).

> When Rust came around, I got really excited about things like the ability to be much more certain I'd handled every error case thanks to monadic error-handling and the ability to encode all sorts of invariants in the type system, thanks to having a compile phase to eliminate the abstraction penalty I was used to.

That led into the focus on zero-cost abstractions, which caused me to start to re-evaluate my conclusion that "reliable + CPU-bound is simply too hard for me, so I'll take reliable and limit myself to other people's C extensions".


I'm quite enthusiastic about it too and I think devs who've moved to managed languages in frustration of unsafe code should give rust a look.
Permalink - Score: 2
.
RE[2]: Try it on linux?
By Bill Shooter of Bul on 2017-03-17 14:04:47
Hmm, Read the article a few times. They did make that assertion, but I don't completely understand why has more of an affect on the Mac Pro than the Mac Book pro. I mean, the Mac pro really has little reason to aggressively conserve energy, and it has more cores. But then again its a bug, not an intended feature.
Permalink - Score: 2
.
RE: Comment by Alfman
By Flatland_Spider on 2017-03-17 17:27:51
Ha. Yeah. :)

Guess what is highly linear? Compiling code.
Guess what is highly linear and easy to cache? Compiling code.
Guess what is highly linear, easy to cache, and has a high possibility of breaking when parallelized? Compiling code.

There are caveats, but in general....

I've had this fight with devs before. What most devs really need is a moderate amount of really fast cores because most of the time the changes are small, and if a build systems is in use, only the changes will be recompiled.

Most devs are application developers with low code churn. Some developers, like kernel devs, do deal with a code base with lots of churn which makes caching hard, and they can take advantage of lots of cores.
Permalink - Score: 1
.
RE: Parallel Programming
By Flatland_Spider on 2017-03-17 17:53:47
Most desktop workflows are stupidly linear, and we can't do much about that because humans. Multithreading vim only gets me so much.

It's nice to say "developers develop software with single thread in mind", but then there is reality. Only so many workloads can be parallelized, and only so much concurreny can be squeezed out of problems. The work doesn't scale because the data can't be broken down enough due to interdependence or the set isn't big enough. Especially end user stuff.

Servers are a different story because they have a lot more going on. Provided the language isn't a barrier to parallelization. There are lots of opportunities to use many cores since a lot of the work has low interdependence between requests.
Permalink - Score: 1
.
Comment by Flatland_Spider
By Flatland_Spider on 2017-03-17 18:16:20
> maybe we just need better compilers?

If you had to point at something, I would guess it would be the preprocessor, which is not the compiler. It probably could do a better job of breaking files into components to reduce what needs to be compiled.

There are a lot of steps involved in turning text files into machine code, so blaming the compiler is a little disingenuous. We never really think about them, but build systems do some heroic stuff to detangle code bases to only recompile needed files. Then there is the compiler toolchain of preprocessor, compiler, and linker.

We have tools that paper over a lot of details in the compilation process, but some serious work has been put into those tools.
Permalink - Score: 1

Read Comments 1-10 -- 11-20 -- 21-30 -- 31-36

No new comments are allowed for stories older than 10 days.
This story is now archived.

.
News Features Interviews
BlogContact Editorials
.
WAP site - RSS feed
© OSNews LLC 1997-2007. All Rights Reserved.
The readers' comments are owned and a responsibility of whoever posted them.
Prefer the desktop version of OSNews?