Base 15" - Mid 2014 vs. Mid 2017 - Development

[udflyer]

OK,

I am an iOS developer and I will upgrade machines sometimes just to get a 20% gain in compile/build time etc...


I just bought the base 2017 15" I have the base 2014 15".

I just did xcode builds on both and my old machine wins for each product I develop on.

This starts from a clean build, followed by a build hitting b at the same time

Project 1 build times:
2014 1:10
2017 1:12

Project 2

2014 :45
2017 :48


Project 3

2014: 2:12
2017: 2:17


On paper the HD should be way faster as well as the processor should be faster.

These times are in minutes and they are neck and neck.


What gives ?


The new machine is slick looking but unless I figure out what is going on it is going back.

 

[leman]

Is the Xcode version the same on both machines? Try compiling using xcodebuild from command line.

I have measures dramatic improvements for running statistical simulations between the 2015 and 2016 models. But it's also possible that compilation process is bottlenecked by something else. Try maybe building some other, larger project?

 

[udflyer]

Is the Xcode version the same on both machines? Try compiling using xcodebuild from command line.

I have measures dramatic improvements for running statistical simulations between the 2015 and 2016 models. But it's also possible that compilation process is bottlenecked by something else. Try maybe building some other, larger project?

I did command line builds and also turned on a flag where Xcode tells you how long everything it does takes. Xcode versions are the same

Done a good 50. After all that for my use case which is all that matters to me is they are a basic draw. Ugggg

 

[leman]

Interesting. Does your project include a lot of small files? It's possible that the file system latency completely overshadows the performance increases of the CPU/RAM/SSD. Did you try it with APFS?

 

[jerryk]

Is the processor really that much faster? They close. I am not at all surprised that the build times have not changed. Both machines have very fast SSDs and you are reading and writing small amounts. So the task is not disk I/O bound. More likely the factor is memory I/O. So with processor throughput nearly the same, getting the same times is to be expected.

 

[leman]

Is the processor really that much faster? They close.

The CPU is definitively faster. Skylake has improved branch predictors over Haswell, more execution units, larger scheduler etc. etc. And it definitely performs better on numerical code.

I can imagine though that a compiler might not benefit from these things. If the compilation process involves a lot of cache misses and unpredictable jumps, the architectural improvements of Skylake might play a very minor role. And of course, there are things such as filesystem latency and I/O latency in general...

 

[jerryk]

I can imagine though that a compiler might not benefit from these things. If the compilation process involves a lot of cache misses and unpredictable jumps, the architectural improvements of Skylake might play a very minor role. And of course, there are things such as filesystem latency and I/O latency in general...

Agree with your analysis. Compiling is largely a translation problem with lookups in memory structure and creating in memory structures that then get flushed out to disk.

 

[ayeying]

The CPU is definitively faster. Skylake has improved branch predictors over Haswell, more execution units, larger scheduler etc. etc. And it definitely performs better on numerical code.

I can imagine though that a compiler might not benefit from these things. If the compilation process involves a lot of cache misses and unpredictable jumps, the architectural improvements of Skylake might play a very minor role. And of course, there are things such as filesystem latency and I/O latency in general...

It has better capabilities due to newer technologies but in terms of faster, it's very subjective unless you're running the processor for something it's "good" at. For general most users and developer tasks it's not that much better or not good enough to warrant an must upgrade.

Skylake is more efficient then the older Haswell especially with power usage but the Haswell is still more powerful and faster on some tasks despite being a processor from 2012. And here, in Xcode, it seems that Haswell is still king vs the Skylake equivalent.

I have a Dell 5510 with the Xeon Mobile E3-1505M processor running at 2.8GHz and 3.7GHz turbo. On paper, it has a larger L3 cache (8MB), nearly the same TDP (45W) and better turbo boost, speed step, etc. compared to my older Late 2013 MacBook Pro running the i7-4960HQ processor that runs at 2.6GHz and 3.6GHz turbo. The L3 cache is smaller (6MB) and runs nearly the same TDP at 47W. Comparing it here, it would seem that the MBP running the Haswell processor would be:

1) Slower, due to lower clock speed.
2) Hotter, due to the higher TDP

However, just a blank test in benchmarks, it's a complete opposite.

* These processors are both Quad core w/ Hyperthreading.

Running Geekbench, at the same time after a clean start up, let the system sit for 5 minutes to let all background processes settle and a clean install w/ all updates and running the latest drivers and OSes produced these results in Geekbench 4.10.

Single Core:

E3-1505M @ 2.8GHz - 4230
i7-4960HQ @ 2.6GHz - 4401

Multi-Core:

E3-1505M @ 2.8GHz - 14721
i7-4960HQ @ 2.6GHz - 15337

That's amazingly different. Now in handbrake, converting a 1080p MKV video to MP4 with the same present also showed a difference with Haswell being king. Keep in mind, there is a possibility that OpenCL played a role; I'm not sure.

E3-1505M @ 2.8GHz - 28:11 Minutes
i7-4960HQ @ 2.6GHz - 27:54 Minutes

Now let's take into account possible throttling due to heat. Nope, no throttling. The CPUs were pinned at it's max turbo speed for all 4 cores the entire time. In fact, the Dell ran vastly cooler then the MBP and the MBP still was able to hold its own even after multiple runs.

Dell CPU max temperature: 79 C / Range observed: 51 - 79 C
MBP CPU max temperature: 98 C / Range observed: 66 - 98 C

The difference isn't big and on a daily thing; I see no difference in speed. I do a difference in power usage though. The Skylake processor seems to be more using way less power then the older Haswell. Overall, if you only care about raw performance; you'll be disappointed due to the fact that it's not that much faster or if at all faster on some tasks. There are some programs and processes that Skylake will beat Haswell out of the water. However, for most users; it's not big enough to warrant a "must" switch.

And for kicks, Apple really screwed up on the nVidia drivers in Sierra. In Civ6, it is faster to use the integrated video card then it is for the nVidia. Running the same settings and resolution:

Intel Iris Pro 5200 - 34FPS average benchmark
nVidia Geforce 750M - 16FPS average benchmark
nVidia GeForce 750M (with nVidia macOS drivers) - 17FPS average benchmark

 

[leman]

Running Geekbench, at the same time after a clean start up, let the system sit for 5 minutes to let all background processes settle and a clean install w/ all updates and running the latest drivers and OSes produced these results in Geekbench 4.10.

I think that this might be your problem: you are comparing Geekbench scores across different operating systems. I know that the company says that the scores should be comparable but that doesn't really seem to work out that well.

There is of course a difference in theory and practice, and slower hardware can turn out to be faster due to more competent software (like the case of FCPX vs. Premiere). And again, a lot of applications won't benefit from architectural improvements in the newer CPUs. For my work, I see a very healthy boost of Skylake vs. Haswell. For your case, it might be different. But please do not use Geekbench as a reference.

 

[The Mercurian]

Single Core:

E3-1505M @ 2.8GHz - 4230
i7-4960HQ @ 2.6GHz - 4401

Multi-Core:

E3-1505M @ 2.8GHz - 14721
i7-4960HQ @ 2.6GHz - 15337

That's amazingly different.

I would not consider those scores amazingly different at all. In fact if you run Geekbench a couple of times on the same machine it could differ by that much if I recall correctly. And as leman pointed out best not to rely on GB anyhow.

Is the Xcode version the same on both machines? Try compiling using xcodebuild from command line.

I have measures dramatic improvements for running statistical simulations between the 2015 and 2016 models. But it's also possible that compilation process is bottlenecked by something else. Try maybe building some other, larger project?

Interesting. If I can ask what kind of stats simulations do you see the benefit with ? I do painfully slow MCMC models every now and then. Have you tested such models ? When the rMBP first came out I bought one to see if I got speed-up versus a 2012 non-retina MBP - even though Geekbench was a better score it didn't translate to any improvement in an MCMC model I was using at the time. Therefore returned within the 14 day window and ended up later getting a 2014 model (which was faster!).

 

[leman]

Interesting. If I can ask what kind of stats simulations do you see the benefit with ? I do painfully slow MCMC models every now and then. Have you tested such models ? When the rMBP first came out I bought one to see if I got speed-up versus a 2012 non-retina MBP - even though Geekbench was a better score it didn't translate to any improvement in an MCMC model I was using at the time. Therefore returned within the 14 day window and ended up later getting a 2014 model (which was faster!).

I only did benchmarks for some simpler stuff, like randomised tests, bootstraps, drawing from distributions etc. I guess that the additional integer port in Skylake comes in useful here. Its interesting that you don't see improvements with MCMC. I guess it would depend on what software you are working with.

Personally, I run larger MCMC jobs only on supercomputers these days so I didn't test it on the laptop...

 

[The Mercurian]

I only did benchmarks for some simpler stuff, like randomised tests, bootstraps, drawing from distributions etc. I guess that the additional integer port in Skylake comes in useful here. Its interesting that you don't see improvements with MCMC. I guess it would depend on what software you are working with.

Personally, I run larger MCMC jobs only on supercomputers these days so I didn't test it on the laptop...

Interesting. I wonder would KabyLake offer further advantages for such stuff ?

As I recall at that time I was running Openbugs under Windows/Parallels and it wa a model that took about 3.5 hours to run. There was very little difference between the non-retina MBP and the 2013 rMBP. My current 2014 rMBP did give improvement on this model. I wouldn't do it it that way now of course - now I would use Rjags under OxS or on a windows hex-core desktop I picked up in the mean time