Dispelling the SSD myth. What is fast? Samsung vs Toshiba

theSeb

macrumors 604
Original poster
Aug 10, 2010
6,963
91
Poole, England
It always amuses me to see how people get hanged up over SSD performance. On this very sub-forum we have people stressing over whether they have the Toshiba or the Samsung SSD, because the Samsung model is a bit faster in synthetic benchmarks. What's even worse is what happens on the other forums, especially the iMac and MBP sub forums. People characterise the Apple SSDs as "crappy" and "slow" and drool over the latest Vertex 3 benchmarks, without taking into account whether they actually need such a drive.

There is no question that the Apple supplied SSDs are overpriced and that is not what I am discussing, but if the drive is user upgradeable do you really need to pay more for a Vertex 3 when an Intel 320 is fast enough for most users and workflows? Never mind the reliability can of worms.

Anand summarises it best in one of his SSD reviews so I'll quote it here.

www.Anandtech.com said:
The majority of our SSD test suite is focused on I/O bound tests. These are benchmarks that intentionally shift the bottleneck to the SSD and away from the CPU/GPU/memory subsystem in order to give us the best idea of which drives are the fastest. Unfortunately, as many of you correctly point out, these numbers don't always give you a good idea of how tangible the performance improvement is in the real world.

Some of them do. Our 128KB sequential read/write tests as well as the ATTO and AS-SSD results give you a good indication of large file copy performance. Our small file random read/write tests tell a portion of the story for things like web browser cache accesses, but those are difficult to directly relate to experiences in the real world.

So why not exclusively use real world performance tests? It turns out that although the move from a hard drive to a decent SSD is tremendous, finding differences between individual SSDs is harder to quantify in a single real world metric. Take application launch time for example. I stopped including that data in our reviews because the graphs ended up looking like this:



All of the SSDs performed the same. It's not just application launch times though. Here is data from our Chrome Build test timing how long it takes to compile the Chromium project:



Even going back two generations of SSDs, at the same capacity nearly all of these drives perform within a couple of percent of one another. Note that the Vertex 3 is even a 6Gbps drive and doesn't even outperform its predecessor.

In doing these real world use tests I get a good feel for when a drive is actually faster or slower than another. My experiences typically track with the benchmark results but it's always important to feel it first hand. What I've noticed is that although single tasks perform very similarly on all SSDs, it's during periods of heavy I/O activity that you can feel the difference between drives. Unfortunately these periods of heavy I/O activity aren't easily measured, at least in a repeatable fashion. Getting file copies, compiles, web browsing, application launches, IM log updates and searches to all start at the same time while properly measuring overall performance is near impossible without some sort of automated tool.

Now this, unfortunately, is where Anand and I differ in thought when it comes to his "storage bench" suites. I understand what he is doing and I respect the site and Anand's work. But the problem here is that certain impressionable people see the benchmarks and do not realise what they're actually saying to them.

Anandtech.com said:
The best we can offer is our Storage Bench suite. In those tests we are actually playing back the I/O requests captured of me using a PC over a long period of time. While all other bottlenecks are excluded from the performance measurement, the source of the workload is real world in nature.

What you have to keep in mind is that a performance advantage in our Storage Bench suite isn't going to translate linearly into the same overall performance impact on your system. Remember these are I/O bound tests, so a 20% increase in your Heavy 2011 score is going to mean that the drive you're looking at will be 20% faster in that particular type of heavy I/O bound workload. Most desktop PCs aren't under that sort of load constantly, so that 20% advantage may only be seen 20% of the time. The rest of the time your drive may be no quicker than a model from last year.

The point of our benchmarks isn't to tell you that only the newest SSDs are fast, but rather to show you the best performing drive at a given price point. The best values in SSDs are going to be last year's models without a doubt. I'd say that the 6Gbps drives are interesting mostly for the folks that do a lot of large file copies, but for most general use you're fine with an older drive. Almost any SSD is better than a hard drive (almost) and as long as you choose a good one you won't regret the jump.
Here is a description of the test

Anandtech.com said:
Not too long ago I tweeted that I had created what I referred to as the Mother of All SSD Benchmarks (MOASB). Rather than only writing 4GB of data to the drive, this benchmark writes 106.32GB. It's the load you'd put on a drive after nearly two weeks of constant usage.. And it takes a *long* time to run.

1) The MOASB, officially called AnandTech Storage Bench 2011 - Heavy Workload, mainly focuses on the times when your I/O activity is the highest. There is a lot of downloading and application installing that happens during the course of this test. My thinking was that it's during application installs, file copies, downloading and multitasking with all of this that you can really notice performance differences between drives.

2) I tried to cover as many bases as possible with the software I incorporated into this test. There's a lot of photo editing in Photoshop, HTML editing in Dreamweaver, web browsing, game playing/level loading (Starcraft II & WoW are both a part of the test) as well as general use stuff (application installing, virus scanning). I included a large amount of email downloading, document creation and editing as well. To top it all off I even use Visual Studio 2008 to build Chromium during the test.
And finally here is the disk busy time chart, which tells us everything we need to know. Clearly some drives are faster than others. Looking at the chart, without considering what it's actually saying, the Vertex 3 and other new SandForce-based drives look very impressive indeed. They are clearly the quickest by far! But hold on a moment… take a look at the chart and consider this benchmark….



It is run using an automated tool and it's playing back a recording of work done by a real user over a couple of days. If I understand what the bold bit says, then it is actually the work done over two weeks. I may be wrong in the exact details, but it's clearly not something that a normal person would be looking to do within a couple of minutes.

Now, as much as I like to think I am pretty good, and I earn decent money doing it, no matter how hard I try I cannot edit images, play games, surf the web, compile my code, copy stuff and install applications all at the same time or one after another within 700 seconds.

There are times when I do a lot and then I do something like actually read the web page, or edit a document or edit my code or swear at the computer in frustration. I am not always doing something where the drive needs to do intensive work. This chart tells us that the difference between the fastest and the slowest drive tested is 869.7 seconds, which is equivalent to nearly 14.5 minutes. Therefore, I've managed to save 14.5 minutes by buying the fastest drive on a workload that is equivalent to a couple of days from a human perspective. Think about that for a moment. So how much time have I really saved?

The point remains, unless you're doing 50 things at the same time and running 5 automated build servers on 5 different virtual machines, which compile lots of code and runs lots of automated tests and downloading, copying, editing photos, and editing a movie on the side too, you, as a real user, are the bottleneck; not the SSD drive.

There are instances and workflows where the fastest SSD is justifiable, but even the "slow", "crappy" drives are fast enough for 98% of you. Do not stress whether you have the Samsung or the Toshiba SSD. Enjoy your new computer because you won't even see the difference, unless booting up in 0.5 of a second and having a better xbench score is really important to you.

Perhaps I've misunderstood what these benchmarks are doing and perhaps someone like HellHammer can chat to Anand and correct me.
 
Last edited:

Borjan

macrumors regular
Sep 28, 2004
222
2
Yeah, that is a longer forum post, but honestly its pretty easy to read in a few minutes.It's sad that it may put people off!

But yes, this is a top post, very appreciated. This all makes a lot of sense, but ironically the people who moan about things often aren't the same people who go around and take time to learn/read.

Hence this post might not reach its target audience!
 

theSeb

macrumors 604
Original poster
Aug 10, 2010
6,963
91
Poole, England
Yeah, that is a longer forum post, but honestly its pretty easy to read in a few minutes.It's sad that it may put people off!

But yes, this is a top post, very appreciated. This all makes a lot of sense, but ironically the people who moan about things often aren't the same people who go around and take time to learn/read.

Hence this post might not reach its target audience!
Thanks for the comments. It's easier and faster for everyone to make their own thread to complain about the SSD travesty. :D
 

urkel

macrumors 68030
Nov 3, 2008
2,783
862
I bought a new red sportscar that says it goes to 180mph. My buddy paid the same amount and got the exact same car but his speedometer says it goes to 190mph.

Just because we may never hit top speed doesn't mean someone shouldn't feel a bit irked that all things aren't the same.

(BTW. I'm not complaining. I got the Toshiba 256 but I also got a $100 sale discoun, $90 reward zone bucks, $100 gift card and a free Pepsi at Best Buy so it's not worth going back to risk getting another. BUT, I can see why others would want to)
 

bigp9998

macrumors regular
Dec 21, 2007
144
0
I bought a new red sportscar that says it goes to 180mph. My buddy paid the same amount and got the exact same car but his speedometer says it goes to 190mph.

Just because we may never hit top speed doesn't mean someone shouldn't feel a bit irked that all things aren't the same.

(BTW. I'm not complaining. I got the Toshiba 256 but I also got a $100 sale discoun, $90 reward zone bucks, $100 gift card and a free Pepsi at Best Buy so it's not worth going back to risk getting another. BUT, I can see why others would want to)
Damn, all they had was Coke Zero at my Best Buy!! At least I got the rest of the deal though....
 

urkel

macrumors 68030
Nov 3, 2008
2,783
862
Damn, all they had was Coke Zero at my Best Buy!! At least I got the rest of the deal though....
Dude, I totally got more calories than you. I'd suggest you exchange it but according to one of those graphs up there then we're supposed to pretend there isn't a difference between Coke Zero and Pepsi.
 

lukekarts

macrumors regular
Mar 16, 2009
155
0
Really good post, and those graphs really go a long way to demonstrate the point. I'm not too concerned what I get now.
 

Philflow

macrumors 65816
May 7, 2008
1,270
3
anandtech says:
So why not exclusively use real world performance tests? It turns out that although the move from a hard drive to a decent SSD is tremendous, finding differences between individual SSDs is harder to quantify in a single real world metric. Take application launch time for example. I stopped including that data in our reviews because the graphs ended up looking like this:
I disagree here. It's quite easy to find differences in true real world settings as Hardwareheaven shows in their reviews: http://www.hardwareheaven.com/reviews/1143/pg9/crucial-m4-256gb-ssd-c400-review-install-times.html

PS. here are several benchmarks where the Toshiba beats the Samsung: http://forums.macrumors.com/showthread.php?t=1198391
 

Abstract

macrumors Penryn
Dec 27, 2002
24,415
124
Location Location Location
As far as I know, people have been saying that Samsung's 256 GB SSD is faster than Toshiba's 256 GB SSD.

I always thought people who were making the blanket statement that Samsung SSDs are faster than Toshiba SSDs just misunderstood and were saying what they thought was true. :confused:
 

Scepticalscribe

macrumors Sandy Bridge
Jul 29, 2008
49,791
33,439
The Far Horizon
Excellent post, OP, and yes, I did take the time to read it from beginning to end. Thanks for posting it, and it made interesting reading.

I'm one of those who would lie in your "98%" category. Like the '98%' referred to, while I am delighted that SSDs will be the norm on computers, I am not hugely fussed about relatively micro advantages of one brand over another. The advantages over the old HDD (of both iPods and computers) are simply too great to ignore the advantages of SSD, while the differences between types of SSD don't really excite me at the moment.

To my way of thinking, Apple SSDs are fine for the moment, and, as with all other technological advances (think of the fall in price of USB sticks for example; a few years ago, a stick of 1GB or 2GB cost nearly €100, now they are easily affordable), prices will fall, probably quite considerably, over the coming years.

Cheers
 

theSeb

macrumors 604
Original poster
Aug 10, 2010
6,963
91
Poole, England
I bought a new red sportscar that says it goes to 180mph. My buddy paid the same amount and got the exact same car but his speedometer says it goes to 190mph.

Just because we may never hit top speed doesn't mean someone shouldn't feel a bit irked that all things aren't the same.

(BTW. I'm not complaining. I got the Toshiba 256 but I also got a $100 sale discoun, $90 reward zone bucks, $100 gift card and a free Pepsi at Best Buy so it's not worth going back to risk getting another. BUT, I can see why others would want to)
It's funny and odd that you've posted this analogy because I was just about to post something similar. First of all, it would appear, thanks to benchmarks, that the difference between the Samsung and Toshiba is more like 190 MPH versus 187 MPH. Is this really an issue? I do understand your point though.

Then taking this analogy further: the difference between the fastest drive and the slower SSDs can best be thought of as two cars, where one can sprint from 0-60 MPH in 4.5 and the other can achieve this in 5 seconds. What if the faster car was also temperamental and overheated in traffic, but drove like a dream on the open roads?

It is nice to be able to accelerate at a more rapid rate and have more power, but do you always take off at full throttle from every stop street? No, quite often you feather the throttle and drive normally. Would the difference in getting from 0-60 MPH have a significant impact on how fast you get from home to work or how fast you can drive to the coast for your holiday? Not really, because you have speed limits and do not always drive at full throttle. You, as the user, are the bottle neck for both the SSDs and the cars.

There will be occasions where there extra 0.5 of a second will make a difference, like, for example, if you take your car to a race track. This is very stressful and is akin to having a very heavy workflow and therefore the fastest possible SSD is justifiable, but it shouldn't be your top priority for usual pottering about town and getting to work. You want something reliable, even if it means you may arrive 2 minutes later.
 

rds

macrumors regular
Aug 9, 2007
148
0
That's only one page. The review has several pages that show clear real world differences.

More examples: http://www.techspot.com/review/402-crucial-m4-ssd/page5.html

I see lots of significant differences that are easy to quantify.
You could link to bar charts all day. Never mind the fact that the difference between the fastest and slowest SSDs on that page is ~1 second and doesn't account for the fact that the fastest has double the storage (thus up to double the memory chips) meaning the test isn't exactly fair.

Without benchmarks, if you switched between a MBA with a Toshiba SSD and a MBA with a Samsung SSD on a regular basis, you would not tell the difference. It's only the fact that benchmarks show a irrelevant difference that people care.

Both SSDs are fast even by SSD standards. The original SSD in the old MBAs, compared to a HDD, was a remarkable transformation that took away the biggest bottleneck in computing speed. In a benchmark, the older SSD is ~5x slower than either the Toshiba or Samsung SSD. In reality, the difference going from a 'slow' SSD to a relatively 'fast' SSD is nothing compared to going from a HDD to SSD. I can notice the difference but only just, compare two relatively close SSDs and you'd probably only tell the difference with massive file transfers.
 

Philflow

macrumors 65816
May 7, 2008
1,270
3
You could link to bar charts all day. Never mind the fact that the difference between the fastest and slowest SSDs on that page is ~1 second and doesn't account for the fact that the fastest has double the storage (thus up to double the memory chips) meaning the test isn't exactly fair.
there's a 6 second difference on 15 seconds in the boot benchmark.

Without benchmarks, if you switched between a MBA with a Toshiba SSD and a MBA with a Samsung SSD on a regular basis, you would not tell the difference. It's only the fact that benchmarks show a irrelevant difference that people care.
Sure. I totally agree.

That wasn't what I was responding to. I was only responding to Anands argument why he has replaced true real world benchmarks with his Storage bench.
 

glen e

macrumors 68030
Jun 19, 2010
2,619
2
Ft Lauderdale
You could link to bar charts all day. Never mind the fact that the difference between the fastest and slowest SSDs on that page is ~1 second and doesn't account for the fact that the fastest has double the storage (thus up to double the memory chips) meaning the test isn't exactly fair.

Without benchmarks, if you switched between a MBA with a Toshiba SSD and a MBA with a Samsung SSD on a regular basis, you would not tell the difference. It's only the fact that benchmarks show a irrelevant difference that people care.

Both SSDs are fast even by SSD standards. The original SSD in the old MBAs, compared to a HDD, was a remarkable transformation that took away the biggest bottleneck in computing speed. In a benchmark, the older SSD is ~5x slower than either the Toshiba or Samsung SSD. In reality, the difference going from a 'slow' SSD to a relatively 'fast' SSD is nothing compared to going from a HDD to SSD. I can notice the difference but only just, compare two relatively close SSDs and you'd probably only tell the difference with massive file transfers.
great post - this is so ridiculous.....but I guarantee you some did not sleep since this "fact" came out - LOL

PS I have the "slow" Toshiba - but somehow I will make it thru the day ok, it'll be a struggle but I'll make it. I have to go now console my wife, she is frantic and in tears over this thing....
 

kelfaij

macrumors member
Jul 25, 2011
34
0
Great post !

I have to say that it is true that 98% of the users won't feel the difference, the frustration come from the fact that 2 people who bought the same product won't get the same quality. it is just about injustice.

The ignorant are always the happiest !

My advice: once you receive your mbair don't check what SSD is inside and enjoy the little guy !!!

I'll try to do so ;-) mine is in the plane to Europe ;-)
 

Hellhammer

Moderator emeritus
Dec 10, 2008
22,166
581
Finland
Like you linked in the OP, this is what people should read before looking at the MOASB scores:

What you have to keep in mind is that a performance advantage in our Storage Bench suite isn't going to translate linearly into the same overall performance impact on your system. Remember these are I/O bound tests, so a 20% increase in your Heavy 2011 score is going to mean that the drive you're looking at will be 20% faster in that particular type of heavy I/O bound workload. Most desktop PCs aren't under that sort of load constantly, so that 20% advantage may only be seen 20% of the time. The rest of the time your drive may be no quicker than a model from last year.

The point of our benchmarks isn't to tell you that only the newest SSDs are fast, but rather to show you the best performing drive at a given price point. The best values in SSDs are going to be last year's models without a doubt. I'd say that the 6Gbps drives are interesting mostly for the folks that do a lot of large file copies, but for most general use you're fine with an older drive. Almost any SSD is better than a hard drive (almost) and as long as you choose a good one you won't regret the jump.
To be honest, unless you are really a pro user (heavy video, photo, music editing, rendering etc) and undertand something about SSDs and computer in general, don't even take a look at the benchmarks. They will do nothing else but confuse your head. The MOASB is nothing unless you actually know what it really does and what it is useful for.

PS. here are several benchmarks where the Toshiba beats the Samsung: http://forums.macrumors.com/showthread.php?t=1198391
Those drives aren't 100% valid in MBA discussion. 2.5" SSDs usually have 16 NAND devices while MBA has four or eight (only in 256GB). That can make a huge difference. One controller can be better optimized for fewer NANDs than the other. Plus, Apple uses a custom firmware which again changes the game.

To show that the firmware really makes a difference, the Samsung one gets around 270MB/s read and 212MB/s write. In the link you posted, the Samsung gets 250MB/s and 245/150/105 MB/s depending on the capacity. Samsung claims 250MB/s read and 220MB/s write for 256GB model.

What we really need is two MBAs: one with Samsung and one with Toshiba. Then some real world tests to see the real difference under OS X.
 

Philflow

macrumors 65816
May 7, 2008
1,270
3
Those drives aren't 100% valid in MBA discussion. 2.5" SSDs usually have 16 NAND devices while MBA has four or eight (only in 256GB). That can make a huge difference. One controller can be better optimized for fewer NANDs than the other. Plus, Apple uses a custom firmware which again changes the game.
I agree there are differences. But they do use the same controllers. So translating the results may not be 100% accurate for the MBA, I still think it's 97% accurate.

I've benchmarked both SM and TS SSDs in MBAs 2010 under Windows. I got very similar random performance as with Kingston V+ 100 and Samsung 470 (that I both reviewed for NBR). Unfortunately I didn't keep the screenshots.

What we really need is two MBAs: one with Samsung and one with Toshiba. Then some real world tests to see the real difference under OS X.
That would be most ideal I agree.

In the absence of that I think we can learn a lot from comparing the controller performance in the SSDs that are technically most close to that.
 
Last edited:

Oppressed

macrumors 65816
Aug 15, 2010
1,265
10
The only time you would perceive a difference would be on long processes such as installing large files or copying large files. As the time of the process increases the evidence of the difference between the two becomes noticeable. For example if we duplicated a 2gig file on both models then one may complete it only a matter of seconds faster. Would you notice this difference? Yes. Would you notice the difference in everyday use? No.
 

theSeb

macrumors 604
Original poster
Aug 10, 2010
6,963
91
Poole, England
some more data, if not added here yet:

barefeats tests different SSDs in MBAs
Thanks for adding this because I was just looking for the thread where I last saw it.

Barefeats said:
Though artificial benchmarks are helpful, we wanted to see how quickly Finder could duplicate the Applications folder on the Desktop. It had over 50 apps/documents totaling over 3GB. We calculated the transfer rate (size/time*2=MB/s). RED bar indicates the fastest.
Let's stipulate that "over 3 GB" is around 3072 MB

Samsung:

3072/T * 2 = 61.9
therefore
T = 99.3 seconds

OWC Aura:
T = 98.1 seconds

Toshiba:
T = 116 seconds
 

Similar threads

  • kreasonos
9
Replies
9
Views
911
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.