A Month with Apple's Fusion Drive
by Anand Lal Shimpi on January 18, 2013 9:30 AM EST- Posted in
- Storage
- Mac
- SSDs
- Apple
- SSD Caching
- Fusion Drive
Fusion Drive: Under the Hood
I took the 27-inch iMac out of the box and immediately went to work on Fusion Drive testing. I started filling the drive with a 128KB sequential write pass (queue depth of 1). Using iStat Menus 4 to actively monitor the state of both drives I noticed that only the SSD was receiving this initial write pass. The SSD was being written to at 322MB/s with no activity on the HDD.
After 117GB of writes the HDD took over, at speeds of roughly 133 - 175MB/s to begin with.
The initial test just confirmed that Fusion Drive is indeed spanning the capacity of both drives. The first 117GB ended up on the SSD and the remaining 1TB of writes went directly to the HDD. It also gave me the first indication of priority: Fusion Drive will try to write to the SSD first, assuming there's sufficient free space (more on this later).
Next up, I wanted to test random IO as this is ultimately where SSDs trump hard drives in performance and typically where SSD caching or hybrid hard drives fall short. I first tried the worst case scenario, a random write test that would span all logical block addresses. Given that the total capacity of the Fusion Drive is 1.1TB, how this test was handled would tell me a lot about how Apple maps LBAs (Logical Block Addresses) between the two drives.
The results were interesting and not unexpected. Both the SSD and HDD saw write activity, with more IOs obviously hitting the hard drive (which consumes a larger percentage of all available LBAs). The average 4KB (QD16) random write performance was around 0.51MB/s, it was constrained by the hard drive portion of the Fusion Drive setup.
After stopping the random write task however, there was immediate moving of data between the HDD and SSD. Since the LBAs were chosen at random, it's possible that some (identical or just spatially similar) addresses were picked more than once and those blocks were immediately marked for promotion to the SSD. This was my first experience with the Fusion Drive actively moving data between drives.
A full span random write test is a bit unfair for a consumer SSD, much less a hybrid SSD/HDD setup with roughly an 1:8 ratio of LBAs. To get an idea of how good Fusion Drive is at dealing with random IO I constrained the random write test to the first 8GB of LBAs.
The resulting performance was quite different. For the first pass, average performance was roughly 7 - 9MB/s, with most of the IO hitting the SSD and a smaller portion hitting the hard drive. After the 3 minute test, I waited while the Fusion Drive moved data around, then repeated it. For the second run, total performance jumped up to 21.9MB/s with more of the IO being moved to the SSD although the hard drive was still seeing writes.
In the shot to the left, most random writes are hitting the SSD but some are still going to the HDD, after some moving of data and remapping of LBAs nearly all random writes go to the SSD and performance is much higher
On the third attempt, nearly all random writes went to the SSD with performance peaking at 98MB/s and dropping to a minimum of 35MB/s as the SSD got more fragmented. This told me that Apple seems to dynamically map LBAs to the SSD based on frequency of access, a very pro-active approach to ensuring high performance. Ultimately this is a big difference between standard SSD caches and what Fusion Drive appears to be doing. Most SSD caches seem to work based on frequency of read access, whereas Fusion Drive appears to (at least partially) take into account what LBAs are frequently targeted for writes and mapping those to the SSD.
Note that subsequent random write tests produced very different results. As I filled up the Fusion Drive with more data and applications (~80% full of real data and applications), I never saw random write performance reach these levels again. After each run I'd see short periods where data would move around, but random IO hit the Fusion Drive in around an 7:1 ratio of HDD to SSD accesses. Given the capacity difference between the drives, this ratio makes a lot of sense. If you have a workload that is composed of a lot of random writes that span all available space, Fusion Drive isn't for you. Given that most such workloads are confined to the enterprise space, that shouldn't really be a concern here.
Facebook
Twitter
RSS
127 Comments
View All Comments
Constructor - Sunday, February 3, 2013 - link
That is an utter non-issue on a Mac.Simply make one last Time Machine backup before the drive switch, then restore from Time Machine afterwards.
Done.
That's even the process when switching to a completely new Mac – in my case I once even switched machines, architectures and OS versions all in one go, from a PowerMac G5 running Leopard to an Intel iMac under Snow Leopard. Completely painless and everything was still there exactly where and as it was on the old machine, down to the last application preference and to the last document.
You should have an up-to-date Time Machine backup at all times anyway, not least because it is so incredibly easy to maintain.
One of the huge advantages of Fusion Drive is that I will be able to build in an additional SSD into my Late 2009 iMac, fuse it with the regular HD and then restore the Time Machine backup to the fused drives exactly and with zero changes to the directory structure, reconnecting the new copy to the Time Machine history so that every directory will retain its full backup history right back to 2008 while still auto-optimizing performance.
That is as close to perfect as it gets.
That they initially charge a few bucks (or Euro) for the added speed, capacity and convenience is perfectly reasonable in my view (I'll get it for free beyond the third-party SSD and some time spent on modifying it myself).
name99 - Friday, January 18, 2013 - link
" I’m personally a much bigger fan of going all solid state and manually segmenting your large media files onto HDD arrays, but perhaps that’s me being set in my ways (or just me being right, not sure which one)."I don't want to complain, Anand, but your criticism is dumb. Like EVERYONE, as soon as it comes to iMacs and Mac Minis, reviewers seem to lose their minds and forget that USB3 exists.
If you want the config you want, the obvious solution is
- buy Fusion mac
- add USB3 SSD
- create a symlink from ~/Documents (or whatever it is you think you really want to be on SSD) to the USB3 SSD.
I do wish reviews spent less time on this nonsense, criticizing in a desperate attempt to find something to complain about, and spent more time on interesting technical issues. For example
- if one creates a 3rd party fusion device via CLI, does it actually engage in this smart transfer of data between SSD and HD? I've seen different claims on the net, and it's not clear to me, if one fuses two USB devices, how the kernel would know that one is SSD and the other is HD.
- can one boot off such a 3rd party fusion device? My instincts would be yes, because one can boot off AppleRAID devices, and this is a later generation of tech, but I've not seen this confirmed. It IS possible that, right now, the mac Mini and iMac have a special version of 10.8.2, and so certain functionalities will not be generally exposed until 10.8.3. If someone has access to an early 10.8.3 release I would, for example, like to see what happens if one takes an older mac mini, and fuses its internal HD with a USB SSD. Can we boot it? Do we get the same sort of data back-and-forth that is described here?
- where in the kernel are the hooks that decide to move data back and forth? If, for example, I create a JBOD of an SSD plus HD (using AppleRaid) I assume I will NOT get this sort of back and forth motion, just the usual Apple hotfile motion. But is that assumption wrong?
- how does Intel's equivalent tech work? In particular, is there actually any HW at all involved (doing what?) or is it just a Windows driver that happens to be written by Intel, and that happens to check you have a particular chipset, before it does all its driver work via the CPU?
name99 - Friday, January 18, 2013 - link
- can one boot off such a 3rd party fusion device? My instincts would be yes, because one can boot off AppleRAID devices, and this is a later generation of tech, but I've not seen this confirmed.To clarify this point.
I know that 3rd party CLI-created fusion drives can boot (and we've seen a few descriptions of this in the comments). What is NOT clear to me is whether these 3rd party drives give the full fusion experience, in particular
- writes going first to the SSD and
- subsequent re-shuffling of data between SSD and HD to match usage patterns.
gotnate - Saturday, January 19, 2013 - link
My personal experience with a "home made" fusion setup in my 3+ year old MacBook Pro is that does seem to be the full fusion experience.I happen to work from home on an iMac with a 240 GB SSD and a 3 TB hard drive that I manually manage. When I first installed my 1.24 GB fusion drive (1 TB HD + 240 GB SSD) I promptly filled up 500 GB. After that, I used the MBP as my primary workstation for a week on the road. On Monday, system performance was miserable. By Wednesday, my workflow was getting pretty snappy. On Saturday, I switched back to my iMac and was surprised to find that it felt a little slower than the MBP. I have been aching to convert the iMac to fusion drive ever since.
hyrule4927 - Friday, January 18, 2013 - link
You seem to be missing a mouse in that picture on the last page . . .lang999 - Friday, January 18, 2013 - link
Another apple-INVENTION i bet they pantent it and sues western digital for copying em.CharonPDX - Friday, January 18, 2013 - link
"About the only difference between how I manually organize my data and how Fusion Drive does it is I put my documents and AnandTech folder on my SSD by default. I don’t do this just for performance, but more for reliability. My HDD is more likely to die than my SSD."So Fusion Drive is essentially a RAID-0. If the spinning drive dies, the whole thing dies. So it doesn't matter how you organize the files. With this, you just let it do its thing and make sure to keep proper backups.
If one of my drives were to die, I'd replace it and restore from backup before doing anything else, anyway!
lan8 - Friday, January 18, 2013 - link
Under the "Putting Fusion Drive's Performance into Perspective" heading, in the sentence: "In trying to come up with a use case than spanned both drives I und a relatively simple one." I believe that odd word "und" is probably a typo of the word "found".mschira - Friday, January 18, 2013 - link
Yea the biggest nuisance is the lack of 256GB SSD option - either standalone or fusioned.Very very disappointing, and a bog step back from the older iMacs!
You think the they improve on their SSD choices, right?
You can't even get a 512 GB one - which is luxurious, but maybe justifiable.
768SSD is a pretty insane size and price,
M.
thaidrez - Saturday, January 19, 2013 - link
Thank you Anand for this very timely and thorough review. Like many people who have a 1 or 3 fusion drive on order from Apple, this was very eye opening or confirming. The reality is that when you stress a fusion drive, it's going to show it's colors in comparison to a 100% SSD solution. I agree that 256GB would have been more appropriate for the flash portion of this fusion solution. As far as my solution, if this helps anyone contemplating this choice. I have nearly 700GB of Media (music, photos, etc) currently spread between an external HDD and a 500GB HDD internally mounted and my 240GB SSD from OWC usually has 127GB of free space on it while hosting my OSX (2011 Mini). Looking closely at that 110+/-GB of OS "stuff", 62GB User, 28GB Applications, 6.3GB Private, 6.2GB Library, 5.5GB System, and then it tapers off from there into the MB range with use at 776 and then bin at 6.2. While I won't be able to tell exactly what is going onto the fusion or not... it seems to me that a lot of bigger files like iDevice backups, old emails, big apps I never use like iMovie and most of CS will probably be moved by Core Services over to the HDD leaving a lot of space on the flash side of the fusion disk. I can imagine that a good majority of "stuff in that 110GB of data does not really get used day to day. What about my Media? Do I really want a 2012 iMac that has a HDD in it? No! But I got the 3TB fusion because I would rather take the 900$ dollars and invest into a 1TB SSD Thunderbolt solution to keep my media on. Something I can keep if I sell the iMac or put onto my MBA or MBP. So I will take my 700GB of media, put it onto my external SSD and let my system, email, downloads, and day to day stuff run off the 3TB fusion. I also have a lot of storage that I used to keep on my old mac pro that I can now move off my NAS and put back onto my fusion, knowing I won't touch it but once every few months or so. So I can't see how using a 3TB fusion would be any different than having a 768GB given my configuration and storage plan. In fact, I see the fusion with a 1TB external SSD as a superior solution. Good luck to all