Huge capacity is nice, but it also increases components, therefore decreasing reliability. Most of the SSD drives that fail actually don't run out of PE cycles.
It might make sense if you are somewhere where server space is unreasonably expensive, but for most use cases it will be better to utilize a number of smaller drives instead, that will decrease density, but will increase performance, reliability and data availability.
100 tb is a lot of data to lose, and a lot of data to restore in case of a failure until you get the replacement drive populated and ready for service.
365 * 5 * ~2.2 DWPD comes to over 4000 PE cycles. That might be pushing it when it comes to contemporary high density mlc nand, which is far less durable than it used to be back when it was made on larger process node.
Usage patterns are also very important. A fifo pattern might do well to extend media life as close to the theoretical maximum. But imagine a situation where a substantial amount of the data is permanently stored and those blocks of memory are essentially out of circulation, the drive will now have less memory to service them promised 43 tb of writes per day, increasing wear on the blocks that are in circulation.
That can actually be mitigated by moving stale data around, which will also be beneficial to avoid running into issues with data retention, although this will increase complexity and possibly decrease performance. But without that scheme implemented, wearing the drive out using the promised "unlimited endurance" is actually quite possible.
"That might be pushing it when it comes to contemporary high density mlc nand, which is far less durable than it used to be back when it was made on larger process node."
that would be true, if it were true. 3D NAND is built, typically, on 40-50nm nodes, far, far larger than contemporary node size.
"But imagine a situation where a substantial amount of the data is permanently stored" -------------------------------------------------------------------------------------------------------------------
Flash is not "permanent"
Who wants to trust 100TB of data to a temporary storage medium?
....and even if it was permanent, I would demand a write protection switch to prevent extortionware from encrypting all my files or other malware from permanently deleting my files
This is not the time to rely on temporary storage for this much data
1) Nothing is permanent. That's what your backups are for.
2) If you're going around switching hardware write protection switches on your drives, you're not the target audience here. You clearly aren't managing enough data. If it's a software switch, does it really matter whether it's part of the drive or the OS? Either can potentially be changed by an attacker, and again, that's what your backups are for.
3) To the OP, there's no reason an SSD can't cycle "permanently" stored data onto new blocks occasionally to aid in wear leveling, even if it isn't actually modified.
"365 * 5 * ~2.2 DWPD comes to over 4000 PE cycles. That might be pushing it when it comes to contemporary high density mlc nand, which is far less durable than it used to be back when it was made on larger process node."
Reading the article would be helpful. These are using 3D NAND, which tends to be made on much larger processes (45nm or larger) than planar NAND. IOW, 3D NAND actually has *better* durability than planar NAND.
Max file count depends on the file system. It will probably not slow down significantly, as it is already capped kinda slow, it doesn't even fully saturate sata, which is most likely intentional. And given the capacity, its internal components probably operate at a tiny fraction of their peak performance, so it will likely have plenty of load headroom before it comes to performance degradation.
So test it, but you're speculating about things that have nothing to do with the drive and everything to do with the filesystem and use case. This drive is going to suffer from the same concerns any 100+ TB storage array would, and if you're not managing that much data already, you're not the target audience here.
The drive won't be slowing down from small files, but the filesystem might. Finding a file in a directory is generally a linear search, and one can imagine there are interesting scaling issues in a drive this size needed to prevent directories from getting large, or possibly even different filesystem types.
This drive is meant for enterprise users that need a lot more than 1x 100TB drive. Like the use case in the picture say, u use this to exchange over 3000 30TB SSDs with around 900 100TB drives. ;) If you need max 100TB, you don't buy one 100TB SSD in a enterprise use case.
I'm a bit confused by the slide comparing their 100 TB drive setup to one with Samsung's 32TB drives for 100 petabyte setups. 70% fewer drives looks right, but how do they go from that to 90% fewer enclosures and 86% fewer racks?
Am I missing something, or did they merge that slide with one for transitioning from HDDs? 6:1 consolidation looks like it should be about right for coming off of the biggest 3.5" HDDs.
They're jumping from 24bay enclosures to 90bay enclosures (4U stuff.. likely something like the Supermicro SC946ED) along the way. A tad disingenuous, if you ask me, since you can put the Samsung drives in the 90bay enclosure just fine, or go slightly denser at 96 disks per 4U with double-cage Supermicro chassis (like the 2028R-NR48N , which has room for 48 NVMe disks in 2U!)
One wonders the exact use case for a drive this big with such a small pipe to it. SAS will help, but this is still almost a cold-storage/tape replacement type of drive - minimal updates to stored data, and rare access to that data. But, when that data is needed, one doesn't have to wait for a tape librarian to pull the tape. I see this as something Facebook could use to periodically build historical stories. If it takes a couple of days to retrieve all the data, that's fine, and it provides responsive access to peoples old photos after a week or so of being actively viewed. Maybe Netflix could keep their long-tail of videos on something like this, or their historical usage data for improved prediction.
Anyone seen this in real life? For *years* Nimbus Data was known to be full of shit, even most of their client claims were simply made up. The worst part was that if they managed to fool some customers into paying for their shitty kits - yes, you had to sign a prepurchase agreement - they (customers) were routinely left with their prepaid demo units & balls in hand even if the ND's spectacular performance promises turned out to be jackshit - trying to sue them to cough up the money/take back their crap took a lot of persistence... After years of abuse of the market and internal staff as I heard they got to the point when they had to shut their SF office because screwing their employees out of their pay got so bad there was no way they can go on there any more, (Also hilarious: few private investors and the CEO owns the company, and as I heard latter was having affair with other mgmt people etc - people were saying it was like a real-life daily Spanish telenovela series. :)) I'm kinda curious: did anyone actually *see* this thing, in working condition?
PS: their original spiel was their "custom" controller, claiming to avoid SATA/SAS - something probably they never really managed to iron out, then NVMe arrived and it became a moot point. If they have half a brain they quickly realized that their shit probably never going to be able to compete with the big guys on speed, so they are better off just sticking a *lot* of cheap flash to their (now) slow-ish gear and sell it through drive suppliers, priced competitively. If that's what happened it makes sense. They can live in a niche for a while without much worry, finally.
Did an eval of one of their arrays years ago, and they tried the whole pay before doing a PoC bit with us. Never got anywhere close to the stated performance figures (got somewhere around 10% of expected performance) no matter what we tried, even using data streams that should have been very favorable for it. Spent a week working everyday with their engineering team troubleshooting things before boxing it up and sending it back which was another fiasco. They tried to refuse delivery then charge us for the equipment.
Really rather surprised they're still around in any form and still using such a tarnished name. When you and your customers (multiple ones) end up in lawsuits against each other its a sign to change business practices.
Yep, that sounds like Nimbus, it's their MO. I never got to the PoC phase, their paperwork reeked so badly at one point I told the owner (Iskovich?) to fuck off (was also suspicious what's going on there if the CEO himself handles a single box PoC.) I wonder if this 100TB drive will have a similar BS factor/hidden catch too... but I agree, I'm also a bit surprised they never been shut down by the authorities, there was ample evidence of knowingly deceiving customers on a daily basis. It was also rumored that their oft-invoked 'famous' customer list was largely made up. :)
We’ve updated our terms. By continuing to use the site and/or by logging into your account, you agree to the Site’s updated Terms of Use and Privacy Policy.
25 Comments
Back to Article
iter - Monday, March 19, 2018 - link
Does it self-brick after 5 years?Huge capacity is nice, but it also increases components, therefore decreasing reliability. Most of the SSD drives that fail actually don't run out of PE cycles.
It might make sense if you are somewhere where server space is unreasonably expensive, but for most use cases it will be better to utilize a number of smaller drives instead, that will decrease density, but will increase performance, reliability and data availability.
100 tb is a lot of data to lose, and a lot of data to restore in case of a failure until you get the replacement drive populated and ready for service.
iter - Monday, March 19, 2018 - link
365 * 5 * ~2.2 DWPD comes to over 4000 PE cycles. That might be pushing it when it comes to contemporary high density mlc nand, which is far less durable than it used to be back when it was made on larger process node.Usage patterns are also very important. A fifo pattern might do well to extend media life as close to the theoretical maximum. But imagine a situation where a substantial amount of the data is permanently stored and those blocks of memory are essentially out of circulation, the drive will now have less memory to service them promised 43 tb of writes per day, increasing wear on the blocks that are in circulation.
That can actually be mitigated by moving stale data around, which will also be beneficial to avoid running into issues with data retention, although this will increase complexity and possibly decrease performance. But without that scheme implemented, wearing the drive out using the promised "unlimited endurance" is actually quite possible.
Ian Cutress - Monday, March 19, 2018 - link
43 TB/day is 0.43 DWPD.0.43 * 365 * 5 = 785 PE cycles
iter - Monday, March 19, 2018 - link
My bad, I was thinking DPDW instead of DWPD, that makes it significantly more feasible.FunBunny2 - Monday, March 19, 2018 - link
"That might be pushing it when it comes to contemporary high density mlc nand, which is far less durable than it used to be back when it was made on larger process node."that would be true, if it were true. 3D NAND is built, typically, on 40-50nm nodes, far, far larger than contemporary node size.
ಬುಲ್ವಿಂಕಲ್ ಜೆ ಮೂಸ್ - Monday, March 19, 2018 - link
"But imagine a situation where a substantial amount of the data is permanently stored"-------------------------------------------------------------------------------------------------------------------
Flash is not "permanent"
Who wants to trust 100TB of data to a temporary storage medium?
....and even if it was permanent, I would demand a write protection switch to prevent extortionware from encrypting all my files or other malware from permanently deleting my files
This is not the time to rely on temporary storage for this much data
chaos215bar2 - Monday, March 19, 2018 - link
1) Nothing is permanent. That's what your backups are for.2) If you're going around switching hardware write protection switches on your drives, you're not the target audience here. You clearly aren't managing enough data. If it's a software switch, does it really matter whether it's part of the drive or the OS? Either can potentially be changed by an attacker, and again, that's what your backups are for.
3) To the OP, there's no reason an SSD can't cycle "permanently" stored data onto new blocks occasionally to aid in wear leveling, even if it isn't actually modified.
phoenix_rizzen - Tuesday, March 20, 2018 - link
"365 * 5 * ~2.2 DWPD comes to over 4000 PE cycles. That might be pushing it when it comes to contemporary high density mlc nand, which is far less durable than it used to be back when it was made on larger process node."Reading the article would be helpful. These are using 3D NAND, which tends to be made on much larger processes (45nm or larger) than planar NAND. IOW, 3D NAND actually has *better* durability than planar NAND.
ಬುಲ್ವಿಂಕಲ್ ಜೆ ಮೂಸ್ - Monday, March 19, 2018 - link
If I were to move a Billion small files to this drive, would it slow down and choke after a few hundred MB?How many years would it take to fill it up when using small file sizes?
Wouldn't a SATA drive of this size only be useful for huge files?
How long can it move tiny files before it slows to a crawl
How long before it hits 50% of initial speed?
20% ?
10% ?
0% ?
iter - Monday, March 19, 2018 - link
Max file count depends on the file system. It will probably not slow down significantly, as it is already capped kinda slow, it doesn't even fully saturate sata, which is most likely intentional. And given the capacity, its internal components probably operate at a tiny fraction of their peak performance, so it will likely have plenty of load headroom before it comes to performance degradation.ಬುಲ್ವಿಂಕಲ್ ಜೆ ಮೂಸ್ - Monday, March 19, 2018 - link
"probably" and "as tested" are two completely different thingschaos215bar2 - Monday, March 19, 2018 - link
So test it, but you're speculating about things that have nothing to do with the drive and everything to do with the filesystem and use case. This drive is going to suffer from the same concerns any 100+ TB storage array would, and if you're not managing that much data already, you're not the target audience here.ಬುಲ್ವಿಂಕಲ್ ಜೆ ಮೂಸ್ - Monday, March 19, 2018 - link
Waste your own money and test it!I won't speculate whether it works "FOR ME" or not....
I already know what works for me
jhh - Monday, March 19, 2018 - link
The drive won't be slowing down from small files, but the filesystem might. Finding a file in a directory is generally a linear search, and one can imagine there are interesting scaling issues in a drive this size needed to prevent directories from getting large, or possibly even different filesystem types.Gasaraki88 - Monday, March 19, 2018 - link
This is so wrong on so many levels.JKJK - Wednesday, March 21, 2018 - link
This drive is meant for enterprise users that need a lot more than 1x 100TB drive.Like the use case in the picture say, u use this to exchange over 3000 30TB SSDs with around 900 100TB drives. ;)
If you need max 100TB, you don't buy one 100TB SSD in a enterprise use case.
DanNeely - Monday, March 19, 2018 - link
I'm a bit confused by the slide comparing their 100 TB drive setup to one with Samsung's 32TB drives for 100 petabyte setups. 70% fewer drives looks right, but how do they go from that to 90% fewer enclosures and 86% fewer racks?Am I missing something, or did they merge that slide with one for transitioning from HDDs? 6:1 consolidation looks like it should be about right for coming off of the biggest 3.5" HDDs.
ZeDestructor - Tuesday, March 20, 2018 - link
They're jumping from 24bay enclosures to 90bay enclosures (4U stuff.. likely something like the Supermicro SC946ED) along the way. A tad disingenuous, if you ask me, since you can put the Samsung drives in the 90bay enclosure just fine, or go slightly denser at 96 disks per 4U with double-cage Supermicro chassis (like the 2028R-NR48N , which has room for 48 NVMe disks in 2U!)jhh - Monday, March 19, 2018 - link
One wonders the exact use case for a drive this big with such a small pipe to it. SAS will help, but this is still almost a cold-storage/tape replacement type of drive - minimal updates to stored data, and rare access to that data. But, when that data is needed, one doesn't have to wait for a tape librarian to pull the tape. I see this as something Facebook could use to periodically build historical stories. If it takes a couple of days to retrieve all the data, that's fine, and it provides responsive access to peoples old photos after a week or so of being actively viewed. Maybe Netflix could keep their long-tail of videos on something like this, or their historical usage data for improved prediction.chaos215bar2 - Monday, March 19, 2018 - link
So, how much does it cost?alanh - Monday, March 19, 2018 - link
Finally, an SSD that you can throw into your TiVo and expect to work long term.Well, if you ever wanted a TiVo that costs more than most cars.
T2k - Monday, March 19, 2018 - link
Anyone seen this in real life?For *years* Nimbus Data was known to be full of shit, even most of their client claims were simply made up. The worst part was that if they managed to fool some customers into paying for their shitty kits - yes, you had to sign a prepurchase agreement - they (customers) were routinely left with their prepaid demo units & balls in hand even if the ND's spectacular performance promises turned out to be jackshit - trying to sue them to cough up the money/take back their crap took a lot of persistence...
After years of abuse of the market and internal staff as I heard they got to the point when they had to shut their SF office because screwing their employees out of their pay got so bad there was no way they can go on there any more, (Also hilarious: few private investors and the CEO owns the company, and as I heard latter was having affair with other mgmt people etc - people were saying it was like a real-life daily Spanish telenovela series. :))
I'm kinda curious: did anyone actually *see* this thing, in working condition?
T2k - Monday, March 19, 2018 - link
PS: their original spiel was their "custom" controller, claiming to avoid SATA/SAS - something probably they never really managed to iron out, then NVMe arrived and it became a moot point. If they have half a brain they quickly realized that their shit probably never going to be able to compete with the big guys on speed, so they are better off just sticking a *lot* of cheap flash to their (now) slow-ish gear and sell it through drive suppliers, priced competitively.If that's what happened it makes sense. They can live in a niche for a while without much worry, finally.
Carpeinferi - Monday, March 19, 2018 - link
Did an eval of one of their arrays years ago, and they tried the whole pay before doing a PoC bit with us. Never got anywhere close to the stated performance figures (got somewhere around 10% of expected performance) no matter what we tried, even using data streams that should have been very favorable for it. Spent a week working everyday with their engineering team troubleshooting things before boxing it up and sending it back which was another fiasco. They tried to refuse delivery then charge us for the equipment.Really rather surprised they're still around in any form and still using such a tarnished name. When you and your customers (multiple ones) end up in lawsuits against each other its a sign to change business practices.
T2k - Monday, March 26, 2018 - link
Yep, that sounds like Nimbus, it's their MO. I never got to the PoC phase, their paperwork reeked so badly at one point I told the owner (Iskovich?) to fuck off (was also suspicious what's going on there if the CEO himself handles a single box PoC.)I wonder if this 100TB drive will have a similar BS factor/hidden catch too... but I agree, I'm also a bit surprised they never been shut down by the authorities, there was ample evidence of knowingly deceiving customers on a daily basis. It was also rumored that their oft-invoked 'famous' customer list was largely made up. :)