April 2, 2009 at 11:16 am
Hi folks, I was forwarded this article by a colleage: Businessweek about Fusion-io, a company that's been capturing a lot of attention. If this really takes off, I can only imagine how storage capacity and speed will affect the world of the DBA. In theory, fewer pages at 3am about failed backups due to space, and less fighting with the SAN administrators about increasing storage.
Anyone have any thoughts on this?
Gaby________________________________________________________________"In theory, theory and practice are the same. In practice, they are not." - Albert Einstein
April 2, 2009 at 11:41 am
The problem with flash storage is that it wears out rapidly when under constant rewrites. Rapidly compared to hard drives, that is. So, while it's really good for files that get written infrequently, like the music on an iPod, it's got problems when used for OLTP databases.
These issues are being resolved, and it's better now than it was a few years ago, but it's still an issue.
Now, if the cost of replacement is low enough to be soaked up in the savings from initial purchase price, then the problem is purely one of adequate safeguards in the system (checksums on pages, backups, etc., all the usual). From that point of view, they probably are a pretty viable contender.
But right now, for example, you can buy a 32 Gig solid-state hard drive for $400 (US) to $500. You can get a 1 Terabyte platter-style hard drive for about $100.
The solid state one is faster, by about 35% per http://download.intel.com/design/flash/nand/extreme/OLTP_Performance_Comparison_Solid-State.pdf. But it's 32 Gig, and the other is 1 Tera, and the smaller disk costs 4-5 times as much for about 1/30th the storage.
And keep in mind that the speed difference is greatly reduced if the flash drives are on a network store, instead of plugged right into the server's motherboard. And when you compare them to the internal platter-style drives that are coming out later this year/early next year (next version of SATA), the speed difference goes the other direction.
One major advantage flash drives have over platter drives is heat and energy cost. They take a LOT less watts, and they generate a LOT less heat. That gives them major advantages in portable systems, where battery life is king, and in some situations with large server rooms where heat and energy costs matter more than hard drive cost.
So, for right now, I think flash is going to continue to do well in portable systems, is going to take over a lot of the market for large datacenters, but will still be a niche product in small datacenters and desktop systems.
That will change. Give it another 5-10 years, tops, and platter-style hard drives will go the way of floppy disks. It's just not quite yet. That's my opinion, anyway.
- Gus "GSquared", RSVP, OODA, MAP, NMVP, FAQ, SAT, SQL, DNA, RNA, UOI, IOU, AM, PM, AD, BC, BCE, USA, UN, CF, ROFL, LOL, ETC
Property of The Thread
"Nobody knows the age of the human race, but everyone agrees it's old enough to know better." - Anon
April 2, 2009 at 11:52 am
Another thing I remember reading somewhere is how, just because you can theoretically rig 10 8-gig memory chips from jump drives, doesn't mean it's faster either. Apparently, the way data is ordered on the jump drive makes it inefficient for access as opposed to convetional drives. I'm sure those issues are also being addressed.
Anyways, all this will be moot and obsolete and we'll all be out of a job when the Cloud comes...all hail the Cloud! 😉
Gaby________________________________________________________________"In theory, theory and practice are the same. In practice, they are not." - Albert Einstein
April 2, 2009 at 12:17 pm
Anyways, all this will be moot and obsolete and we'll all be out of a job when the Cloud comes...all hail the Cloud!
That is very good question because to replace engineering you need actual engineering I have been researching the Clouds all I have seen is hype and some web page generating API which is not actual engineering. There are reasons the Object databases failed no engineering and scalability issues.
All I see is hardware consolidated and CMS installed and a lot of hype, some one is funding tectonic shift in software without basic engineering. So I am going to Cloud Camp I want to see actual engineering or the emperor have no clothes.
The question is how did web hosting transformed to Cloud?
:Whistling: :hehe:
Kind regards,
Gift Peddie
April 9, 2009 at 4:20 pm
Gaby Abed (4/2/2009)
Another thing I remember reading somewhere is how, just because you can theoretically rig 10 8-gig memory chips from jump drives, doesn't mean it's faster either. Apparently, the way data is ordered on the jump drive makes it inefficient for access as opposed to convetional drives. I'm sure those issues are also being addressed.Anyways, all this will be moot and obsolete and we'll all be out of a job when the Cloud comes...all hail the Cloud! 😉
Gaby,
I have got a good one for you the article below says there are no DBA in the Cloud so yep the crazy developers are still saying the DBA is data storage keeper. When Oracle gets to 50% of Microsoft revenues all these idiots will be vapor. Who will do the product development Algebra and Calculus?
http://visualstudiomagazine.com/features/article.aspx?editorialsid=2576
Kind regards,
Gift Peddie
April 19, 2009 at 6:09 pm
There is a review of one of the Fusion-IO products here. No SQL Server-specific benchmarks I'm afraid - would be very curious to see any of those if anyone knows of one?
If the capacity were a bit larger than 160GB I'd seriously consider using one of these for local TempDB hosting on our servers.
Also following the OP's theme, for those that haven't see it here's an interesting use of solid-state storage with SQL Server at the larger end of the scale:
http://erlang.org/pipermail/erlang-questions/2007-December/031843.html
http://www.eveonline.com/devblog.asp?a=blog&bid=286
EVE Online is a popular online game that has the distinction of hosting the single largest universe (eg non-"sharded") of any current MMORG. The architecture behind it is interesting, and they use one of the largest SQL Server setups I've come across to host it. The reached an IOPS bottleneck a couple years ago where their then current disk hardware couldn't keep up so they switched key parts out with these. DRAM-based rather than Flash, but with much better performance. Not cheap either 🙂
Regards,
Jacob
April 19, 2009 at 9:15 pm
There are different types of solid state drives, and some are supposed to last nearly as long as HDDs in writes and reads. I've tried to interview EVE online, but haven't been able to get responses from them.
I do know some MVPs have been testing SDDs in some situations, but I haven't heard back. I'd like to know what the long term response is for them, in addition to the performance.
I think if you were testing these, you'd want to backup to regular disk regularly until they've been proven as good long term solutions with failure detection.
April 20, 2009 at 4:02 am
Hey everyone,
Just to add to the whole solid-state reliability thing: we serve two thirds of all the web pages viewed in New Zealand daily and use four Ram-San 400s for the hottest data tables.
I'm not saying we're all that or anything; it's just a bit of real-life data for you :-).
Cheers,
Paul
Paul White
SQLPerformance.com
SQLkiwi blog
@SQL_Kiwi
April 20, 2009 at 9:16 am
EVE Online also uses RAM SAN products, just recently upgraded to the latest and greatest.
April 20, 2009 at 10:01 am
i read up on RamSAN thanks to you guys;
I'm jealous, quite frankly.
seems to be faster than a bat outta...well
but it's not a cheap solution either. that's some big money to get a terabyte or more of storage.
I guess your agencies need some serious I/O to decide to buy that system. congrats.
Lowell
April 20, 2009 at 3:17 pm
Well yeah it's all cool an' all, but as you say - expensive. SSDs aren't a panacea of course - we just found they solved a particular problem quickly, which was kinda important at the time.
Cheers,
Paul
Paul White
SQLPerformance.com
SQLkiwi blog
@SQL_Kiwi
April 20, 2009 at 5:15 pm
Paul White (4/20/2009)
...Just to add to the whole solid-state reliability thing: we serve two thirds of all the web pages viewed in New Zealand daily and use four Ram-San 400s for the hottest data tables.
...
Hey Paul - question for you since you're the first person I've met that has actually used these firsthand: since they are volatile devices how do you go about populating them with your data? Do you preload specific tables or copy whole DBs into them as SQL Server starts up? And if so what about any write-back guarantees for committing the changed data back to the real source disks - how does that work?
Regards,
Jacob
April 20, 2009 at 6:15 pm
Jacob Luebbers (4/20/2009)
Hey Paul - question for you since you're the first person I've met that has actually used these firsthand: since they are volatile devices how do you go about populating them with your data? Do you preload specific tables or copy whole DBs into them as SQL Server starts up? And if so what about any write-back guarantees for committing the changed data back to the real source disks - how does that work?
The RAMSANs are always-on just like the SAN, it doesn't depend on SQL Server being up. They were treated just like any other disk really. A normal file-copy-and-attach or restore (I forget which) was all that was required to move the database from SAN to RAMSAN.
The RAMSANs are battery-backed and they also de-stage their data to fast locally-attached traditional storage. The theory is that the battery provides enough time to de-stage all the uncommitted data at any time. Because the storage is battery-backed, I/O can be safely reported as committed to SQL Server when it enters memory, just like a traditional SAN with a write guarantee.
In the event of a catastrophic failure, we have the sort of DR you might expect; however the current design is such that the loss of the RAMSANs would not bring the site down - it would just lose certain functionality which is not critical to the core business activity. An example would be the 'people who looked at x also looked at y' kimd of idea.
Paul
Paul White
SQLPerformance.com
SQLkiwi blog
@SQL_Kiwi
April 20, 2009 at 7:09 pm
Thanks for the info Paul. So since they have local disk-based storage with auto-destage capability they are really just a traditional SAN silo with enough cache to hold the entire contents of the on-disk data, right?
Regards,
Jacob
April 20, 2009 at 7:23 pm
Jacob,
I guess you could think of it that way. It's probably not 100% technically accurate, but whatever works is all good!
Paul
Paul White
SQLPerformance.com
SQLkiwi blog
@SQL_Kiwi
Viewing 15 posts - 1 through 15 (of 30 total)
You must be logged in to reply to this topic. Login to reply