Fast Storage for LL databases
|
|
SuezanneC Baskerville
Forums Rock!
Join date: 22 Dec 2003
Posts: 14,229
|
08-27-2007 22:33
Maybe some of these would speed up SL's operations a bit. http://www.channelregister.co.uk/2007/08/28/memory_appliance_gear6_cachefx/From: someone Gear6 fits into that elite, blissful class of start-ups that have an easy to digest premise and infrastructure-friendly gear.
The Silicon Valley-based firm ships a pair of caching appliances. These RAM-based boxes plug right into existing Ethernet networks and work as complements to disk-based shared storage systems. As a result, applications that depend on accessing large data sets tend to enjoy dramatic performance improvements by getting much of their information straight from the speedy appliances rather than always going out to disk.
Gear6 likes to focus its pitch around I/O operations per second or IOPS. Its base CACHEfx G200 appliance boasts 250,000 IOPS with 1.6GB/s of throughput and less than half a millisecond of latency. This system takes up half a rack, has 250GB of capacity and costs $400,000 with the hardware, software and support. The G400 appliance is twice the machine across the board.
At 250,000 IOPS, Gear6 boasts that it can beat out a high-end NAS (network attached storage) system from, say, NetApp by 5X. The company also says that its half a millisecond latency easily beats out the typical 2 to 8 milliseconds of latency experienced by customers with demanding, clustered software. All told, Gear6 promises a 10X to 30X performance improvement for applications with large data sets.
So, you can imagine oil and gas, video, bio-tech, financial services and database companies wanting to have a look at the gear. And, in fact, the likes of Sony and GX Technology, a player in the oil and gas field, have already bought Gear6 systems.
While the hardware may seem expensive, it will prove attractive enough to high performance computing users and big business types that demand the most out of their hardware at just about any price. How many would it take to put all the databases on things like this? 
_____________________
-
So long to these forums, the vBulletin forums that used to be at forums.secondlife.com. I will miss them.
I can be found on the web by searching for "SuezanneC Baskerville", or go to
http://www.google.com/profiles/suezanne
-
http://lindenlab.tribe.net/ created on 11/19/03.
Members: Ben, Catherine, Colin, Cory, Dan, Doug, Jim, Philip, Phoenix, Richard,
Robin, and Ryan
-
|
|
Malachi Petunia
Gentle Miscreant
Join date: 21 Sep 2003
Posts: 3,414
|
08-27-2007 22:55
From: someone While the hardware may seem expensive... I was using machines similar to the Gear6 stuff in 1994. This isn't new technology, but it is costly. On the other hand, it works as promised. |
|
Osgeld Barmy
Registered User
Join date: 22 Mar 2005
Posts: 3,336
|
08-27-2007 23:33
yea i remember in back in 94 also we were using bricks of ram connected to scsi and battery backups 1gb of storage (which was pimp in 94) fast as a racecar falling off of a cliff wouldn't trust it for anything persistent, altho technology has pretty much covered that price =  as your article states 400 grand for 250 gb, where as a 300gb high speed scsi (altho granted not as fast as a brick of ram) is about 5 - 6 hundred |
|
SuezanneC Baskerville
Forums Rock!
Join date: 22 Dec 2003
Posts: 14,229
|
08-28-2007 14:30
From: Malachi Petunia I was using machines similar to the Gear6 stuff in 1994. This isn't new technology, but it is costly. On the other hand, it works as promised. What were those machines? Did you see the articles about the 500 Ghz processor? I suppose you were using those back in 1994 too, eh? How many Gear6's would it take to hold SL's databases? I found that article interesting not only for the device's specs but the price was, umm, interesting.
_____________________
-
So long to these forums, the vBulletin forums that used to be at forums.secondlife.com. I will miss them.
I can be found on the web by searching for "SuezanneC Baskerville", or go to
http://www.google.com/profiles/suezanne
-
http://lindenlab.tribe.net/ created on 11/19/03.
Members: Ben, Catherine, Colin, Cory, Dan, Doug, Jim, Philip, Phoenix, Richard,
Robin, and Ryan
-
|
|
Malachi Petunia
Gentle Miscreant
Join date: 21 Sep 2003
Posts: 3,414
|
08-28-2007 15:12
From: someone What were those machines? They were Network Appliance boxes; I forget the model and specs. The impressive part was that across the LAN they were faster than local hard drive accesses. Of course, processors are faster and cheaper now as are drives. I saw the "5x NetApp" statistic above and recall the maxim "lies, damned lies, and benchmarks" but I really haven't kept up with that market. I don't think LL has disclosed what their storage needs are, but the prices back then were as relatively stunning as they are now. |
|
Lex Neva
wears dorky glasses
Join date: 27 Nov 2004
Posts: 1,361
|
08-29-2007 09:43
Millions of dollars. Last I heard, the asset database was at 15TB... that was two years ago. |
|
SuezanneC Baskerville
Forums Rock!
Join date: 22 Dec 2003
Posts: 14,229
|
09-01-2007 23:22
The Tilera is the other piece of hardware that caught my eye lately, the current model is a 64 core chip with 1024 core being hopefully on the way. The current version is maybe a bit slow.
I doubt NetApp actually had devices of comparable specs to the Gear6 in 1994. The big high speed storage systems they had back were hard drives with a few gigabytes of ram cache, not all solid state like the Gear6.
_____________________
-
So long to these forums, the vBulletin forums that used to be at forums.secondlife.com. I will miss them.
I can be found on the web by searching for "SuezanneC Baskerville", or go to
http://www.google.com/profiles/suezanne
-
http://lindenlab.tribe.net/ created on 11/19/03.
Members: Ben, Catherine, Colin, Cory, Dan, Doug, Jim, Philip, Phoenix, Richard,
Robin, and Ryan
-
|
|
Osgeld Barmy
Registered User
Join date: 22 Mar 2005
Posts: 3,336
|
09-02-2007 00:12
From: SuezanneC Baskerville The Tilera is the other piece of hardware that caught my eye lately, the current model is a 64 core chip with 1024 core being hopefully on the way. The current version is maybe a bit slow.
I doubt NetApp actually had devices of comparable specs to the Gear6 in 1994. The big high speed storage systems they had back were hard drives with a few gigabytes of ram cache, not all solid state like the Gear6. altho i should be used to it, you totally missed my post i was using 1gb full size 5 1/4 inch solid state drives in 1994, yea its not the same speed nor storage at the Gear6 but wheres your 3ghz dual core bit of uberness desktop back then? right ... point is, they did exist, they cost about 30 grand then, and their uber wicked fast, problem is A they cost too damn much B they are not super reduant (sorry we lost your inventory after a 3 week power outtage) C no matter how fast your disk is, your CPU is whats doing all the hard work really from my experience what we see as lag in asset management is not disk access, cause once its actually found its just a blip, but more in cpu usage you can have the worlds fastest disk's but if your processor (cluster) is bogged down you will never see its advantages for giggles lets say theres only 4tb of asset data (its much more than that) thats 6.4 mil in just disk storage for about 6.4 mil, they could drop their devotion to lameoid home/desktop computer technology, and move into some serious computer power such as sgi/cray i bet a cupple cray X1E clusters could solve both asset data bog, and physics http://www.cray.com/downloads/X1E_datasheet.pdfoh and the awnser to "what do you do with a 15 lb brick of ram that has a scsi port?" is "swapspace / application cache" |
|
SuezanneC Baskerville
Forums Rock!
Join date: 22 Dec 2003
Posts: 14,229
|
09-02-2007 01:40
Osgeld, I didn't miss your post, I just didn't have a response for it. I understood what you said and didn't disagree or have a question.
In 1994 I was making signs, pretty much like I am now. I have no employment related experience with any computers except home and office desktop style computers. I read Google News science and technology section, and sometimes I think something is neat sounding and want to mention it to someone in hopes of sharing the feeling of pleasure at the state of technology and in this case sorta stare google-eyed, so to speak, at the thought of so much ram. I don't have anyone else to share such stuff other than the people on the internet such as the readers of this forum.
Sigh, I just read some more about the 500 Ghz processor, that turns out to be a disappointment after reading the distorted summaries I'd read to start with.
_____________________
-
So long to these forums, the vBulletin forums that used to be at forums.secondlife.com. I will miss them.
I can be found on the web by searching for "SuezanneC Baskerville", or go to
http://www.google.com/profiles/suezanne
-
http://lindenlab.tribe.net/ created on 11/19/03.
Members: Ben, Catherine, Colin, Cory, Dan, Doug, Jim, Philip, Phoenix, Richard,
Robin, and Ryan
-
|
|
Osgeld Barmy
Registered User
Join date: 22 Mar 2005
Posts: 3,336
|
09-02-2007 02:07
From: someone
An K8 can do 2 FLOPS/cycle , P4 the same ...
From: someone
Cray X1E Supercomputer Sample Configurations
1 AC* Cabinet
Peak Performance 576 GFLOPS
1 LC* Cabinet
Peak Performance 2.3 TFLOPS
not to mention the cray can transfer 34 gb/s tru its northside bus |