Performance Comparison: Openspace vs. Regular sims
|
|
Wildefire Walcott
Heartbreaking
Join date: 8 Nov 2005
Posts: 2,156
|
07-22-2008 07:36
As I promised last week, I have published the results of a series of controlled tests I ran over the weekend on two standalone class 5 islands. You can read about the testing methods and see the raw benchmarking data at my blog (the link is safe for work so long as you stay on that page): http://wildefirewalcott.blogspot.com/2008/07/performance-comparison-openspace-sims.htmlI went to great lengths to prevent contamination of test results, and ran each test case three times after sim restarts to make sure the numbers were consistent. I'd post the whole thing here, but the formatting differences between BBCode (as if it even worked here) and Blogger are too different. I will, however, post the "Conclusions." From: someone Based on the final results, I believe we can safely conclude the following:
* Since an EMPTY openspace sim actually performs a little better than an empty regular sim on the script benchmarks, it appears that openspaces are tweaked to allocate more processing power to scripts to make up for their inadequacies.
* When running under a normal script load (based on a sample of currently-in-use openspace sims), however, script performance of a class 5 openspace sim is nearly 1/3 the performance of a regular sim. (Hey, it could be worse!)
* Since the "with objects" tests barely affected the full-sized sim's results, while they profoundly affected the openspace sim's final numbers, it's clear that full-sized sims have a lot more headroom for script processing; these tests didn't even touch a regular sim's threshold.
* Unless the large discrepency with test 5 on the openspace sim is attributable to some other factor (running on newer/faster hardware than the other tests?) the influence of server neighbors (other sims residing on the same physical server), seems able to have a non-trivial effect on openspace sims, while the interference appears to be less marked on full sims.
* Performing a user-initiated restart on a sim (openspace or full) does land the sim on another physical server most (if not all) of the time, and may even land you in another co-location facility.
* Even though the openspace sim scored poorly compared to the regular sim once scripted objects were added to the mix, the fact that sim FPS and time dilation remained admirably high shows that an openspace sim can still be perfectly usable and otherwise "healthy" even as script performance begins to suffer. |
|
Oni Horan
Registered User
Join date: 11 Jan 2008
Posts: 89
|
07-22-2008 07:45
From: Wildefire Walcott * Performing a user-initiated restart on a sim (openspace or full) does land the sim on another physical server most (if not all) of the time, and may even land you in another co-location facility. ehh? thats news to me... interesting |
|
Wildefire Walcott
Heartbreaking
Join date: 8 Nov 2005
Posts: 2,156
|
07-22-2008 07:47
From: Oni Horan ehh? thats news to me... interesting This only happened once during the tests, and only with the regular sim. I wonder if perhaps that one co-location site was full so I was pushed to the first available slot at a different facility? It was a big surprise to me, though. |
|
Kyllie Wylie
J-Rocker
Join date: 7 Mar 2008
Posts: 489
|
07-22-2008 08:28
I'm curious to learn what LL is using as servers now.
I'm sure what people refer to as "class 5" is in reality a mix of servers running on Core 2 Duo or Quad cores...... I mean I seriously doubt the CPU that was discribed in the 2006 blog as the "Class 5" server is even available anymore.
As LL has stated they added 44% more land over the last year or so, they would have had to buy quite a few new servers over the last year to run them.
|
|
Meade Paravane
Hedgehog
Join date: 21 Nov 2006
Posts: 4,845
|
07-22-2008 08:38
Minor pet-peeve.. It's a "sim running on a class 5 server" and not a "class 5 sim".
Yeah, I'm OCD.. Sorry.
_____________________
Tired of shouting clubs and lucky chairs? Vote for llParcelSay!!! - Go here: http://jira.secondlife.com/browse/SVC-1224- If you see "if you were logged in.." on the left, click it and log in - Click the "Vote for it" link on the left
|
|
Debbie Trilling
Our Lady of Peenemünde
Join date: 17 Oct 2006
Posts: 434
|
07-22-2008 08:44
Very useful & interesting info. Thanks
_____________________
http://wiki.secondlife.com/wiki/User:debbie_Trilling
|
|
Sassy Romano
Registered User
Join date: 27 Feb 2008
Posts: 619
|
07-22-2008 08:51
Seems like a reasonable place to ask my question of the day...what *exactly* is time dilation measuring or representing?
|
|
Meade Paravane
Hedgehog
Join date: 21 Nov 2006
Posts: 4,845
|
07-22-2008 09:08
From: Sassy Romano Seems like a reasonable place to ask my question of the day...what *exactly* is time dilation measuring or representing? It's a measure how well the physics stuff on the sim is doing vs real time. A 1.0 means that sim physics are pretty much running in time with reality. Lower numbers means that the sim is slipping behind - when you see people/stuff rubber-banding, that usually means that the time dilation has dropped.
_____________________
Tired of shouting clubs and lucky chairs? Vote for llParcelSay!!! - Go here: http://jira.secondlife.com/browse/SVC-1224- If you see "if you were logged in.." on the left, click it and log in - Click the "Vote for it" link on the left
|
|
Sassy Romano
Registered User
Join date: 27 Feb 2008
Posts: 619
|
07-22-2008 09:20
thanks, I simultaneously had this bright idea of googling too  http://wiki.secondlife.com/wiki/LlGetRegionTimeDilation |
|
Argos Hawks
Eclectically Esoteric
Join date: 24 Jan 2007
Posts: 1,037
|
07-22-2008 09:28
Did the full sims have 4 times the scripts and objects of the openspace sims you were testing? If not, then 1/3 the scripting performance on an openspace sim is actually better than the 1/4 that should be expected.
_____________________
Step 1: Create virtual world
Step 2: ???
Step 3: Profit
|
|
Wildefire Walcott
Heartbreaking
Join date: 8 Nov 2005
Posts: 2,156
|
07-22-2008 09:41
From: Argos Hawks Did the full sims have 4 times the scripts and objects of the openspace sims you were testing? If not, then 1/3 the scripting performance on an openspace sim is actually better than the 1/4 that should be expected. Both sims had exactly the same scripts/objects, and yes, I agree. It does seem like openspaces actually are tweaked a little for better script performance (presumably at the cost of some less crucial resource) even though they still falter in the long run. |
|
Argos Hawks
Eclectically Esoteric
Join date: 24 Jan 2007
Posts: 1,037
|
07-22-2008 09:48
From: Wildefire Walcott Both sims had exactly the same scripts/objects, and yes, I agree. It does seem like openspaces actually are tweaked a little for better script performance (presumably at the cost of some less crucial resource) even though they still falter in the long run. Maybe I'm missing something. I don't see how that's "faltering". If the openspaces are running on 1/4 of a processor, but they are operating scripts 1/3 as well as a full sim, that's really good performance. Going up from the 25% that's expected to 33% is a huge performance boost. At that rate, you could take a project that was planned for 1 sim, spread it out over 4 openspaces, and get a huge increase in performance without spending any extra money.
_____________________
Step 1: Create virtual world
Step 2: ???
Step 3: Profit
|
|
Desmond Shang
Guvnah of Caledon
Join date: 14 Mar 2005
Posts: 5,250
|
07-22-2008 10:05
This is by far, the best comparison test I've seen since these things came into existence years ago.
Very, very, very, very well done!
_____________________
 Steampunk Victorian, Well-Mannered Caledon!
|
|
Wildefire Walcott
Heartbreaking
Join date: 8 Nov 2005
Posts: 2,156
|
07-22-2008 10:18
From: Argos Hawks Maybe I'm missing something. I don't see how that's "faltering". If the openspaces are running on 1/4 of a processor, but they are operating scripts 1/3 as well as a full sim, that's really good performance. Going up from the 25% that's expected to 33% is a huge performance boost. At that rate, you could take a project that was planned for 1 sim, spread it out over 4 openspaces, and get a huge increase in performance without spending any extra money. By "faltering" I mean "not always meeting user expectations." I do agree that it looks like openspaces are doing better than they could be in terms of performance. EDIT: And thank you, Desmond. |
|
Ceera Murakami
Texture Artist / Builder
Join date: 9 Sep 2005
Posts: 7,750
|
07-22-2008 10:40
Thank you, Wildefire! By the way, I can confirm that the Lindens did at one point state clearly that OpenSpaces sims and Regular sims never inhabit the same regular server. It's always dedicated to either one type or the other. Can't nail down a link for that, however.
What I really wish we could manage to do would be to get 4 OpenSpaces sins at once under test conditions, knowing all 4 were on the same core one server, and verify once and for all if lagging the hell out of one of those four causes the other three to take a performance hit. The only reference I have seen to performance cross-talk was relating to a discussion on the viability of allowing *unoccupied* OpenSpace Sims in a core to temporarily allow part of their script processing allocation to go to occupied ones running under load. But the implication was that if anyone did a TP into that unoccupied sim, it would reclaim its script time allocation. There was no indication in that discussion that this was how it actually worked, but rather it was a "would it be OK if we did this?" discussion.
_____________________
Sorry, LL won't let me tell you where I sell my textures and where I offer my services as a sim builder. Ask me in-world.
|
|
Wildefire Walcott
Heartbreaking
Join date: 8 Nov 2005
Posts: 2,156
|
07-22-2008 11:13
From: Ceera Murakami Thank you, Wildefire! And thanks for the palace! I furnished it with all manner of sex beds, AOs and temp-rezzing jumping dolphins. Kinda like Graceland. |
|
Tali Rosca
Plywood Whisperer
Join date: 6 Feb 2007
Posts: 767
|
07-22-2008 11:14
So the conclusion as I've picked it up seems to be that openspaces are pretty much exactly a quarter-powered sim - and in fact a fair bit better than that, script-wise.
With prims, and by extension textures, is seems to be no problem, since they are limited by the sim. In fact, the smaller texture load can often make things a bit easier on the client, but that is not something intrinsic to openspaces; merely a side effect of the limited amount of objects in it.
Scripts, while actually having at least as much power as could be expected, can become an issue, since it's easier to forget that you're "quarter-powered", and approach "normal" sim use with those, and people can teleport in with running scripts.
It would seem to me from these numbers that the number of avatars should have a fairly big impact, but IIRC, Desmond reported that that wasn't really the case. Do openspaces have a "secret power reserve" to run avatars, or is it just that avatars don't really hit the sims that hard, being largely a bounding box with a lot of funky rendering client-side?
|
|
Ceera Murakami
Texture Artist / Builder
Join date: 9 Sep 2005
Posts: 7,750
|
07-22-2008 12:16
From: Wildefire Walcott And thanks for the palace! I furnished it with all manner of sex beds, AOs and temp-rezzing jumping dolphins. Kinda like Graceland. *laughs* And somewhere, the "Son of Heaven" who built the original palace may be rolling over in his grave, fast enough to power a city-sized turbine! Must have been quite a sight...
_____________________
Sorry, LL won't let me tell you where I sell my textures and where I offer my services as a sim builder. Ask me in-world.
|
|
Darien Caldwell
Registered User
Join date: 12 Oct 2006
Posts: 3,127
|
07-22-2008 12:23
From: Wildefire Walcott As I promised last week, I have published the results of a series of controlled tests I ran over the weekend on two standalone class 5 islands. You can read about the testing methods and see the raw benchmarking data at my blog (the link is safe for work so long as you stay on that page): http://wildefirewalcott.blogspot.com/2008/07/performance-comparison-openspace-sims.htmlI went to great lengths to prevent contamination of test results, and ran each test case three times after sim restarts to make sure the numbers were consistent. I'd post the whole thing here, but the formatting differences between BBCode (as if it even worked here) and Blogger are too different. I will, however, post the "Conclusions." Thanks, you've confirmed everything I've been saying for the last year.  |
|
Desmond Shang
Guvnah of Caledon
Join date: 14 Mar 2005
Posts: 5,250
|
07-22-2008 12:44
From: Tali Rosca It would seem to me from these numbers that the number of avatars should have a fairly big impact, but IIRC, Desmond reported that that wasn't really the case. Do openspaces have a "secret power reserve" to run avatars, or is it just that avatars don't really hit the sims that hard, being largely a bounding box with a lot of funky rendering client-side? Avatars do hit hard. But in ways that aren't necessarily noticeable (or might be VERY noticeable) depending what you are doing. If you are running scripted vehicles, or other such things - yep, you'll feel it alright. If you aren't running many scripts, if there aren't many textures stuck on prims all around you, and you are just chatting... then yes you can run dozens and dozens of avatars and *once it all loads* you'll see sim FPS and time dilation recover to incredibly good levels, even in an openspace. I'm talking class 5 here, on class 4's... erm... don't try this If done slowly enough, you may be able to load up to 60 or 80 avatars with essentially zero hit at all to the "just hanging out and talking" kind of region use. But again, it depends what sort of performance you are expecting: smooth physics (or don't care), fast scripts (or don't care) - all these things matter different amounts to different people. Many don't even realise when scripts are loaded down to a crawl, because they aren't using anything requiring fast script timing and just can't tell.
_____________________
Steampunk Victorian, Well-Mannered Caledon!
|
|
HoneyBear Lilliehook
Owner, The Mall at Cherry
Join date: 18 Jun 2007
Posts: 4,500
|
07-22-2008 12:52
From: Ceera Murakami Thank you, Wildefire! By the way, I can confirm that the Lindens did at one point state clearly that OpenSpaces sims and Regular sims never inhabit the same regular server. It's always dedicated to either one type or the other. Can't nail down a link for that, however. "Normal regions run on their own dedicated CPU, but the Openspace regions run four per CPU; as you would expect, this limits their performance. Openspaces only ever share with other Openspaces on a server. It is therefore important to understand what these regions are. They are provided for light use only, not for building, living in, renting as homes or use for events. As a stretch of open water for boating or a scenic wooded area they are fine, but we do not advise more serious use than this and will not respond to performance issues reported should you not use them in this way."
_____________________
Virtual Freebies now has its own domain!
URL=http://virtualfreebiesblog.com
The Mall at Cherry Park - new vendors, new look!
|
|
Talarus Luan
Ancient Archaean Dragon
Join date: 18 Mar 2006
Posts: 4,831
|
07-22-2008 14:21
From: Desmond Shang But again, it depends what sort of performance you are expecting: smooth physics (or don't care), fast scripts (or don't care) - all these things matter different amounts to different people.
Many don't even realise when scripts are loaded down to a crawl, because they aren't using anything requiring fast script timing and just can't tell. Well, the problem is that many people who are renting these don't know what they want or what to expect. They are often coming from full mainland sims where they know a certain X level of performance. Then, when they have their 20-person party with friends wearing full-prim avatars or lots of prim / scripted attachments, let alone on the same CPU with another group having a 100-person major event, they get all upset wondering why the performance is so horrid, get disgusted, and leave, all when it was spelled out clearly by the Lindens when these things were released that they were not MEANT for that level of use. Some people are savvy enough to understand the issues and therefore come into the game knowing full well what to expect, and thus have little issues, but these people are not the majority, since now, an OS sim rental is just an IM away to everyone. |
|
Wildefire Walcott
Heartbreaking
Join date: 8 Nov 2005
Posts: 2,156
|
07-22-2008 14:48
From: Darien Caldwell Thanks, you've confirmed everything I've been saying for the last year. Well you're not off the hook, missy! I'm still waiting for your mono and physics testing reports! |
|
Wildefire Walcott
Heartbreaking
Join date: 8 Nov 2005
Posts: 2,156
|
07-22-2008 14:53
From: Tali Rosca It would seem to me from these numbers that the number of avatars should have a fairly big impact, but IIRC, Desmond reported that that wasn't really the case. Do openspaces have a "secret power reserve" to run avatars, or is it just that avatars don't really hit the sims that hard, being largely a bounding box with a lot of funky rendering client-side? In addition to Desmond's follow-up to your question, I'd add that the contents of the sim also determine the effect that the avatars present in the sim will have. For example, if you've got a cube with a hundred textures inside it cycling images every couple of seconds, every avatar within viewing distance of that cube is going to be getting asset downloads, and the sim plays a part in that. 80 avatars in an empty sim are going to have a much better experience than 80 avatars in a densely primmed and scripted area. |
|
Wildefire Walcott
Heartbreaking
Join date: 8 Nov 2005
Posts: 2,156
|
07-22-2008 14:54
Thanks for that citation! |