misinterpreting data for fun and profit

In a Hotline to Linden thread there was a request for some information about the SL v1.7 performance problems. It was pleasing to see that they were actually trying to measure the impact of the problem. Unfortunately, the interpretation claims an improvement when the data presented complete obscures any meaningful interpretation. Here is part of the interpretations that is particularly flawed:It is hard to show in how many ways that interpretation is wrong, but I'll try.

First, the measure "fraction of the grid that is running slowly" is both a really coarse measure and one where the definition of "slowly" is pretty pessimistic. As far as I can tell from the graphs, the criteria for a "slow sim" is one that is running at under 20 sim FPS (out of a possible 45) - in other words, the "slow sims" are those that are running under half the speed that they could. How far below 20 FPS could these "slow sims" be running? Anywhere from 20 FPS to 0 FPS and where 20 FPS may be tolerable, 1 FPS isn't. How many of the "slow sims" are "really slow sims"? We don't know because the cut-off at 20 FPS hides distinction between "slow" and "really slow". Remember that the most a client can show is visual FPS is the Sim FPS.

According to the graphs shown, 1.6 would have up to 90% of the sims performing at some degree of "slow" at peak times. Also according to the graphs, 1.7 seems to show that on the whole the number of slow-to-very-slow sims is now peaking at far lower than they did in v1.6 with the worst case being about 40% of the sims running at some number under 20 sim FPS. What do those numbers mean? In the best times shown on the v1.7 graph, one out of every ten sims is performing somwhere between 20 and 0 FPS; during the worst times, 4 out of 10 sims is running somewhere between 20 and 0 FPS. What these graphs also hide is which of the sims are running poorly. Using the data provided, an extreme example would be that during the worst period 400 (out of about 1000) sims, the ones containing activity are performing at 1 FPS. This may not be true, but you couldn't tell whether or not it was from the data provided. Finally, the red "regression" lines in the charts are essentially meaningless.

Given the way the data are shown, 1.7 could be a better experience for all, or it could in fact be worse for all players. Given the anecdotal experience shown on these forums, I'd have to surmise that the data are giving a misleading picture of improvement where there is none and possibly degradation of performance for all active players.

Yeah, no kidding.

if sims coolsim, excitingsim, awesomesim run 20% slower but sim dullashellsim runs 400% faster, does that mean things are better?

And considering that scripts run slower now, I wouldn't go around touting sim fps as a measure of accomplisment.

None of this apparently addresses the texture taking so long to show up, even if I have seen or applied that texture before. To say nothing of the totally miserable time of trying to shop for textures now.

coco

Nice. I can still kick sim fps right in the nuts by doing a certain something. It takes either a reboot or a half hour to get the sim fps back to 45. IM if you don't believe me.

Yay progress!

You mentioned this before, the post mentioned reproduction, and getting down on your knees and making something rear it's head.

Whatever are you talking about?

20 years ago, half of all Americans were heavier than the median weight. This is still true today, so therefore we can conclude that Americans are *NOT* heavier today than they were 20 years ago.

I tend to believe the anecdotal evidence over the nonsense statistics.

1.7 is pathetic. If LL thinks its OK, I'm outa here.

* applause while eating a french fry *

It looks like those worse than meaningless statistics have been promoted to an announcement.

I've seen a lot of people come at a lot of issues with wildly divergent perspectives on these forums but even when I've wholeheartedly disagreed with posts here, I could usually understand the motivation prompting the poster.

This couldn't be a public relations move as these aren't public forums. It cannot be an "ignore the man behind the curtain" posting as most people don't care if there is a curtain or who may be behind it. It cannot be a "move along folks, nothing to see here" as everyone can see that there is indeed something to see here (or not see, so to speak).

I have to conclude that they actually believe themselves.

I can't say that I've ever felt more like Alice as she passed through the looking glass.

I would really really really like to give them the benefit of the doubt, cut them a little slack, and let them get on with fixing the problems. But its looking ominously like a denial that any problems exists.

As I type this, there are over 3,000 users in-world. They must be doing something.

I remember the good old days in the forums, when the drahma was whether SL must be made more appealing to ordinary people. What about the poor people with no talent. Well, guess what!

The statistic I'd like to see is how the $L upload texture cash flow. Up to 1.6 levels, LL? Hmmmm? Whatever those 3,000 people are doing, it sure as hell isn't uploading textures.

I would wager that sex balls work as well as ever, so maybe most people are obliviously humping away in their skyboxes. I'm a hands-on kind of guy, so I prefer RL sex balls. I don't see the virtual point.

I can attest also that annoying security scripts are alive and well. Now you can get teleported home from property you can't see yet. That's a real treat.

Is it me, or is LL trying to appeal to a different crowd? What's the LL business model? It just doesn't make sense.

Yes, they are changing their model. It's a build button instead of an event button now, for example.

Yup. Actually I uploaded a texture just yesterday, the texture image popped up clear as day in a window... I immediately applied that texture to a prim and waited about 3 minutes for it to materialize.

Well, I wouldn't dis ANY attempt at communication, ever, from anybody. And from game devs, it's always good, and better than no communication, even if it is limited in scope. Praise communication of whatever quality, and more is more likely to appear.

But I do conclude that they actually believe themselves. That's why I wonder if they see what we see. This business about textures, for instance - I just mentioned in another thread that the reply to someone's post in hotline about them was, can you tell us when and where this happened? I'm thinking, just go into the GAME, ANYwhere.

Which makes me think that not only do they believe themselves, but they are actually seeing different things from what we are. Or just aren't actually in game enough to be able to notice the difference between now and 1.6. I mean, on my computer, I am always slower than others, regarding textures, uploading (often doesn't work at all), rezzing, everything, you name it. But the differences between 1.6 and 1.7 is hugely obvious to me, and crippling, especially in terms of building.

It's not like a nice new update, or even like an update with problems we can all be patient with. I will keep being patient, but there are so MANY problems, and they are so dire in so many ways (regarding people's businesses, and now this new inventory poofing thing), I worry.

Then again - you know, they are at the helm, and they might very well be in a STATE OF TOTAL PANIC. Working their heads off overtime, knowing they've got all these new players, and things are worse than ever. (Or since February of 2005, when I came onboard, anyway.)

So they might be bending over backwards not to spread that same panic to us, and that may be largely why they sound rather too calm, to the point of obliviousness.

coco

So you can please some of the people none of the time and all of the people a few times and a few of the people once and one person seldomly... seldomly can you please some of the people but a few of them are seldom pleased.

One person is never pleased .

But you can please one person...

Unless its Prokofy.

Consider - Discuss.

I have considered and discussed, and just said I figured the problem had increased exponentially, and for more people. Thus - poop hitting fan.

coco

I was taking the piss outta malachi's thread title... do I have to make a disclaimer on every fucking post?

Sorry, Siggy. I am too literal at times.

coco

I have not known any of my scripts in non-busy areas run slower. I believe the change is that the server sacrafices script speed to keep a person's fps from slowing down. Maybe I'm misinterpreting their explanation and the speeds I'm experiencing though. I'm intentionally running the main script I use in a fairly empty sim.

If the script is running slower when the sim is full, then this is not that bad of a thing. If Tringo is trucking along same as ever, but no one can see their boards change, what is the point anyway?