Bot count, the sequel: 52%

Anya's studies are interesting within the limits of sample size and bot-determining criteria, which limitations she fully acknowledged in both of her posts. And she has not made exaggerated or inappropriate claims as to what they show about grid-wide activity. She's made some hypotheses, and provided reasoning to support the tentative extrapolations she's offered in those hypotheses.

The difference between Anya and all of the folks in these threads complaining about how Anya's studies don't show this and Anya's studies don't show that is: Anya is actually out there conducting studies.

Anya's first study was criticized because it was done at a time of high concurrency. So she went out and did another study at a time of low concurrency. She didn't change her sample because any smart researcher knows you don't change two variables at once.

Anya is doing interesting and responsible studies. From her posts it is amply clear that she understands the limitations of what can and cannot be concluded from her work. I for one am very interested and grateful to her for this hard work. Anyone who questions her methods is free to devise their own bot-identifying criteria and sampling methodology and get to it. I look forward to seeing your results as well, and seeing how they differ from or are similar to Anya's.

But until there's data to compare to Anya's, all the protestations about how "the results would be different if ..." are just conjecture and hypothesis at best. And an untested hypothesis is not worth very much compared to a hypothesis supported by and reshaped in the face of actual data.

This is a red herring.

There's nothing in this data or methodology that suggests it would be used to identify whether individual avs should be banned. That would be like using a RL traffic survey, ostensibly for planning or identifying safety issues, to prosecute individuals.

I'll rephrase what I wrote:- ... but this sort of sweep on its own isn't going to give any such indication.

Indeed.

Phil, and the rest of you, if you don't like the results, if you think her criteria are wrong, do your own studies.

Why? We aren't reinventing the wheel here. I've been involved in statistical surveys (DC Jail, for the DSS, various studies for traffic for bicycling, various lab studies as part of what one of our divisions did routinely) and the requirements for a minimally acceptable survey weren't met here. We documented the survey points and for anyone with the necessary access or that were doing a survey of surveys, they could review our raw data and reassess that data against the method used in other surveys and compilations of raw data. Most of the surveys collected the data and compared it against the other collected data to see if there was conformity (you rarely get uniformity). How was that done here?

Its nice to play the 'gotcha games' in SL and on the forums, but people, any one that does this for a living (and I don't - I have just been involved, including sitting through the sessions on stat results that put me to sleep) could explain why this is no step at all, except a bit beyond opinion. Not that I have a problem with it really. I don't think the issue merits the work that would be required in RL for a real survey. But lets cap the talk about this being a real survey or a real study of any significance. Its not. FWIW, every survey started with a well defined analysis of the scope and area of the survey and a specific description of the expected range of possible results, with an explanation of how that did or did not extrapolate to the general population (absent that last item, generally it was considered for most of our purposes to be useless and not worth bothering to try for grants - occasionally it was still useful).

Of course I am not going to do one myself. I don't think it is worth the effort. Unless you actually were able to do all the above and get the kind of waivers that I think LL would have to give you to just do the survey groundwork, the results would go no further than this one did - everyone would still be left with their favorite arguments. To get the range and waivers necessary, you would probably have to sign a protocol with LL, and you probably would never be discussing the results on this forum...

Most of the above is just my opinion, but I've had better surveys shot to pieces like a stop sign in hunting season by the real experts.

Why?

Rha, that's an argument for never trying to learn anything that isn't already known. Feel free to choose to remain ignorant, but don't attack a reasonable attempt to get some data.

These results may turn out to be skewed. At this point, we can't tell. But with more input, we can.

And you don't need a "sure fire" way. You only need a way with a reasonable level of confidence.

Have you ever actually seen a bot farm? They're pretty obvious.

That depends, I guess, on what your motivation is in dumping on Anya's studies. But, regardless, if the only data that's available is low quality, but it's the only data available, it's still the most credible data around.

If people get together to gather more data, and decide on what is and isn't counted as a bot, I suggest leaving campers completely out of it because there is no way of knowing whether or not an av that doesn't respond is a bot. It could be individual who has turned in for the night, or watching TV, or out for the evening, etc., or it could be a camping bot. I don't count individuals as bots, regardless of what they doing.

I'd also suggest leaving manequins and models out of it because they are functional bots.

I'd suggest simply counting those that are obviously bots - groups of avs in boxes, in lakes and places like that.

I wouldn't describe my posts as "dumping". I've discussed Anya's posts rationally. My motivation is to show that the data *on its own* isn't anything to go by. I don't have a pro-bot motivation as some people who don't know my views probably think I do. I'd like to know how many traffic bots there are on average, but Anya's data on its own doesn't give a reasonable indication of that and I've shown why, that's all.

I did say that it would have been much better if the second sweep had been through a different set of sims, but it's just a repetition of the very limited first sweep.

Argent, the only thing worse than NO data...is incorrect data. This raw data is no more or less accurate than what the Lindens give us.

None of the people here need to run their own surveys, because none of us decided to bring bad information to the table.

Edit: I'm going to rephrase my own post. Anya's information isn't BAD...I think it's just incomplete.

Hey, some of the best parties are in boxes, lakes, and places like that!

I don't think you can base the bot criterion on the location of the avi, anymore than you can suppose that someone who doesn't answer is a bot.

/me didn't mean to be rude here, Anya. Though I don't agree with your numbers, you're one of the few who are trying to get them.

/me sends Anya a cookie.

I haven't followed the links to see if the data posted actually qualifies as a complete set of raw data, but presumably that would be easy enough to do for future surveys. As for the comparison, what was the first set of data compared against? Obviously there had to be a first, with no previous data to compare. You didn't throw it out, you waited until more data was collected.

You can't expect methodologies that are used in established survey areas to apply exactly in areas where new ground is being broken.


These are good points, but I don't think they're that far out of reach.

The scope was limited to one particular continent. I'm not sure what percentage of sims on that continent were surveyed, but I'm sure that's easily obtainable. The timing of the survey is well-defined, though not necessarily well chosen. I think it's fair to conclude that the survey looked at 100% of the avs on the selected sims within the given timeframe. I haven't looked a the raw data to know whether the time is included, but it could certainly be included the next time the survey is performed.

Since this survey is exploratory, the range of possible results doesn't seem an appropriate or necessary question. One could give the trivial response "between 0 and 100% satisfying the bot criteria", but that's not useful.

How does it extrapolate? It doesn't. But rather than invalidating the results so far, this is merely identifies one of the open issues that must be addressed before applying this survey to the entire grid, or even just the mainland subset of the grid.

I disagree with your assertion that this is no step at all. When the methodology itself is breaking new ground, it doesn't seem at all unreasonable to begin with a preliminary survey before addressing or solving the issues you raise.

Look at it this way. When Dewey beat Truman, it didn't mean the survey was useless. It was extremely useful for learning how to do a better job of surveying next time.

Count them separately, perhaps, if you can't tell if someone's a non-responsive camper or a bot. Count the decorative bots and campers separately from the bot farms and camping holes, too. But count them all. If the result is:

48% active avatars
5% inactive decorative avatars (models, dummies, gardeners, etc)
47% inactive clumped avatars (camping chairs, bot farms, etc)

that tells a different story than if the result is:

48% active avatars
37% inactive decorative avatars
15% inactive clumped avatars

This is what's called a "follow-on study". Since this is open-source research, "submit a patch" by doing the study yourself if it matters to you. If it doesn't matter enough for you to "submit a patch", you're probably already spending more time on it than it should be worth to you just posting about it.

Don't forget those hidden inside invisiprim spheres and randomly distributed in all three dimensions (look for their shoes).

I haven't seen any indication in this thread that this is incorrect data, or even that it's too small a sample to be useful. It's based on a statistically significant sample size, over a population of sims that should be representative of the mainland. It may not be representative of estates, but that's also a useful follow-on study.

LL has already analyzed this data. Lindens have mentioned on several occasions that bots make up about 10 - 15% of the total user hours and that the the ratio of bots to human users has stayed steady for many months. They've also mentioned that this is backed up by the rate of spending per user hour and other related stats.

Seems like as good a guesstimate as any other. The grid is vast and varied. I don't think that sampling a slice of the mainland is going to produce any stats that are accurate for the grid as a whole.

Phil is right - What we really need to do is get rid of the reasons for having traffic bots and the problem will be eliminated or at least greatly reduced. I'm in favor of just eliminating the Traffic counts all together, but removing Traffic's effect on the search results might be enough.

You also haven't seen how the particular band of sims was selected - none of us have.

It's not a statistically significant sample at all. You could say that the population of Arizona is a statistically significant sample of the U.S. population but, if you'd polled them all before the election, you would have concluded that McCain would now be the President-elect - easily. That's because support for the two candidates was unevenly spread through the U.S. For a statistically significant sample, you would have needed to poll some form each state. It's the same with SL's population - it is unevenly spread through the grid and a band of sims doesn't cover it on its own. Not only that, but we don't know how the band of sims was selected. It may have been selected because it had plenty of green dots, some or many of which gave the appearance on the map of likely being bots. That's a possibility - we just don't know.

added:
I think Arizona is McCain's home sate - I meant his home state, anyway.

With all due respect, I think there is a complete break between that and what constitutes a real study (comment: the OP never claimed it to be a scientific study AFAIK, and only went as far as using it to extrapolate it to the grid, which is entirely their right). Having seen several people working on their masters or doctorates keep coming back after discussing their survey structure with the consulting prof, the groundwork issue IS HUGE. You don't just wander off, collect raw data and call it a survey. You don't do it in business (one of the major steps in consulting projects is defining the context to which the project applies), you don't do it in lab work (efficacy for one purpose and context is useless for another), and you don't do it in real surveys. Absent that first necessary step, the data is no more than darts thrown against a board.

And I never heard that either the raw data or the survey could be done poorly, just because it was the first. Again, I think we are comparing real surveys against one that was done casually, so I don't mean that to be overly harsh.

That said, generally no one discussed the data or survey or project results without having read the prep work (which is why I never had an opinion at the meetings). You couldn't discuss the data or the results unless you knew the methodology, and that only makes sense (in most cases) within a context. Test results for a tendency for a certain type of cancer is different than the same test for a person having that cancer, as an example. So how many giving opinions here feel that they have enough information that they could go out and do exactly the same survey with a conformity (again, not uniformity) of results? If so, do you believe that there is enough definition and data records of the process that it could be used and compared against future studies? IMO, no, not exactly, and no. So also IMO, no progress.

It was probably a fun little thing to do and IMO was probably seen through to the end because it was producing results in line with what they expected. I still see it as nothing beyond another set of arguing points and it only shows how few common assumptions we have to start a marginally biased 'neutral' survey.

Thanks again for your efforts Anya.

Keep up the good work.

I'd approach the problem differently.

Leave out all mention of terms like "bot" and "camper". They're loaded terms that provoke debate. At least for bot, it's unlikely that there's a set of criteria that's 100% accurate based on data available to everyone.

Instead, simply define categories by letter. Perhaps Category A is a group of one or more avs in an empty skybox or platform. Category B is the same av sitting on an object for over an hour with no open chat. Category C is an av in a club for over an hour with no open chat, but not sitting on an object. We might still argue over the significance of the categories, but at least there won't be any debate over what the numbers are measuring.l

These are just sample definitions. I'm sure there's a better way to define them.

Given the size of the continent, "a four-sim-wide path, coast to coast" doesn't leave much room for variation. If you want to do a quick check on the data, you could simply scan all 1-sim-wide chunks on the continent, look for a contiguous set that has about 600 avatars and 219 sims, and you know the band.

If the distribution of parcels across the mainland tended towards any kind of clumpiness that meant that a roughly rectangular collection of 200+ sims had a significantly different distributaion of land use than any other, that would itself be interesting... because sims really don't have THAT much differences between them, like the difference in terrain and climate between New York and Arizona. The biggest discrepancies would be the FIC sims, the snow sims, and the coral sims around the edges of the north continent. Corsica is typical of most of the mainland, with a mixture of flat green and hilly green and stone sims, and coastline... and I don't think there would be any swath across Corsica that would be all that different from a similar swath across Nautilus or the Southern continent or most of the main continent.

I am not remaining ignorant, because I choose to not do a survey that I know would not be valid and just

and I sure as heck would not post it on a forum, without some proof to back it up

This is what gets me, people seem to be taking her word as gospel, that she can do not wrong, and that her numbers are fact, then attack those of us who feel the opposite

that her numbers are flawed, and most likely the flaw is human judgment and error, and because we choose to not do the same thing it makes us ignorant...

ok... if that is the way you see it, then so be it

I stand by my opinion, her 2 survey's or whatever they are called, are in my opinion, not worth the time or effort of putting them out there (even if the time and effort were to make up bogus numbers)

I am not saying she made it up, but there is always that possibility, which is why I do not trust it, or put any emphasis on it

*shrug*

obviously others will disagree, and therefor we will be on opposite sides of the fence

(for the record, I do not know the OP, never spoke to them in my recollection, and do not even recall seeing any posts of theirs before this bot survey thing)


and thank you for insulting my intelligence asking if I have ever seen a bot farm...

I have been in SL nearly 3 years, yes I have seen bot farms, I have also worked for a guy that set up massive camping dance things, which some were bots some were people, part of my job was to determine which were ppl and eject the bots...

on a small scale (2 sims max 100 campers each sim) it was still not easy to determine without a doubt if it was a bot or person.. I did my best, and sometimes was in error...

however I am not stupid... thank you

You could be correct, but those are assumptions, not supported reasoning. There could be a significant clumpiness if all but eight sims were bordering open water or included disproportionately some of the features you mentioned. The age of the sims could create significant differences. We don't know and until we do, it is a potential source of error in survey and/or extrapolation to the whole and is nothing more than opinion, one way or the other. Phil says it does, you apparently think it doesn't, and I think unless we actually know, it doesn't make any difference, its all just arguing points. Certainly that it is mainland and not including any theme structured islands means that it only holds well for the mainland (IMO).