Guidelines for Those Posting Surveys

Ok, call me a wimp or whatever . . . but it frankly distresses me whenever a survey, however poorly designed or explained, gets posted here and the poster gets flamed. I don't disagree with most of the responses to my suggestion that we be "kinder," in the sense that I agree that these surveys are usually poorly conceived, and those posting them frequently display little interesting in learning about the app first-hand. But it still bothers me, and it seems moreover a poor expenditure of OUR time and effort not merely answering poorly thought-out posts, but also flaming them.

So, rather than simply watching one survey thread after another getting toasted here, I thought I'd put together a list of guidelines for those posting surveys here. Ideally, it would be nice if this could be made a "sticky" (Millie, are you there??), but at the least, I am going to post the results of this on the SLED list, in the hope that the educators there pay attention.

Please add your own thoughts and ideas to this, and I'll edit the original post to make it more complete or "correct."

"Residents Answers" is a forum designed to permit those who have questions about Second Life the opportunity to seek advice and assistance from the more experienced residents of Second Life. Those who frequent this forum are enormously generous of their time and advice, and are frequently more than happy to assist those who request assistance with well-designed and thoughtful surveys or questions of an academic nature.

However, it is important to bear in mind a few important points if you wish to post questions or links to online surveys here:

"Similarly, some of the first educators and researchers in SL were a bit clumsy in their approach. There were early incidents in which Residents felt—rightly—that they were being spied upon or treated like guinea pigs rather than human beings. Imagine how you would feel if an entire class of students set up right next to your vacation hideaway, left trash lying around, logged your conversations, and posted them on the Internet with criticism and mean-spirited comments. This is exactly what happened to some early Residents. Things have improved greatly in the years since then, but sometimes SL Residents are still subject to that kind of inconsiderate behavior. For example, a constant stream of researchers (often students) posts on the Second Life official forums requesting that Residents please take their surveys. Most of these surveys include the same questions that have been asked over and over. Often they aren't spell-checked, or refer to SL as a game—a sure way to irritate a substantial number of Residents. Some Residents make a hobby of critiquing these surveys, pointing out how the questions and their wording make it clear that the researcher isn't familiar with Second Life. Some post outright that they are sick of badly planned surveys and suggest that the researchers log in and do their own research."

(From The Second Life Grid: The Official Guide to Communication, Collaboration, and Community Engagement
by Kimberly Rufer-Bach
ISBN-13: 9780470412916; with thanks to Marianne McCann)

Here are a few basic ground rules to posting successful and well-received surveys on the Residents Answers.

1) Forum residents have little patience with surveys or posts that reveal a lack of thought of effort. If you expect people here to take the time to fill out your survey, you owe it to them to ensure that your post(s) and the survey itself show some thought and effort. Watch for typos, spelling errors, and other simple mistakes that may seem to demonstrate that you have not spent much time composing them.

2) Surveys and questions on the forum can be a useful supplement to information gained from a first-hand experience of Second Life, but they are no substitute for it. Forum residents will invariably respond negatively to posters who have not taken the time to familiarize themselves to some degree with Second Life itself. Do not post here with avatars that are only a few days or a week old: give yourself time to visit Second Life, and speak to residents in-world before posting here. In your post, take a sentence or two to explain how you have explored or learned about the application first-hand.

3) Watch your nomenclature. Second Life is not a "game" in the way that, for instance, World Of Warcraft is, and residents will generally respond negatively to the suggestion that it is. Similarly, those who are part of Second Life are called "Residents": don't call them "players," for the reasons outlined above. Incorrect nomenclature shows a lack of familiarity with the application.

4) Provide some context for your survey. Are you an undergraduate, a graduate student, a professor, a researcher, or someone seeking information on starting a business in Second Life? What are your qualifications, if any? What are you going to do with this information?

5) Take the time to put together a good survey. Repetitive or vague questions will lead many to give up on the survey before they have finished it. So too will questions that reveal an ignorance of the basics of Second Life.

6) Post the results of your survey on the Residents Answer forum, and inform the residents that you will do so in your initial post. Residents are far more likely to contribute to a survey or questions if they have assurances that they will be privy to its results.

7) Remember that the Residents Answers is a community in its own right, and be sure to demonstrate your respect for that community, and those who are members of it. One simple way to do this is to respond to people who post on your thread: too often, surveys are simply "dropped" by people who never return to answer questions, thoughts, or critiques that are offered by residents.

Bear in mind that the community of residents who are active in the Residents Answers are NOT necessarily very representative of the resident population of SL as a whole. Use of this forum for surveys can potentially skew your results, as a broader cross-section of residents in-world would like respond collectively in a quite different way.

Show the residents of this forum that you take your own research work as seriously as you are asking them to take it, and you will likely achieve a much more satisfactory response to your resquest for information.

What this does not address is that the mere act of conducting a survey here is inherently flawed and therefore inherently poorly thought out. The demographic of the Resident Answer forum readers/posters is not representative of a meaningful cross section of the Second Life Resident population, so any survey results are going to lack reliability to form conclusions for which the survey is being conducted (except in the unlikely event that they are looking to study the Resident Answers community which has never been the purpose of any survey to date).

Ahhh! Thank you! This point (which was brought up in one of the survey threads still current) was one I had thought of, but forgot to add. I'll do so now.

There are scripters that make survey devices that reward folks for taking the survey. The device can gather info from the UUID of the participant.

Someone set up a 'Survey World' parcel and send these students there to rent a machine.

That's a really excellent idea. Does anyone know where to find such a machine? (I think Prok has a posting on it, which I'll have to try and locate again.)

Just need to type "SUrvey" on XstreetSL.com and there are a few. But any well-versed scripter can throw one together that meets specific needs. And the results automatically go to their web-based database where it can be viewed anyway they want.

But most of these student survey peopel do not know what SL has to offer so they don't realize there are easier ways to get what they want from SL.

Isn't it easier to have a sticky that says: "These forums is NOT for posting surveys..." in the title. With the reasons why under. Them those who still post surveys do it at their own risk.

Because frankly, I don't want to do (lazy) students homework for them twice a week, and having "guidelines" here makes it "legal" to post surveys.

But you don't have to, Marianne. No one is going to make you even open such a thread, let alone read it and participate.

Personally, I think that, if such surveys are approached thoughtfully and intelligently, they are an entirely legitimate part of the function of this forum.

And the fact is that, sticky or no, people ARE going to post surveys here. If they do, shouldn't we ensure that they are as well prepared and informed as possible.?

I ignore most of them. I am guilty of replying to some. As I am to you Scylla, I ignore most of your posts.

I disagree with you that they have a place here, in this part of the forum. Of course I know you have your opinion and that my arguments aren't going to change it. And vice versa.

If they had a own forum here for surveys it would be better.

Guidelines for those posting surveys - DON'T!

Pep (Can a mod make this a sticky please?)

That would have been my choice of wording, yes. Just DON'T.

Aside from that, Scylla, do you seriously expect people to read these, even if it is made a sticky? Do you seriously expect those same folks we mock regularly for their poorly phrased surveys and not even taking more than the 30 seconds it takes to create the account to give attention to procedural detail?

I don't think they will, just like that "no products for sale here" sticky is regularly ignored.

I think we need a separate board for Surveys and Research - then bone fide researchers can post there and those willing to help them can do so.

I just wanted to join this conversation as a researcher, to add my perspective. I can understand that feeling of being scrutinized from the outside, and it can't be pleasant. And if the surveys feel like an interruption to your community's conversations, that would also be irritating I expect. It can also be hard for researchers, who wish to support the SL (and, in my case, WoW) communities through research, who come to forums that are decidedly not pleased to see us. At the same time, we're expected not to discuss the topics of our surveys, which we've spent months developing and researching and are foremost on our minds most days. So, any conversations we might start by asking the survey, we have to let it take its course without jumping in and joining the community in a conversation we're enthusiastic about.

I agree with someone who said forums are not a representative sample. It also seems to me that surveys in SL might reach a limited number of residents and not be a valid crosssection of the whole SL community - but perhaps people with more SL experience than me feel differently about that.

As a suggestion: at one SL forum, I was asked to introduce myself to the community in an Introductions section, and put the survey in the Help Wanted section. I found this worked well for both me and for that community - they got to know me a bit, I didn't feel like I was just invading their forum. So, I like Deira's suggestion about a dedicated forum, perhaps inviting folks to introduce themselves first.

What makes you think that "The SL Community" actually needs to be "supported by through research"?

You wouldn't go to Israel and ask the residents there to take a survey about where they display their crosses, would you? That's exactly how most of these surveys come across to us, an obvious disregard and lack of interest for the culture they claim to be studying.

I think for me, that's the biggest issue. If researchers do spend months developing a survey, how is it so many of them are ignorant of the sub-culture? A couple of hours in these forums should tell them we are not gamers and SL is not a game (though it can be to the few who want it to be). Also most people don't like to reveal real life info for obvious reasons. And not everyone is in world for sex, and not everyone portrays themselves as human, and not everyone -- well you get the idea. There are hundreds of preconceived notions that are simply wrong and it's too difficult to set all these clueless tourists straight.

I actually don't mind being a lab rat. I have responded to some surveys when they are not insulting. SL is one of the strangest ways people interact when you think about it. We ourselves tend to raise questions to each other about our SL experiences. It's the researchers' blatant lack of understanding of the sub-culture that puts most of us off.


.

Well, thanks for responding with an utterly gratuitous insult, Marianne. You felt it was important to tell me that you usually ignore me because . . . ?


I have no problems with the suggestion that there be a separate forum for this sort of thing. The problem is that there isn't one, and LL is hardly likely to create one now.


No, I don't expect that all will read it, yet alone abide by it. But some will, our friend Anthony in this thread being, I suspect, a case in point. And if they do ignore it, we have a place to point them to, and more legitimacy to our complaints.

Were we to get a sticky that simply said "DON'T," I'd be cool too, although I still think that well-constructed and conceived surveys are a legitimate tool (used properly), and have a legitimate place in this forum.

The point is, people, that the status quo sucks: we will continue to get these awful surveys, and will continue to respond with an incivility and hostility that reflects very poorly on this forum.

I once suggested just having a separate forum for surveys. People who wanted to participate would go there. I forget the specific reasons, but most respondents to my idea hated it. Now, the idea is moot since these forums are in the final stages of abandonment.

This would be my recommendation as well.

I really can understand SOME of the survey requests and, if we all think about it, we really would prefer that the rest of the world out there really would gain a true and valid understanding of SL and stop with all the media horror stories. Not to mention that I am really so very tired of people that just do not have a clue telling me that my SL time is unproductive or a sign of some problem in my life. The better the rest of the world comes to PROPERLY understand the virtual worlds and their various usages, the better we all will be.

It would be far better though, for us and the researchers, to create a separate sub-forum for it. That would make it easier for many to ignore and flaming of badly written surveys or request would be perfectly legitimate.

IT does - if only to reduce the number of sensationalism media articles and the constant portrayal of us 'onliners' as perverted or warped or mentally disturbed or just plain disfunctional (more so than the rest of society).

They are not going to get that clue with a proper understanding...by doing a survey, LittleMe. It can't be reduced accurately to several dozen questions, formatted incorrectly, with inaccurate choices.

No, but IF they are doing proper research, the survey will be only one of many aspects to their research. Surveys are a valid method for a portion of research on a topic, and even if they know the survey is only getting to a certain type of resident/player, as long as they know that, they can view the results appropriately.


The biggest issues for me are putting up a survey and request that actually indicates time and thought went in to it, and showing respect for the community in which they are trying to study.

What do you suppose the odds are.....that any of these people actually did REAL research, beyond a survey and a google search?

Most of the people posting surveys here are not doing it as part of some research project that will produce a meaningful paper in a scholarly journal. Most are students who are doing a research project for the class. It's not the final findings of the survey that are important; the important thing is for students to go through the process of doing research, experiencing the problems and complexities, and understanding the limitations of what methods they are using. It's practice and education so that one day some of them can work on those big research projects that result in widely-cited and debated papers in scholarly journals.

*laughs* But see, these threads turn out just like the rest of the forum posts.

Less than a tenth of a percent probably and unfortunately.

However, in reading Anthony's opening post on his request, reading the first page of his survey and on reading his comments on other researchers' works, I would be willing to bet quite a bit of real money that his research on this topic encompasses much more than this survey.