Ratings: Do They Serve Their Purpose?

The rating system is, of course, designed to show what others think of you. But at this point it seems to me like it has decended into a high score contest of sorts, considering very few people actually rate fairly, and neg. ratings are given out only for serious offences. Is this right, or should it be changed to act more like a Whuffie?

You need to ask yourself that? Pretty much any community based system sucks. Does voting work IRL? Does slashdot karma really measure how interesting your posts are? No.
Rule #1 of Comp Sci is Never Trust The User. You cant expect the people to organize themselves. There are karma whores in slashdot and there are rating whores in SL.

To be perfectly honest, I've made a point of negatively rating the objects in-world lately that I think are crappy. Behavior ratings are easy to give: if you're shooting me, negative. If you're social with me, positive. If neither, then pfft.
If you have an appearance that suits the voice, or if you seem to be setting up an appearance that is thematic or your appearance seems to have taken a lot of work, ooor if it just isn't "offensive" to me somehow, then positive rating. Also, for appearance, I like to rate based on gesture usage, too.
I rarely rate on building unless I see something I like. But I'm not hesitant to negrate someone's building if their builds are eye sores. So. I like the rating system. It's a good way to give evidence to someone that I do/do not like what they've made. Sometimes just saying "I don't like what you built. " gives eh, a very er... offensive reply back... so yeah.

do ratings mean anything or are they just so people can look at your profile and decide if you're cool or not? what about calling cards? everytime i go to an event i get buried by cards then i never see those people again (except everytime they log on). most never even say hi. what's the point?

What are ratings good for? Money! Certainly I wouldn't bother to give out ratings, if it didn't mean a material reward for the person I'm rating. I'd much rather just compliment them. Calling cards also bring in money. It's a bit strange. I am guilty of asking for calling cards at events, simply to support what I'm building. It's not something I would do otherwise, until I'd had a chance to chat with someone.

So, to me ratings are not a contest, but simply a way of keeping myself and people I like financially solvent.

Yes, ratings are good for money.. but I propose a sort of "rating decay" Two problems with ratings are 1) Rating mining resulting in unfair amounts of money given to select residents and 2) Negative Permanence, when you are rated negatively, you are marked for life. A public criminal record would be an adequate parallel to illustrate this point, which is what negative ratings can become.

The solution:

I believe the rating system should be infused with a "decay" property. When you get a rating, say, a month later it would be stricken from the record as it is no longer relative to your current level of performance in SL. Rating miners would have a harder time maintaining their grossly exaggerated ratings, as well, the evils you took part in the past would not stain your new, repentant self.

Any comments? suggestion? Does this idea deserve it's own thread?

feniks touched on this in another thread...

I have enough trouble knowing how much I get for a bonus now. If rating decayed, fluctuated, etc it would be even worse. You might as well just roll dice for your stipend.

Personally, I try to at least put a little thought into them. As long as someone is nice, and looks like they put some thought into their appearance I will give positive Appearance and Behavior ratings after talking to them a while, attending their event, etc. Yes sometimes I give return ratings for people who rate me randomly but I will rarely initiate that myself.

Building rating I am much more strict about, and it seems that others are too juding by my numbers and how they stand relative to other people. I don't give +Building unless I've seen them build, obviously. Whether it's a house, a chair, or attachment doesn't matter, but at least something they built or scripted. Being able to rate through builds makes that much easier now. I also do this on a sliding scale. Someone knew who's doing some building in the Morris sandbox, and they are competent and doing good work for their level is easier to give a +1 to, sort of as encouragement. Someone sitting there with +150 Building, making a simple bench probably won't.

Perhaps if the different ratings were weighted against each other a bit, such as Building being worth a touch more since it seems harder to get, and if you are building actively you need the money more. Whereas getting a high Behavior ("Nice guy, that" or Appearance ("Oooh she's cute" is easier, so they are worth slightly less than a build rating.

Just some rambling from a bored code monkey at work.

No.

There's something I think I'm missing in these ratings threads. Rate-mining. I read in another post that rate mining is like going to a lot of events, exchanging calling cards with everyone you can, and to rate others for their looks or manners in the hope they will rate you back. Is that correct? It seems that a lot of people look at that sort of thing as being bad.

Huh?

I thought the -purpose- of a lot of meet and greet events were put on just for that. Metting new people, trading cards, and handing out a few ratings to the people who seemed nice and/or put some real work into their AV's.

Am I missing a piece of this puzzle? Seems pretty fair to me all the way around. I help out someone elses cash flow, and hopefully they think enough of me to help with mine while we all say hi to one another.

This is wrong? How exactely is trying to get your own ratings up while helping others do the same wrong? This is the question that gets to me, how is this a bad thing?

It seems hard to knock this system unless you really don't like being social and get ticked off that others are rated higher.

I like to rate people, especially if their av or project is something I like.

I like the idea of calling cards, but I admit they can get out of hand. Within my calling card folder, I made one for friends - since I'm fairly new its small - but it's people I've 'sat' down and talked with in SL.
I would be for being able to edit the calling cards once you have them, so you know why you got it, like:
* random exchange
* event exchange
* Oooh! cool costume
* neat build!
* annoying - stay away from!

I think this would help me remember who people are - especially if their AV changes a lot.

I think people will remember me - there's not too many big green Fairy Troll Princesses around. LOL
Unless of course I change the look.

Because, Jellin, whenever two people rate each other they are taking money away from the rest of us. That's how the system works, and that's why you lose a lot of bonus when you are away from the game for a little bit. It's become a race, and if you stop running to catch your breath everyone catches up. I dont like this sort of frenzied competition, especially in a place that describes itself as a collaborative environment.
I am pretty social, I have loads of friends but jesus, theres 1500 people in SL, how many friends do you expect me to be able to make?
I have better things to do than fly all the way to stage 4 to attend yet another crappy event, namely building, scripting, and yes, socializing, not with some random trial user that will be gone in a week, but with my good old friends that I have known for the past 6 months.
The point of this argument is moot anyway since the lindens already have a new system planned, just check the "tiered stipends" thread on the feature feedback forum.

OK, I understand Eggy, if person X is doing great, is popular, gets good ratings, then he's taking money out of person Y's pocket cause they don't try as hard. OK, got it now, thanks.

I don't agree with it in the slightest, but I understand it.

Jellin what you posted as rate mining isn't rate mining because of these words: "rate others for their looks or manners"

Thats not rate mining. Rate mining is rating someone with the main or sole purpose and intention of getting a rating in return. You do not care whether they look good or have good manners. You are rating them so they will rate you back. You may never see them again, may not know anything about them.

Why is this bad? Bonuses are paid out in the following manner:

( Your Ratings points / All ratings points in game ) * total pot being given away.

(Its a little more complicated because I think it seperates the values for each rating and calling cards etc but this is the basic idea.) The total pot is fixed, and your ratings points is smaller than all given in game and thus you get a percentage of the total pot. Does that make sense? So what rate mining does is water down the system - it unfairly raises the 'All ratings points in game' number. Which means everyone gets a smaller amount of money. It means that the more 'All ratings points in game' there are the less value a single rating has. And it gets to the point (where it is now I think) that a single rating is near worthless. To make any difference in $$ you get you need many ratings - which means more rate mining which means votes are worth less so you need more etc etc etc.

---------

And yes I know that 'Total pot being given away' isn't quite fixed. In terms of an 'instance' that we are looking at it may as well be though. It fluctuates based on how many users are in world, and we can assume for this equation/example that it is constant.

--------

And back to the topic! Ratings are intended as a way for one person to say to another person "You deserve more of the server resources". And no, I don't think they serve that purpose any more. I think it worked semi well with a small player base durring beta, but as the player base grows, as the numbers are influenced by trial users, I think it has lost much if not all its original meaning.

People used to go around and rate people, and then say, "Now rate me" and if they didn't, then they would neg rate that person. Given, that's the most extreme case of rate mining, but it's how it happens sometimes.

The more subtle way is people will go someplace with lots of AVs, and rate everyone, regardless of their opinion, and then just *expect* a rate back.

I've had some people get mad at me for not rating them back when they rated me... but I'm usually either mentally vacant, or just haven't been wowed to the point of rating them.

*curses, Ama is too quick for me*

If we are keeping the rating system, I'd like to be able to rate a person more than once.

I've only been on a month and I've seen multiple awesome av's and builds by the same person. But I can't rate them again.

so if i rate someone who's nice, or who looks great, or who built something that knocked my eyes out then people are going to assume i want something in return and think i'm a jerk? that's fun. unless you're a psychic friend please don't assume you know what someone's thinking or why they do something. cause you don't.

if people are going to think you're an insincere ass for giving them a complement then why bother? let's all stop rating entirely. nothing would change if we stopped. if your ratings are just a percentage of a total amount of money and if you're increasing your rating while everyone else is increasing theirs then you'd all keep the same percentage of the pot. and no one will get offended because you think they look good.

you now have a little text field to justify your rating to the other person, please use it always

Nastasja, I actually assume all ratings I get are not rate mining. I rarely do return rates. I rate people as I see they deserve rating. If I get a random rating I think maybe they saw my Arcades or thought one of my posts was insightful (I have been told that as the reason for a vote or two) and am thankful. But I don't think that rating me is a good reason to rate someone. And I'm fairly low on the boards as far as ratings for someone who has played this game for about 10 and a half months.

Topic reply:

No

You know the old chestnut about classifying pornography - "I know it when I see it" - ? That's how I feel when presented with this discussion. I know the rating system is broken when I use it...

Let me briefly comment on how I use them... most of the time, I've found that someone who's making an effort to participate has either created a nice avatar or is behaving themselves, and generally both. So, rather than cost myself the extra $1 per person, I'll rate both behavior and appearance positively at the same time. Moreover, if someone isn't behaving, there's almost no way I'll be generous enough to rate them on just appearance.

The building rating I save until I actually get to see something built by them. This way, somewhere down the line, I can "meet" them again, and hopefully receive my own good ratings in return sometime (mine are all good, of course ).

All nice and rational... and it stinks...

Why should it cost me money - why should money ENTER THE EQUATION? - to rate someone?

Why should my three positive ratings for someone like Resident #1 - a longtime resident who has created dozens of spectactular builds and costumes and helped me out on numerous occasions - be equated to the three positive ratings I gave to the new person I met last week who's gotten off to a good start?

<edit/removal of less-useful ranting>

Okay, I haven't analyzed the system, I haven't thought much about a redesign, and I know whining isn't going to solve anything.

That said, here's a few suggestions to toss around:

* No more charges for rating - residents get a pool of ratings points they can distribute.
- - This pool could be recharged on a cycle, weekly probably.
- - This pool could be part of a larger pool distributed among residents, or each resident could get a base allotment, like the stipend.
- - This pool could be larger or smaller for residents who meet certain criteria. Trial users could have less. Residents who perform an official SL function - Mentors, Liasons, perhaps "Judges" of some kind? - could have more points usable within strictures.

* Residents could rate other residents multiple times, within restrictions
- - Allows perhaps for multiple positive and negative votes, perhaps both (!); a resident with two positive votes and two negative votes from the same voter, instead of a plain "neutral".
- - Votes could be "given" on a permitted schedule; example - the second vote of a person could be after a week following the first vote, the third vote after another month, the fourth after another two months, etc., up to the vote limit. This would help prevent abuse where a resident maxes out their votes for another resident immediately, without justification
- - Votes could be "retracted" within a certain time
- - Alternatively, residents could rate each other linearly, on a scale from say +5 to -5, with each increment allowed to be changed after a period of time as above

These sorts of suggestions are meant to be more favorable to residents who produce consistent behavior and/or builds over periods of time, and to reduce the impact of abusive behavior by constraining multiple votes over time.

/Soapbox. Comment invited...

=Sinclair

What I don't get is why users are TOLD about being rated. Why isn't it hidden as much as possible?

What would it be like if others did not get notification of being rated by you--and, not only that, but if the public profile ratings were delayed so that changes all take effect at the same time for everyone?

Summary:
No notification for rating someone, and especially no + or - notifications.

Delayed modification of a player's rating scores, updated once a week (or so). This is to make it more difficult to determine WHO is rating you.

Furthermore, eliminate the silly $1 charge to alter this.


What's wrong with this system? Seems very simple, would require very little work to do, and appears nifty.

(btw I didn't read the entire post above so if there's any repeat, nevermind....hehe)

without notification, how can ratings influence behavior? especially negative ratings. If I dont know you are rating me negative, why would i have any incentive to change the behavior or address the problem. If I dont know I'm getting rated positive, how will the rating be positive reinforcement?

If I just looked at all the ratings at the end of the week, how would I know if i got the ratings hanging out at the welcome area, participating in an event, hosting an event, etc. As it turns out hosting events garner more positive ratings for me. So is it rate-mining, if I decide to host more fun events because of the positive feedback received? I think the ultimate objective of the system is achieved in this instance. Reward social behavior thate benefits SL and discourage anti-social behavior. I agree it doesn't do much for content creators. This is definitely a system geared toward socializing.

To answer the topic question, depends on what you think the purpose of rating system is. If its to encourage socializing, then i think it does serve its purpose. If the purpose is to balance resources among all the types of activities in SL - building, scripting, modelling, texturing and socializing - then no. It does a very poor job of representing builders (and scripters and texturers who dont focus on avatar design)

I feel they do serve a purpose if people use the system correctly. Gaming it by "rate mining" is an unfortunate thing, but we all know it happens every day. If you rate me for something I have contributed to the world, or the way I look and act, that's really nice of you! Does this necessarily mean I have to return it? Well no one is holding a gun to my head I suppose.... Chances are I'll rate you back just for being such a nice person, especially if I have met and/or seen you before.

It's completely understandable that a lot of people who play Second Life have goal-oriented drives-- many of us are vets to the MMOG scene and have a burning desire: to do the best, be the best, have the most money, the best equipment blah blah blah. Once upon a time I did too, I admit it! These days, I just want to have fun with my friends and meet new ones; ratings are merely an afterthought.

The system is fine: it is the people that actively choose to mistreat it that are not serving our purpose as a community.

That's definitely a valid point, good answer. They definitely help discourage bad behavior, etc. But I'm not sure if ratings are intended to influence behavior so much as they are to reward people who deserve to be rewarded (get more server resources).

If they are supposed to stop bad behavior...they don't seem to do that good of a job IMO. But I don't think they should be attempting to do this really, it seems almost simliar to trying to create laws to enforce moral values...it won't work. And one main reason it doesn't work in this case is the fact that rating someone negative will probably result in a revenge vote...kinda cripples the system, I think.

I'm not saying eliminate all feedback...leave the profiles alone. I'm saying take the "who" out of the equation. With the new rating upgrade you can now put a comment with the ratings, so that could be your potential feedback. As a comment someone could decide to say "welcome area", etc., and still hide their identity. More importantly, if someone is getting excessive negatives for a category constantly, they deserve them. And I'd be very surprised if they didn't know *why*....I'm sure they'd have a pretty good idea, really. If someone behaves like an idiot in one area, they probably do in more than one. It's not hard to avoid getting lots of -'s, just use your common sense.

I guess what it comes down to is that I think it's stupid to punish the whistle blowers that rate negative when they see something wrong. And if you know who rated you negative, with the ability to "revenge vote", that's basically what happens. If there's another solution to stopping revenge voting other than hiding people's identities, I'm all for it, but I can't think of one.

Really good observation, honestly I hadn't thought of that, but I'd rather see bad behavior punished than see it ignored.
(by behavior don't just mean the one category but all)

As a side thought, I think another reason people rate negative less is because they only reserve it to punish annoying or offensive appearance/builds/behavior, and not punish lack of talent in an area. If I was bad at building or whatever (Uh I'm not, am I? ), and I was trying my best....I would hope people wouldn't rate me negative for just that.

On the other hand if there's a giant 30 meter statue of a gigantic evil clown trashing the sim's appearance, go ahead and rate -.

Actually your best "whistle blowing" option for a troublemaker is the abuse report. On the other hand, when you express personal dislike and affect someone's income in the process, then they *should* have a chance to respond. If you can dish it out, then you should be prepared to take it. I LOVE that the new rating systems identifies who and what. I think that makes the rate givers (especially negative rate givers) more responsible.