Working on a reputation system, looking for feedback

I've started working on this some time ago. It became apparent later that this wasn't such a new idea, but everybody else seems to concentrate on griefers. I don't think things are nearly as clear cut as "good" and "griefer", and my approach follows for that.

What I have is a reputation system: A database containing a long list of "Alice likes Bob", and "Carol hates Eve". These entries are created by using a scanner that allows rating (note, this has nothing to do with SL's rating system, it's completely separated) people. This is not a ban list, as it's able to represent very different attitudes towards the same person by different groups. As an example, furries and goreans don't seem to mesh very well, but there are plenty members of both that hate the general "griefers" group.

My system works by taking a starting point, building a rating tree up to two levels deep from it, and calculating scores based on the distance from the root. Every person has their own perception.

For example, let's suppose the following data:

Alice likes Bob
Bob likes Carol
Bob likes Alice
Carol hates Eve
Eve hates Alice
Eve likes Mallory
Mallory likes Eve
Dave likes Grant

Depending on where we start from, we get a different picture. From Alice's point of view, Bob is good, and Carol is good because Bob likes her. In Alice's group of friends she's going to have a good positive rating, and Eve is evil because Alice doesn't like her.

From Mallory's point of view, Eve is good, but Alice is evil because Eve doesn't like her.

Dave knows nothing of whatever feud there is between Alice and Eve.

Scores are currently calculated by adding the scores from people that you or your friends rated positively, scaling them by the distance, and adding them up. For example:

Alice
    Bob +10
        Carol +1
            Eve -0.1
    Eve -10

Your own rating is always final. If you like, or don't like somebody there's nothing to discuss. So for Alice, Eve's score is -10, and Bob's is 10. Carol's score is +1 because Alice relies on Bob's opinion. And if Alice hadn't met Eve yet, she'd see -0.1 as Carol doesn't like her. The influence on the score is less the further you get from the starting point (Alice).

In-world, it works like this: You get a scanner that shows a list of nearby people. You click on people and select a rating. The ratings are sent to the server, and the server is contacted to calculate the scores of everybody nearby. So for instance, if Carol has an unpleasant experience with Eve and rates her negatively, if Alice comes near Eve, she will then see a negative score for Eve even though they never met before. She'll be able to ask the scanner where that score came from, and make deductions based on that.

Additionally, I will provide an API to query the server, so that you can do whatever you want with the data. I plan to have a door that refuses to open to people with bad ratings. It could also be used to simply emit warning about potential undesirable people, or kick them out.

The API is going to have options to restrict it to use only your opinions, so that you can use it as a server-side list for whatever you want. Banning people, making scripts that greet or give extra functionality to those who you consider friends, etc.

Currently, there's only one rating category, "behavior". I also intend to add more, like scripting, building, and perhaps buying and selling. Suggestions for categories are welcome.

The system will be available to everybody who wants it, not just large places.

This is still under development, but it already works. Basic functionality is fully functional, and being used by me and several people I gave copies to test. If somebody wants to test it as well, just ask. The API isn't ready yet, but will be soon.

I'd like feedback on this: How does the overall concept sound? Are there any features people would like to have? If people like the idea, I'd like to know how they're planning to use it. I'm willing to consider any special needs and add support for them if possible.

I would love to test this.

Sure, just IM me in-world.

Count me in too, this is an awesome idea. It resembles something EVE-ONLINE uses that is a bit more manual, that automatically changes the color of the bracket around the player when you see them. If they have a big red flashy box around them, it's go time.

Dale says "Your own rating is always final. If you like, or don't like somebody there's nothing to discuss. So for Alice, Eve's score is -10, and Bob's is 10. Carol's score is +1 because Alice relies on Bob's opinion. And if Alice hadn't met Eve yet, she'd see -0.1 as Carol doesn't like her. The influence on the score is less the further you get from the starting point (Alice).
"

Ok, there was someone i Didn't like, and she didn't like me.

Now we are Great friends.

Our Feelings changed about one another as our relationship matured How does your system Take that into account, Or does it just assume that Love, or it's Opposite are Forever?

This is the sort of idea that would Fly on the Teen grid, The Format seems very "Highschool" to me. Children Base all of their ideas about someone on what makes them Popular with their crowd, "X and Y don't like Z so i shouldn't either". I Prefer to Judge people as Individuals Based upon my Own experiences with them. I've got some Very rewarding relationships that way.

Angel.

Oh, sorry, I didn't explain well enough. Your ratings can be changed at any time. What I meant is that for purposes of scoring, your own ratings are absolute.

Meaning, if you rate somebody, their score for you is always going to be +10 or -10. It doesn't bother to check your friends' opinions on the assumption that if you like somebody, you just do, even if all your other friends hate them.

This sounds interesting. =)

While you can do this, I'm not saying that this is what you have to use it for. I gather data and provide scores, it's up to you to decide what to do with them. You could see a negative score, ask where it came from, and IM the person who gave the rating to ask about it.

I think the system can be used for other purposes. These are the ones that came to mind so far:

1. Griefer control, by groups rating each other positively to get warned when a griefer just kicked out of an area moves to another.
   
2. Allowing a group of trusted people to moderate a location, without giving any single person absolute power: Make an alt account, use it to rate positively several trusted people, restrict search depth to 1, and require a score of -2 before kicking. At least two admins will need to rate a person negatively for them to be kicked.
   
3. Unlimited length ban list shared between different locations. Say, you have multiple pieces of land all over the grid. Restrict search depth to only use your own ratings, put scanners that remove people with negative ratings on every parcel you own. Then a person getting banned on one of them will automatically get banned from all the others.
   
4. The above in reverse, instead of banning you could use the same system to grant extra privileges.
   
5. While it's not designed for such a thing, you could even use it to record who visited a location by rating them automatically, then removing the ratings at the end of the day.
   
6. Use it for access control. For example, I plan to integrate it into my door. People with low enough scores will always be denied entry, while people I rated positively will be allowed even if I locked the door (people with no rating won't be able to get in).
   
7. Combinations of the above
   

I'm sure other people will come up with other uses for it. I'm not going to restrict usage for any purpose that doesn't negatively impact my server.

I also plan to provide multiple ways (user selectable) to calculate the scores. My philosophy is that I provide the tools, you decide how to use them.

Sims Online had a similar system built in.

It was notoriously used for griefing. Griefers would form large groups of users and threaten to enter negative ratings against anyone who annoyed just one of them. With there being at least two "griefing groups" known to be active this could be a real threat even if not for the possibility of people entering alts. It has the common problem, that people who want to do nothing but game the system probably do not care if they are negative rated, but "innocent" people do and can be targeted.

I would certainly suggest that the ability to rate should be restricted to people with Payment Info on File to prevent alt attacks. Also because this is a network effect thing, I hope you would be planning to keep maintaining and improving it since if it becomes established it will completely fill a slot in the world.

The potential harm cuased by people simply looking to "get someone" far outweighs any good ("Anti griefer" any reputation system would do.

I see no benefit for a system that would aid stalkers, disgruntal ex lovers, and plain mean-spirited people cuase anguish to the objects of their misguided affections.

In First Life your reputaion is through word of mouth I dont see why Second Life should be any different.

Id be all for paying people 1L$ for every rating point they have today , then eliminating LL ratings entirely.


I think the real problem is drama is too prevalant in SL and people will gang together to ostrasize unpopular people - which they already do; But with any negative reputation system that being voted off the island would carry over to that person finding another place to fit in.

I say this even though such a system wouldnt necessarily effect me -

Back when there was LL negrating I only even got one neg rate and she later recinded it.

Online reputation systems have, by and large, been extremely fallible. Some have been absolutely useless. Personally, I haven't seen one yet that I think will work or that wasn't easily gamed. That said, however:

http://terranova.blogs.com/terra_nova/2003/12/reputation.html



In Second Life, the ability to create endless free alts has essentially destroyed any hope of having a reputation system worth its salt. Group reputations may be more reliable, but even then it is easily gamed.

Wouldnt something like that mean every user would have to use it ?

I see this as a problem.

Forcing anyone to be on a non Linden "good/bad" list would pretty much equate to harrasment.

And if participation is voluntary people would always unvolunteer soon as it was working against them.

If it becomes a "wow your hiding something cuase you wont opt in" deal then its discriminatory.

The thing is, it's not 100% opt-in. I can rate you (if you're within the sensor range) even if you don't wear the HUD. It's nice to see that "oh, this person is a friend of a friend" or "oh, one of my friends hates this guy", but a paper trail of why someone has gotten a score would be cool.

I don't know how it works in TSO, but negative ratings against yourself by random groups don't necessarily mean much. Yes, it could be used by a griefer group to coordinate attacks, but then that's nothing new, they already use IRC and message boards already. Even 500 negative ratings against you are meaningless to people not connected to the griefers.

I'm planning to implement alternative scoring algorhitms, such as "the friends of my enemies are my enemy", by inverting the ratings of people you don't like.



Well, I will have to charge for it due to the cost of keeping something like this running if it gets large enough, so that should restrict the amount of unverified people that can use it. So if somebody is interested in giving it a try, ask while I'm still testing it!

No. There's certainly an advantage to getting your friends to use it, but the people being rated don't need to have it.

Correct. Wouldn't be of much use otherwise.



Yep, there will be a way to add a comment (there isn't one yet). Still, IMO, if you see somebody negatively rated and have doubts about it, just IM your friend and ask him why he rated this person that way.

I will also add more rating categories, so that you can rate a lot more specifically.

you completely miss my point.

I am totally against this service.

I would probably AR someone who put me on any published "good/bad" list. Even if I was one of the "good" -- I see too much potential character assasination here.

I think Dale is making a well intentioned effort and has made a thoughtful approach to the situation.

I just think its a BAD idea.

Why have a reputation system at al - I left school many years back..

/Tina

This isn't a list, the overall structure is more of a graph. Everybody has their own "list" and chooses how to connect to others. The full database isn't published anywhere, and to find what somebody else thinks you have to rate them.

I would like to see a reputation rating system that involved some sort of absolute value for statistical comparison. I would suggest Tom Selleck's character from the film "Quigley Down Under."

Because the value of Quigley's reputation never varies, we can easily see by comparison where each of us stands in the Quigley Reputation Spectrum (QRS).

For instance, the well-respected avatar Starax Statosky may have a reputation rating of four (4) Quigleys, which would be represented as follows:


Whereas a less popular avatar such as myself may not rate as highly:

(Please note the introduction of the "Half Quigley," a value used to create finer granularity in our QRS measurements.)

Notorious griefers can be properly rated by using "Negative Quigley" values:


Clearly, a Quigley-based reputation ratings system will be far superior to anything anyone else can come up with. Don't bother to try and refute it. Like Quigley, this value is absolute.

I'l trade you a stewie and a Brian..

For a Peter and Evil Monkey.

It won't, because a verified account can buy L$ and give them to unverified alts.

Please make it check for actual verification. Please?

There is no good or bad, only numerics, and you would have to have an alt that trusts me to see your own rating.
It's a really great idea, I can pick out people my friends conflict with and react accordingly, as well as meet new people my friends get along with.

I can do that, I just don't see how unverified users break anything. Maybe I'm just missing something obvious, so please explain how you think it'd be a problem

Say, I make an alt account now, and go rating negatively every person around. That will make absolutely zero effect for anybody that doesn't rate me positively. Your ratings exist in a void until people start rating you, and it's them who are choosing to rely on your opinion. Should they decide they don't like you after all, they can remove the rating, and your opinion again becomes irrelevant.