Let's introduce a user reputation system

[Eso]

A large part of MacRumors is speculating about rumors in the discussion forum. Users enjoy debating about the validity and likelihood of rumors, ultimately staking their claim about whether or not they believe them. The problem is that there is never any meaningful resolution to the discussion. True or false, discussions about rumors end up forgotten and all the points users have argued amount to nothing.

What's missing is a persistent element based on the sensibility of a user's discussion - an effect that doesn't die with a single thread. We should have a system that uses a metric to rate how accurate a user has been in predicting the validity of rumors.

The premise is that for every rumor headline, there be an associated feature in which a user can vote on the validity of said rumor. When the time comes that the rumor is confirmed true or false, MacRumors can award points to a user's "Reputation" based on their affiliation. These points will be displayed on each post along with the other information for each user.

It can implemented like a more descriptive version of the current positive/negative rating system. It can possibly be used to replace it. It can be a simple plus/minus system (+1 point when correct, -1 point when incorrect) or it can be a system of bidding points based on how confident a person feels (like final jeopardy).

It would be a very fun way expand the website and has the following benefits:

1) Rewards users with a metric recognized in the community based upon their accuracy on past rumors.

2) Lends weight to opinions within a discussion.

3) Based completely on users opting-in - no one is required to vote to participate in a discussion or put their reputation on the line.

4) Isn't based on users rating each other so it doesn't become a mechanism of relatiation, popularity, etc.

5) Can be used with the system of site contributors, potentially allowing them to reset negative popularity (plus/minus) or reset their points (bidding system) each year.

EDIT: intended to include point 4 but didn't remember it until the post below

 

[miles01110]

1) Accuracy of rumors based on previous performance is usually mentioned in the news item. Very few credible rumors come from the forums.

2) Why should one opinion be "weighted" more than another? It's an opinion.

3/4) What good reason do you have for doing this? Various rating and reputation systems have been suggested multiple times. They don't add anything of value to the site other than turning things into a popularity contest.

It would be a very fun way expand the website and has the following benefits:

1) Rewards users with a metric recognized in the community based upon their accuracy on past rumors.

2) Lends weight to opinions within a discussion.

3) Based completely on users opting-in - no one is required to vote to participate in a discussion or put their reputation on the line.

4) Can be used with the system of site contributors, potentially allowing them to reset negative popularity (plus/minus) or reset their points (bidding system) each year.

 

[Apple OC]

sounds very complicated and does not seem like something the majority of members would want ...

start me out with a -1 if you want.

 

[iSaint]

Been done.

 

[Blue Velvet]

Eso, I like the thoughtful work that you've put into this and that you're willing to put an idea on the table, especially with some consideration given to try and work around the 'popularity' argument which comes up time and time again when reputation systems get discussed.

However, in many ways, the metric of how many times someone gets a rumour 'right' just isn't that important to a large proportion of forum members, especially many long term members who have their own ideas about who is worth listening to on any given topic or not. Many of them might read the news threads to find out what's going on and what people are thinking, but rarely participate in the scrum.

Personally, if we were thinking of the reputation of people, I would rank forum members who actually directly help people out with technical, software and hardware problems and follow through, far far higher than someone who clicks on a button to express an opinion on a rumour.

Besides, many news articles may not be about rumours but instead might be about actual substantiated news, so it would seem pointless to employ opinions on these as meaning anything useful. Someone voting on whether a OS release is a good or bad idea hardly qualifies them to get a higher ranking than someone who doesn't vote.

Another problem is this idea:

When the time comes that the rumor is confirmed true or false, MacRumors can award points to a user's "Reputation" based on their affiliation.

What if the rumour turns out to be partially true? What if there are currently no technical means to retrospectively award 'points' to almost half a million members? Is this where Arn and co should be spending limited time and resources to develop this system or would attention be better spent elsewhere on other forum and site features? Also, enabling contributors to clear their slate for $25 per year would make the idea of a reputation system completely redundant, how would you know who to trust?

Just from my experience, these are just a handful of problems with your idea that immediately come to mind. Keep the ideas coming, though.

 

[nastebu]

fun idea

I like it very much. I can imagine sets of polls that you can vote for fun, and then your results are summarized in your profile. Think of the way Arn always starts a thread after Apple media events evaluating which rumors were true and which were not. Instead of just doing that after the events, there could be a set of questions and everyone would have a chance to make their bets.

One thumbs up from me.

 

[Creative One]

Too much work.

 

[Eso]

However, in many ways, the metric of how many times someone gets a rumour 'right' just isn't that important to a large proportion of forum members, especially many long term members who have their own ideas about who is worth listening to on any given topic or not. Many of them might read the news threads to find out what's going on and what people are thinking, but rarely participate in the scrum.

I think it could be somewhat analogous to member titles based on a user's total number of posts. Most people probably don't care what their title is. Some people will take "newbies" less seriously than others.

In either case, member titles, avatars, and join dates are form of recognition for members of the forum. Points could be used in much the same way. Just like member titles, some people won't care what points a poster has and others will. Perhaps the term "Reputation" isn't the best. Maybe "Rumor Rank" or something (or "Gullibility" ).

In other words, the intent is to award a specific recognition for members (that want to participate). Just like member titles are known to be based on total posts, this would be known to be based on 'rumor' stories. I didn't really mean a system to rate a user's general contribution to the site. What I meant by "giving weight to user's opinions" was just in context of discussing rumors. For example, given the track record of Kevin Rose and John Gruber - when both weigh in on an rumor, which are you more likely to believe?

Besides, many news articles may not be about rumours but instead might be about actual substantiated news, so it would seem pointless to employ opinions on these as meaning anything useful. Someone voting on whether a OS release is a good or bad idea hardly qualifies them to get a higher ranking than someone who doesn't vote.

Right; news items wouldn't be appropriate to include a vote. By the same token, some rumor items could have multiple votes. Let's take one of the recent stories, for example. iPad 2 with High Resolution Screen and SD Card Slot, iPhone 5 with A5 Processor? There's essentially three major features up for discussion:

1) Will the iPad 2 have an SD slot?

2) Will the iPad 2 screen have a greater resolution than the current iPad?

3) Will the iPhone 5 be a dual-mode CDMA/GSM model?

Out of the 346 posts, 21% of them either directly said aspects of the rumor were true/false or posted with the assumption that it was true/false (that includes you up there, Apple OC ) The percentage of unique members making such a post is higher because many of them posted several times in the thread, obviously not saying what they believed of the rumor every time.

How many of these particular posters are likely to be interested in some kind of poll/vote which can potentially earn them a badge or point for their profile? Of all the other posters that discussed the rumor without saying they believed it or not, how many of them may be interested in voting on the rumor for fun?

It difficult to say how broad interest in something like this is, but I don't think it's insignificant. Currently there are awards based on seniority/posting history: avatars, member titles, forum access, etc. Since a large portion of the website is rumor-centric; having a rumor-centric award system could be a popular addition.

What if there are currently no technical means to retrospectively award 'points' to almost half a million members? Is this where Arn and co should be spending limited time and resources to develop this system or would attention be better spent elsewhere on other forum and site features?

That's certainly is a concern. I'm also aware that forum software is basically a package deal and it could be difficult to add. I have, however, used several forums that use point-based system for members. One gaming forum in particular works in basically the same way as this system would work: members have points that they can use to vote on who they think will win a match. If they are right, they are awarded points; if they are wrong, they lose points.

Also, enabling contributors to clear their slate for $25 per year would make the idea of a reputation system completely redundant, how would you know who to trust?

Hopefully I have explained what system I imagine a little bit better. Contributors being able to clear their points would be more analogous to being able to use the "MacRumors Demi-God" member title.

 

[Blue Velvet]

^ The Gullibility Index.

Thanks for responding to my devil's advocacy. How things tend to work around here is that any ideas like this get batted around out here, perhaps discussed amongst mods and admins... and if they pass muster, then added to the list of things to do. Some of these things can take months or even years to even reappear in some form.

Personally, I'm not sure that this idea will be given much priority, but that's just a hunch based on previous experience as I have no say in this process. Saying that, it would also certainly generate an interesting amount of data for those who like stats and graphs as well.

 

[Dezinformatsiya]

User reputation systems are a sure-fire route to turning your community into the echelons of histrionics. Accuracy, trustworthiness and reputability are, I've observed in other sites introducing rep systems, usurped by hackneyed humour and textual displays of crowd-pleasing. There is always an exception, and user rep systems used correctly would be excellent forms of self-regulation, but it is quite a risk to take.