Announcement: PRSI moderation review and hate speech/discrimination rule changes

Discussion in 'Site and Forum Feedback' started by HexMonkey, May 4, 2019.

  1. HexMonkey Administrator


    Staff Member

    Feb 5, 2004
    New Zealand
    Based on community feedback and to help us ensure we can best serve members, we've completed a thorough review of moderation in the Politics, Religion, Social Issues (PRSI) forum. Additionally, and as a result of both this review and community feedback, we've made some changes to clarify and strengthen our rules around hate speech, discrimination and group slurs. If you're interested in just these changes but not in the full review, you can see them at the bottom of this post.

    Background and scope

    Over time, the proportion of moderator time spent in the PRSI forum has greatly increased. Since this forum is not the focus on the site, we discussed options to reduce this workload, including closing the forum completely. In April 2017, we decided to keep the forum open, but implemented a new policy that tightens moderation in the PRSI forum. In particular, members making three violations of the Rules for Appropriate Debate. within a six month period will generally permanently lose access to the PRSI forum, even though they may retain rights to the rest of the forums. The goal of this change was to focus moderator attention on the most important parts of the forums, without removing a forum that many members find valuable.

    Reviewing this policy, and moderation in the PRSI forum as a whole, helps us to investigate whether the policy has been effective in its goals, as well as addressing potential concerns that members have raised over the last year.

    The review sets out to answer the following questions:
    • Is the policy meeting its goals of reducing moderator workload in the PRSI forum?
    • Is there any political bias by the moderators, collectively or individually?
    • Are processes being followed correctly by the moderation team, and is moderation fair and consistent?
    • Are there changes to rules or policies that would make rules clearer to members or improve the way the forums are moderated?
    In order to do the review, I analysed the following:
    • Moderator documentation of the violations that lead to removal of PRSI access for all members since the new policy was enacted
    • The underlying posts, reports and (where applicable) appeals for the above violations for the first 50 members who had their PRSI access removed
    • Posts made in the PRSI forum by every member who had their PRSI access removed, in order to infer their political leaning to investigate possible moderator bias
    • Post report statistics for the PRSI forum over the last several years
    • PRSI forum activity in relation to the above
    Is the policy meeting its goals of reducing moderator workload in the PRSI forum?

    The primary goal of the new policy was to reduce moderator workload in the PRSI forum, with secondary goals of keeping the forum open to members who find it valuable and encouraging constructive discourse via the Rules for Appropriate Debate.

    Since the policy was enacted, 80 members have had their access to the PRSI forum removed, or an average of about three and a half per month. Over the first year of the policy, there was an increasing trend in the number of members with access removed per month, as would be expected as it transitioned in (for example, no one had their access removed in the first month due to the short timespan in which they could have violated the rules for appropriate debate). Since then, the trend has been less clear, but overall there appears to be a decreasing trend in the last year, suggesting some improvements in forum decorum. However, it's also possible that some of the recent improvement was at least in part due to it coinciding with a post-election period in the US where debates may have been less contentious.

    Revocations By Month.png

    For the next part of the analysis, I examined reports made in the PRSI forum. Perhaps of most relevance is the subset of these that are related to the Rules for Appropriate Debate (RfAD), which relate most commonly to rules about personal attacks and trolling - with only the very rare exception, these have been the reasons that members received "strikes" that were considered under the new policy. These reports are detected by keyword analysis in the reasons members provide when reporting posts, and are likely an underestimate of the true number, but provide useful data when examining trends.

    The number of RfAD reports within the PRSI forum has been fairly stable since the new policy was enacted, but decreased slightly in the last six months:

    Report Counts.png

    For a more complete picture, another relevant variable is the PRSI forum activity in general. We don't have data on posts made in the forum over time, but have approximated data for the number of new threads created per month. Using this, we can calculate the estimated number of RfAD reports per PRSI thread to get an idea of the level of discourse:


    There is an increasing trend in this rate for most of the period since the policy was enacted. This is due to the relatively more stable rate of reports while the forum activity in general has dropped significantly since the 2016 election (but is still significantly higher than 2015). There has been, however, a significant drop in this rate since late 2018, which coincided with a lower rate of reports rather than a drop in forum activity.

    It is difficult to make any concrete conclusions from this data, as the period since the policy was enacted coincides with an unusually divisive period of US politics. This makes it hard to tease apart the effects of the policy and underlying changes in political discourse in society at large. For example, while the above graph shows a generally increasing trend in RfAD reports per thread from mid 2017 to late 2018, it's possible there could have been an even steeper trend had we not enacted the policy. Subjectively, staff have noted a lower workload from dealing with contacts regarding PRSI moderation.

    Is there any political bias by the moderators, collectively or individually?

    Our policy is that we allow all political opinions as long as they are expressed within the forum rules. Nevertheless, I reviewed the moderation records to examine whether there was any bias in practice.

    Looking at PRSI moderation as a whole, I saw no evidence of bias. 53% of members with PRSI access removed had a conservative political leaning, compared to 47% with a liberal political leaning. The distribution of severity of violations was also similar on both sides. Although we don't have statistics on the political leanings of PRSI members as a whole, these numbers do not suggest any systemic bias.

    Looking at particular moderators, I looked at the individual violations that lead to removal of PRSI access and noted the moderators who actioned each of these and the political leaning of the moderated members. I compared the statistics for each individual moderator to the overall moderation team statistics above, looking for statistically significant differences using a 95% confidence interval. I also examined many of the underlying violations, and found that the severity of the underlying violations handled by each moderator was similar regardless of political leaning of the moderated members. Overall, I found no evidence of political bias by any of the moderators.

    We can't rule out bias in terms of what members report (e.g., members of a certain political persuasion being more likely to report posts they disagree with), nor do we have any evidence that this does happen, as we don't track data that would be needed to investigate this. However, even if it were the case, the findings above suggest that it does not appear to impact moderation in any noticeable way. Also of note is that reporting a post doesn't necessarily mean it will result in any moderation, since moderators review reported posts for compliance with the rules rather than blindly acting just because a report has been made. In 2017, roughly two thirds of reports in the PRSI forum resulted in some moderator action, and this dropped further to 56% in 2018 - considerably below the 2017 forum-wide rate of 82%.

    Are processes being followed correctly by the moderation team, and is moderation fair and consistent?

    Of the 80 members who have had their PRSI access removed under the policy, 30 contacted us either to request clarification on the moderation or to appeal it. Appeals frequently included attacks against us and accusations of bias, and we note that contacts taking this aggressive tone tend to reinforce the reasons that access was removed rather than help the member contacting us. However, in four cases, we reinstated PRSI access on appeal where we discovered that we had erred. Some members also requested access to be reinstated after their access having been revoked for over a year. Access was also reinstated in some, but not all, of these cases - we consider these requests on a case by case basic and use discretion based on a number of factors, such as the severity of the original violations, any evidence to suggest that the member has changed their behavior, and whether the member is active in other parts of the forums.

    All 80 PRSI bans correctly met the criteria of the three violations occurring within a six month period (although we reserve the right to make exceptions to this time criteria to fit with the spirit of the rule). The average time period between the first and third violations was 85 days (just under three months).

    There was some inconsistency around handling of strikes that had overlapping timeframes, e.g. where the third violation was posted before receiving a warning for the second violation. Four PRSI bans occurred under these circumstances, while another eventual ban included a case were a strike wasn't counted because of this. This doesn't include additional cases that have likely occurred where the member still has PRSI access, which I didn't review. There were also two members who received a warning minutes before they posted their next violation, so they might not have seen or read the warning yet. After discussion with staff, we have decided that a standard policy needn't be applied here, but rather discretion based on the severity of the violations and whether there would be a reasonable expectation for the member to be aware they were violations the rules for appropriate debate.

    I examined approximately 150 individual violations to check whether they met the criteria as outlined in the policy. The vast majority met the criteria under the RfAD. In some of the early months of the policy, there were rare cases of moderation for other issues being considered as a strike, although there were no cases of this since the start of 2018. Overall, despite rare errors, moderation was consistent.

    Is the policy fair and consistent?

    About 600 members have had posts reported in the PRSI forum since the policy was enacted. Of the 20 most reported members, 65% have lost PRSI access. For the other 35%, I confirmed that none of them should have lost access under the policy. Examining some of the reports made against these members who did not lose access, reports appeared to have been handled correctly, and were most often not rule violations, or were for more minor rule violations that don't fall under the Rules for Appropriate Debate. Sometimes members would have a large number of (rejected) reports made against their posts by a small number of people of differing political views. These findings suggest that the policy is correctly targeting the most problematic members who create a disproportionate amount of moderator workload, while also confirming that members are not unfairly targeted just because those who disagree with them report their posts.

    Although the policy was generally applied correctly, a secondary question is whether there are any modifications we can make to the policy to better achieve our goals. From a moderator perspective, we would like to spend fewer resources dealing with PRSI reports. From a community perspective, we would like to facilitate better conversations amongst members, as well as have the community as large feel confident in the moderation processes and staff.

    Changes to rules and policies

    As a result of the review and subsequent discussions with other staff, we are making the following changes and clarifications:
    1. We are creating a more explicit rule prohibiting hate speech, and group slurs/discrimination, under both the general rules and RfAD. Previously many of these violations were classified under different rules, such as trolling. Having a separate rule make our warnings about violations clearer to members, as well as making it clearer what we do and don't allow. The new rule also increases the scope of what we consider a violation, in line with changing community expectations. The new rule is as follows:
      The new rule takes effect for any posts made after this announcement.
    2. Refusal to cite sources, which is part of the RfAD, will not generally be counted as a PRSI strike. However, it is still moderated under the forum rules as normal, and persistently violating this rule may be counted as a strike. This is consistent with how we have implemented the policy in practice, but has not previously been communicated to members.
    3. If we decide to reinstate a member's PRSI access after it was previously removed, there will be a reduced tolerance of rule violations; two violations of the RfAD within six months will result in a permanent loss of access from the PRSI forum, with no option of later reversal except in the case of moderator error.
    4. Although errors have been rare, we are making some minor internal process changes to reduce the risk of errors and improve oversight.
  2. MacNut macrumors Core


    Jan 4, 2002
    While I like the explanations given and the added rule, can I make a suggestion in the strike process. I'm not even sure if it is physically possible in the forum software.

    Can there be a private record given to a member to list the severity of an infraction and also have it noted that a strike was given. Just randomly saying, you got a strike, even if it might be 5 months and 28 days after the last strike seems like a form of entrapment. To me there needs to be a floating scale by severity of the strike and not treat them all equal. After 6 months does a strike come off the board?
  3. HexMonkey thread starter Administrator


    Staff Member

    Feb 5, 2004
    New Zealand
    Warning messages will always specify when moderation falls under the policy. We discussed considering severity, but concluded that it would add significant complexity to both enforcement and communication of the policy, and it wouldn't change the result in the vast majority of cases. We do have some discretion under the policy to require fewer or more violations if warranted (eg due to severity), although this is rarely used or needed in practice.

    Yes, after six months a violation is no longer considered under the policy. However, it's still part of a member's moderation history, so can be considered under the general approach of moderation escalation that applies throughout the forums. Most cases of removal of PRSI access occur in a much shorter period than six months, so edge cases close to the six month cut off are rare. In such cases we might start to look at severity or other factors and consider using discretion if it makes sense. The main thing we're concerned about is whether we think a member is likely to change their ways, rather than moderating as a form of punishment.
  4. LizKat macrumors 601


    Aug 5, 2004
    Catskill Mountains
    Thank you so much (and the others participating in the review) for taking the time to look into these matters so thoroughly. I'm sure I'm not the only one appreciative of a moderated forum like PRSI and they are not easy to come by these days.

    Also grateful for the very existence of this subforum, as a venue that permits us a semi-public general discussion of some of the thornier situations that end up subject to moderation. :)
  5. StralyanPithecus macrumors regular


    Sep 27, 2018
    Personally I will close those forums. I don’t see the point of having them in a tech site. Enough forums of that kind are around the web for those in need, as for me PRSI forums doesn’t exist, don’t care, don’t read, not interested at all in them.
  6. raqball macrumors 68000

    Sep 11, 2016
    I have a question on this. Should I ask here or PM a moderator?
  7. annk Administrator


    Staff Member

    Apr 18, 2004
    Somewhere over the rainbow
    If you have a question specific to @HexMonkey's post that doesn't involve specific moderation, it's fine to post it here.

    If you have a more general question about the rules, not specifically related to @HexMonkey's post, you can ask in Site and Forum Feedback.

    If you have a question of any kind that involves specific moderation, send it via the Contact us form.

    I hope one of those answers your question!
  8. raqball macrumors 68000

    Sep 11, 2016
    Thanks... Just tying to keep my nose clean...

    1. It's no secret to many in PRSI that I view BLM as a hate group. They are a group and their supports are no doubt offended by my opinion of them. A violation of the new rule?

    2. Not rehashing moderation but an example. A transgender thread in the past mentioned a male (not transitioned) winning a cycling race when competing as a female. I recently saw another instance where a male weight lifter competed against woman and smashed their records. I was going to post about it but didn't want to get placed in MR jail so I didn't.. This is a group and also a sexual orientation scenario that no doubt will be offensive to some if an opinion on it is posted. Would this be considered a violation of the new rule?
  9. annk Administrator


    Staff Member

    Apr 18, 2004
    Somewhere over the rainbow
    I'm guessing that it's not possible to give an answer without specifics, but I'll defer to @HexMonkey.
  10. raqball macrumors 68000

    Sep 11, 2016

    Yeah I agree specifics would be important but figured I'd ask in an attempt to keep my nose clean and to stay out of MR jail...

    As written, I assume both are now violations as they are groups and no doubt some would find an opinion on them as offensive.. Maybe not so much the BLM scenario but definitely the transgender one...
  11. HexMonkey thread starter Administrator


    Staff Member

    Feb 5, 2004
    New Zealand
    I'll start off with a disclaimer that the comments below are my interpretation of the rules, but other staff may have different ones. Moderation is a team effort and we often discuss borderline cases and reach a consensus over time.

    I'll start off on a general note. Under the new rule, it can apply both to common protected classes like race, gender, religion, etc, as well as other groups like political parties, activist movements, ideologies, etc. The former group is more likely to be classified as hate speech, while the latter is more likely to be group slurs. The former will, on average, be a more serious offence, but the latter is still a violation because it's not constructive to the conversation. If you say "Republicans are all evil" or "Anyone who voted for Hillary Clinton is an idiot", that's not going to do anything to advance whatever argument you're making, will offend other forum members, and will likely take the discussion off topic. Many of these sorts of comments were already covered by other rules, so there's not a significant change here, but more a clarification.

    Now, to your specific example. There are probably contexts where it's appropriate to post your opinion on this. But, I would imagine that the majority of contexts would fall under this rule (and previously, other rules in the majority of those). A couple of factors to take into account are a) the term hate group has a specific meaning, that does vary slightly by source, but generally refers to promotion of hostility or violence against other classes of people, where those classes are defined by immutable characteristics such as gender, race, sexual orientation, etc; and b) it's a highly emotive term. The combination of these mean that applying it to a group that doesn't meet the generally accepted definition, or does so only at a stretch, is likely to be contentious and offensive to some. On the other hand, applying it to a group that clearly meets the criteria, such as the KKK, would be fine. Another factor would be why the comment is being made. In most cases, you could make the same argument you're trying to make without comments that are likely to offend others or take the conversation off topic. If the topic itself was about whether a group could or should be classified as a hate group, then it would be on topic, and potentially legitimate to post your opinion there if it advanced the conversation, but - as always - depending on the exact context and how your opinion is expressed.

    Again, the context is important, but as a general topic I don't see a problem with this per se - there's legitimate viewpoints on both sides. Making slurs against transgender people within that thread, would, of course, be a violation of the rules.

    The inclusion of the words "that a reasonable person would find offensive" in the new rule is quite deliberate. "Reasonable person" is a legal term with a specific meaning. We've included it to show that common sense applies. A single person being offended doesn't mean that your post violates the rules; conversely, your post can violate the rules without everyone or even a majority finding it offensive.

    Hope that clarifies things!
  12. tobefirst macrumors 68040


    Jan 24, 2005
    St. Louis, MO
    Thank you, @HexMonkey, for the thoroughness of your research and explanation. I've been (very?) critical of mod/admin explanations and posts in the past, but this is top-notch.
  13. raqball, May 6, 2019
    Last edited: May 6, 2019

    raqball macrumors 68000

    Sep 11, 2016
    I appreciate the response and thanks for the clarification.

    I am still left a bit confused over the classification of hate groups. I do agree the KKK and others like them are hate groups, no confusion there but I also believe BLM on par with them. The white supremest groups target others based on race whereas BLM and other BIE's target Law Enforcement (a group). Both mentioned groups (KKK and BLM) spew hateful messages and promote violence against their targets. This my opinion of course but it is based on factual events, data and FBI reporting.

    Either way, I do thank you for the response and I'll try my best to remain out of MR jail.

    FWIW, I've never thought the moderators were bias and they appear pretty even handed to me. Now I will stop asking questions as I am sure you need a break. Compiling all the data in the OP must have taken significant time and effort....
  14. ericgtr12 macrumors 65816


    Mar 19, 2015
    I think @HexMonkey makes a valid point. For example, it's my view that Blue Lives Matter can be just as divisive as any of them, targeting those of race. Does that mean they should ban that topic as well? Certainly not. In either case it sounds like they're saying as long as the content does not violate the rules then it's allowable. To me this seems fair.

    Speaking for any of us who have been in MR jail, it's been because of the way we interact with each other in a manner that breaks the rules that lands us there typically, not so much hate speech in general. At least in my experience.

    A lot of thought obviously went into this rule change, with it also comes that much more supervision. In one way I don't envy you and still question why you would keep it, on the other it's hard not to respect the effort.
  15. raqball macrumors 68000

    Sep 11, 2016
    I was asking about groups identified as hate groups. If Blue Lives Matter have been then add them as well. It's my understanding they have not... BLM has been identified by the FBI as a BIE group...

    I am not asking to ban BLM discussion... My question was more along the line of me:

    1. Addressing them as a hate group
    2. Which will upset those who support them
    3. Which will lead to post reports
    4. Which will lead to MR jail

    They are a group and my view that they are a hate group (also a BIE) could be construed as me attacking their supporters.

    I'll let the powers that be decide but I am still unsure of this new rule and how it applies as any group can be offended by a label especially if they don't like it....
  16. Scepticalscribe Contributor


    Jul 29, 2008
    The Far Horizon
    Thank you, @annk, and @HexMonkey - who have both taken the time and trouble to post on this subject matter, here, on this thread, with considered posts and responses, and to the mods, and admins, who have clearly given the subject matter further considerable thought.
  17. ericgtr12 macrumors 65816


    Mar 19, 2015
    I will say that I myself and others are all guilty of this on a daily basis when it comes to party affiliation. It's simply used as a way to make indirect digs at one without pointing fingers at them and people here at MR have become quite good at it. For example "Why do Liberals hate this" or "Why do Conservatives hate that" is in practically every single thread in there. I'm not exactly sure how that dynamic will be changed but I can imagine moderating it will be a nightmare.

    If this is not the case then maybe I misread but if it is then it should really be highlighted IMO.
  18. HexMonkey thread starter Administrator


    Staff Member

    Feb 5, 2004
    New Zealand
    I want to be careful here because I don't want to get into a debate about whether a particular group should be classified as a hate group here any more than is necessary to answer your question - further discussion on that should be in the PRSI forum. But I'll clarify that in my previous post I referred to definitions being specific to immutable characteristics (gender, race, etc) which does not include other mutable groups (eg profession). This is in contrast to our rules, which also cover the latter types of groups under group slurs.

    In the case of BLM, applying an emotive term to them that doesn't fit its commonly accepted definition is likely to cause offence, particularly since it's most often associated with groups diametrically opposed to their primary stated goal and there's a lot of historical context involved. That's not to say that such opinions are never allowed, particularly if it's the topic of discussion, and in a general sense it's not unheard of for groups to become what they fight against, so the latter point doesn't preclude such accusations should they be warranted.

    I accept that it's not as clear cut as would be ideal, which is often the case when balancing competing rules and philosophies. That's not a problem that's unique to us, as you'll find in almost any case that reaches the US Supreme Court. As a general guideline to avoid violating the rule, if there's a way to make your point that avoids using emotive terms, then you probably should. If there's not and it's on topic, then it's probably ok, depending - of course - on context and how it's expressed.

    In isolation I don't think those examples would be classified as group slurs. They might be, however, if they imply certain group slurs in context.

    An important factor we often consider is whether wording is specific to some members of a group or the entire group. The latter is much more likely to be problematic as it's clearly targeting everyone who identifies with that group. Sometimes phrasing is ambiguous, but by being a bit more careful with phrasing you can ensure you're much less likely to be misinterpreted.
  19. raqball, May 6, 2019
    Last edited: May 6, 2019

    raqball macrumors 68000

    Sep 11, 2016
    Thank you for the additional clarification... I believe I fully understand the intent of the new rule now.. I appreciate you taking the time to add the additional explanation.
  20. Doctor Q, May 6, 2019
    Last edited: May 6, 2019

    Doctor Q Administrator

    Doctor Q

    Staff Member

    Sep 19, 2002
    Los Angeles
    The "better to beg forgiveness than to ask permission" adage isn't good advice in a moderated forum like this one.

    When you are puzzled about whether your specifically worded draft of a post, containing negative comments about a person or group, would be within the rules, you aren't sure whether or not to post it or rephrase it, and threads like this one don't provide enough guidance, here are two tips:

    1. Before making a post, you are welcome to use the Contact form to ask if the specific comments would be OK. This obviously won't be convenient every time you want to make a post about something controversial, but the guidance provided should clarify how the rules apply to the proposed post as well as to others like it, so you'll better understand how the rules are applied in general.

    2. You can probably judge whether comments are acceptable yourself by considering the goals of the Forum Rules. They are designed to foster discussions that will be interesting and/or useful to everyone, keep threads on topic, and avoid having them deteriorate into flames, shouting matches, or personal feuds.​
  21. Mr_Brightside_@ macrumors 68030


    Sep 23, 2005
  22. Gutwrench Contributor


    Jan 2, 2011
    Omg, Bans Extrapolated.

    I’m somebody. I’m somebody. It’s this type of spontaneous publicity that makes people. I’m going places now!

    If @chown33 could only see me now.
  23. yaxomoxay macrumors 68040


    Mar 3, 2010
    Me :)

    This is the thing that is not clear to me, still. Is it three violations in general, on the same post, thread, what? For example, suppose I say something I shouldn't (nothing horrible, but a violation), like "I love this hate group!". Then someone replies to me that it's a stupid group, and then I reply : "you're wrong! I love them!". Does this count as two violations? Or as one?
  24. Doctor Q Administrator

    Doctor Q

    Staff Member

    Sep 19, 2002
    Los Angeles
    I'd consider it to be one, perhaps even if you posted the same thing in two threads at about the same time. But if you received a warning from the moderators after the first post and then made the second post, that could be counted as two.
  25. TheFluffyDuck macrumors 6502

    Jul 26, 2012
    "Hate speech" is ambiguous, as is what is a "reasonable person". These need stricter definitions or run the risk of being abused as blanket terms. For example which are these following are hate speech, and valid criticism, I bet if you had a poll people would find different ones:

    "I hate one eyed one horned flying purple people eaters"
    "I think one eyed one horned flying purple people eaters, have a cultural disposition to eating people".
    "I think we should have stricter boarder controls before letting in one eyed on horned flying purple people eaters into this country".
    "There are several documented issues (see Wooley et al, 1958) that show one eyed one horned flying purple people eaters are prone to increased eating based violence".
    "I think there is an evolutionary biological difference between one eyed one horned flying purple people eaters, and two eyed one horned flying purple people eaters".
    "One eyed one horned flying purple people eaters are in secret plot with Trump and Hillary to bring down the USA".
    "You are just dog whistling to one eyed one horned flying purple people eaters supremacy".
    "One eyed one horned flying purple people eaters are better at flying than people".
    "People with views like one eyed one horned flying purple people eaters should be banned".
    "One eyed one horned flying purple people eaters just spread hate on their platform"
    "If you care about your race you must exterminate one eyed one horned flying purple people eaters".
    "One eyed one horned flying purple people eaters throughout history have had privilege, and thus must step aside for minorities".
    "Your just a fragile one eyed one horned flying purple people eater"
    "One eyed one horned flying purple people eaters are trying to take our guns".
    "It is well known that Hitler also shared views that one eyed one horned flying purple people eaters have".
    "There are just two kinds of purple people eaters, one-horned and two-horned flying purple people eaters".

    Some of those are hateful, others not so much. But if you have blinkers and find offence everywhere, then all of them are. Thats not reasonable. We need a definition of hate speech and what is a reasonable person.

Share This Page

39 May 4, 2019