As many of you already saw the thread talking about moderation on WSO, I thought it would be helpful to start a new thread that gives everyone updates on where we are relative to the plan I posted there... as you can see, this morning we already sent a lot of updates live. We will be keeping a close eye on all of these.
Below are some details on what is coming and a rough timeline. So as to discourage trolls, I will be intentionally vague on where certain thresholds are + the exact rules. All of these are subject to change/removal as we see how it impacts everyone and how some of them get abused (which we know will eventually happen).
UPDATES THAT ARE LIVE [10/3/22]:
- Posting to the Off Topic forum now requires a certain type of profile...so any members outside this "allowed range" will lose the ability to post to the off topic forum (this is a tiny Band-Aid, not a perfect measure... but it's a start to limit the members that tend to debate politics and/or are the most extreme). No, it won't stop most of the users that like to debate, but it is likely to limit the obvious trolls. [DONE]
- New members will also NOT be able to post to the Off topic forum for an unspecified amount of time + until they earn a certain number of bananas (again, this one is minor, mostly as a way to discourage troll users from creating new accounts to start shit-posting right away again after they have just been blocked). [DONE]
- New automated unpublishing rules based on a certain criteria (+time based) to be reviewed by mods. This one should help more meaningfully since it is automated but we will need to keep an eye on and see what the flow looks like... I suspect that it will help get the worst comments and threads off the site faster until it can be reviewed later. This requires us to build in some new features to our current moderation table + make sure our permissions are dialed in on mod role but we think we can have it live by mid week [DONE]
- Autoblock members that hit certain criteria. This has been all manual in the past, so it would be good to automate this so that the worst actors eventually lose their account if they are not adding enough value [DONE]
UPDATES COMING LATER THIS WEEK [10/3/22]:
- (likely by Thursday) New violation notice will go live and be used to help remove controversial topics faster. Like @m1 mentioned, religion and politics are good examples : ""Violation Magnet: Hi user, while your post itself may not be directly against our official terms and conditions, it has been removed because it will lead to a high number of violations (or already has). As such, to save WSO from going bankrupt and having a team of thousands of moderators, we had to remove your post. We appreciate your contributions and hope you understand this decision. Best Regards, WSO Mod Team" [THIS IS NOT TOO HARD AND IS JUST A WAY TO MAKE SURE PEOPLE KNOW WHY SOMETHING WAS REMOVED...WE MAY NOT NEED TO DO THIS TOO OFTEN IF THE CHANGES ALREADY IMPLEMENTED ABOVE HELP REDUCE VIOLATIONS]
- Hive Moderation which is used by other large content businesses. By using their API, we can potentially use their AI mod tools as a first line of defense to either flag more or just autoblock content that scores too high on various metrics: https://docs.thehive.ai/docs/classification-text This will require more dev work to integrate with our system, but I'm hoping we can go live with something before end of October [SOME TECHNICAL WORK NEEDED W HIVE TEAM SINCE THERE ARE CHARACTER LIMITS THAT WE NEED TO WORK AROUND. THIS WILL LIKELY PUT ALL NEW COMMENTS AND POSTS IN A QUEUE SO AS NOT TO DELAY POSTING BUT TO GET THEM REVIEWED WITHIN ~5-10 MINUTES AFTER]
UPDATES COMING IN 2+ WEEKS:
- Team of 5 volunteer mods that can flag and remove content and recommend bans based on user history - hoping to have a few ready to go by end of next week so the automated flagging/unpublishing doesn't drown us
- Better visibility of moderation for comments. Instead of just unpublishing comments that are against our T&C, we will try to "grey them out" and/or have some sort of indicator that a comment was removed...this way, if there are replies nested under that comment it won't make the discussion very confusing...especially in topics that are controversial (still TBD whether they will just be removed entirely)
ON THE TABLE BUT MAY TAKE LONGER:
- Allowing users to "mute" specific other users that they don't like or are offensive according to them
- Potentially requiring certification status to post anonymously. I'm open to this but I want to see how everything trends with the above changes into November before we make this one.
That's all for now,
Patrick
ps - have one call scheduled on Monday w a potential mod but would like more. If you have a long history on WSO of avoiding all political/controversial topics and focusing on helping members, I'd love to speak with you. [email protected]
==================================================================
UPDATE [10/6/22]:
- Hive moderation (using API) testing is complete and ready for roll out to production server on Monday Oct 10, 2022. Scoring explanation here: https://docs.thehive.ai/docs/classification-text
- Posts scoring a 1 will be flagged automatically but stay up by default. This will allow mods to review it to determine if it should stay live or get removed. Mod will review and either send violation notice which then unpublishes the content OR will clear the flag which will keep it up.
- Posts automatically flagged at level 2 or 3 will be unpublished and sent to flag table for review and user will be notified that their post had a potential violation and was removed. Mod will review and either send violation notice OR will clear the flag which will republish it.
- This will not be perfect but it's really more as a line of first defense...I tested it an it does a decent job at flagging obvious violations but it misses nuance (as expected)
- New MOD ROLE IS READY. Actually called "Content Reviewer" - we had to create a new role in order to make sure permissions were dialed in and that the new content reviewers only had the right level of permissions. We are still working on documentation to help the mods, but I could still use some volunteers here (only a few I've had calls with).
- We have increased shifts of internal mods to help us deal with the 200% increase in the # of potential violations already hitting the flag table with the new systems already in place. I think it's working to get violations off faster, but it's not perfect
- New Violation Magnet PM created (now LIVE) so mods can send violation notices to members that post threads to incite debate, get a reaction or prove a political point...
COMING SOON [Posted this 10/6/22]
These next set of initiatives (after Hive on Monday and what we've already rolled out) have me the most excited because I think they will make a meaningful difference long term and are more WSO specific...
- Better transparency...greying out but leaving "frame" of comments that have been removed so responses to those violations are less confusing and there is more clarity when something has been taken down because of a violation
- "No bump forever" flag on threads will no turn into "No bump Forever + No Anonymous" which means two things
- 1. Any new comments will not bump the thread up to the top of the recent activity trackers [how it is now]
- 2. Any new comments after flag is applied will force participants to post under their username [this is new]. Even though admins can see what members are posting even if they use anonymous feature, the goal here is for people not to hide behind the anonymous feature on the most controversial/sensitive topics in order to try and encourage more respectful debate, especially in these threads
- Adjustment to Silver Banana weightings from Topics and comments in the Off Topic forum...tbd if we can do this, but should help lower the benefit of riling up your "team" and getting a bunch of SBs as a result at the expense of another member
Thank you for your patience and help with these ideas...
Patrick
==================================================================
UPDATE [10/10/22]:
- Hive moderation ("first line of defense") API is LIVE. This means every new post or comment is run through this API and scored. See above for explanation
- New Moderator ("Content Reviewer") role is READY. I will start granting this new role this week to a few select members
- No Bump forever flag applied to threads has now become "No bump forever + No Anonymous flag"... this means once this flag is applied to a thread, everyone must use their username to post and can't do it behind an additional layer of anonymity. While this is not a huge difference, we think it ay help reduce violations on the margin and lead to a slightly more respectful tone on average...
- NOT live but still high priority = leaving a placeholder for moderated comments so people know when something was moved (not just the person with violation notice)
- Re-weighting of SBs granted in off topic forum
...After this week I'd like to take an honest assessment of the changes that are currently live and see if we think the new system is doing a good job (or not) and where we may be able to improve things further.
Thanks,
Patrick
==================================================================
UPDATE [10/13/22]:
- LIVE = now there is a placeholder for moderated comments so people know when something was moved (not just the person with violation notice)
- Hive moderation ("first line of defense") API is adjusted so that only level 3 violations is automatically unpublished (to reduce false positives). Removed potential violation PM notice for level 1 and level 2 violations since this was causing confusion. They are still auto-flagged and reviewed by mod team but the user will only be notified if their content is unpublished (level 3 potential violation or actual PM violation)
- New Moderator ("Content Reviewer") role has been granted to 3 members. I would still like an additional 3 more mods to help us since the number of flags is now ~3-4x what it was prior to these changes. Potential violations still sitting for too long in the flag table.
- Fixed anon posting issues in the mobile app related to some of these changes...new restriction was not being applied correctly there.
- Re-weighting of SBs granted in off topic forum still to come...
...still very much in the learning and tweaking stages.
Thanks for your patience,
Patrick
==================================================================
UPDATE [10/17/22]:
- Removed any automatic unpublishing from Hive moderation. There was still too many false positives (w level 3) so Hive is still checking every piece of content, but it's just flagging potential violations and we're leaning on mod team to get violations off the site (some won't be immediate)
- To deal with dramatic increase in flags, we have brought on 2 additional staff members to mod team (internal to WSO) to help have better coverage of the flag table throughout the day + evenings + weekends
- New tool in mod arsenal to help deal with troll members... top secret
- Re-weighting of SBs granted in off topic forum still to come...likely in ~2 weeks
Overall, I think we're doing a much better job than before. We've blocked ~3 members in the last 2 weeks that refused to reduce their violations... some that have been around for years but were creating 80%+ of the work... others we've warned and so far they've been better / more helpful.
Thank you for your continued patience...it feels like we're starting to turn a corner.
Thanks,
Patrick
UPDATES THAT ARE LIVE [10/27/22]:
==================================================================
- Boosted data for the mods on the flag table to make better decisions (types of violations flagged by Hive, etc)
- Started violation tracking by user so that we can build a history of violations and start automating some rules for repeated offenders that should decrease the mod burden over time
- We are still going to re-weight the SBs granted in off topic forum but it is still to come...still likely in ~2 weeks
- In other news, with better mod team we are also now reviewing every company database submission BEFORE publication to help reduce fakes (and incentives for fakes). we will also introduce more sanity checks that should increase the signal: noise ratio dramatically in coming weeks...
Overall, I feel like the efforts are paying off...the conversations overall seem to be much more respectful - even in some violation magnet threads that were left up. I'm hoping this continues but I understand we'll need to stay on top of it.
Thanks,
Patrick
Thank you for the update. Agree or disagree, just want to share my support for you and the team. I appreciate all your hard work in continuing to build and grow this awesome community. Happy to be helpful in any way I can.
Thanks rabbit. Much appreciated.
UPDATES THAT ARE LIVE [10/27/22]:
==================================================================
Overall, I feel like the efforts are paying off...the conversations overall seem to be much more respectful - even in some violation magnet threads that were left up. I'm hoping this continues but I understand we'll need to stay on top of it.
Thanks,
Patrick
Literally 1984
>Comparing R#ddit and 4chinz in the year of our Lord 2022
ISHYGDDT
Another idea that came to me while reading through a controversial topic.
Instead of our "No Bump Forever" flag just stopping new from comments from bumping the thread, also make that flag enforce "No Anonymous"...this means that for any additional comments after that flag is marked on a topic, we won't allow anonymous comments anymore. It removes a layer of anonymity and I think will make people more careful on how they are wording posts. Good example was the "Sexism in the Office" topic...I think it would have been more productive and respectful with the anonymity of usernames, but not broad anonymity.
Just another way to deemphasize these topics and make people more thoughtful before posting something rude/aggressive.
I support this. Anonymity is GOOD for cases like job shopping where you don't necessarily want to self dox even if the risk is low. It's not meant to be a shield to just run your mouth off - plenty of other places on the internet to do that
Not going to pretend that I've sat here and thought this through carefully, but here's my $0.02 as a longtime user who has for years refused to post under my anonymized user name for fear of what my real-life connections would think of me, if they knew I was on this site:
WSO has the same problem that American society writ large does right now. Namely, the discourse is being completely hijacked by the political and the provocative, no matter what the topic at hand is - this is because we collectively refuse to deprive tribal political nonsense of air, placing primacy on "free speech" instead, I guess. These days, odds are that if you randomly sample a given thread there is some idiot (or idiots) positively sucking the air out of the virtual room by force-fitting any discussion at all (no matter how unrelated) into some childish "left vs right" paradigm of viewing the world. I really do think that serious professionals have left the website over time, because they are averse to the cringe-inducing, ugly results of "free speech" being a priority, which affords more airtime to insufferable morons who simply insist that you listen to their meandering, contorted reasoning as to why XYZ topic is really about the libs, or the MAGAts, when the adults in the room are not trying to have that discussion at all (because it's childish and irrelevant).
It's so completely played and tired, and I think the best move for WallStreetOasis is to simply ban people who are on the website to beat these topics to death. Tribal politics has fuck all to do with what we are here to discuss most of the time (or at least, it used to / ought to). And in the rare instance that someone wants to have a serious discussion about the philosophical/political nexus with finance, I am sorry but I am willing to sacrifice that potential discussion for the greater good of the site. Or condition such discussions on having such and such a silver banana ratio.
The bottom line is that you have to choose Patrick. Is this a site for serious and aspiring-to-be-serious professionals to discuss professional topics in the same way they would in a conference room or call (i.e. with courtesy, mutual respect, intellectual honesty, good will, etc.)? Or is this yet another sprawling internet property where the lowest common denominator of internet society is allowed to hide behind anonymity and (genuinely or performatively) spew their smarmy, idiotic political takes for the sake of their own ego? For real, watching most of these users discuss politics is like watching a child throw a tantrum when their mom gives their newborn sibling attention - just absolute hysterics and desperate repeated insistence that every one and every thing acknowledge their dogshit take on "why XYZ is just another indictment of the libs", or of Trump, or whatever other boogeyman they obsess themselves with. It has nothing to do with what I thought we were all here for.
I can tell you this - when I became a user many years ago I was impressed with the Brofessors and APAEs etc who would keep it professional and show me how to do the same. Now when I come on this site, most discussions are just boring inside jokes/rivalries/catiness along political lines, between a handful of users who post endlessly about their bad social takes to the exclusion of most/all else and it's so eye-roll inducing that I just stopped posting under my name, or at all. This has gone from a site that I once thought was very useful and happy to contribute to, to one which I unfortunately am ashamed to be on most of the time. I think of WSO like a friend that had a bad break in life and has really strayed from themselves in the years since. And while I am loathe to write said friend off because of our history together, it's also getting to the point where I just can't associate with this behavior anymore.
Thank you for the honesty and not for giving up on WSO entirely. I'm hopeful the efforts we're putting in will help encourage more post from the APAE's, brofessor and Compbanker's of the world and less from the others. I don't think we can eliminate the political takes entirely or prevent some users from trying to bend topics into us vs them, but I am confident we can do better.
The Hive moderation tool which is now being integrated and tested by our devs should serve as a really nice first line of defense (for the really horrible crap) and we expect that to be live before the end of this week.
Yes, it was and it will be again. And for the record, I still think we can keep things fun/interesting.
The problem with relying to heavily on ratio is that with tribal politics we see some pretty impressive ratios from some pretty cringe-inducing members. But I just had an idea of how to restore some semblance of balance to those ratios (ie dramatically deemphasize SBs granted in the Off topic forum)...idea keep flowing and my devs might kill me :-)Thanks and stay tuned,
Patrick
ps - funny side note is your comment was initially unpublished because of some of the block words in your post
Only 2 days live and Hive moderation helping us a lot... most of the false positives were from our own block words which we've adjusted. This tool is actually really helping...below is an image of various violations, the count of violations scored 1, 2 or 3...this is only for 1 classifier (sexual)...there are a lot more under "hate" "bullying" etc...
Getting better...
Thanks,
Patrick
Is anyone really complaining about the religious and philosophical discussions?? It seems to me that the top problems are discussions on politics and diversity hiring.
Further, I think it's super cool to interact with other people in the industry who are deeply thinking about their role in the world and how their high net worth careers are balanced with their personal beliefs and convictions. It really makes me feel about better about the industry and others who work in it.
Without these discussions, you are giving a big thumbs up to secular materialism which is very prevalent on the site (posts on BMWs to buy as a first year analyst to $20K watches). I don't know if some people have become tone deaf, but do you realize how cringe those discussions look to someone visiting the site (especially journalists with bad feelings for the sector already.) I think it's much cooler to see a MD arguing about their beliefs and realizing that these are real people with real values, not some stereotype straight out of the worst Hollywood movie.
Politics, religion, etc are always 2% of the content but 90% of the butthurt. There's no point keeping them on the site. Do you want to pay for extra mod bandwidth to handle?
The large social groups I ran with ~300k+ users had the same problems. Is it a bit of a lazy shortcut? Yeah. But this is a small business and everyone pays $0 for access to the forum.
I know that "avoiding religion and politics" is common advice for polite conversations but let me repeat again, are religious battles 90% of the problem on WSO specficially?
98% of the problem is discussions about politics and diversity hiring. 2% may be some contentious religious discussions which at times are also very valuable and insightful. So, I don't see why they are being singled out here as I've seen practically zero complaints on WSO about religious discussions whereas people have been asking for some moderation on politics / racism for years now.
Here's an example of another very real issue that comes up literally every week on WSO.
Random kid posts about being burned out, finding success but not meaning in life and asks what they should do to fix this?
Am I now not allowed to tell him the honest truth of my experience that religion has helped pull me out of that exact same rut? Am I know only allowed to tell him or her to excercise more and drink green juice and get more sleep???
+SB agreed
How does the off-topic ban work? I have 0 responses to my meat question - is it a shadowban, or will those banned from posting to Off-Topic unable to post a post in the first place?
You wouldn't be able to post... Again, that rule will really only stop the worst trolls.
That being said, we are likely to adjust the value of silver bananas coming from off topic discussions down significantly to help us determine more accurately who is collecting SBs from off topic debates and who is more likely collecting SBs from career advice
I think that's a great idea. Not all SBs are created equal (speaking as a squirrel whose majority of SBs were from Off Topic lol)
Gotcha
What do you think of adjusting the Banana to MS points as well? Instead of +3 for banana and -2 for MS, make it +3 for banana and -4 for MS or even -5.
WOW - andrew tate was not wrong.
“WSO off topic is paradise”
...or my hell if you look at it from my perspective ;)
Great work from the team, Patrick!
These initial measures will hopefully clean up the worst posts. Although I am not as active as I once was, I do feel a certain responsibility of upping the level of discussion, especially for all non-Off Topic subjects. It's our collective effort to make the changes we want to see on this forum.
Thank you - trying our best here to improve the culture and tone of our community.
UPDATE [10/13/22]:
...still very much in the learning and tweaking stages.
Thanks for your patience,
Patrick
UPDATE [10/6/22]:
COMING SOON
These next set of initiatives (after Hive on Monday and what we've already rolled out) have me the most excited because I think they will make a meaningful difference long term and are more WSO specific...
Thank you for your patience and help with these ideas...
Patrick
The diversity hires are trying to censor me! AGHHHHH!H!!!!!
Lol. I know that's meant to be a joke, but I do think controversial current topics like ESG or diversity hiring is worth discussing and debating, as long as it's done in a respectful and professional manner. Some of the blatant troll posts though need to go.
fyi, for those curious...the system is working as intended. some really annoying shit blocked immediately and we're trying to allow other stuff to stay up with no bump but the thread asking about dating outside your race had to go (sorry, that is just asking for trouble). Maybe once team gets our feet under us we could allow for that but right now we still have a lot of angry members trying to push the limit :-)
It will be even better once everyone can see that a comment has been removed so that it is less confusing...sorry that has taken longer but goal is to have version 1 of that live this week.
Thanks,
Patrick
The ethnicity one is still up, but the height one got taken down.
was reposted from determined member...removed (again)
Big W, hope we can get rid of all these troll/spam accounts
Honestly these updates have way too many false flags. So many innocuous comments are getting completely removed. I get that certain content and trolls have degraded the rep of this website, but it's honestly annoying to even use this place anymore as even one swear word often gets a comment chucked. This seems like an overreaching effort imo to stop a few bad apples at the sake of open communication on this platform. I and many others I know will probably stop using the website if these ridiculous restrictions remain permanent. Even reddit doesn't have this type of moderation
I responded personally to why your comment had been removed. Yours was a clear violation... If your referring to other innocuous comments that are getting removed and then republished, I agree it can get better and we can have less false positives.
my comment got removed for saying I do not see how you could call elon an idiot. pls fix thx
I know the comment you are referring to, but that's not the one I'm talking about. I've had comments where I'm just answering normal finance/technical questions, and the automation tool unpublishes my comment and says potential violation and needs to be reviewed. I didn't use any swear words, and it's 100% finance. The flagged comments don't get reviewed, presumably because of backlog of too many false positives, and you effectively just wasted a bunch of time to type something out to help someone and it gets removed. What's the point of even typing anything on this website if it can easily end up getting tossed? Even harmless comments asking about dating scenes get cut. I really hope this updates aren't permanent or there is an upgrade with the automod that doesn't strike down so many posts.
I'd say a lot could be fixed by aggressively moving off topic threads to the off topic forum. The IB forum has sort of become a general forum instead of IB focused, and that's a problem. People are asking ER questions in the IB forum. And then you have blatant trolls trying to rile people up - some of them are funny, but also probably not the best professional look either.
Candidly, I wouldn't be shocked if a number of the trolls came from Buzzfeed itself - place bait and then write an article about it. After the last few years, if people still think "journalists" like them have any ethics, I can't help you.
We have someone who reviews topic placement every single day, but that is only done 2x/day so it may be why stuff sits in IB forum for a while. Usually, however, we do get them moved to the proper forum...
UPDATE [10/17/22]:
Overall, I think we're doing a much better job than before. We've blocked 3 longstanding members in the last 2 weeks that refused to reduce their violations... some that have been around for years but were creating 80%+ of the work... others we've warned and so far they've been better / more helpful.
Thank you for your continued patience...it feels like we're starting to turn a corner.
Thanks,
Patrick
What's been up with the MS bombs. Have seen posts where almost immediately the post or any comment gets posted it gets 2-5+ MS's almost automatically. Not a big deal, but just curious if it's a bug or if WSO's just been extra angry lately.
It's a glitch
Quaerat porro sit totam veritatis commodi sit soluta. Ut nam minus dolorem molestias et numquam earum culpa. Dignissimos et occaecati quia fugiat. Laboriosam amet qui ab dolorem id vel. Totam et totam nisi omnis architecto sit. Dolores et maiores architecto occaecati aut.
See All Comments - 100% Free
WSO depends on everyone being able to pitch in when they know something. Unlock with your email and get bonus: 6 financial modeling lessons free ($199 value)
or Unlock with your social account...
Dignissimos cumque quaerat voluptatum sed. Similique sed quod nemo veniam. Cupiditate odio et similique voluptatem. Qui consequatur eos autem doloremque odio voluptas nulla. Quo et soluta distinctio officiis rerum et praesentium quis. Ipsa sed pariatur dolorem debitis nihil voluptas sint.
Nulla consectetur facilis impedit in qui aspernatur deleniti. Ut sit facilis ullam. Qui minus ad libero accusamus illum.