Transparency on star ratings

[engswee]

Dear SCN team

Currently, unlike other actions (like, bookmark, etc), star ratings on contents do not provide transparency on the user who made the rating. The only exception is if the user chose to leave a comment together when making the rating. As such, this system behaviour leaves room for abuse.

For example, in the space where I'm active in, there are frequent surges in points by a certain member on the leaderboard (I'm not sure how long this dates back, but it's been very frequent this month). If I were to look at the points history, it shows a surge of likes and 5 star ratings on the user's content happening very frequently. As the "like" actions are transparent, this can be traced to user profiles that are newly created in SCN with no other level of participation. However, since the star ratings are anonymous, there is no way to know who made them.

As SCN moderators might be busy and not always able to spot these behaviour, IMHO having transparency on this rating system will allow other members to spot these suspicious trends and raise them to moderators for further action.

Regards

Eng Swee

[BenedictV · Accepted solution]

Some people are really good at abusing anything. I never knew the ratings could be played this way to hike our points.

I like the 'Silent Ratings' the way they are now. Saves us the trouble and embarrassment of having to explain our rating for content. But you do bring in a valid point here.

A criteria may be established to be able to rate content. Like attaining a minimum reputation level before you can actually start rating content.

Benedict

[Colleen · Accepted solution]

I like the 'Silent Ratings' the way they are now. Saves us the trouble and embarrassment of having to explain our rating for content.

How is it embarrassing to justify why you gave a particular rating? This is a professional forum and having ratings visible would have a lot of benefit to the community - especially when ratings are poor.

Constructive feedback is necessary for authors to improve content. Also, I have different levels of trust for some community members (am aware of their professional reputation and skills) - if those members gave high or low ratings I would pay attention to the feedback.

[BenedictV · Accepted solution]

Maybe my choice of word was wrong, but not everybody conducts themselves in a professional manner when it comes to feedback. Given the number of "why my post rejected?" discussion and the way the OP reacts to moderator messages, I doubt we can have a professional exchange of words when we down rate a content. I would like to avoid such a situation if that happens.

The irony would be that, people who are able to take a criticism in their stride would generally be the ones to post good content. Just saying

[Colleen · Accepted solution]

but not everybody conducts themselves in a professional manner

Then there are repercussions for lack of professionalism in a professional community:

1. Moderator warns them/rejects content
2. They are no longer welcome in the community (escalate to Global Moderators who lock the account)

In the case of rating, it would be unprofessional behaviour if there was evidence of some going through and points gaming (rating a friends content 5 stars for everything) or bullying (constant target of a member and rating their content 1 star regardless)

Where it comes to making comments - these are independent of putting a rating (although it would be helpful that members were encouraged to offer feedback as to why they gave their rating). I can see the challenge in making comments mandatory here - either users won't rate content OR we'll get an endless thread of good job/nice work which add more value than what a 4 or 5 star rating implies.

In reality, if the piece of work does not meet quality standards, the content will be removed. In SCN 3.0 (on it's way) there may be pre-moderation on blogs and documents to ensure the material meets a standard.

A few tangent here, but I see more benefit in removing anonymity of ratings.

[matt · Accepted solution]

In the case of rating, it would be unprofessional behaviour if there was evidence of some going through and points gaming (rating a friends content 5 stars for everything) or bullying (constant target of a member and rating their content 1 star regardless)

Precisely because of the lack of accountability and transparency, this could be going on all the time, and we'd have no way of finding out.

[engswee · Accepted solution]

Hi all

Sorry for the delay in my response after starting this thread, it was a long weekend over here.

Jurgen has already DM-ed me stating that:-

- this invisible rating is a known area for points cheating

- there is a log in the gamification system to find out who made the invisible ratings, however it is complicated to retrieve and review

- with 1DX just round the corner, it is unlikely that this behavior will be changed

I'm glad to know that my concern is also shared by others here. I still do not agree with the so called "reason" for having anonymity for ratings due to potential unfavorable ratings, especially as others have mentioned that this is a professional forum. Having transparency for every action taken will encourage everyone to be wiser (hopefully) as each and every action taken or word spoken/written reflects upon oneself professionally.

As a matter of fact, I think the gamification element to the rating system itself is inconsistent. It is only possible to award points for 4-5 stars but no point reduction on low star ratings. Firstly, there is already another mechanism to award points for good content, which is to "Like". Subsequently, apart from gaining points for 4-5 star ratings, I'm not sure if there are other benefits to the rating system, i.e. is it used in the backend to bring to surface higher rated contents during search? As a comparison, the rating system in StackOverflow, helps push up answers that are useful.

Anyway, I sure hope the 1DX system will not have similar loopholes (not sure how it's gonna work on that platform.)

Rgds

Eng Swee

[engswee · Accepted solution]

On a separate note, I just read the announcement from regarding the .

From the way things are going, it seems likely that the aforementioned person might end up with a Topic Leader recognition. If that indeed comes to pass, I'd personally conclude that SCN's Topic Leader contest/badge is effectively rendered meaningless, and the gamification/point system is therefore pointless (pun intended).

[Colleen · Accepted solution]

Hi Eng Swee

Global Moderation validate the top 3 to check if there are any concerns of points gaming, etc. If there are allegations of this, the SCN community team/Global Moderators will check through system logs to see where ratings are coming from and investigate.

I won't go into the details of these processes as they are private (also if an allegation is made against someone it remains confidential so no risk of slander or bad reputation if allegation is unfounded).

If anything, a shorter competition period may make it easier for us to all observe patterns of points gaming.

If you believe there to be an issue with a current member it might be worth CCing the Moderators of that space and asking one of them to direct message you. Alternatively, press alert moderator on one of the items and add a comment as to your concern. This is a way to initiate the investigation as a member.

Regards

Colleen

[engswee · Accepted solution]

Hi Colleen

Noted. Thanks for the clarification and advice. Glad to know that such processes are in place.

Rgds

Eng Swee

[caroleighdeneen · Accepted solution]

Yes, we indeed have a process to vet the topic leaders once we identify the top point earners. If you believe someone should be investigated for cheating, can help.

[Colleen · Accepted solution]

Hi Caroleigh

I didn't mention Moshe as you can't DM without following each other. I figured Alert Moderator is the better approach to initiate a complain/raise concern

Regards

Colleen

[engswee · Accepted solution]

Dear Caroleigh

Thanks for your response.

Just to clarify, prior to opening this thread, the space moderator has been alerted and have mentioned that it will be investigated. The purpose of this thread was more to highlight the potential room for abuse of the system that can hopefully prevent future occurrences.

From Colleen's previous reply, I've realised that there is risk of slander should my suspicions be proven untrue. I've avoided any direct references in this thread in my attempt to highlight this issue. I hope I have not caused any unnecessary risk of slander or bad reputation. If I have, please feel free to moderate my posting/responses as deemed necessary.

Rgds

Eng Swee

[caroleighdeneen · Accepted solution]

Sure, that is the right first step:)

[caroleighdeneen · Accepted solution]

I think these are important issues to discuss. And helpful feedback from a user centered design perspective. I have referenced this thread in planning sessions with the 1DX team, as we consider transparency in the future platform:)

[Jelena]

In absence of "dislike" button, low rating is the only mechanism on SCN to voice your objection to the content that does not really violate ROE but is of low quality. Unfortunately, there is no guarantee that some disturbed author won't engage in a vendetta on the SCN members who "offended" him/her by giving a low rating. Hence I support leaving the rating anonymous.

But in the spaces I visit I actually don't see any foul-play evidence in the ratings. They usually quite match my own perception, so this means they are right.

[engswee]

Hi Jelena

Thanks for your input. Jason's reply in the linked thread above has also mentioned the need to "protect" those who might rate content unfavorably, and I get where you are coming from.

However, as I've mentioned in my earlier response, I don't see how the ratings affect the content at all apart from gaining points for 4-5 stars. From a system perspective, it is not transparent if the ratings affect the search results on SCN (which most don't use anyway and rely on standard/custom Google search). From a user perspective, we don't see the rating until we have completely read the content as it is placed far at the bottom. Even if we were to object to the content by way of an anonymous low rating, it does not send any alert to the author and it just appears as an averaged-up rating. Some authors might not even notice an anonymous low rating.

On the other hand, there is already a mechanism to reward good content by awarding points through Liking. This mechanism helps surface good content (i.e. Top Liked content of a space), and visible from the Content tab (how many likes for a blog/document) even before the content is read. Compared to the rating system, any abuse of this for point cheating is transparent and easier to detect.

As already pointed out by Jurgen, I don't expect things to change in the current platform with IDX imminent arrival. It's just my two cents that hopefully be considered for the future platform.

Rgds

Eng Swee

[former_member183349]

your attention really brings alert to all scn members. What you had stated here is absolute correct

information Mr.Eng Swee Yeoh.

Regards,

Vivek

[former_member183424]

Indeed. Your concern is absolutely correct. But there is different reason to make it anonymous. In a thread I read it from Jason, here it is About "Silent Raters" | SCN

[matt]

I totally agree!