🕸Fergus Duniho wrote on Sun, Mar 11, 2012 01:25 PM UTC:
Ben Reiniger wrote:
I think the inventor shouldn't rate the game;
if they want to mention what they think of the game, it should go into the
page somewhere (in the introduction would be good).
Judging by the data I collected on the frequency of ratings in the comment system, there is a lot of rating inflation going on, and it can't all be due to inventors rating their own games more favorably than more objective critics. In fact, I can screen for that. Here are the results of non-anonymous ratings by people other than the inventor:
Excellent is still the most common rating, followed by Good and Poor. We can also look specifically at how inventors rate their own games. Here are the results of non-anonymous raters rating their own games:
This is a smaller sample size, but the pattern is not significantly different. Pride might motivate inventors to rate their own games higher, but that doesn't account for others also giving their games an Excellent rating more often than other ratings. Perhaps the other raters feel they would be hurting the inventor's feelings if they rated a game anything less than Excellent. Perhaps the word 'excellent' has lost its comparative connotations. We may use the word too liberally, even for things that aren't really that much better than other things.
One of my main concerns in designing a new rating system is to avoid the rating inflation common in the current rating and comment system. One thing I think will help is to replace qualitative rating terms (Poor, BelowAverage, Average, Good, Excellent) with personal preference terms (Dislike, Like, Favorite). I think Favorite works better than Excellent, because it is more clearly a comparative term than Excellent is. Judging from the data I just presented, I don't think that banning inventors from rating their own games would do anything to reduce rating inflation.
Furthermore, given that an inventor is usually the most experienced player of a Chess variant, or at least the one person who knows the game better than anyone else does, I think the opinion an inventor has of his own games is valuable information, and it should be included in the system.
Ben Reiniger wrote:
Judging by the data I collected on the frequency of ratings in the comment system, there is a lot of rating inflation going on, and it can't all be due to inventors rating their own games more favorably than more objective critics. In fact, I can screen for that. Here are the results of non-anonymous ratings by people other than the inventor:
Array ( [Excellent] => 782 [Good] => 698 [Poor] => 137 [Average] => 27 [BelowAverage] => 18 )Excellent is still the most common rating, followed by Good and Poor. We can also look specifically at how inventors rate their own games. Here are the results of non-anonymous raters rating their own games:
Array ( [Excellent] => 18 [Good] => 13 [Poor] => 3 [Average] => 2 [BelowAverage] => 1 )This is a smaller sample size, but the pattern is not significantly different. Pride might motivate inventors to rate their own games higher, but that doesn't account for others also giving their games an Excellent rating more often than other ratings. Perhaps the other raters feel they would be hurting the inventor's feelings if they rated a game anything less than Excellent. Perhaps the word 'excellent' has lost its comparative connotations. We may use the word too liberally, even for things that aren't really that much better than other things.
One of my main concerns in designing a new rating system is to avoid the rating inflation common in the current rating and comment system. One thing I think will help is to replace qualitative rating terms (Poor, BelowAverage, Average, Good, Excellent) with personal preference terms (Dislike, Like, Favorite). I think Favorite works better than Excellent, because it is more clearly a comparative term than Excellent is. Judging from the data I just presented, I don't think that banning inventors from rating their own games would do anything to reduce rating inflation.
Furthermore, given that an inventor is usually the most experienced player of a Chess variant, or at least the one person who knows the game better than anyone else does, I think the opinion an inventor has of his own games is valuable information, and it should be included in the system.