Essentially, posts are ranked on two dimensions: agreement, the degree to which you agree with the posts arguments and conclusions -- and quality, the degree to which the post the post conforms to certain expectations in a well-mannered, good faith discussion.
Any ranking or moderation involves one degree of subjective judgement: you have a post and you have a dimension along which to rank it (according to certain criteria), where do you put it? This is true even if the criteria are completely objective, e.g. if you asked people to moderate posts in terms of spelling or grammar, which are fairly objective, moderation would introduce a subjective quality (e.g. people don't recognize a typo, some will think a couple of grammar mistakes in a long post shouldn't lead to a down vote, others will disagree).
Post quality has fairly objective criteria, but they are never really spelled out. Nevertheless, it's usually fairly easy to recognize trolls and flamebaits and overall bad quality posts, independent of your own position or even the existence of a position in a certain context.
Agreement introduces another level of indirection, in a manner of speaking: not only does the act of assessing the criteria involve a subjective effort, the criteria themselves are completely subjective, ie. it completely depends on the moderator's own opinion in the context.
Why elaborate on this? Well, in the first place I think it's intellectually interesting how the two dimensions differ in a fairly fundamental way. But there are practical consequences to this, as well.
Simply adding up up/downvotes means we can't distinguish between a post that has not received any moderation and one that has received an large amount of moderation that has cancelled out itself. This seems all right for "quality" moderation: when good-faith moderators can't decide on where to put a comment on this scale, then it's probably not clear where applying the objective-but-not-spelled-out criteria leads to, and it's probably a post that's neither particularly trolly or insightful -- the system works! But when good-faith moderators hugely conflict on an "agreement" moderation, this means the moderators themselves are in conflict, and the post articulates this in a way that makes them agree or disagree, e.g. by identifying the central contested issue. And despite being a +300 & -300 post, it sits there at +/- 0: I don't think the system works very well here.
I'm not sure why we're trying to moderate both dimensions on a single scale on HN. I guess it's an elegant system, because it compresses so much information and complex judgements down to just a single up- or a downvote per person. On a large scale, this should yield good results, right? Well, I don't buy it. In effect, every post is a mini-poll mixed with a quality moderation -- and this is what we use to, essentially, delete comments. Ideally, we'd have two moderation options: a post quality moderation that's summed up, used to hide trolls and flamebaits and not otherwise shown, and an agreement moderation that's not summed up but displayed for everyone to see. "Agreement" votes wouldn't affect karma, as people should neither be awarded for having a popular opinion nor be punished for voicing an unpopular one.
Ideally, we'd have two moderation options: a post quality moderation that's summed up, used to hide trolls and flamebaits and not otherwise shown, and an agreement moderation that's not summed up but displayed for everyone to see.
I upvoted you because I'm in agreement that this is how it should be in an ideal world. I just don't think it is helpful in practice.
The problem here is that people tend to either pick "Agree/High Quality" or "Disagree/Low Quality". These two axes are not orthogonal within the voter's mind.
Given a high correlation, the extra axis only increases complexity without much benefit.
Suppose you had two axes on which to vote (Quality/Agreement) but you only got one vote? If people were forced to pick which axis they speak of, maybe it could allow for more discussion on otherwise controversial topics? I'm imagining a scenario were registering your agreement/disagreement is easy, ala reddit, but saying something about the quality of a post is harder, maybe some threshold of standing in the community, ala hackernews.
Any ranking or moderation involves one degree of subjective judgement: you have a post and you have a dimension along which to rank it (according to certain criteria), where do you put it? This is true even if the criteria are completely objective, e.g. if you asked people to moderate posts in terms of spelling or grammar, which are fairly objective, moderation would introduce a subjective quality (e.g. people don't recognize a typo, some will think a couple of grammar mistakes in a long post shouldn't lead to a down vote, others will disagree).
Post quality has fairly objective criteria, but they are never really spelled out. Nevertheless, it's usually fairly easy to recognize trolls and flamebaits and overall bad quality posts, independent of your own position or even the existence of a position in a certain context.
Agreement introduces another level of indirection, in a manner of speaking: not only does the act of assessing the criteria involve a subjective effort, the criteria themselves are completely subjective, ie. it completely depends on the moderator's own opinion in the context.
Why elaborate on this? Well, in the first place I think it's intellectually interesting how the two dimensions differ in a fairly fundamental way. But there are practical consequences to this, as well.
Simply adding up up/downvotes means we can't distinguish between a post that has not received any moderation and one that has received an large amount of moderation that has cancelled out itself. This seems all right for "quality" moderation: when good-faith moderators can't decide on where to put a comment on this scale, then it's probably not clear where applying the objective-but-not-spelled-out criteria leads to, and it's probably a post that's neither particularly trolly or insightful -- the system works! But when good-faith moderators hugely conflict on an "agreement" moderation, this means the moderators themselves are in conflict, and the post articulates this in a way that makes them agree or disagree, e.g. by identifying the central contested issue. And despite being a +300 & -300 post, it sits there at +/- 0: I don't think the system works very well here.
I'm not sure why we're trying to moderate both dimensions on a single scale on HN. I guess it's an elegant system, because it compresses so much information and complex judgements down to just a single up- or a downvote per person. On a large scale, this should yield good results, right? Well, I don't buy it. In effect, every post is a mini-poll mixed with a quality moderation -- and this is what we use to, essentially, delete comments. Ideally, we'd have two moderation options: a post quality moderation that's summed up, used to hide trolls and flamebaits and not otherwise shown, and an agreement moderation that's not summed up but displayed for everyone to see. "Agreement" votes wouldn't affect karma, as people should neither be awarded for having a popular opinion nor be punished for voicing an unpopular one.