Can you elaborate on how a test might cause permanent damage? Keep in mind that it should be clear whether significant damage is occurring quite soon after it rolls out. (Eg. Whether comments are getting approved, rate of comments, etc.)
That's certainly a possibility. It would show up in HN's metrics. (perhaps not visible to you and I, though)
On the other hand, you might also see more people coming in the door. What if this makes conversation on HN so much better that more people want to get involved?
The one thing that is certain with the new system is that if your comments are abusive, they will probably not get endorsed. Perhaps at first they will (protesting 1k+ karma users) but eventually anyone who is endorsing poor quality comments would likely lose the ability to endorse.
At a minimum, I think this a really interesting experiment.
Eg, if this policy precludes 1-on-1 dialogue between sub 1k users, that would (arguably) be a dis-service. It would be "permanent" in the sense that the rules# logically don't allow a two party conversation (ie, absent a 3rd party "endorsement"). So this meets the test of "damage" and "permanence" at some threshold.
The thing is, this is a kind of change which makes it difficult to know the real impact since the users who would have a problem with it are the ones losing their voice (until a karma elite user intervenes, of course). That's one of the reasons most people are opposed to this.