Everyone's first thought when they read something is whatever the social norms say you're supposed to think (peer review = good, publishing without peer review = not science somehow?), but shouldn't you stop and wonder why the esteemed scientist wrote that line instead of just dismissing it? Otherwise you are only chiming in to enforce a norm that everyone already knows about, which is pointless.
One of the really refreshing things about reading older research is how there used to be all these papers which are just stray thoughts that this or that scientist had, sometimes just a few paragraphs of response to some other paper, or a random mathematical observation that might mean nothing. It feels very healthy. Of course there were far fewer scientists then; if this was allowed today it might be just too crowded to be useful; back then everyone mostly knew about everyone else and it was more based on reputation. But dang it must have been in a nice to have such an unrestricted flow of ideas.
Today the notion of a paper is that it is at least ostensibly "correct" and able to be used as a source of truth: cited in other papers, maybe referred to in policy or legal settings, etc. But it seems like this wasn't always the case, at least in physics and math which are the fields I've spent a lot of time on. From reading old papers you get the impression that they really used to be more about just sharing ideas, and that people wouldn't publish a bad paper because it would be embarrassing to do so, rather than because it was double- and triple-checked by reviewers.
We still have lots of stray thoughts, responses and observations, now they just happen on blog posts, on social media and in other non-peer-reviewed venues. The Internet has driven the cost of publishing to 0, and peer review is the only thing left that makes academic publishing qualitatively different. If anything, publishing your thoughts online is better than publishing a traditional paper in every single way except for peer review.
Well, publishing online also has a reach problem. The nice thing about journals is that they consolidate all the material on a subject. Arxiv does this for some fields (and I guess similar aggregators in other fields) but really it is nice to have the thoughts still be _curated_, like a magazine, without necessarily being to a citeable/publishable standard.
>shouldn't you stop and wonder why the esteemed scientist wrote that line instead of just dismissing it?
I go to forums like Hacker News and Reddit and regularly see software engineers who are outraged about having to have their code reviewed and even more outraged about actually having to implement feedback from their reviewers rather than receiving a rubber stamp.
I go to work and see the effects on product, team, and world of what would happen if those coders were allowed to bypass supervision.
So no, even someone who is intelligent and good at what they do should have peer review.
You are talking about something different than I am. I'm not saying you should have un-peer-reviewed major research. But there are other types of communication that are useful but do not need to be rigorously vetted. (not that peer review is all that good at the vetting anyway)
Im not a scientist and do not know how these things work out, but wouldn't it be possible for scientist to simply publish their papers online without peer review if that is what they want?
The only for work to have an impact is if it gets exposure. Publishing in journals got you an audience, but that audience is gatekept by peer review, which has its problems.
So sure, you could publish but the chance of having an impact was low. Thankfully that's changed a bit with arxiv.
Nothing stops them, some people do do that. Two examples that come to mind are Aella's research on fetishes[1] and Scott Alexander's research on birth order effects[2]. But you don't get academic credibility by publishing online without peer review, and it's much harder to get university funding.
Very true and it's wonderful. But only a thing in some fields, as I understand it. In the past that was the role that a lot of papers played but the conflation of publications and citations with career advancement messes that all up.
