ouch. but also, surely that has to happen to the majority of papers, meaning the snap judgment call of effectively one person greatly colors the quality of the whole process. as a conference organizer this is something i worry about. is there a better process proposed out there - that respects the constraint that high value people have limited time to review things?

I think preprints provide a good way to solve this problem, but it requires some cultural shifts.

Imagine a world where all papers start their life as preprints. Researchers read these preprints (like arxiv) and comment on them (like PubPeer). Journal editors then search for the best papers (or hear via word of mouth within their field) and journals compete for the rights to publish the most interesting papers. When an agreement is reached, the journal organised reviewers and asks for any changes to the paper.

I may be missing something critical but I dream of a day when the actual people producing academic output have power over journals who produce nothing but fees.

Historically journal controlled peer review didn’t exist and science still progressed perfectly well.

It’s an appealing vision. Distributing the early review process across time and the whole community would solve a lot of problems with the current model.

I do suspect that this would strengthen the winner-takes-all dynamics in academic publishing. Like whoever has the biggest twitter following gets all the attention, then gets Nature and Science competing for their paper, which allows them to get funding to hire a publicist, who increases their profile.

I suspect the opposite since winner-take-all-dynamics are partly driven by gatekeeping. A famous scientist (like Francesca Gino) gains a following _because_ she is anointed by journals and institutions, which causes positive reinforcement via better jobs / grants / book opportunities etc. If top journals reject a researcher, they languish until they get lucky or the value of their work becomes unquestionable (like Katalin Karikó).

In my utopia, publication success would be driven by interest within a field which means lesser known researchers who haven't had breaks in high impact journals are more likely to break through, because the community acts as the gatekeepers, not a small group of journal editors.

sadly no, it is an unsolved problem of scholarly publishing imo. on the one hand you have the reputable journals following the traditional publishing process that take pride in their high rejection rates -- these require a large percentage of desk rejections to avoid flooding their reviewers with sub-par papers. thus they'll inevitably have some quality papers fall through the cracks + some flashy sub-par papers making the cut.

on the other hand you have the pay-to-publish journals that have a financial incentive to push as many papers through peer review -- these thrive on sub-par papers that are technically just barely 'good enough', but the upside is that the real good ones will also make it through. however, they inevitably face reviewer fatigue, and the most valuable ones will quit reviewing if they often send them low-quality papers. so basically once in a while they'll publish top notch research without being aware of it.

i'm not aware of any middle-ground solutions out there and it certainly feels like a tough problem to solve.