Yeah, but the contract lab doesn't need to be filled with geniuses (like Young perhaps) just to reproduce a now-mundane result from a few years ago, right? We're not talking about coming up with cutting-edge new experiments, just reproducing an old result using lab techniques that have probably become far less "cutting edge" in the intervening years...

I am highly suspicious of a claim that some experiment from a few years ago can still only be replicated by a tiny number of the top experts.

I don't think you can find any claim on the order of "only a few top experts can do this..." in the article, the objections claimed to be against specific contract shops. As I understand things, biology experiments are still a matter of craft, of doing things by hand. Like any such thing (say, wood working or computer programming) there are going to be levels of skill involved with the lowest level of qualified people often being rather bad (for example lowest level contract programming shop). An analogy that springs to mind is whether one would want one's video game ported by the cheapest porting operation the business.

Of course, all this is me talking about what I can glean from the article compared to what others imagine the article says - until and unless someone with specific knowledge of the complaints and their validity or lack-there-of appears.

There will either be sufficient documentation to show whether the experiment was repeated accurately or there won't be.

But as the op mentions, lab techniques vary and not everything can be documented. It's reasonable for a scientist to ask for a certain level in a lab attempting reproduction. I don't know if these scientists are demanding too much and have something to hide or if there are problem with the lab involved. However, the article actually mentions numerous scientists expressing misgivings.

>It's reasonable for a scientist to ask for a certain level

Maybe, but a scientist should be able to specifically describe what that 'level' is.

Instead, this article seems to present many individuals who think appealing to an abstract 'level' of their own choosing and to which they seem to not be able to describe sufficiently to anyone else (which is fishy, because all of this serves, intentionally or not, to prevent anyone from attempting to replicate their experiment).

In my view of science, there should never be a case where 'misgivings' about a replication attempt should ever be expressed. Either the experiment will be accurately replicated or it won't, in which case we can probably determine if the replication was not accurate or if the original experiment was flawed (either in design, execution, and/or analysis).

Science gives us the tools to remove things from the influence and design of opinion and to examine them from viewpoints that are as free from subjectivity as possible. Once you start moving into the realm of opinion (expressed misgivings) you are leaving the realm of science and moving into philosophy, religion, or worse.

So it's important for the reproduction attempts to be as high quality and rigorous as we would hope the original studies were. And it behooves scientists to make sure that these attempts are legitimate, unbiased and equitable, and to investigate any experimental flaws and biases of the experimenters before the results.

Misgivings about a reproduction attempt don't indicate denial of the validity of the scientific process, but recognition of the volatility of scientific news media. The unfortunate reality is that both sides of this effort, both original researchers and reproduction attempts, are subject to a great number of biases and restrictions. Subjective opinion does deeply affect the lives of scientists, and it's not possible for even the best scientists to live in a bubble of scientific purity and assume things will work out.

Interesting proclamation.

If a result can't be replicated after many 'highly visible' attempts then the result should be called into serious question.

>The scientific community may be able to poke holes in the reproduction attempt and sort through the damage, but the media and the court of public opinion certainly can't. Not until long after the reputations of possibly faultless scientists have been ruined irreparably.

This sounds like unwarranted fatalism to me. If the result was not reproduced because the experiment was not actually reproduced...I don't see what the issue is here.

>So it's important for the reproduction attempts to be as high quality and rigorous as we would hope the original studies were. And it behooves scientists to make sure that these attempts are legitimate, unbiased and equitable, and to investigate any experimental flaws and biases of the experimenters before the results.

This is why it is critical for any researcher that desires credibility (and more importantly: explanatory power) to detail their work accurately enough that someone else can exactly replicate their experiments in order to provide independent verification of their claims.

The best way to ensure that replication attempts are 'legitimate, unbiased and equitable' is to ensure your work is good enough that someone can actually (as opposed to merely attempting to) reproduce it.

>Misgivings about a reproduction attempt don't indicate denial of the validity of the scientific process, but recognition of the volatility of scientific news media.

Maybe.

>The unfortunate reality is that both sides of this effort, both original researchers and reproduction attempts, are subject to a great number of biases and restrictions. Subjective opinion does deeply affect the lives of scientists, and it's not possible for even the best scientists to live in a bubble of scientific purity and assume things will work out.

This sounds like something scientists need to work towards resolving.

I'm suddenly having flashbacks, something about cold fusion and how groups with the right competence were indeed able to reproduce the results, for some value of "reproduce".