This is nice, simple experiment design. However, I'm really surprised they only ran this once - or indicate trial counts and indicate range/distribution. I'd be curious if video transcoding is so consistent that a single measurement is enough to draw a conclusion; certainly network/storage transfer is not. Sure, time and bandwidth are not free...
Hi DMV - we ran the transcoding tests a few times, but transcoding performance is pretty steady across multiple runs (here and elsewhere).
Network obviously isn't; the numbers here include about a dozen test runs. We should make that more clear. Even a dozen isn't enough to be a scientific test, so hopefully we (or someone else) will do more benchmarking in the future.
