> For each top-level domain, our system picked a random nameserver published for each of the top-level domains and queried a random domain name that we picked for it. We then grouped the results by region and logged the data every 10 seconds.
Am I misunderstanding what they're doing or is this completely misleading? If they're only testing one randomly chosen nameserver, the results are much less likely to be a good indication of the speed of an average request for that TLD. Why not average across all of them?
Also, as they kind of suggest near the end, caching is probably good enough that this is very rarely a problem anyone needs to worry about.
I think they are saying on each probe, a random nameserver and then random site are chosen. So overall it’s randomizing over all nameservers and sites, to make the median a more useful metric of the overall TLD.
In other words, what they wrote suggests it’s not just one fixed nameserver chosen per TLD, rather randomly chosen each time they will make a request.
This methodology doesn't really consider what recursive resolvers tend to do either.
Recursive resolvers will keep track of performance of the authoritative servers they use and send more queries towards the servers where they get faster responses.
For popular TLDs like .COM/.ORG, it's almost guaranteed your recursive resolver will have enough data to pick a fast authoritative. If you're using a
.bike domain though, I'd guess it makes some difference.
Am I misunderstanding what they're doing or is this completely misleading? If they're only testing one randomly chosen nameserver, the results are much less likely to be a good indication of the speed of an average request for that TLD. Why not average across all of them?
Also, as they kind of suggest near the end, caching is probably good enough that this is very rarely a problem anyone needs to worry about.