"the average error was a staggering 24 minutes"-- The average is a terrible measure of centrality for data that is unbounded at one end, because one outlier (the 42 year off clock) skews the measure disproportionately; the median would be much more appropriate to report. That's unusually sensationalist number crunching for The Economist.
I'll bet that out of the clocks that doctors actually use to decide on treatment (tallying every device is a bit unfair; if a blood pressure monitor on some cart in a closet has the wrong time of day who cares) 80% of them are within a few minutes of the right time. And at the end of the day, doctors are supposed to be able to reconcile conflicting information, it's part of their training. This is a little bit like tallying up every textbook lying around a high school and getting concerned that the average book is three editions out of date and has 142.8 errors--calm down, most of those are not in active use and have some faith in teachers to check their material.
The research tested 1700 devices. A single device that is off by 42 years (reset to Unix epoch?) would have skewed the average by over 9 days. So unless a number of other devices had similarly extreme errors in the opposite direction (and even if they did, I'm pretty sure the researcher would have only looked at the absolute magnitude of the error, not their direction), I think it would be safe to assume that extreme cases were excluded from the average.
Bnd you're right, 80% of them were within 30 minutes of the right time.
> "One in five were off by more than 30 minutes"
In fact, given that the article mentions daylight saving time as a major problem, I'm not surprised that the average error approaches half an hour. At any given time, a lot of machines would be running in the wrong timezone.
doctors are supposed to be able to reconcile conflicting information
They're supposed to wash their hands, too. (http://boingboing.net/2010/03/09/study-says-us-doctor.html)
As a guy who makes dozens of mistakes a day, the ones I know about because a computer catches them, I'm really not reassured by the idea that human judgement makes up for machine error.
It's pretty unlikely to be a median; if 20% of machines are 30 minutes off and 24 minutes is a median, then 30% of machines are between 24-30 minutes off, which would seem to be a strangely narrow band.
More likely, the 24 minutes is a mean average after excluding outliers. One of the standard ways to compute outliers is to measure the inter-quartile range---the "distance" between the 25th percentile value and the 75th percentile value---and call everything that is more than 1.5*IQR past the 75th percentile an outlier. You can do this without computing a mean first, and you can use it to exclude values before computing a modified mean.
When you're timing something that needs accuracy, you use a timer not a clock. If you need to use a clock, you use one clock not all the clocks. Yes, I work in the medical industry. Yes, we generally use timers not clocks. Yes, our clocks are all wrong too.
If you really do need a solution to the "time" problem (accurate time all the time), you need to issue everyone a smartphone.
> "Surely connected devices could use one centrally calculated timesource?"
... like NTP?
Sorry, but I'm not sure what point you are trying to make here. Anything with a computer, even a microcontroller, and basic network connectivity can use NTP, it is an extremely simple protocol.
Anything that doesn't connect to a network could be kept in sync via a wireless pulse/tick, though then you have security issues unless you cryptographically sign the clock signal. Though I'm not sure why anybody would try and screw up a clock signal like that other than general asshattedness.
That aside, I'm unsure why they are using local time for logging data anyhow. If the folks managing the hospital were rational, shouldn't there be a regulation to use GMT and 24-hour clock? My home town is right on the Eastern/Central timezone split, and I know it causes all kinds of problems in day-to-day life. I just hate that kind of ambiguity.
> If the folks managing the hospital were rational, shouldn't there be a regulation to use GMT and 24-hour clock?
People are not rational, that's the problem! Just look at any web app that can't get timezones right.
In addition, most Americans swear by EST, CST, PST, etc. and have no idea how many hours behind GMT they are. Record everything in GMT and soon there will be a similar article documenting how medical professionals routinely misinterpret GMT timestamps. Especially in March and November.
When lives are at stake, people tend to be more amenable to changes like GMT and 24-hour clock. If the military can do it, a civilian organization with sufficient motivation (insurance terms, regulation, etc) should be able to make the same shift.
However, I was referring more toward the recording system than the staff. If the recording systems were all using unix time stamp with a GMT system clock synchronized over NTP, there would be no ambiguity as to when a record was added to the system. End user systems could be responsible for keeping track of what time zone the user is in, and could convert as necessary. The important thing is that the "One True Datum" have the appropriate stamp.
> End user systems could be responsible for keeping track of what time zone the user is in, and could convert as necessary.
That would require routine updates to the timezone database, which is a bit more complicated than NTP. (Yes, DST rules change all the time. The U.S. changed its rules just a few years ago.)
Hopefully, the rise of cheap and powerful ARM SoCs will make it impossible for device manufacturers to make excuses about not running a proper OS on their machines anymore. I would love to see more Linux and BSD on medical devices.
I'm pretty certain that CDMA towers, at least, had exceptionally accurate clocks for something related to the technology (synchronizing the chip code maybe? it's a guess).
Not sure if this applies to GSM as well, but CDMA used to be an acceptable clock source for a Statum 0 clock source.
You technically don't even need accurate time, you need consistent time. If all your clocks are consistently slow by 10 minutes, there's less of an issue then if some of the clocks are slow by 10 minutes.
Since the EMR is updated constantly, data from devices
whose clocks are way off would simply never be recorded.
Others might bury current information in older files.
Worse, they may insert the data into the EMR when the
patient they concern has left the given clinical
environment and another has come in.
1. Why does the central EMR trust timestamps from the devices?
Daylight saving time corrections twice a year require
tedious manual tweaks that the MD PnP Program, an
initiative supporting networking standards for medical
devices, estimates cost American hospitals over $17m
annually. At Massachusetts General, a patient-monitoring
system deletes an hours' worth of data when rolling back
from 2am to 1am every autumn, while drug pumps are kept
permanently on standard time, so they are (at least) one
hour off for half the year.
Hi. Written EMRs and built medical devices (10-12 years exp between them). In many cases messages from devices and external systems are processed asynchronously, so there isn't much choice but to trust remote timestamps.. I may be getting data minutes or hours later. Just a distributed system challenge.
The time zone issue is a human one. Historically clinicians schedule doses at "Every 3 hours" or "Once per day, 8am". Humans figure out what that means, but a dose scheduling system that shifts 8am an hour after daylight savings (because it interpreted the schedule as "every 24 hours" is going to run aground with nurses when it tells them they're late). However, every 3 hours is pretty obviously an interval and likely shouldn't shift by 30% of the interval!
This isn't a distinction that clinicians are used to making, but as someone who had to write some of the first dose-scheduling systems focused on clinical accuracy, we ran into this challenge and did a lot of thinking about which schedules really meant 'intervals' of specific units of time and which ones meant 'due at these calendar times every day'.
It's an interesting example of a problem that introducing technology introduces/exposes while showing up to fix other things.
Hi. Written EMRs and built medical devices (10-12 years exp between them). In many cases messages from devices and external systems are processed asynchronously, so there isn't much choice but to trust remote timestamps.. I may be getting data minutes or hours later. Just a distributed system challenge.
So I'm guessing you might solve that by asking the device what its internal clock is at the time of synchronization, and adjust the logs by the offset between device time and EMR time (with manual device time changes included in the log)?
Sure. If there was a 'time of synchronization'. There's generally not. The device often doesn't know when/if/who is consuming it's data. Likewise the EMR systems don't generally have any access to the device other than receiving a stream of messages some non-deterministic after they were produced.
As someone who works with devices which require good timekeeping, I'll add another question: if the devices are communicating with a central server, why doesn't that protocol include a time update from the server?
I wrote software for medical devices for 3 years. Saying 'just use NTP' is all well and good, except that getting Internet access inside of a hospital is usually impossible. So of course we offered a configurable setting so that they could use an internal or firewall-allowed server. I think one hospital took advantage of that setting. Everybody else was just wrong.
Configuring every single device to use NTP is a hassle. Do medical devices on an intranet use DHCP? DHCP is capable of handing out NTP server addresses, although I don't think it's very common. Another hacky option would be to override "pool.ntp.org" on the intranet's DNS server so it points to an internal server.
I'm glad that there is some more discussion of this issue, as medical devices are some of the main high technology products in my part of the United States, and my son (also a Hacker News participant) worked as an intern at one of the medical device companies here last year. Medical device businesses are much more regulated as an industry than are most Web-based SaaS businesses.
I just hope this doesn't result in poorer security between outside networks (including the internet) and medical devices. I'd argue there should be an "air gap" between the two, but I'm probably more paranoid than most.
Certainly it's fine to use NTP, but I'd want to use some kind of GPS-receiver NTP appliances at each site (possibly redundantly).
I'll bet that out of the clocks that doctors actually use to decide on treatment (tallying every device is a bit unfair; if a blood pressure monitor on some cart in a closet has the wrong time of day who cares) 80% of them are within a few minutes of the right time. And at the end of the day, doctors are supposed to be able to reconcile conflicting information, it's part of their training. This is a little bit like tallying up every textbook lying around a high school and getting concerned that the average book is three editions out of date and has 142.8 errors--calm down, most of those are not in active use and have some faith in teachers to check their material.