Unfortunately the thing to think about here is that as these things become more mainstream, there will be less people who are even skilled enough to control a car. So in the long run, if the system works well, any part of the design that relies upon human intervention is doomed. Ultimately your stopgap would become the equivalent of giving the wheel to the child in the backseat: more dangerous than just letting the computer do its best.
On the other hand, it will be best case a decade or two before these things are out on the road en masse and even then only the youngest drivers wouldn't have gained many hours of driving experience. Meanwhile, in that time both automated car and sensor tech will have advanced quite a bit. It's only been a decade since that first DARPA challenge and five years since google started their project, after all.
