What if programming consists of only refactoring? Then there is no separate "refactoring step", just programming and neglect. This is what Bret Victor is getting at. It is about finding the right medium to work in.
We have that already i.e. coding to a test. It sucks because you never seem to grasp the entirety of a program but instead just hack until every flag is green. It doesn't prevent entropy either. Only thing that prevents code entropy is careful and deliberate application of best practices when needed i.e. a shit ton of extreme effort.