Unless you did it already and I missed it, my vote would be for the external bus sequencing. The 8086's multiplexed A/D bus and its 4-clock (!) cycle have always seemed needlessly complicated to me, especially when compared to the 68k's async bus or the 6502's "?!$!@ it, just expose the internal latches" designs. Clearly Intel was thinking it was getting value here somehow (well, something more than pure pin count optimization), but it really just seems like they were adding behavior they didn't need.
Yes, the bus sequencing is very complex. Memory accesses turn out to have a 6-clock cycle, but two cycles are overlapped with the end of the previous access, so it's even more of a mess than you'd think.
I think a large part of it was pin count optimization; Intel had weird beliefs about keeping the pin count down. (E.g. Intel was really resistant about using even 18 pins for the 8008, which was way too few.) Another issues was the 8086 supported two different bus protocols: "minimum" and "maximum" mode, where "minimum" was straightforward and easy to use, while "maximum" provided much more information but needed another chip to decode the signals. (I don't know if anyone used "minimum" mode.) Finally, the 8087 needed a lot of information about what was going on inside the 8086's prefetch queue, so that needed more bus signals.
> Intel had weird beliefs about keeping the pin count down.
They might seem weird by today's standards, but by late 1970's standards, a '40 pin' package was significantly cheaper for others to incorporate into their designs than another size. There were a lot fewer "standard socket sizes" available back then, and anything that wasn't already available entailed a lot of expense to manufacture, resulting in a very high initial cost (comparatively) to recover the capital expenditure to bring the new size to production.
Intel's beliefs about pin count were weird even for the time. See Federico Faggin's oral history (p55-56) for a discussion about how making everything 16 pins was like a religion at Intel, damaging functionality, at a time when other companies had 40 or 48 pin packages. The Motorola 68000 had 64 pins.
One of the magazine ads for the original Macintosh highlighted this. Iirc, a 68k in all its glory was shown next to a plastic DIP 8088. It made enough of an impression on my ~12 y/o self to still picture it today.
Seeing that view, for the first time in twenty years I thought about the curious stickiness of a well-used mouse ball, and cleaning the gubbins off the counter wheels, prompted by the cursor starting to stall and jump on the screen.
