There are vision defects that are attributed to misshapen lenses/cornea that seem to be common with astigmatism, and it's those defects that flare up under the conditions I listed in the OP.

I don't have conclusive proof, of course, but in my experience it is those symptoms that get exacerbated.