This problem is very real. There is at least an ok way to avoid it getting too bad: make lots of View and ViewModifier structs, with each one having smallish amounts of code. Split them up to the extent that it feels silly.

As an added bonus, this is also the way to get good performance at run time. It helps minimize how many views need to get re-evaluated when something changes.