It directly impacts livelihood. It's physical in the sense that it's a real world, tangible harm. This is not at all the same thing as somebody merely saying something critical on Twitter. There's a clear distinction here.
There is a distinction but it's not the one you were originally making. Which seems important in a discussion about kinds of harms. Firing someone is not assault.