> You will absolutely get into some kind of trouble (social or legal) if you put a camera on a sidewalk and start filming whoever goes by.

IANAL.

You should not get into any legal trouble if you're doing it by yourself. There is no expectation of privacy in public spaces in the US.