The key difference is that push notification's backing "service" are out of band with the subscribers.

Applications can subscribe to push notifications and get them even if they're off (the OS runs them and gives them the notification). WebSockets are closed if the app closes.

A little to your point, the backing notification service might rely on WebSockets or polling, or a mixture of both, in implementation.