Having access to the source code doesn't mean it's "open source". Open source has to do with being able to reuse the code yourself, not to being able to read it.
But the server literally pushes the source to me, and I don't typically explicitly accept any agreements.
Why cannot I simply use a readily available feature of my browser and take a piece of code and just use it?
Especially if there is no licence agreement right in that JS file.
Unlicensed code does not grant permission to use, in fact it's the exact opposite.
The server is not pushing the source to you so that you can take it and use it in your own projects. It's pushing the source because that's how the browser will run the program.
Of course you CAN take and use the code. But you MAY not.
