Sure. Do you personally read through every line of source? Maybe you trust the repository managers to do so, and limit yourself to only very-popular projects - do you completely trust the trust-chain that lets them submit new code? Maybe you do - remember when RubyGems.org was hacked? Or when [many sites] lost their private crypto keys? What's to stop the same thing from happening, and pushing a critical update with an exploit? Maybe you reduce the frequency you check for updates to mitigate this kind of vulnerability - oops, now you're more vulnerable to new exploits.
If you're not watching every step, every time, you're gambling the same way you're gambling with malicious code in a browser (though I'll admit it's lower frequency). Your privacy/security is in the hands of whoever is part of the chain you trust, and their security practices, completely aside from new exploits that could affect you directly. Open Source, binary, it's all the same in the end unless you're perfect in your observational skills. Sandboxing limits that trust. I'll even grant that it's technically possible to do the same thing with processes in Unix, which you could be doing - but it's hard, error-prone, and essentially nobody does it except the stragglers who haven't switched to virtualization (which is essentially sandboxing).
You're being paranoid. With perfectly justifiable reasons - everything you listed is possible, plausible, and related things have actually happened. But you're not applying the paranoia evenly.
If you're not watching every step, every time, you're gambling the same way you're gambling with malicious code in a browser (though I'll admit it's lower frequency). Your privacy/security is in the hands of whoever is part of the chain you trust, and their security practices, completely aside from new exploits that could affect you directly. Open Source, binary, it's all the same in the end unless you're perfect in your observational skills. Sandboxing limits that trust. I'll even grant that it's technically possible to do the same thing with processes in Unix, which you could be doing - but it's hard, error-prone, and essentially nobody does it except the stragglers who haven't switched to virtualization (which is essentially sandboxing).
You're being paranoid. With perfectly justifiable reasons - everything you listed is possible, plausible, and related things have actually happened. But you're not applying the paranoia evenly.