Really nice idea to only show warnings if they are relevant. It's indeed annoying if you need to upgrade lodash only to make your audit tool not show critical warnings because of some function that is not used at all.
This is not open source, though? It does make a big difference for some whether you're able to run the check offline or you're forced to upload your code to some service.
One feature I'd love in such tool would be to be able to get the relevant parts of the changelog of the package that needs to be upgraded. It's not responsible to just run the upgrade command without checking the changelog for breaking or relevant changes. That's exactly why upgrades tend to be done very late, because there is a real risk of breaking something even if it's just a minor version.
As with all other Semgrep scanning, the analysis is done locally and offline -- which is a major contrast to most other vendors. See #12 on our development philosophy for more details: https://semgrep.dev/docs/contributing/semgrep-philosophy/
Relevant part of the changelog is a good idea--others have also come out with statistical approaches based on upgrades others made (eg dependabot has a compatibility score which is based on "when we made PRs for this on other repos, what % of the time did tests pass vs fail")
(Note: I helped write that. We're building a similar service to the r2c one.)
You're right that patching is hard because of opaque package diffs. I've seen some tools coming out like Socket.dev which show a diff between versions. https://socket.dev/npm/package/react/versions
But, that said, this is still a hard problem to solve and it's happened before that malware[0][1] has been silently shipped because of how opaque packages are.
Looking at package diffs is super important because of the rise of "protestware". For example, a maintainer of the event-source-polyfill package recently added code which redirects website visitors located in Eastern European timezones to a change.org petition page. This means that real users are being navigated to this random URL in production.
It’s very unlikely that users of event-source-polyfill are aware that this hidden behavior has been added to the package. And yet, the package remains available on npm many months after it was initially published. We think that supply chain security tools like Socket have an important role to play in warning npm users when unwanted ‘gray area’ code is added to packages they use.
There are definitely other approaches that don't require code to be uploaded anywhere. For example, we (https://rezilion.com) work with your package managers to understand what dependencies your program has, and then analyze that metadata on the back end. Net result is still to be able to see what vulnerabilities are truly exploitable and which are not.
This is not open source, though? It does make a big difference for some whether you're able to run the check offline or you're forced to upload your code to some service.
One feature I'd love in such tool would be to be able to get the relevant parts of the changelog of the package that needs to be upgraded. It's not responsible to just run the upgrade command without checking the changelog for breaking or relevant changes. That's exactly why upgrades tend to be done very late, because there is a real risk of breaking something even if it's just a minor version.