Well that's not quite true. You can do a few things:
1. Reduce dependencies and features of dependencies
2. Use a faster linker like mold
3. Use a faster compiler backend like cranelift(if possible)
4. Use the parallel compiler frontend(again if using nightly is possible)
5. Use sccache to cache dependencies
But i do get what you mean. Especially in CI the build times are often long