Not to engage in the spergy one-upmanship that usually happens now, but one of the best hackers I know got me building kernels without loadable module support of any kind in the 90s after demoing a similar attack. I build my colo machine kernels without most hardware drivers outside of disk and ethernet. (n.b. that you must also patch to deny writes to /dev/mem and /dev/kmem (even for uid0), as you can use these to insert code into a running kernel even without module support.)
HBGary's leak included a subcontract involving development of a USB kernel driver exploit to inject a rootkit into kernel memory. Just saying ;)