(if you have a place to keep this extra secret where such an attacker can't get it, why not just put the passwords there?)
The argument is that you can put the key into the code base or into the deployment environment. Passwords are user changeable and can't go there. And, just maybe, you only lose one or the other but not both.
(NB: I'm not defending the pepper, but at least one very major software security consulting firm uses this argument. I don't know how much time they spend doing pen tests and getting a feel for when the pepper would have actually saved anyone's ass, and when I pressed I only got a vague mumble about customer confidentiality.)
Oh, another argument is that sometimes the attacker gets to your DB at time t1, and your codebase at time t2, and if you have a system that changes your pepper regularly the t1 may be out of sync at t2.
All in all, it seems like a lot of work (that could be spent elsewhere) for little gain. I've been burned enough to know that any tweaks to a crypto system to "make it better" sometimes shoot you in the foot instead.
The argument is that you can put the key into the code base or into the deployment environment. Passwords are user changeable and can't go there. And, just maybe, you only lose one or the other but not both.
(NB: I'm not defending the pepper, but at least one very major software security consulting firm uses this argument. I don't know how much time they spend doing pen tests and getting a feel for when the pepper would have actually saved anyone's ass, and when I pressed I only got a vague mumble about customer confidentiality.)