Of course not. He's going to go botnet shopping and try to log in through a compromised machine with an IP address in the same town, in the same address range.

He might even be able to use the same IP as you, if you happen to be using a gateway with many machines behind it, and one of them is compromised.