If users aren't asked for access to their address book, this is still a potential public relations nightmare, since many people will inevitably be distressed that a company will be able to know who was in your address book even if they don't know those people's phone numbers.