difference is that if they keep something from my suitcase, I will know. I have no such guarantees when they access my phone unless I see what they are doing.
That means, that you should expect your device and all access tokens on that device compromised.
I mean, they could cut copies of any keys you have on you, so you don't really have any guarantees there either, and you should therefore expect your locks to be compromised. Or they could photocopy any paper documents, etc.
That means, that you should expect your device and all access tokens on that device compromised.