On the first request. All subsequent requests will be cached.
Oh, and if the requestor has a large DNS cache upstream, it's already done.
Oh, and if the browser used a pre-fetch, that's already done.
Oh, and if you have already invested your branding effort across 100 sites, maybe you don't want to re-do all that?
Oh, and if you need to cut 50ms from your first time page load, have you considered dropping all the trackers and analysis JS loads? Can you deliver your first page without any JS at all? Can you do it without a database lookup?
Those are all things you should do before killing anyone.
If you haven't done already, take a look at don't delivery improvements. Serve don't files locally, subset them, and use variable fonts and woff2. I'm a micro performance enthusiast myself and it's my #1 optimization with the biggest gain.
