Total immigrants = EU immigrants + rest-of-word immigrants

If you have a cap to total immigrants and can’t limit EU immigrants, the open spots have to come from ROW

Right. And while there is no legal cap to immigration, de facto there are limits to how much immigration is feasible in a given period.

Switzerland also makes it more difficult because immigration is done at a very local level of cantons

Traditionally, it was more local than that: It was done on a community level. But it is becoming increasingly an administrative process because the previous system was too obviously arbitrary.