I'm not sure about the different GPU issue. But if that is an issue, the model can be made deterministic (probably compromising inference speed), by making sure the calculations are computed deterministically.