No, because the device they are accessing needs to be specifically built for this use case. So you would need to give the user a malicious device then use a malicious webpage. The attack scenario isn't impossible, but is far less then those that are possible with WebUSB.
While not impossible, I just can't think of a good reason why you'd need them to access a malicious webpage when you've already gotten them to install a malicious device.
