For voice assistants to be misused only takes malicious intent or incompetence on the server end since they hear and transmit (Perhaps with activation phrase) by design.

For my smartphone to be similarly vulnerable I would need the server side plus malicious or incompetent intervention on the client end to enable voice activation (I have voice activation disabled).

Plus my smartphone hardware is not as capable of picking up voices around the house plus I am not actively positioning it in several rooms to do so.