Eventually, we might not be the best at that either. Imagine AI companions that can not only perfectly emulate human behaviour, but can produce simulations of empathy, compassion, humour, etc. perfectly tailored to an individual's psyche.
When we imagine intelligent AIs keeping humans as pets, we tend to do so in analogous terms to how we treat animals: E.g. in sparse, constrained environments, like cages and zoos. But those environments are designed with animal level intelligence and instincts in mind. AIs will probably design habitats intended to placate human instincts. And a big part of that will be keeping us psychologically happy, which will involve providing simulated companionship.
If I was confident in a GAI's ability and willingness to emulate me, then I might be willing to grant it my identity, after I die. My work would carry on, and accelerate, while I would still get to have the final experience of death, for better or worse. The people who depend on me would not be abandoned, and the people who like me, might like the new me even better.
We might become a species that undergoes metamorphosis from a carbon based body to a silicon based body. How much of a caterpillar remains in a butterfly, when it emerges/ascends?
