True, and the successful ones usually require an external source of information. For AlphaGo, it is the simple algorithm which decide who is the winner of a game of Go. For GAN, it is the images labled by human. In these scenarios, the critic is the medium which transforms external information into gradient which optimized the actor, but not the direct source of that information.