Also worth mentioning Murphy's WIP textbook[0] focused entirely on RL, which is an outgrowth of his excellent ML textbooks.

[0]: https://arxiv.org/abs/2412.05265