There seems to be a guide for openssl here [0] but it seems kinda complicated. This discussion inspired me to add name constraints support to rcgen [2]. If you aren't afraid to write Rust, you should give using it a try.

[0] https://www.marcanoonline.com/post/2016/09/restrict-certific...

[1] https://tools.ietf.org/html/rfc5280#page-41

[2] https://github.com/est31/rcgen/commit/059cc19fcd1b8bb57feed5...