I'd argue that is mostly because 1) people follow audiobook or TV series pronunciations and 2) most discussions happen online and not in verbal form.
This is definitely a problem when it surfaces. For example the Stormlight Archive [1] series has two voice actors narrating the audiobook, and they don't even agree between them how to pronounce half the made up names.
As someone who has listened to The Stormlight Archive (and The Wheel of Time with the same two narrators), the differences are absolutely there, but they're relatively small.
Fantasy novels predate the widespread popularity of audiobooks. It used to be quite expensive to distribute a large enough volume of audio. The old "books on tape" cost a lot of money, were frequently abridged, and only existed for the most popular titles.
This is definitely a problem when it surfaces. For example the Stormlight Archive [1] series has two voice actors narrating the audiobook, and they don't even agree between them how to pronounce half the made up names.
[1] https://en.wikipedia.org/wiki/The_Stormlight_Archive