With the right of free speech comes the responsibility to extend that speech to others. Because if we leave who is heard to simply "who can speak the loudest", we won't find out when we're wrong.

See also: Mill's "On Liberty", available now in every Intro to Philosophy textbook.