The main benefits are tax free gifts between partners and filing jointly, both of which seem very reasonable and wouldn't be of value to single people.
The actual tax breaks most people think about are tied to dependents in your household, not marriage.
Pretty much all of the legal benefits of marriage are contractual, not financial, and come at no cost to the public.
Things like spousal medical rights, a joint estate, etc don't come at the expense of anybody else.