Honestly, as an American (California n almost all of my life), I find both “by accident” and “on accident” to generally sound stilted, though “by accident” seems natural in certain broader constructions, and “on accident” fits in the unique case of a direct contrast with “on purpose”. Generally, though, “accidentally” is the term that sounds right.

In America it's a regional thing. "By accident" is east coast, "on accident" is more midwest and west coast (though given that the west coast is full of transplants, there's a mix in usage". I've lived in CA for 16 years, but grew up in NJ and MD, and "on accident" still sounds weird to me.

I'm from CA and I hate "on accident", it sounds dumb and wrong.

You should eschew both and say 'accidentally'.

I do hear 'by accident' but always (as a native speaker of British English, born and living in England) consider it an error.

I have at least heard of 'on accident' too, but interestingly it barely registers on Google's ngram viewer, even with 'American English' selected:

https://books.google.com/ngrams/graph?content=accidentally%2...

I'm being prescriptivist here, but to my (West coast American) ear "on accident" is wrong, caused by confusing "on purpose" and "by accident."

To my ear "on accident" sounds wrong and kind of unsophisticated for lack of a better word

Huh, I speak AmE and "by accident" sounds more natural to me too.