I do. Not because it makes me feel special to have a lot of it (I don't have but a handful of it) but because it serves as a measure of how you're contributing to this community.
A downvote should be a warning that you've wasted this community's time.
An upvote tells you your contribution was appreciated by at least some members and that you've contributed something useful.
Wanton upvotes (aka "karma whoring") simply tell you that you've figured out how to game the system.
Sure, but on most sites that have a "karma" type of system, it doesn't serve as that great of a measure of how you're contributing to the community - it serves as a measure of how much the community agrees with your opinions.
I mean if everybody only used the (up|down)vote arrows as a measure of how much something was worthy of submission, or how much a comment was contributing to discussion it would be great, but you can't rely on people to do that (and they often don't), so the meaning of karma approaches nil.
HN seems to be a bit better about this than most sites with such a system (reddit and slashdot particularly), but I suspect that's because its user base is (or was, I'm not too sure recently) pretty small.
What's the point of karma? It's not money, it's not friendship, and it's not interesting; so who cares?