Yes, it irks me too. I consider myself a "hack" when it comes to programming. I tinker, I play, I build, but I have mostly surface knowledge. "Hacker" is a term that should be reserved for real programmers who actually know what they are doing and dig deep into vast reserves of knowledge to get around barriers that us mere hacks would come to a halting screech at.
Well, IMHO being "tinkerer" is an important trait for a "hacker" :).
I was thinking about the hacker == kids who DDoS mastercard or steal infos from the PSN. Some of them might be hackers, but that's not what a hacker is. Bruteforcing PSN accounts using a stolen DB is not really the mark of a hacker's job for me.