I think there are better ways to make the world a better place than wanting someone you don't like to fail. Besides, there's often an ulterior motive - not just for wanting that person to fail, but also for telling yourself that they are "bad for society".

Where one cannot love, one should – pass by. - Nietzsche