One of Matt Levine's common refrains is "Insider trading is not about fairness, it is about theft." There are always market participants with different levels of information. The problem occurs when you misappropriate confidential information you were entrusted with for personal gain.
I respectfully disagree with Matt Levine's framing. Yes, confidential information belongs, in a statutory sense, to the company, and using it for personal gain instead of company gain is a kind of theft, similar to an executive who uses their company car for personal trips, or any employee who might browse Facebook on a work laptop or take a personal call in a company office. But this argument is akin to saying that no senior executives should ever incur personal criminal or civil liability for the actions that they take in the name of the company, because it is the "company" that did it and not the individual. Most people (particularly after public implosions like 2008) find this argument reprehensible. In fact of the matter there are individual executives who do act in their personal interest. Should stock buybacks be illegal because it happens to be very much in the personal interest of the executive whose bonus depends on a rising stock price, while shareholders may have benefitted more from the executive focusing on day-to-day operations, thus the executive "robbed" shareholders because of making a decision to focus on the wrong thing? It's not reasonable to expect executives to "wear different hats" where one day they act out of their personal interest and the next they are some kind of selfless worker who works for The Company and whose only interests are the Board's and shareholders'. Executives are holistic individuals, they make decisions as holistic individuals, they were hired with the expectation that they would make decisions as holistic individuals, and in and of itself there is nothing wrong with that.
Rather, the theft is from the market participant who is getting the raw end of the deal. Legal trades require consent. If you sell someone a hard disk that you know crashed and is unusable, under the guise that it is working, you committed fraud. If you sell someone a new car that it turns out is going to break down almost immediately, you are subject to lemon laws and must refund or replace the car. If you had sex with someone in a case where consent was not in place, including in cases where the sex was pleasurable but the other person was defrauded to who you were (i.e. not their long-time partner), that is rape. And if you take stock that you're pretty confident is going to lose 80% of its value in three days when poor earnings will be announced, and offload it to an unsuspecting victim, no I don't think you can make the case that there is genuine consent here.