So to design a better controlled experiment, the OP should write a statistically significant number of emails (>100...maybe 1000+) without sending them. An app should perform a double-blind study by randomly choosing half those emails to go to multiple recipients (all but one being fake) and choosing the other half to go to one recipient only.
"without sending them" is unnecessary. Modify your email client to randomly choose (AFTER you write your email) whether or not to CC Alex. Or just flip a coin after writing it and before hitting send.
How about the idea of writing 100+ e-mails without sending them?
You'd be better off having a mail plug-in randomly selected whether to add a cc at the time of sending rather than hoarding them.
Your mechanism is valid as a trial but not particularly workable in the real world.
You might want to also control for factors such whether the mail is internal or external (assuming it's a company), message length (longer messages might be more or less likely to be properly read and responded to), message importance (you'd expect more replies to higher importance messages), attachments and so on.
Anyone see any holes in this approach?