From what I have read/watched, Rogan didn't refuse to interview Harris and offered to do the same multi-hour interview he does with every guest.
Harris just wanted him to fly to another city and do a 1-hour interview in their studio. To make an exception for a single guest seems unfair and I don't blame Rogan for not agreeing.
