Eh, anecdotally I spent about three years thinking the same and figured I'd use my spare time to become a writer. But you can't really take out the time you put in your day career (8x5) out of your private life unless you don't have much going on in your private life.
If I'd gunned in my 40hrs and spent my private time chilling instead of slacking in my 40 and then trying to squeeze out 10-15/wk to focus on writing, I'd have been better off in retrospect.
This is highly personal obv, YMMV. But I think your day job should be the one you want to compound your skills in.
I spent my 20s with your attitude and now regret it as I see my peers at a level or two above me.