If you're claiming a transformer model is a Markov chain, this is easily disprovable by, eg, asking the model why it isn't a Markov chain!

But here is a really big one of those if you want it: https://arxiv.org/abs/2401.17377