One could say whatever one wants but I think that in this case it’s actually a conjunction and I doubt anybody has ever said “I per se I” (Edited to add: actually that makes sense - for the letter i - but in a different context: spelling and not alphabet recitation. “c, a, r : car”, “i : I”. _letter_ (per se) : _word_)

https://www.englishclub.com/efl/podcasts/interesting-facts/a...

At the end of the alphabet, students would say “X,Y,Z and, per se, & (and)”.

Re "I doubt anybody has ever said I per se I" (in the context of alphabet recitation) may I draw your attention to this article[0]:

> Up until as recently as the mid 1900s, it was standard practice when reciting the alphabet to use the Latin phrase per se (literally “by itself”) to differentiate between individual letters of the alphabet—like A, I, and O—and single-character homographic words—like a, I and O.

> So the letter A would be read as “A per se A”, to ensure it was distinguished from the indefinite article a. The letter I, similarly, would be “I per se I” to differentiate it from the pronoun I. And the letter O would be “O per se O” to differentiate it from the interjection O!

[0] http://haggardhawksblog.blogspot.com/2015/03/ampersand.html

Thank for the link but that seems misleading or just wrong. “A per se A” means

    “a” (per se = by itself = as a letter) [spells the word] “a”
    

Nobody would recite the alphabet as

    a per se a, bee, …, aitch, i per se i, jay, …

and ending with

    … zee and per se and

means

    … zee and “&”

It was not, at claimed there,

    Y, Z, and & per se &

In that case the modern name would be _andampersand_!