Is that "math" though, or just a particular construct in the system of math? I could imagine an alien mind that does not break things down across integer boundaries, when you consider that any labelings and groupings are arbitrary. That 1 "hat" you are wearing plus that 1 "hat" your buddy is wearing are actually not 1 and 1 but a conglomeration of fields/particles in an arbitrary configuration that can be torn or shredded or turned into plasma.
You could construct such a mind in a simulation but I doubt one would arise naturally in the physical constraints of our universe. Approximate discretization is everywhere.
At the end of the day, you've still identified that particular configuration as a "hat"- a configuration forming a discrete unit. Without it, you have no concept of object permanence and no basis for comprehending any form of logic at all. After all, even within plasma, there are discrete units which are configurations of other discrete units.
But these discrete units in the plasma are also model constructs of ours rather than fundamental properties of nature. The point is that any discrete lines we draw are ours, and not necessarily nature's. Of course we have no concept of object permanence without these, because concepts and objects are discrete models we create ourselves. It's tautological.