Yeah but all the elemental matter in the universe is like three dust specks floating in a lecture hall. You can ask about the dust specks or the lecture hall interchangeably.
Great point. I almost missed this. It really seems like there should be another separation in the article I linked, between "Universe" and "Milky Way."
TL;DR: Stars burn it up.