It may be better to pigeonhole people depending on which sex chromosomes they have (and how many of each), rather than what reproductive organs they have.

This helps cater for aneuploid people, as well as people who have underwent gender changes.