The pointers that the programmer controls live on the shadow stack and that's how they get updated. Another term for this is a "GC handle"