I'm not sure that useMemo really is an effect in the sense that useEffect is. The useMemo callback is invoked synchronously during render, rather than getting pushed onto an effect queue like useEffect does.
Even useLayoutEffect goes on an effect queue, and if it results in changes, will need to render a second time, while a changed memo calculation happens entirely during the single render pass.