Confusingly this does exist but is super weird to get to.

Select some photos and then select merge. The pop up that follows will let you merge only the exact duplicates.

So you can first merge all your exact duplicates, and then when that is done manually review the visually similar matches.