You can only "pause" your watch history, as they call it, for some reason. And I keep having mine become unpaused constantly which I find interesting. I'm not sure if it's one of my clients doing it automatically or if Google just reenables the setting from time to time.
My watch history has been off for almost a decade now.
They may weasel you into activating it via some other route that states in the fine print that they need to activate your watch history to provide [random almost unrelated feature].