You don't have to remember something perpetually for it to be worthwhile. I've got a young dog, and I've spent 1-2 hours (shared with partner) every single day for the last 18 months outside walking. I don't remember all of those walks, even some of the ones that were very enjoyable at the time. But I enjoyed them at the time, (or maybe I didn't, it was pouring rain etc).

Video games are an easy target, but you can make the same argument about any lesiure activity. Something doesn't have to have an output to be worth spending time on.