I feel there may still be a way solve it using existing hardware -- rather than go build entirely new hardware for it.

Some low effort ideas I cna think of -- a wake word instead of button press; a flick of the wrist; or maybe press the watch to your chin; There must a few more elegant solutions possible if smarter people put their minds to it.