Could be interesting, but if the app operates in the background, how would you prevent actions from happening?
In some households lights, shades and TV would all be going on and off while dad is just talking to you (waving arms frantically while trying to get a point across)
I'm putting a sweater on, lights go to half while I push my arm through the sleeve, do they go off when I put my arm down?
My wife put her arms up to get into her dress and the shades went up.
And if the app does not work in the background, then you have to access it every time you want to do something simple.
At least with voice, a simple "hey google" or "Alexa" then the command would just become part of the routine.
Man that looks annoying.
If we are talking watch what you say might be an issue. But if you use the phone. Hopefully you are smart enough to leave it down for a second while waving your arms