This doesn't have to do with the ATV4, but the 3 actually, and actually the physical button press on the remote. May not solve your issue, but good to know if you do some Pro Control.

Bottom line, make sure when a button is pressed, it is firm, using the whole finger print area of the thumb. What I noticed with a client a week or two back was I could press any button on a PC remote for an ATV 3 and things worked fine. When the lady of the house pressed the button, certain functions would not work. We went back and forth between our hands a few times. Finally she said let me watch you do it. What she was doing was only using a fingertip to click the edge of the button, similar to what one might do on a smartphone. She adjusted her button press and it was all good. I had not seen that one on the ~50 systems I have out in the wild...