I've read a bit that some people got some bad remotes that require some buttons to be jammed in pretty hard to work. It's possible that it's a bad remote. Personally, I just press it no harder than any other button and it responds fine. You might want to send Logitech Support an e-mail or call them and tell them about this and see what they'll suggest.

They'll probably send you a replacement...