The Harmony is an activity based remote. You should use the Device mode only for debugging & testing commands. If you find yourself going into device mode too often, then it's time to map that command into the desired activity(ies). In the activity customize buttons, you can map any command from any participating device to any button.

The Off button (that looks like a power button) is the global built-in Off activity. The built-in Off activity is like an activity without any device. If you need to turn off a device in device mode, you have to use its PowerOn/Off or PowerToggle command. Note: You can also map a device power command into its activity(ies).