I've never really done that a lot over a period of time to any piece of equipment, but I don't think you'd have a problem. Also, you should only have to turn the tv off then on, not necessarily unplug it.

However, the bigger question is what's causing the color shift? You're right, it is from a magnetic influence. Turning the TV off then on will "degaus" (sp?) it to get rid of the funking colors.

When you swivel the TV, does the side with the bad color end up near a speaker? A strong magnet? I'd try to remove the offending item instead of constantly turning the TV off and on.