Watoosi,
I do both IT and AV in my business and would never give up either customer base.
Are you burned out on the AV side?
This may not be the norm but my AV jobs are way easier and more profitable than the IT side.
Sounds like your margins may be too low either on the pricing side or the cost of labor of your contractors.

Where are you located?