Not a good idea. There are actually several firmware fixes that NEED to be applied to TVs. This has ranged from power issues, input issues, and many others. The latest Samsung firmware had a fix for HDR content displaying 'more' correctly (or better, depending on how you look at it).

Do all the firmware updates at installation. Then disconnect devices from the network, that are not going to be using it. In the case of needing IP control, you can block those devices from the WAN, still allowing local IP control on the LAN.
Every service call, check for firmware updates on all equipment. This just should be standard practice these days. Devices these days need to have their firmware or you run the risk of running into more problems than not. Whether you are aware of the issues or not.