Different compression codecs.
A streaming service can use H.265 coming in. Then their box has to convert it to the lossless HDMI for output, which uses a lot more bandwidth for the same video signal.

This is how some of that discrepancy happens.

Service providers can bring out new boxes every year (if need be), with the latest high efficiency codecs.
A connection standard like HDMI however, takes years to standardize. And years for each revision to update. Then they have to be backward compatible with the older connections, which many times cripples the ability to use new high efficiency codecs. So as we push more & more down that HDMI pipe, the bandwidth needed on the cable, goes up exponentially. Because they are locked into an old codec from decades ago now.

If we suddenly switched to a super high efficient codec for HDMI, the bandwidth requirements would be a fraction of what it is now - but then no older devices would/could work with the new HDMI connection standard.

[edit]:
To answer the other part of your question, it still takes a LOT of bandwidth even with higher codecs. So most any streaming service is going to have compression and loss of picture quality.
The only way to get "the best" picture quality right now, is either thru a UHD bluray disc using a UHD bluray player.
Or download the content, then play it from a drive. This is the case with video games and game consoles. Or movies/shows with some movie server devices.