This used to be possible with a feature in ETV called HLS Hybrid, though it has been removed. There have been requests to bring something along those lines back for audio codecs, as you mention, and for loudness normalization. I'm definitely open to making it possible again.