Re: [Flac-dev] floating point
Another point for floating point support: see how WinZip has started using WavPack, I think that was a smart move. It was already the case with RAR & its multimedia compression, I think modern packers need to be multimedia aware. This is very risky, though, and of course requires total lossless compression (assuming it won't try to encode wavs that use codecs). At the same time it's already too late for zip, it's already used too much to touch its format. But for installers (that come with their own decompressor anyway), it can be very interesting. Well, if our installer (nsis for now) had built-in multimedia compression, we wouldn't even need to bother. ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] floating point
Note: the reason 32bit float was a requirement for me was for use in FL Studio. I'd be the first to say that no one needs to store audio in more than 16bit, but if the user has 24 or 32bit audio (32bit float would be to get unlimited headroom, while with 24 you'd have to set it up yourself or normalize), I wanna compress that transparently. My 2 goals were -to compress our demo songs for our online installer (even though we also use ogg where it's ok, and of course we don't go above 16bit here) -to let the user compress all of the audio files his project uses, to share online (even if he recorded 12bit worth of noisy audio in a 32bit float wav, it's still his problem and I wanna compress this losslessly) ..so both are related to internet sharing. We also got users asking for FLAC support to spare space on their HD, but IMHO that's not a valid reason as these days $100 buy a terabyte. The loading times for long tracks is certainly not neglectable. I've finished implementing WavPack support, we will add FLAC as read-only later I think. I agree that FLAC or WavPack wouldn't be nice for audio track streaming, but that's mainly because there's no CPU to waste unpacking several compressed streams, if room on the HD isn't a problem anymore. That reminds me of a similar lossless format that was optimized for realtime decoding, for use in samplers. Why not, afterall it's soundbanks that are the most RAM eaters in a plugin host. But with compression around 50%, it doesn't help that much, you would still eventually need disk streaming. However, to -download- such big soundbanks from the internet, it's perfect. Side note: so far we had been using the Vorbis ACM codec (made by Freddie Fish) to compress our demo songs. It worked, aside from several bugs. So we recently got a developper "fixing" and enhancing it. My main problem was: a sample designed to be loop-sliced or sampler-looped may not survive lossy compression. So the idea was: I tell the codec where my "special" points (markers & loop points) are, and it preserves the original audio around it. That worked, but for drumloops that contain lots of such markers, the resulting audio file grew enough and it started to be less interesting (still better than lossless compression, but not THAT much better anymore). I still believe that such a dream (for us) codec can be made, using FLAC (or WavPack) to store the original bits of audio. But now I'm realizing the other problems -apparently the Windows audio compression manager doesn't "thunk" 32bit codec, thus it's not very future-proof to rely on a 32bit codec while things are changing -apparently the ACM doesn't seem to make it possible to pass floating point audio to a codec (because it doesn't pass WaveFormatExtensible, but I may be wrong on this one) -not many audio editors use the ACM perfectly, so not many can read such compressed wavs, defeating the point of storing them in a popular format -of course it's not portable, as it requires the codec on the target platform So now I think we're gonna stop using the codec & focus on OGG/WavPack formats. But the drawback is that the format/extension is different, meaning that if a project uses WAV files, it needs to be adapted to use the other format, which is not always possible. While the beauty of using the ACM codec was that it didn't change anything to the file's header. I think the best would have been Microsoft enhancing their ACM instead of letting it die. Because what's a better standard than WAV to store audio even today? > On Aug 14, 2009, at 14:24, Paul Davis wrote: >> On Fri, Aug 14, 2009 at 5:05 PM, Josh Coalson wrote: >>> it's unlikely flac will ever support floating-point samples >>> natively. the main application for it is audio engineering, which >>> demands easy editing and very high speed for both encoding and >>> decoding above everything else. >> >> thats not why floating point is used. >> >> the highest current feasible bit resolution for digital audio samples >> is 24 bits. most converters don't even fully provide that. 32 bit >> floating bit allows bit-for-bit storage of all 24 bits of the original >> sample as the mantissa, along with more bits available in the >> exponent. this provides astronomical "headroom" for use when summing >> multiple samples. if you only use 24 bits and add two values very >> close to the maximum 24 bit value, you will overflow. even if you use >> 32 bits, its not imposible to construct workflow scenarios where you >> would overflow. if you do this in float format, you lose a little >> precision in the answer, but you cannot overflow. >> >> this is why floating point is used. > > You have successfully described why the music production industry has > settled on float for processing, but this has nothing to do with > storage formats for analog-to-digital audio recording and playback. > Programs like Logic Pro will transparently convert from 24-bit source > files to 32-bit
Re: [Flac-dev] floating point
On Aug 14, 2009, at 17:14, Paul Davis wrote: > You're assuming that you never store intermediate mixes that may > actually have overflowed a 24 or 32 bit representation. I would agree > that this might not be best practice, but its actually not as > un-useful as it might at first appear. I covered intermediate mixes in the second half of my response. But few consumers of audio are actually interested in production, so they don't need intermediate formats. > i'm not an advocate for Ardour using FLAC as a native format. i just > don't like telling users "it can't be done". Ardour can certainly support FLAC for Bounces or any other final output. It wouldn't even be hard to support FLAC as well as Ableton Live does, and perhaps even go beyond what Ableton supports. Saying "it can't be done" is a gross oversimplification. A more accurate statement would be "FLAC is not the ideal format for all possible files that Ardour might work with, but FLAC is certainly a perfect choice for most input and output files." The typical DAW allows a Bounce in 24-bit or even 16-bit, and it is assumed that the user understands the consequences of choosing a narrow bit depth like 16-bit. Ardour would not even need to guide the user to avoid FLAC for intermediate storage, at least not any place where 24-bit and 16-bit are other format options. It would be sufficient to have a chapter in the manual discuss the benefits of using higher bit depths for intermediate mixes. I haven't used Ardour, but if certain functions like Track Freeze are available, then Ardour could limit the options to 32-bit-clean formats. If the GUI makes bouncing of stems obviously distinct from a final bounce, then Ardour could again guide the user towards 32-bit for intermediate mixes while allowing the choice of FLAC for final delivery formats. Brian Willoughby Sound Consulting ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] floating point
On Fri, Aug 14, 2009 at 6:47 PM, Brian Willoughby wrote: > When the ADC source and the DAC destination are both limited to 24- > bit fixed point integers, it makes absolutely no sense to store > recordings or final mixes in 32-bit floating point representation. > The headroom you speak of is completely unavailable when storing the > output of an ADC into a file. Likewise, headroom is wasted when > playing back a fully mastered piece of audio to a 24-bit DAC. > Headroom is only an issue when you want to work on the audio and > change it, which is something that 99% of audio consumers do not > bother with or even understand. You're assuming that you never store intermediate mixes that may actually have overflowed a 24 or 32 bit representation. I would agree that this might not be best practice, but its actually not as un-useful as it might at first appear. > inappropriate. It would be quite interesting if someone were to > create a lossless format which can handle 32-bit float, but I don't > believe it has been done yet. wavepak (pack?) does. In most respects, this would mostly be > a tradeoff between storage space and processing power, since > processing a compressed file is usually too expensive for most DAW > software, and so they always uncompress source files before using > their data. Thus, even a space-saving format like FLAC would be > pointless for Ardour, since you'd still need to take up disk space > for an uncompressed copy of the data (like Ableton Live, which > supports FLAC). its always a tradeoff between CPU cycles and disk space. you don't need an uncompressed copy if you're prepared to burn the CPU cycles. i'm not an advocate for Ardour using FLAC as a native format. i just don't like telling users "it can't be done". ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] floating point
On Aug 14, 2009, at 14:24, Paul Davis wrote: > On Fri, Aug 14, 2009 at 5:05 PM, Josh Coalson wrote: >> it's unlikely flac will ever support floating-point samples >> natively. the main application for it is audio engineering, which >> demands easy editing and very high speed for both encoding and >> decoding above everything else. > > thats not why floating point is used. > > the highest current feasible bit resolution for digital audio samples > is 24 bits. most converters don't even fully provide that. 32 bit > floating bit allows bit-for-bit storage of all 24 bits of the original > sample as the mantissa, along with more bits available in the > exponent. this provides astronomical "headroom" for use when summing > multiple samples. if you only use 24 bits and add two values very > close to the maximum 24 bit value, you will overflow. even if you use > 32 bits, its not imposible to construct workflow scenarios where you > would overflow. if you do this in float format, you lose a little > precision in the answer, but you cannot overflow. > > this is why floating point is used. You have successfully described why the music production industry has settled on float for processing, but this has nothing to do with storage formats for analog-to-digital audio recording and playback. Programs like Logic Pro will transparently convert from 24-bit source files to 32-bit float internally, to avoid the headroom issues you speak of. There is no requirement that the files be stored as 32-bit float because you can still do all processing in 32-bit float even with 24-bit file sources. When the ADC source and the DAC destination are both limited to 24- bit fixed point integers, it makes absolutely no sense to store recordings or final mixes in 32-bit floating point representation. The headroom you speak of is completely unavailable when storing the output of an ADC into a file. Likewise, headroom is wasted when playing back a fully mastered piece of audio to a 24-bit DAC. Headroom is only an issue when you want to work on the audio and change it, which is something that 99% of audio consumers do not bother with or even understand. >> flac is designed as a consumer audio format. it trades ease of >> editing for a featureful, robust transport layer more suited for >> playback, and encoding speed for more compression and faster >> decompression. > > flac seems more popular at present among high end audiophiles than > mere consumers. its very regrettable that it doesn't support floating > point natively. many of our users (http://ardour.org/) have asked > about using FLAC as an option for recording format, but we have to > explain that its not viable because of the lack of floating point > support. and yes, that is audio engineering :) Audiophiles are still consumers, in that they do not produce music, they merely consume it. You are correct that audiophiles tend to live in the gray area between typical consumers and professionals, but they still don't need 32-bit float. Users of Ardour are beyond audiophiles, because they're actually producing music. There would be two different sets of needs for storage formats. For final mixes and delivery of finished audio, 24- bit file formats like FLAC are perfect. These files are in a format ready to send to a 24-bit DAC for listening. Any headroom beyond 24- bit is pointless for a delivery format. For one thing, a 32-bit file, compressed or not, would need to be dithered before playing on a 24-bit DAC. Ardour users would have a different set of needs for intermediate mix files such as stems and "frozen" tracks. These would indeed be best as 32-bit float, and an optimized format like FLAC would be inappropriate. It would be quite interesting if someone were to create a lossless format which can handle 32-bit float, but I don't believe it has been done yet. In most respects, this would mostly be a tradeoff between storage space and processing power, since processing a compressed file is usually too expensive for most DAW software, and so they always uncompress source files before using their data. Thus, even a space-saving format like FLAC would be pointless for Ardour, since you'd still need to take up disk space for an uncompressed copy of the data (like Ableton Live, which supports FLAC). About the only positive use for lossless compression of 32-bit float would be sending stems from digital mixing houses to digital mastering houses, while saving on disk space. Generally, saving space is not critical at that professional level, since there's usually enough of a budget to cover the cost of increased storage space. I suggest that you tell your users to select FLAC only for final mix bounces, and direct them to another format for intermediate storage of audio which will be processed further. Brian Willoughby Sound Consulting
Re: [Flac-dev] floating point
--- On Fri, 8/14/09, Paul Davis wrote: > On Fri, Aug 14, 2009 at 5:05 PM, Josh Coalson > wrote: > > it's unlikely flac will ever support floating-point > > samples natively. the main application for it is audio > > engineering, which demands easy editing and very high speed > > for both encoding and decoding above everything else. > > thats not why floating point is used. I'm not saying that, I'm saying float is mainly used in audio engineering which has different requirements and tradeoffs > > flac is designed as a consumer audio format. it > > trades ease of editing for a featureful, robust transport > > layer more suited for playback, and encoding speed for more > > compression and faster decompression. > > flac seems more popular at present among high end audiophiles than > mere consumers. its very regrettable that it doesn't support floating > point natively. many of our users (http://ardour.org/) have asked > about using FLAC as an option for recording format, but we have to > explain that its not viable because of the lack of floating point > support. and yes, that is audio engineering :) for export it might be useful, but for recording and editing, a simple, fast codec and easily editable stream format is much more suited. you cannot easily snip an arbitrary section out of a flac or wavpack stream. a very simple float-only codec could be very fast and probably still get compression within 10% of say wavpack. if by audiophiles you are talking about people listening to the end product, properly mastered 16 bit material is mostly indistinguishable from even 24 bit. 32 bit int is beyond overkill. 32 bit float is not even necessary because the dynamic range is fixed in the end result. ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] floating point
On Fri, Aug 14, 2009 at 5:05 PM, Josh Coalson wrote: > it's unlikely flac will ever support floating-point samples natively. the > main application for it is audio engineering, which demands easy editing and > very high speed for both encoding and decoding above everything else. thats not why floating point is used. the highest current feasible bit resolution for digital audio samples is 24 bits. most converters don't even fully provide that. 32 bit floating bit allows bit-for-bit storage of all 24 bits of the original sample as the mantissa, along with more bits available in the exponent. this provides astronomical "headroom" for use when summing multiple samples. if you only use 24 bits and add two values very close to the maximum 24 bit value, you will overflow. even if you use 32 bits, its not imposible to construct workflow scenarios where you would overflow. if you do this in float format, you lose a little precision in the answer, but you cannot overflow. this is why floating point is used. > flac is designed as a consumer audio format. it trades ease of editing for a > featureful, robust transport layer more suited for playback, and encoding > speed for more compression and faster decompression. flac seems more popular at present among high end audiophiles than mere consumers. its very regrettable that it doesn't support floating point natively. many of our users (http://ardour.org/) have asked about using FLAC as an option for recording format, but we have to explain that its not viable because of the lack of floating point support. and yes, that is audio engineering :) --p ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] floating point
--- On Fri, 8/7/09, Didier Dambrin wrote: > Hi, > > I've tried to find info about unofficial 32bit > float support in FLAC, and found several conversations. > > Most of them were talking about a 24bit limit, > but from the manual I guess that this limitation is gone, > as it supports up to 32bit integer. > > So my question is, what would be the best way, or > what is a common way to FLAC-encode floating point > audio? it's unlikely flac will ever support floating-point samples natively. the main application for it is audio engineering, which demands easy editing and very high speed for both encoding and decoding above everything else. flac is designed as a consumer audio format. it trades ease of editing for a featureful, robust transport layer more suited for playback, and encoding speed for more compression and faster decompression. ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] floating point
Brian Willoughby wrote: > Your statement is misleading as written. It should say > >> Try using WavPack, http://www.wavpack.com/ >> >> This can losslessly compress 32-bit floating >> point WAVE-EX files (which FLAC does not do), and faithfully preserves >> every chunk (which FLAC does do perfectly). It is also free. Brian Willoughby wrote: > You are correct, Didier. FLAC preserve every chunk, precisely. ... > I think that > Martin is speaking from out of date experience. FLAC 1.2.1 preserves every chunk perfectly/precisely, except for the 'fmt ' chunk. This is from an e-mail exchange with Josh in September 2007. This doesn't matter to most people but, unfortunately, messes up the ".amb" format used for Ambisonic B-Format. The ".amb" format is a standard WAVE-EX file except that it uses a non-standard GUID in the SubFormat field of the 'fmt ' chunk. (Two GUIDs have been registered for this use, for integer and floating-point.) WavPack preserves the GUID perfectly. Regards, Martin -- Martin J Leese E-mail: martin.leese stanfordalumni.org Web: http://members.tripod.com/martin_leese/ ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] floating point
On Aug 7, 2009, at 21:48, Didier Dambrin wrote: > FLAC doesn't preserve every chunk? I thought it did. I only gave a > quick try > but it seemed to have preserved even the most obscure chunks. > Let me check: it even seems to preserve "MIDI note associated to > marker", > which is a very unknown metadata used by SoundForge (& even defined > in a > buggy way), so I assumed it was saving them transparently. You are correct, Didier. FLAC preserve every chunk, precisely. WAV and AIFF define a chunk in a very generic fashion, such that any chunk can be preserved regardless of its contents. FLAC does not interpret any chunk except the one holding the audio data. The optional chunk preserving code does not treat any chunk differently, thus it cannot preserve some chunks and not others. I think that Martin is speaking from out of date experience. > Btw, what do you think of this? > http://www.hydrogenaudio.org/forums/index.php? > s=95a0210a0ba3304eca44ac3bd57990cb&showtopic=73895 > (didn't know where to post this, that forum seemed related) That article is very naive, or at least the way it is described is very naive. Real music does not repeat in terms of whole frames. Frames are a completely artificial creation of the digital world, and frame timing does not correspond to the timing of music repetitions in music. Because music represents an analog signal, the repetition could occur at a fraction of a frame, or even a fraction of a sample. Compressing a drum loop would require a lot of tricks to detect the repetition unless the frame size were somehow luckily aligned with the tempo. Maybe a song with 70.3125 BPM or 140.625 BPM could be compressed this way, but most music will not have such a precise tempo - in fact, tempo may drift if a live band is recorded. > So I thought: imagine a pre-processing coupled with FLAC. It would > take > frames out of the whole song, and try to cross-correlate them with > the song > itself. When it finds strong matches (under a certain threshold, and > starting with a couple of matches), the frame is saved to a pool, > and it's > subtracted from the song. > Then you FLAC the (small) pool, and the song, full of near-silent > spots (& > silence where pure repetitions occured). > At decode time, you unFLAC the pool and the song, and you add back the > frames from the pool to the song. This might work, but you would have to be very lucky to find matches given the block size of FLAC (or the frame size of any format, for that matter). But, you're right, if you can predict the waveform with reasonable accuracy, then you can reduce the size. FLAC and many other compression algorithms do, in fact, use this technique. They look at the music, predict future samples, and then encode the difference between the predicted value and the actual value. It's doubtful that you could find a better algorithm at predicting the waveform, but if you do, then FLAC will work well with your added processing layer. > I haven't experimented yet, but let's say I try to correlate frames > with the > song, and I get something like 20 near-repeats, I may end up with a > very > silent "song leftover", still as long as the song, but maybe in > 4bits worth > or something? But it would also have bumps of original audio (that > didn't > find any matching frame). > The thing is, I don't really know how FLAC compresses so I don't > know if it > would compress the "leftover" so much better. It's doubtful that you could find such repetition, given that the frame size has nothing to do with the tempo of the song, and repetition in music are based on tempo. But, if you could find a match or even a near match, then FLAC would compress the difference better than the original. > And I don't really know how much matching frames you'd find out in > music out > there, it would be very genre-dependent. But I'm surprised that no one > really investigated this (there were old discussions in that > forum). Sure, > streaming is important, but it's common to fully download a song. The repetition would not be genre-dependent, but would be tempo- dependent. I suppose you could say that certain genres might have a prevalent tempo, but there is enough variation within each genre to make the problem as big as non-genre-dependent matching. People have investigated this, but perhaps not at the macro level as is being discussed here. I think you'll discover that finding a match within a song is very difficult. You could perhaps start with BPM detection code, and then try to find repetitions based upon tempo, but even if you find matches this way, you still would need to find some way to squeeze the repetitions into whole frames, which are not divisions of tempo. Feel free to experiment. The FLAC library makes it possible for you to work at the high level without writing everything yourself. > At the same time this
Re: [Flac-dev] floating point
On Aug 7, 2009, at 20:58, Martin Leese wrote: > "Didier Dambrin" wrote: > ... >> I like FLAC on the paper because of its metadata preservation, in >> that riff >> tag, which is critical for my needs. > > Try using WavPack, http://www.wavpack.com/ > > This can losslessly compress 32-bit floating > point WAVE-EX files, and faithfully preserves > every chunk (which FLAC does not do). It is > also free. Your statement is misleading as written. It should say > Try using WavPack, http://www.wavpack.com/ > > This can losslessly compress 32-bit floating > point WAVE-EX files (which FLAC does not do), and faithfully preserves > every chunk (which FLAC does do perfectly). It is also free. Brian ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] floating point
Yes, WavPack is second in my list. FLAC doesn't preserve every chunk? I thought it did. I only gave a quick try but it seemed to have preserved even the most obscure chunks. Let me check: it even seems to preserve "MIDI note associated to marker", which is a very unknown metadata used by SoundForge (& even defined in a buggy way), so I assumed it was saving them transparently. Btw, what do you think of this? http://www.hydrogenaudio.org/forums/index.php?s=95a0210a0ba3304eca44ac3bd57990cb&showtopic=73895 (didn't know where to post this, that forum seemed related) Brian wrote in a previous answer: "Another consideration is that flac compresses quiet audio files more than loud audio files." So I thought: imagine a pre-processing coupled with FLAC. It would take frames out of the whole song, and try to cross-correlate them with the song itself. When it finds strong matches (under a certain threshold, and starting with a couple of matches), the frame is saved to a pool, and it's subtracted from the song. Then you FLAC the (small) pool, and the song, full of near-silent spots (& silence where pure repetitions occured). At decode time, you unFLAC the pool and the song, and you add back the frames from the pool to the song. I haven't experimented yet, but let's say I try to correlate frames with the song, and I get something like 20 near-repeats, I may end up with a very silent "song leftover", still as long as the song, but maybe in 4bits worth or something? But it would also have bumps of original audio (that didn't find any matching frame). The thing is, I don't really know how FLAC compresses so I don't know if it would compress the "leftover" so much better. And I don't really know how much matching frames you'd find out in music out there, it would be very genre-dependent. But I'm surprised that no one really investigated this (there were old discussions in that forum). Sure, streaming is important, but it's common to fully download a song. As I wrote, I tried various compressors on a drumloop repeated 4 times, and none could benefit from this. It's only a matter of statistics to know how much often repetitions happens in music out there, but it could work more intelligently, like by normalizing matching frames maybe, to even detect repetitions at different levels. Or work in the freq domain. Afterall, music is based on repetition. And repetition is compression's best friend. At the same time this wouldn't be very interesting for my need, which is to compress short samples. Now here too there could be a similar algo, if it's tonal, cross-correlation would detect matching frames, only at a smaller level. Imagine if you convert a violin sound into a pitch period somewhere in its middle, and the residual from that the subtraction of that pitch period in repeated frames. I think the residual would be rather quiet. > "Didier Dambrin" wrote: > ... >> I like FLAC on the paper because of its metadata preservation, in that >> riff >> tag, which is critical for my needs. > > Try using WavPack, http://www.wavpack.com/ > > This can losslessly compress 32-bit floating > point WAVE-EX files, and faithfully preserves > every chunk (which FLAC does not do). It is > also free. > > Regards, > Martin > -- > Martin J Leese > E-mail: martin.leese stanfordalumni.org > Web: http://members.tripod.com/martin_leese/ > ___ > Flac-dev mailing list > Flac-dev@xiph.org > http://lists.xiph.org/mailman/listinfo/flac-dev ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] floating point
"Didier Dambrin" wrote: ... > I like FLAC on the paper because of its metadata preservation, in that riff > tag, which is critical for my needs. Try using WavPack, http://www.wavpack.com/ This can losslessly compress 32-bit floating point WAVE-EX files, and faithfully preserves every chunk (which FLAC does not do). It is also free. Regards, Martin -- Martin J Leese E-mail: martin.leese stanfordalumni.org Web: http://members.tripod.com/martin_leese/ ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] floating point
I don't think there is a common practice for what you describe. 32-bit float only has 24 bits of accuracy at any one time. The tricky thing is that these 24 bits shift around in precision, based upon the value. Audio is normalized to +/-1, but samples below +/-0.5 would have 25 effective bits, and each halving of the sample value adds another bit. You could convert your 32-bit float audio file to 32-bit integer, but you would need to scale such that +/-1.0 takes the full range of the 32-bit integer (keeping in mind that -1.0 is valid but +1.0 is not, thanks to twos-complement coding). there would still be loss compared to the 32-bit float original, but only for samples very close to 0.0 in absolute value. One important question here is now to store a valid 32-bit integer sound file, and whether this is supported by the native flac command line. I kinda doubt that there is a standard for this. So you might need to write your own tools based on the FLAC libraries. If you were to read a valid 32-bit float audio file and convert each sample to 32-bit integer properly, then you should be able to hand off buffer of 32-bit integer samples to the FLAC library and create a valid flac file. Another consideration is that flac compresses quiet audio files more than loud audio files. By expanding the 32-bit float to 32-bit integer in the fashion I've described - which is the only way to preserve the maximum resolution - you will end up with a flac that is much larger than the equivalent 24-bit integer file. I suppose that is obvious, but I wanted to point it out anyway. It should be much better with 32-bit integer audio than it would be with pure noise, though. P.S. DTS Music Disc format put 5.1 surround into a 16-bit WAV. Normally, this would not compress very well with flac. However, the DTS guys realized that some users might play this WAV without a DTS decoder, and the white noise could destroy sound systems. So the DTS format drops two bits from every sample, making the white noise 12 dB quieter. An interesting side effect of this is that flac is able to compress this much more than regular white noise because of the 12 dB reduction. I only mention this because it is instructive about how the amplitude of the audio input affects flac's compression performance. Brian Willoughby Sound Consulting On Aug 7, 2009, at 07:42, Didier Dambrin wrote: I've tried to find info about unofficial 32bit float support in FLAC, and found several conversations. Most of them were talking about a 24bit limit, but from the manual I guess that this limitation is gone, as it supports up to 32bit integer. So my question is, what would be the best way, or what is a common way to FLAC-encode floating point audio? The first idea is obvious, we have a 32bit float, FLAC supports 32bit integer, and it's lossless. So I'd just make it encode 32bit float, it's lossless afterall. 2 problems, first I would suspect it wouldn't encode very well, since the data wouldn't be "audio" anymore, right? It would be like trying to FLAC-encode random data. So I tried FLACing white noise, and comp ratio was pretty poor, confirming this. Interestingly, it still did a better job than ZIP! The second problem would be that no other tool would read them correctly anyway. The second idea is to truncate & use it as a lossy encoder, which can be "audibly lossless" anyway. But what's a common practice here? I would be tempted to leave 1 bit of headroom above 0dB, or maybe 2 bits. Normalizing before encoding could be an option, but the gain would then have to be inserted in a tag. & finally, what bit depth? 24? 25, 26? 32? I know there's no right answer, just asking if there are common practices. ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] floating point
You may be right. I haven't managed to test yet, as right now I'm prospecting options, haven't started coding yet. I've tried FLACing a 32bit integer wav but the binary wouldn't read it. But I'm not sure if it's the right format, I actually saved it using an old CoolEdit Pro that had an option to save 32bit float in a 32bit integer format (for compatibility reasons), so it normally had the same wave format as a 32bit integer wav. And indeed, loading that sample into Audacity, so interpreted as integer, was very saturated but still a little audible, so not white noise. It's hard to decide which encoder to pick because, on one hand it's very possible to use FLAC to store this in a lossy way but with zero audible difference. And let's be honnest, even 16bit is more than adequate for this. On the other hand, it wouldn't be acceptable to do that for, say an undo system in an audio editor - the undo should restore exactly the original data. I like FLAC on the paper because of its metadata preservation, in that riff tag, which is critical for my needs. Up to now I've always tried to stay away from other file formats, to prefer codecs & keep with WAV files. But the ACM in Windows has aged, isn't much supported anymore, and from what I understand it's not very WaveFormatExtensible-ready, thus it would too have problem with floating point audio. Many choices out there. > Didier Dambrin wrote: >> Hi, >> >> I've tried to find info about unofficial 32bit float support in FLAC, and >> found several conversations. >> Most of them were talking about a 24bit limit, but from the manual I >> guess that this limitation is gone, as it supports up to 32bit integer. >> >> So my question is, what would be the best way, or what is a common way to >> FLAC-encode floating point audio? >> >> >> The first idea is obvious, we have a 32bit float, FLAC supports 32bit >> integer, and it's lossless. So I'd just make it encode 32bit float, it's >> lossless afterall. >> 2 problems, first I would suspect it wouldn't encode very well, since the >> data wouldn't be "audio" anymore, right? It would be like trying to >> FLAC-encode random data. > I suspect FLAC may perform somewhat better doing this than it would > compressing purely random data. If you plot floating point numbers > against their binary representation interpreted as a (big endian) signed > integer, I believe the resulting line will vary monotonically, apart > from a discontinuity at zero. This should mean that the data still > looks a bit like audio to FLAC, and that it will still compress > reasonably well. > > Christopher Key ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] floating point
Didier Dambrin wrote: > Hi, > > I've tried to find info about unofficial 32bit float support in FLAC, and > found several conversations. > Most of them were talking about a 24bit limit, but from the manual I guess > that this limitation is gone, as it supports up to 32bit integer. > > So my question is, what would be the best way, or what is a common way to > FLAC-encode floating point audio? > > > The first idea is obvious, we have a 32bit float, FLAC supports 32bit > integer, and it's lossless. So I'd just make it encode 32bit float, it's > lossless afterall. > 2 problems, first I would suspect it wouldn't encode very well, since the > data wouldn't be "audio" anymore, right? It would be like trying to > FLAC-encode random data. I suspect FLAC may perform somewhat better doing this than it would compressing purely random data. If you plot floating point numbers against their binary representation interpreted as a (big endian) signed integer, I believe the resulting line will vary monotonically, apart from a discontinuity at zero. This should mean that the data still looks a bit like audio to FLAC, and that it will still compress reasonably well. Christopher Key ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
[Flac-dev] floating point
Hi, I've tried to find info about unofficial 32bit float support in FLAC, and found several conversations. Most of them were talking about a 24bit limit, but from the manual I guess that this limitation is gone, as it supports up to 32bit integer. So my question is, what would be the best way, or what is a common way to FLAC-encode floating point audio? The first idea is obvious, we have a 32bit float, FLAC supports 32bit integer, and it's lossless. So I'd just make it encode 32bit float, it's lossless afterall. 2 problems, first I would suspect it wouldn't encode very well, since the data wouldn't be "audio" anymore, right? It would be like trying to FLAC-encode random data. So I tried FLACing white noise, and comp ratio was pretty poor, confirming this. Interestingly, it still did a better job than ZIP! The second problem would be that no other tool would read them correctly anyway. The second idea is to truncate & use it as a lossy encoder, which can be "audibly lossless" anyway. But what's a common practice here? I would be tempted to leave 1 bit of headroom above 0dB, or maybe 2 bits. Normalizing before encoding could be an option, but the gain would then have to be inserted in a tag. & finally, what bit depth? 24? 25, 26? 32? I know there's no right answer, just asking if there are common practices. Thanks.___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
