[Flac-dev] compression
hi can we expect much better compression (like the step to v. 1.2.x) in future versions of the flac encoder or are we at maximum compression level now? thx ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] Re: multiple core support
2007/9/9, Josh Coalson [EMAIL PROTECTED]: --- Harry Sack [EMAIL PROTECTED] wrote: 2007/9/8, Josh Coalson [EMAIL PROTECTED]: it actually is complicated. the libFLAC api is not suited to a multithreaded design because the i/o is stream-based, not file- based. flac(.exe) is the file-based wrapper around libFLAC that allows it to work on files. the way libFLAC buffers data is also impossible to parallelize without significantly changing the api. why was this approach used? because the tradeoffs I described required for arbitrarily parallel encoding significantly complicate the api and implementation. libFLAC was not design for multicode file encoding on PCs, it is a reference design that is also being used in embedded devices running 100MHz, low memory, all kinds of different OSes, etc. euh :) Just make a flag or something in the api that enables the code for low cpu power devices and/or streaming and disable this when encoding on fast pc (so no streaming)? So it uses file based code then for encoding of files on pc's with multiple cpu's and stream based code for all other devices and streaming :) Then you have both in the API! The API design seems to me not very smart because it's not flexible and you're stuck in the future (like now for multiple core support) I don't see any reason why you wouldn't make it all based on files and not on streams :s It's just a major disavantage in my opinion an api cannot be all things to everyone. sure it can: you just have to make some different cases (low cpu power device/streaming or multi-core cpu) and call the apprioprate functions you keep making this assertion but if you actually tried to implement it (and I hope you will) the problems we are all bringing up will quickly become obvious. your own lame-mt example is not an incremental improvement but a significant rewrite of lame (and also does not have nearly the performance advantage of process-level parallelism, see http://www.hydrogenaudio.org/forums/index.php?showtopic=50862) It's multi-threaded and that's what I mean: people here say an audio codec can't be written multi-threaded and it will give no performance boost: 1) that's false: se LAME MT 2) performance boost up to 30%: see LAME MT again For very large files you must wait not so long before it's encoded vs. the original LAME. it would take a specialty file-based encoder using an independent frame encoder to do and even that is not trivial. so we can assume that those API changes will never come and the flac encoder will never have multiple core support? you can assume libFLAC will probably not have it. if you can modify libFLAC to make a multithreaded encoder like flac-mt that would be neat and probably useful to some people. until that time there is not much point repeating the same assertions which are just going to aggravate people. Josh Building a website is a piece of cake. Yahoo! Small Business gives you all the tools to get online. http://smallbusiness.yahoo.com/webhosting ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] Re: multiple core support
2007/9/8, Scot Thompson [EMAIL PROTECTED]: No, streams should stay. Audio is NOT a file based process -- it's a stream. You can't listen to an entire song simultaneously. You organize it into files for later use, but you listen and record from a stream. Stream-based storage is practically REQUIRED for an audio codec. It's not random access, it's sequential. You can put wrappers around it to make it convenient for file storage and conversions, but the codec itself must be streams. Multi-core support may not be practical for a variable-length encoding. How would you know where to write the next block when you don't know what size the first block is going to be? The functionality for that is not trivial and is not currently implemented in the API. Maybe somebody will write a multi-core file-based wrapper for you, or maybe you could try writing one yourself. Or if you disagree with Josh about the direction of FLAC you can write your own codec. But your nattering on and on about how you think the API isn't right doesn't help at all and is very annoying. I just say that the way it is now it's IMpossible to make multiple core support, that's all. It's just a fact :) Harry -- *From:* [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] *On Behalf Of *Harry Sack *Sent:* Saturday, September 08, 2007 6:06 AM *To:* Brian Willoughby *Cc:* flac-dev@xiph.org *Subject:* Re: [Flac-dev] Re: multiple core support 2007/9/8, Brian Willoughby [EMAIL PROTECTED]: Ralph, The problem is that there is no clear advantage, at least in terms of multiple cores, to the approach you're asking about. In order to allow each stage of the codec to overlap, you need smart buffering between each stage. That adds code and complexity which isn't there currently. So you end up making the system do more work in the hopes that there will be some overlap. Basically, later stages get blocked waiting for their input buffer to fill, which means that you're not really getting very much overlap at all, but plenty of multi- threading overhead. At least that's the predicted result - I admit that nobody has tried this, to my knowledge. this is because of the limitations/design problem of FLAC API in particular. When the developers had made a smart decision and based everything on file based I/O you would get a HUGE performance boos when using multiple threads divided between multiple cores, because they only thing to do was to split the file output in different threads. But it's not clear to me why everything was based on streams... Harry Brian Willoughby Sound Consulting On Sep 7, 2007, at 18:25, Ralph Giles wrote: On Fri, Sep 07, 2007 at 04:59:50PM -0700, Josh Coalson wrote: it actually is complicated. the libFLAC api is not suited to a multithreaded design because the i/o is stream-based, not file- based. flac(.exe) is the file-based wrapper around libFLAC that allows it to work on files. the way libFLAC buffers data is also impossible to parallelize without significantly changing the api. It seems like buffering (especially compressed) blocks and writing them to the stream in sequence wouldn't be a problem. Is there something in the way the blocking decisions are made that makes it hard to divide the input audio this way? -r ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] Re: multiple core support
2007/9/8, Josh Coalson [EMAIL PROTECTED]: it actually is complicated. the libFLAC api is not suited to a multithreaded design because the i/o is stream-based, not file- based. flac(.exe) is the file-based wrapper around libFLAC that allows it to work on files. the way libFLAC buffers data is also impossible to parallelize without significantly changing the api. why was this approach used? The API design seems to me not very smart because it's not flexible and you're stuck in the future (like now for multiple core support) I don't see any reason why you wouldn't make it all based on files and not on streams :s It's just a major disavantage in my opinion it would take a specialty file-based encoder using an independent frame encoder to do and even that is not trivial. so we can assume that those API changes will never come and the flac encoder will never have multiple core support? thx --- Harry Sack [EMAIL PROTECTED] wrote: 2007/9/7, Avuton Olrich [EMAIL PROTECTED]: On 9/6/07, Harry Sack [EMAIL PROTECTED] wrote: it's really not complicated I think: only api changes to write on any Please get started writing a patch, immediately. I'm just an IT student and I have no time for that :) I also didn't study the flac API in detail but I know it's perfectly possible because I made a avi encoder running on multiple threads once and it's exactly the same for audio data. -- avuton -- Anyone who quotes me in their sig is an idiot. -- Rusty Russell. ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev Building a website is a piece of cake. Yahoo! Small Business gives you all the tools to get online. http://smallbusiness.yahoo.com/webhosting ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
[Flac-dev] Re: multiple core support
it's really not complicated I think: only api changes to write on any position in the file if that's not possible already with existing function. I'm not sure if decoding can have multi-core support: you need an api for writing pcm files in different parts then and this is maybe more difficult to check if it's valid pcm data since the decoder can only check for valid flac streams , not pcm streams i think. you can make recording also support multi cores: just let cpu 1 fill a pcm data buffer while cpu 2 encodes it. In this case you have a more reliable encoding with less cpu usage of your system (it's spread among 2 cpu's) 2007/9/7, Brian Willoughby [EMAIL PROTECTED]: Hmm, You're actually correct, when you put it that way. Because the audio blocks are coded independently, you could speed things by having the encoder (or decoder) do a little up-front processing on the metadata, then use the overall settings to divide the file into sections and run the FLAC process in parallel. However, I think it is generally frowned upon in some circles to create one thread per processor, rather than making the division dependent upon some other factor that is actually independent of the number of processors. Apart from that caveat, someone could certainly develop an encoder or decoder which uses the FLAC library API to implement a threaded encoder/decoder - well, assuming that the API can write to subsets of a file, instead of requiring that the entire file be written from start to finish - and also assuming that the API can run in multiple threads and write to the same file. By your method, decoding would be faster, too. Not everybody decodes just to listen. Some of us keep original masters in FLAC format for archival purposes, but need the whole thing decoded before we can mix and master. Anything which speeds up decoding would help get work done faster. On the flip side, another thing to consider is that not all encoding is done with files that already exist. If you are recording live to FLAC, you cannot multi-thread the encoder because you don't have any audio besides the current block. So, decoding for playback and encoding for recording are two examples which cannot be multi-threaded. But file format conversion could be developed to take advantage of multiple processors. It would require a lot of tweaks to the library, perhaps. Note: PC games are multi-threaded because they're already doing several different things at once. Sometimes the complexity is actually reduced by dedicating a thread to each part of the game logic. And precisely because there are so many different things going on, multi-threading can speed things up when they overlap in unpredictable ways. Games would be harder to write in a single- threaded way. Meanwhile, encoding/decoding FLAC is easier to write in a single-threaded design. Brian W. On Sep 6, 2007, at 16:42, Harry Sack wrote: yes, i totally agree but I'm talking about the ENcoder :) You can perfectly 'cut' the pcm stream 'in half' and let 1 cpu encode one part , the other 2nd part (or with 4 cpu's in 4 parts, ...), because it's just a stream one frame after the other, as far as i know (correct me if i'm wrong). Many pc games synchronise multiple threads on 2 (even 4!) cores (much More complicated then reading frames, compressing, writing compressed frames like the encoder does), so i think making the encoder support 2 cpu's is perfectly possible. of course adding multi-core support to the decoder is silly because it's not necessary (any single core cpu is fast enough to play 1 file and it would only slow it down)! 2007/9/7, Brian Willoughby [EMAIL PROTECTED]: Any software which supports multiple processors must be multi- threaded. The process of designing multi-threaded code adds complexity to the software, so there must be a good reason to go through all the trouble. The procedure for decoding a single file could only benefit marginally from being multi-threaded, because most operations would need to wait until the previous operation was completed. One section of the decoder might be able to process several blocks, but the overall result would still need to wait until all blocks are completed. Besides, managing dynamic buffers between each step of the decoding process would actually require the decoder to do more, meaning it would slow down to an extent. There would pretty much be no difference in the total time, whether single processor or multiple processors. The best way to take advantage of multiple processors is to decode multiple FLAC files at the same time. This will take full advantage of your system, provided that the disk bandwidth and memory bandwidth can keep up. Brian W. ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
[Flac-dev] FLAC: compression ratio
hi flac-dev list! I see, when compressing CD-audio tracks, I can reach up to 60% (ratio = 0.6x) of the original WAV file after compression. I was wondering if the FLAC codec could become as good as reaching 50% of the original WAV file in the future or if we are already at the (almost) maximum compression possible? thx ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
[Flac-dev] FLAC: general question
hi dev'ers is it true flac is developed in linux by Josh and later compiled in windows? I was also wondering if Josh implemented the flac encoder/decoder all by himself ? just some questions because I'm interested in flac in general :) thx ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] FLAC: library for C#
2007/6/14, Josh Coalson [EMAIL PROTECTED]: --- Erik de Castro Lopo [EMAIL PROTECTED] wrote: Harry Sack wrote: Hi FLAC dev's list, I'm looking for a library for the C# language (Microsoft .Net Framework 2.0or higher) to play FLAC files and/or maybe do some other things like getting the file duration, file properties, ... of FLAC files. The library must preferably be free (open source is not required, but is always nice). Why not just build FLAC as a DLL and then call into the DLL from C#? Harry, if that is good enough, there are already pre-build DLLs with headers in the sourceforge download section. but aren't they C++ headers en lib's? I have no idea how I can include a C++ header in a C# project. It this even possible? thx Josh Get the Yahoo! toolbar and be alerted to new email wherever you're surfing. http://new.toolbar.yahoo.com/toolbar/features/mail/index.php ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
[Flac-dev] FLAC: library for C#
Hi FLAC dev's list, I'm looking for a library for the C# language (Microsoft .Net Framework 2.0or higher) to play FLAC files and/or maybe do some other things like getting the file duration, file properties, ... of FLAC files. The library must preferably be free (open source is not required, but is always nice). I hope somebody can help me with this! thx Harry ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
Re: [Flac-dev] compression ratio
2007/5/14, Josh Coalson [EMAIL PROTECTED]: --- Harry Sack [EMAIL PROTECTED] wrote: hi i was wondering if it's possible to tell me what the theoretical best compression ratio the flac encoder can do, because i was wondering what the future of flac will bring us. So this question is probably best addressed to Josh: can we suspect much better compression ratio in the future or is the encoder already at his *almost* peak compression level. I think you're talking about average compression ratio because it depends on the particular sample. to make significant improvements (3%) would require changes to the format that wouldn't work with older decoders, which I don't plan on ever doing. i think that's a very good decision! if you look at the comparison here: http://flac.sourceforge.net/comparison.html you will see that FLAC is already within 3-4% of the most aggresive codecs. so in the future we can expect mostly speed improvements for encoder/decoder? Thanks Josh Boardwalk for $500? In 2007? Ha! Play Monopoly Here and Now (it's updated for today's economy) at Yahoo! Games. http://get.games.yahoo.com/proddesc?gamekey=monopolyherenow ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev
[Flac-dev] compression ratio
hi i was wondering if it's possible to tell me what the theoretical best compression ratio the flac encoder can do, because i was wondering what the future of flac will bring us. So this question is probably best addressed to Josh: can we suspect much better compression ratio in the future or is the encoder already at his *almost* peak compression level. thx in advance! ___ Flac-dev mailing list Flac-dev@xiph.org http://lists.xiph.org/mailman/listinfo/flac-dev