Re: [Bug-gnupod] Patch to support ReplayGain / mp3gain
On 5/8/09 2:26 PM, H. Langos wrote: Could you send me a sample of an iTunes generated RVA2 tag? Sure. I'll fire up iTunes later and send it to you privately. Yeap. And if mp3gain can't be changed to do write RVA2 tags by itself, maybe there is a way to make mp3gain only do the analysis and pass its result to a programm that can write those RVA2 tags? That would be possible using: $ mp3gain -o -q -s s foo.mp3 FileMP3 gaindB gainMax AmplitudeMax global_gainMin global_gain foo.mp3-7-10.5038780.534185255145 "Album"-7-10.5038780.534185255145 Do you know if the author of mp3gain is still active? reachable? alife? I have no clue.. in the thread from 2004 they mention he was unresponsive. So how do we get mp3gain's analysis results into RVA2 / XRVA tags? By changing the mp3gain C code. It's open source after all. I just don't have the stomach or time for it at the moment. Perhaps using the tagging code from normalize will make it easier. However, there could be a good reason not to attempt it. From http://jwz.livejournal.com/370342.html?thread=4232358#t4232358 Anway, whats not clear to me is if the RVA2 value could be used with ReplayGain. Both ReplayGain and RVA2 are applied as a constant value across the complete track, but I'm not sure the units are the same as the RVA2 tag. Are you sure about the non desctructivnes of mp3gain? You can run mp3gain in several modes, either (often reversible) changing the volume of the file or just tagging it. I use it only for tagging. From the Debian man page: mp3gain actually changes your file’s gain only when you use one of the options -r, -a, -g, or -l. If none of these options is given, only a tag denoting the recommended gain change is written to the file. If you've got a strong stomach you might want to read the whole thread: http://jwz.livejournal.com/370342.html Great read, thanks. If this is all true, than your patch for gnupod is in vain as the volume of those files was already changed by mp3gain. Reading the APE tag and acting on the information would either double the adjustment or cancel it out, depending on wether mp3gain stores the information on what it did, or how to undo its action. Interesting question. Since I only tag my mp3s, and never let anything change the audio frames, I'm safe no matter what. The following shows others are too. The ReplayGain APE tags always contain the dB change needed to get the volume to the reference level. When mp3gain has altered the file, the RG APE tags will be close(r) to 0. It will also store undo information in the MP3GAIN_UNDO APE tag. $ cp -p org.mp3 cleaned.mp3 $ mp3gain -s d cleaned.mp3 $ mp3gain -s c cleaned.mp3 cleaned.mp3 $ cp -p cleaned.mp3 altered.mp3 $ mp3gain -r altered.mp3 altered.mp3 Applying mp3 gain change of -7 to altered.mp3... $ mp3gain -s c altered.mp3 altered.mp3 Recommended "Track" dB change: 0.035000 Recommended "Track" mp3 gain change: 0 Max PCM sample at current gain: 11529.519104 Max mp3 global gain field: 248 Min mp3 global gain field: 138 $ mp3gain tagged.mp3 tagged.mp3 Recommended "Track" dB change: -10.50 Recommended "Track" mp3 gain change: -7 Max PCM sample at current gain: 38780.534185 Max mp3 global gain field: 255 Min mp3 global gain field: 145 Recommended "Album" dB change for all files: -10.50 Recommended "Album" mp3 gain change for all files: -7 $ cp -p cleaned.mp3 tagged.mp3 $ mp3gain -s c tagged.mp3 tagged.mp3 Recommended "Track" dB change: -10.50 Recommended "Track" mp3 gain change: -7 Max PCM sample at current gain: 38780.534784 Max mp3 global gain field: 255 Min mp3 global gain field: 145 $ cp -p altered.mp3 undo.mp3 $ mp3gain -u undo.mp3 Undoing mp3gain changes (7,7) to undo.mp3... $ mp3gain -s d undo.mp3 $ cmp -l cleaned.mp3 undo.mp3 $ Cheers, Richard ___ Bug-gnupod mailing list Bug-gnupod@nongnu.org http://lists.nongnu.org/mailman/listinfo/bug-gnupod
Re: [Bug-gnupod] Patch to support ReplayGain / mp3gain
On Thu, May 07, 2009 at 09:16:51PM +0200, Richard van den Berg wrote: > On 5/7/09 5:06 PM, H. Langos wrote: >> What's important is the fact that there seems to be a standard that defines >> how volume adjustment data can be stored in id3 tags. > > Nice research. RVA2 is definitely the standard to use, and gnupod > already has support for it. See, I wasn't aware of that. Eventhough I have been changing a lot of code less than 3 lines above that area. :-) BTW: I just added XRVA support as it is a too simple to let it pass. > The problem is that there is not a lot of > software that writes RVA2 tags. (The only ones I know about are iTunes > and normalize). Could you send me a sample of an iTunes generated RVA2 tag? Would be interesting to see what they actually use. I.e. which identification string they use, which channels they adjust and if they add any peak data. (The first 20kb of the mp3 file should easily be enough unless it contains album art.) > mp3gain should really be changed to write RVA2 tags > instead of APE ones. Yeap. And if mp3gain can't be changed to do write RVA2 tags by itself, maybe there is a way to make mp3gain only do the analysis and pass its result to a programm that can write those RVA2 tags? >> So I wonder why the author of mp3gain decided to store the information in an >> ascii APE tags comment instead of using the existing standard. (He >> definetly was aware of it as he states in the FAQ that he "uses David >> Robinson's Replay Gain algorithm to calculate how loud the file >> actually sounds to a human's ears.") >> > > mp3gain is one of the first implementations of the Replay Gain > algorithm for mp3 files. According to http://jwz.livejournal.com/370342.html?thread=4232358#t4232358 it is the only implementation. Every other implementation seems to be cut'n paste. > Please note that the algorithm for calculating > the volume adjustment in itself has nothing to do with the way of > storing it. ACK that. > I think mp3gain predates the RVAD/RVA2 standards so they > went for an APE tag instead. Well, AFAIK the original proposal of David Robinson already contained the RGAD tag definition. That would have been easy enough to use or expand on. Using "Radio" for track gain and "Audiophile" for album gain would have been easy and apparently correct. > It would have been nicer if they had stuck > it in a custom X??? ID3 frame instead. It would have been easier to port > it to RVA2 now. Do you know if the author of mp3gain is still active? reachable? alife? >> I took a look at the source code of normalize and it seems to be able >> to write to id3v2.3 using XRVA and id3v2.4 using RVA2. Maybe this would >> be >> interesting for you too? http://normalize.nongnu.org/ >> > > I used normalize for a long time before I switched to mp3gain. Normalize > is nice, but the mp3gain algorithm is better. So how do we get mp3gain's analysis results into RVA2 / XRVA tags? > And I like the concept of > not altering the data, but just tagging it and only changing the volume > during playback. It seems normalize can do that now too. The way I used > normalize (before mp3 support was added) was letting it change the > volume of the wav files before encoding them to mp3. Are you sure about the non desctructivnes of mp3gain? I've read up on it and it looks more messy the deeper I look ... ... http://jwz.livejournal.com/370342.html?thread=4241574#t4241574 >> mp3gain doesn't just write tags, it modifies the mp3 frames >> somewhat destructively, and adds APE tags so it can reverse >> it. This is rather backward. > > Well, there is no destruction going on. It just saves a gain value > in the mp3 frames. Which has the benefit that it works on all mp3 > players without special replaygain support and using the info in the > APE tags, it can be completely undone. > The APE thing is weird. I would prefer it used ID3v2. I asked the > author why he chose APE. His answer was that at the time he heard > lots of horror stories about ID3v2 problems. If you've got a strong stomach you might want to read the whole thread: http://jwz.livejournal.com/370342.html If this is all true, than your patch for gnupod is in vain as the volume of those files was already changed by mp3gain. Reading the APE tag and acting on the information would either double the adjustment or cancel it out, depending on wether mp3gain stores the information on what it did, or how to undo its action. Seems like you should * backup an mp3, * run mp3gain on it and * run a "cmp -l file.mp3 backup.mp3" then * try to undo the changes that mp3gain made and * once more do the "cmp -l file.mp3 backup.mp3" just to make sure that it realy undid the changes... good luck -henrik ___ Bug-gnupod mailing list Bug-gnupod@nongnu.org http://lists.nongnu.org/mai
Re: [Bug-gnupod] Patch to support ReplayGain / mp3gain
On 5/8/09 12:14 PM, Frank Blendinger wrote: You might want to take a look at this python script: http://mpd.wikia.com/wiki/Hack:ape2id3.py It will read the ReplayGain settings from APE tags and add it to ID3v2 tags. It has worked without any problems for me so far. That script seems to create custom TXXX tags. This is not a standard and it surprises me that players support it. I know rockbox (alternative iPod firmware) does as well. Besides, I don't need to store the data I already have in APE tags in yet another format. This would require me to always run a conversion script after I run mp3gain, and I would have to add the script to abcde (the ripper script I use) as well. So to sum this up, we could have to deal with: * APE tags * RVA2 in ID3v2.4 or XRV/XRVA in ID3v2.3 or RGAD in ID3v2.3 * iTunNORM as Comment in either ID3v2.3 or v2.4 Did I miss anything? Yes, the following TXXX tags: TXXX:replaygain_album_gain TXXX:replaygain_album_peak TXXX:replaygain_track_gain TXXX:replaygain_track_peak So how will gnupod handle this? I suppose the iPod will just use iTunNORM and ignore anything else. Will gnupod use a present iTunNORM comment or will it always create one from ID3v2/APE information? The patch I created will favor ReplayGain info found in an APE tag over iTunNORM since RG is a superior algorithm. RVA2 tags are converted to the iTunesDB volume value when "--max-vol-adj" is used with gnupod_addsong.pl. It is combined with normalization info (if also present) when Sound Check is enabled on the iPod. What will be used if both ID3v2 and APE is present? I read about a (possible?) --ignore-ape option, but I didn't really get what the default behaviour will be. The new default behavior will be to read APE tags, including ReplayGain information. ID3 tags with the same name of APE tags will overwrite the APE tags. ID3v2 tags are favored over ID3v1 tags. Perhaps we should also read TXXX:replaygain_album_gain and TXXX:replaygain_track_gain from ID3v2 tags to replace the respective APE tags. I, for one, would propose to use RVA2 if present and APE only as a fallback, as it is more standardize and also more often present in actual files, APE does not seem to be supported by many (software) players (and I guess not by a single hardware device), while some do support ReplayGain tags in ID3v2. My Neo 35 running OpenNeo reads RG info in APE tags just fine. As do my various Squeezeboxes (powered by SqueezeCenter). One of the problems is that normalization data like RG APE tags and iTunNORM comments is sometimes combined with RVA2 volume information. iTunes is one of the software that does this, and SqueezeCenter also combines the tags (but there is some debate to change that). So favoring RVA2 over RG info should really be selectable by the user. As explained above, iPods will combine the two when both are present in the iTunesDB and Sound Check is enabled. And one last thing: what about support for the music formats covered by the gnupod_convert_* scripts? Those can carry ReplayGain tags, too, so it would be nice to keep/convert them when the audio gets converted. The tags of non-native file formats are spit out by the GET_META part of the gnupod_convert_* scripts. If they spit out the correct soundcheck value, that would work. So if you can show us how to get ReplayGain information from those non-native files, we can probably make it work. So far I only have experience with OGG Vorbis files. There is tool similar to mp3gain called vorbisgain to calculate the gain data and write the tags, and they can be read with the vorbiscomment tool. So I guess it should not be that hard to read those tags as it is done with the common artist/title/... tags and add them to the converted mp3 files. gnupod_convert_OGG.pl uses Ogg::Vorbis::Header which may or may not support the RG tags. If you send me an ogg file (privately) that contains RG info I can check (and patch gnupod_convert_OGG.pl). I have read that there is also support for ReplayGain tags in FLAC files, but I have no experience with that. We'll leave that for someone that actually uses FLAC (I don't). Cheers, Richard ___ Bug-gnupod mailing list Bug-gnupod@nongnu.org http://lists.nongnu.org/mailman/listinfo/bug-gnupod
Re: [Bug-gnupod] DESC field in iTunesDB
On 5/8/09 11:51 AM, H. Langos wrote: Meanwhile code that used to be in gnupod_find has been generalized and moved into the new FindHelper.pm and that is now probably be better place for %FILEATTRDEF. I agree. It was only until I dug deeper that I realized that code in iTunesDB.pm was really only used in FindHelper.pm. Smart playlists can have a "desc" attribute? I didn't know that. Does tunes2pod retain it? If so, could you send me the relevant lines from your GNUtunesDB.xml ? I will probably not do anything with it but I'm curious. I created it with iTunes, and it filled the playlists with the songs currently on the iPod. From GNUtunesDB.xml: limitval="25" liveupdate="1" matchany="1" moselected="" name="Veere" plid="292683058" > [snip] limitval="25" liveupdate="1" matchany="1" moselected="" name="Random" plid="3323171954" > [snip] But since they are liveupdate smart playlists, after adding more mp3s using gnupod (not from /somedir/) and without running iTunes, they did show up in the playlist named "Random". Which is exactly what I want. Could be. Could you test if the "desc" attribute in combination with the "lyrics_flag" attribute result in displayed lyrics? Apart from the "desc" attribute i wouldn't know where the lyrics could be stored. I forced the lyrics_flag to 1 in mk_mhit() but I still don't see the description. Pressing the center button I still cycle through: volume -> position -> artwork -> rating Maybe you can minimize the change by simply add the information that you need to the __merge_strings call. That way you can use the desc attribute both ways. (unless your mp3 collection has a lot of garbage in the comment tags. mine has.) Yeah, I was thinking the same thing but I'm afraid the 1% of mp3s that I did not rip/tag myself contain all sorts of tags I don't care about. Maybe I can query my SqueezeCenter MySQL database to see how bad it is. Cheers, Richard ___ Bug-gnupod mailing list Bug-gnupod@nongnu.org http://lists.nongnu.org/mailman/listinfo/bug-gnupod
Re: [Bug-gnupod] Patch to support ReplayGain / mp3gain
Hey guys, I'm very pleased to know you are working on ReplayGain support in gnupod. I have just recently started to add ReplayGain tags to my music collection, as mpd has started to support that. Now it seems I can also benefit from this in my iPod, very nice. On Thu 2009-05-07 21:16, Richard van den Berg proclaimed: > On 5/7/09 5:06 PM, H. Langos wrote: >> What's important is the fact that there seems to be a standard that defines >> how volume adjustment data can be stored in id3 tags. > > Nice research. RVA2 is definitely the standard to use, and gnupod > already has support for it. The problem is that there is not a lot of > software that writes RVA2 tags. (The only ones I know about are iTunes > and normalize). mp3gain should really be changed to write RVA2 tags > instead of APE ones. You might want to take a look at this python script: http://mpd.wikia.com/wiki/Hack:ape2id3.py It will read the ReplayGain settings from APE tags and add it to ID3v2 tags. It has worked without any problems for me so far. >> So I wonder why the author of mp3gain decided to store the information in an >> ascii APE tags comment instead of using the existing standard. (He >> definetly was aware of it as he states in the FAQ that he "uses David >> Robinson's Replay Gain algorithm to calculate how loud the file >> actually sounds to a human's ears.") > > mp3gain is one of the first implementations of the Replay Gain > algorithm for mp3 files. Please note that the algorithm for calculating > the volume adjustment in itself has nothing to do with the way of > storing it. I think mp3gain predates the RVAD/RVA2 standards so they > went for an APE tag instead. It would have been nicer if they had stuck > it in a custom X??? ID3 frame instead. It would have been easier to port > it to RVA2 now. This load of different tags is really annoying. I wish people would just agree on a standard for things like this and stick to it, instead of reinventing the wheel all over again. But I guess it's too late for that already, as different players and devices are supporting different tags. So to sum this up, we could have to deal with: * APE tags * RVA2 in ID3v2.4 or XRV/XRVA in ID3v2.3 or RGAD in ID3v2.3 * iTunNORM as Comment in either ID3v2.3 or v2.4 Did I miss anything? All of those could potential be present at once in a file. There can only be either an ID3v2.3 or an ID3v.4 as far as I can tell. But there can always be that iTunNORM comment in addition to the RG tags, and also additional APE tags. I have actually seen files in the wild that have all of this present. [1] So how will gnupod handle this? I suppose the iPod will just use iTunNORM and ignore anything else. Will gnupod use a present iTunNORM comment or will it always create one from ID3v2/APE information? What will be used if both ID3v2 and APE is present? I read about a (possible?) --ignore-ape option, but I didn't really get what the default behaviour will be. I, for one, would propose to use RVA2 if present and APE only as a fallback, as it is more standardize and also more often present in actual files, APE does not seem to be supported by many (software) players (and I guess not by a single hardware device), while some do support ReplayGain tags in ID3v2. And one last thing: what about support for the music formats covered by the gnupod_convert_* scripts? Those can carry ReplayGain tags, too, so it would be nice to keep/convert them when the audio gets converted. So far I only have experience with OGG Vorbis files. There is tool similar to mp3gain called vorbisgain to calculate the gain data and write the tags, and they can be read with the vorbiscomment tool. So I guess it should not be that hard to read those tags as it is done with the common artist/title/... tags and add them to the converted mp3 files. I have read that there is also support for ReplayGain tags in FLAC files, but I have no experience with that. Greetings, Frank [1] I use this to check the files: For ID3: eyeD3 foo.mp3 | egrep -i '(gain|iTunNORM|ID3 v2)' -A1 For APE: mp3gain -s c foo.mp3 Is there anything nicer to do that? Do I miss any potential gain tags? I thought about putting together a small script to check all possible tags and show them in a nice format. Any hints/suggestions are welcome. -- Frank Blendinger | fb(at)intoxicatedmind.net | GPG: 0x0BF2FE7A Fingerprint: BB64 F2B8 DFD8 BF90 0F2E 892B 72CF 7A41 0BF2 FE7A signature.asc Description: Digital signature ___ Bug-gnupod mailing list Bug-gnupod@nongnu.org http://lists.nongnu.org/mailman/listinfo/bug-gnupod
Re: [Bug-gnupod] DESC field in iTunesDB
On Thu, May 07, 2009 at 06:23:01PM +0200, Richard van den Berg wrote: > On 5/7/09 1:50 PM, H. Langos wrote: >> I wasn't aware of a limitation to 40 characters. This seems a rather >> arbitrary limit imposed on the user by itunes. It doesn't even resemble the >> original id3v1 comment tag limit as that was 30 characters. >> ( http://www.id3.org/ID3v1 ) > > My bad. I took the width value of %FILEATTRDEF in iTunesDB.pm to mean > the width of the field in the iTunesDB. It turns out it is only used as > display width in FindHelper.pm. The mk_mhod() / mk_mhit() functions that > actually convert the fields to iTunesDB format do not seem to impose any > size limit. I probably should move that %FILEATTRDEF stuff over to FindHelper.pm . Initially I thought I could combine the iTunesDB format information that is in the code of iTunesDB.pm with the information regarding its presentations that I added in %FILEATTRDEF. But I don't have the time to dig into the itunesdb format deep enough to combine those. Meanwhile code that used to be in gnupod_find has been generalized and moved into the new FindHelper.pm and that is now probably be better place for %FILEATTRDEF. > I haven't been able to read the "desc" field I set on my mp3s using > gnupod_addsong.pl + mktunes.pl either on my 5G iPod or using the iTunes > GUI. Seems like that only works for audio podcasts. > I can use it in smart playlists though. Smart playlists can have a "desc" attribute? I didn't know that. Does tunes2pod retain it? If so, could you send me the relevant lines from your GNUtunesDB.xml ? I will probably not do anything with it but I'm curious. > Googling shows that my iPod > should be able to display lyrics, but I guess the lyrics_flag needs to > be set first. Gnupod doesn't seem to do that for mp3s (which I guess is > a bug). Could be. Could you test if the "desc" attribute in combination with the "lyrics_flag" attribute result in displayed lyrics? Apart from the "desc" attribute i wouldn't know where the lyrics could be stored. >> If you need to get the information that currently is encoded in the file's >> path into the file itself I would recommend using a comment tag. > > I thought about that, and mass tagging my collection is not a big deal, > but I'll have to redo it every time I add something, or move files from > one directory to another. Hacking gnupod is much easier, and I won't > have to change my habits. :-) I'll see if keeping a customized version > of gnupod is better than changing the way I tag/select my mp3 collection. Maybe you can minimize the change by simply add the information that you need to the __merge_strings call. That way you can use the desc attribute both ways. (unless your mp3 collection has a lot of garbage in the comment tags. mine has.) cheers -henrik ___ Bug-gnupod mailing list Bug-gnupod@nongnu.org http://lists.nongnu.org/mailman/listinfo/bug-gnupod
