Why not just add a menu option "sync metadata" that disables playback and
warns you of the consequences of having files open in other programs before
writing the metadata?
Ewan.
On Apr 1, 2012 8:41 AM, "Ilkka Tuohela" <ilkka.tuoh...@gmail.com> wrote:
>
> Not even file access or locking, but modifying a file mapped with mmap on
> the fly.
>
> BTW this raises another issue: if any other program outside mixxx modifies
> file tags on disk while the track is loaded to mixxx, same problem with
> file mmap garbage can happen, if the mmap is directly to on-disk file!
> There is no way to prevent other programs from modifying a file while mixxx
> is playing.
>
> Of course, if mixxx does not mmap the raw file directly, but decodes the
> file to a buffer and plays from that, the mmap issue with tags should not
> happen in the first place.
>
> I think what some other programs do here is to:
>
> - open original file for reading, feed it to decoder
> - write/copy the decoded raw wav data to a static memory buffer (one for
> each deck)
> - close the original file
> - play the original wav file completely from memory buffer, ignoring any
> changes to file on disk
>
> In deckadance this has raised another known bug, since they have
> artificially limited the memory buffer to quite small value by modern
> standards and can only load files less than about 12 minutes long (64MB
> buffer per track?). If we would go this way, I would prefer a user
> customizable maximum size for track read buffer, my laptops have 4-16GB of
> memory and I don't mind giving the DJ program half of it so let me have 1GB
> read buffer for each deck if I want to allocate the memory!
>
> I completely understand why you would prefer mmap, but the issue raised
> here can't be avoided while using mmap on the file on disk.
>
> On 1 Apr 2012, at 06:00, Scott wrote:
>
> > My mistake, I just looked at #728197 and if I understand correctly it is
> > a file access / lock issue.
> >
> > On Sat, 2012-03-31 at 22:46 -0400, RJ Ryan wrote:
> >> We actually use TagLib for all of our metadata processing needs -- we
> >> didn't roll metadata writing ourselves or anything like that.
> >>
> >>
> >> On Sat, Mar 31, 2012 at 10:18 PM, Scott <nappingcrac...@gmail.com>
> >> wrote:
> >> This sounds a bit like NIH (not invented here) syndrome.
> >>
> >> I suggest using id3lib to do the actual file manipulation and
> >> simply
> >> hook Mixxx's front end into that API. This would work for at
> >> least ID3v1
> >> and ID3v2 tags.
> >>
> >> id3lib http://id3lib.sourceforge.net/ has been around for a
> >> while and is
> >> used by many tagger / player programs. EasyTAG uses it, and
> >> I've used
> >> that program without problems for years on tens of thousands
> >> of files at
> >> a time.
> >>
> >> This would leave the quality control to a group focused
> >> specifically on
> >> manipulating audio file tags and metadata.
> >>
> >> -Scott
> >>
> >> On Sat, 2012-03-31 at 22:40 +0100, Adam Davison wrote:
> >>> Hi,
> >>>
> >>> I think Albert's general point was more that if you have any
> >> memory
> >>> corruption anywhere in Mixxx, then the results of our write
> >> to the
> >>> file are undefined. Even if we fix this specific memory
> >> corruption bug
> >>> then we have no way to guarantee there aren't more lurking.
> >> In fact I
> >>> could probably guarantee you that there are memory
> >> corruption bugs
> >>> lurking in code the size of Mixxx + all the libraries living
> >> in our
> >>> process space.
> >>>
> >>> One safe way to do it perhaps would be to have a very small
> >> helper
> >>> program which handled the writing in a different process.
> >>>
> >>> Adam
> >>>
> >>> On 31 March 2012 18:54, Ilkka Tuohela
> >> <ilkka.tuoh...@gmail.com> wrote:
> >>>>
> >>>> Maybe I'm asking the obvious, but why wouldn't following
> >> process work just fine:
> >>>>
> >>>> - Store new tags to database fields
> >>>> - Make a copy of the file on disk to temporary file,
> >> original being still mmaped or not doesn't matter
> >>>> - Write tag changes to the temporary file
> >>>> - If writing temporary file was successful (disk space was
> >> enough etc), unlink old file and rename new file to original
> >> name
> >>>> - If writing temporary file was not successful, remove
> >> temporary file and report error, maybe revert database fields?
> >>>>
> >>>> At least on unix-like systems, the mmap will hold a
> >> reference pointing to the original deleted file, and the
> >> changes to tags can't affect it, preventing corruption. This
> >> will use extra bytes from disk for all rewritten files (mmaped
> >> bytes + new file with new tags) as long as the mmap FD is
> >> open, but since we don't hold hundreds of files open in
> >> player, the extra disk space used should not be an issue and
> >> is released anyway on fs level when the FD is closed by mixxx
> >> (song has been played).
> >>>>
> >>>> Additionally, if we wish to show the updated tags in
> >> decks, I think the tags should be and are based on database
> >> entries not file tags and should reflect the current info
> >> correctly (as long as there is a slot to update the details in
> >> player on the fly).
> >>>>
> >>>> I have no idea if this works similarly on windows
> >> filesystems though, but I think so.
> >>>>
> >>>> *hile*
> >>>>
> >>>> On 31 Mar 2012, at 18:13, Owen Williams wrote:
> >>>>
> >>>>> I did develop an initial crappy workaround that prevents
> >> many of the
> >>>>> memory-corruption situations. Basically, if Mixxx can
> >> know for sure
> >>>>> that the music file is not open, it's safe to update the
> >> metadata. The
> >>>>> trick is figuring out if in fact that file is open or
> >> not.
> >>>>>
> >>>>> I don't believe there's any risk in corrupting the file
> >> on disk. The
> >>>>> corruption I experienced was only in terms of playback --
> >> still a big
> >>>>> problem because it would cause static during sets.
> >>>>>
> >>>>> Mixxx is a nice interface for tag editing, though, so it
> >> might be worth
> >>>>> pushing up the importance of that bug.
> >>>>>
> >>>>> (one solution might be queuing the metadata update in the
> >> file reader
> >>>>> object itself, and then writing the data in the
> >> destructor. Is it safe
> >>>>> to assume only one file object per file, though?)
> >>>>>
> >>>>> Owen
> >>>>>
> >>>>>
> >>>>> On Sat, 2012-03-31 at 10:38 -0400, Albert Santoni wrote:
> >>>>>> Hi Keith,
> >>>>>>
> >>>>>> I'm sorry you had a crappy experience with this - Our
> >> intention was to
> >>>>>> reduce the risk of data loss.... which is the opposite
> >> of what
> >>>>>> happened to you.
> >>>>>>
> >>>>>> Mixxx did not write write metadata to any music files
> >> until 1.9
> >>>>>> because we were concerned that we didn't have the
> >> quality control in
> >>>>>> place to ensure we would never ship a library-destroying
> >> bug. Just
> >>>>>> imagine if we had a bug that caused your MP3s to get
> >> corrupted...
> >>>>>>
> >>>>>> And it turns out this fear wasn't totally unfounded.
> >> After 1.9
> >>>>>> shipped, Owen tracked down a subtle memory corruption
> >> bug related to
> >>>>>> Mixxx writing to MP3s, and although there were no
> >> reports of corrupt
> >>>>>> files, file writing and memory corruption are not a
> >> great combination.
> >>>>>> (You might be able to make an argument that Mixxx is
> >> more susceptible
> >>>>>> to heap corruption programming errors which may not
> >> affect the actual
> >>>>>> file writing itself if it's done on the stack. I'm sure
> >> the
> >>>>>> TrackInfoObjects are kept on the heap though, so maybe
> >> worst case you
> >>>>>> could just end up with corrupt metadata. But never say
> >> never...)
> >>>>>>
> >>>>>> As Daniel pointed, it was due to bug #728197 that we
> >> disabled metadata
> >>>>>> writing for Mixxx 1.10.0 and made a quick 1.9.2 release.
> >>>>>>
> >>>>>> If you have any suggestions on how we can communicate
> >> that metadata
> >>>>>> isn't written or want to work on #728197, you know how
> >> to get in
> >>>>>> touch... :)
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Albert
> >>>>>>
> >>>>>>
> >>>>>> On Sat, Mar 31, 2012 at 4:32 AM,
> >> keithsalisb...@gmail.com
> >>>>>> <keithsalisb...@gmail.com> wrote:
> >>>>>>> Sorry guys, normally I wouldn't say this, but something
> >> happened today
> >>>>>>> which I really don't understand why this choice has
> >> been made, maybe
> >>>>>>> someone can explain.
> >>>>>>>
> >>>>>>> So in the last week or so, I've spent many hours
> >> tweaking the metadata
> >>>>>>> in my dj library, sorting out track titles, artists,
> >> labels, keys,
> >>>>>>> bpms all sorts of goodness which makes a record box
> >> more than just a
> >>>>>>> bunch of files.
> >>>>>>>
> >>>>>>> It was my assumption all this goodness was being baked
> >> into the files.
> >>>>>>>
> >>>>>>> Yesterday as part of building a new version I moved
> >> my .mixxx config,
> >>>>>>> and associated database etc out the way to create a new
> >> one.
> >>>>>>>
> >>>>>>> And here it is, to my surprise and quite honestly my
> >> shock, my files
> >>>>>>> we ALL back to their tatty messed up disorganised state
> >> - ALL my time
> >>>>>>> and effort had been reverted.
> >>>>>>>
> >>>>>>> Now you'll notice I was smart enough to backup my data
> >> before creating
> >>>>>>> a new database, so I've not really lost anything,
> >> however I AM upset
> >>>>>>> that this happened. I would expect more from Mixxx. I
> >> actually trusted
> >>>>>>> Mixxx more than Rhythmbox or those other media browsers
> >> to do the
> >>>>>>> right thing by my files.
> >>>>>>>
> >>>>>>> So what's the deal here - I can understand a concern
> >> allowing users to
> >>>>>>> edit their files - sure - but there should be a
> >> config/preferences to
> >>>>>>> allow me to choose. And more over, I should be made
> >> aware that all
> >>>>>>> this goodness is not really in my files. Somehow.
> >>>>>>>
> >>>>>>> My point is, I'm a power user, but I'm sure many users
> >> out there have
> >>>>>>> edited their files believing they're updating the
> >> files, and only when
> >>>>>>> they change computers, or rebuild their computer or
> >> whatever, that
> >>>>>>> they find out all that data is lost.
> >>>>>>>
> >>>>>>> Thoughts?
> >>>>>>>
> >>>>>>> Keith
> >>>>>>>
> >>>>>>>
> >>
>
> ------------------------------------------------------------------------------
> >>>>>>> This SF email is sponsosred by:
> >>>>>>> Try Windows Azure free for 90 days Click Here
> >>>>>>> http://p.sf.net/sfu/sfd2d-msazure
> >>>>>>> _______________________________________________
> >>>>>>> Get Mixxx, the #1 Free MP3 DJ Mixing software Today
> >>>>>>> http://mixxx.org
> >>>>>>>
> >>>>>>>
> >>>>>>> Mixxx-devel mailing list
> >>>>>>> Mixxx-devel@lists.sourceforge.net
> >>>>>>>
> >> https://lists.sourceforge.net/lists/listinfo/mixxx-devel
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>
>
> ------------------------------------------------------------------------------
> >>>>> This SF email is sponsosred by:
> >>>>> Try Windows Azure free for 90 days Click Here
> >>>>> http://p.sf.net/sfu/sfd2d-msazure
> >>>>> _______________________________________________
> >>>>> Get Mixxx, the #1 Free MP3 DJ Mixing software Today
> >>>>> http://mixxx.org
> >>>>>
> >>>>>
> >>>>> Mixxx-devel mailing list
> >>>>> Mixxx-devel@lists.sourceforge.net
> >>>>> https://lists.sourceforge.net/lists/listinfo/mixxx-devel
> >>>>
> >>>>
> >>>>
> >>
>
> ------------------------------------------------------------------------------
> >>>> This SF email is sponsosred by:
> >>>> Try Windows Azure free for 90 days Click Here
> >>>> http://p.sf.net/sfu/sfd2d-msazure
> >>>> _______________________________________________
> >>>> Get Mixxx, the #1 Free MP3 DJ Mixing software Today
> >>>> http://mixxx.org
> >>>>
> >>>>
> >>>> Mixxx-devel mailing list
> >>>> Mixxx-devel@lists.sourceforge.net
> >>>> https://lists.sourceforge.net/lists/listinfo/mixxx-devel
> >>>
> >>>
> >>
>
> ------------------------------------------------------------------------------
> >>> This SF email is sponsosred by:
> >>> Try Windows Azure free for 90 days Click Here
> >>> http://p.sf.net/sfu/sfd2d-msazure
> >>> _______________________________________________
> >>> Get Mixxx, the #1 Free MP3 DJ Mixing software Today
> >>> http://mixxx.org
> >>>
> >>>
> >>> Mixxx-devel mailing list
> >>> Mixxx-devel@lists.sourceforge.net
> >>> https://lists.sourceforge.net/lists/listinfo/mixxx-devel
> >>
> >>
> >>
> >>
>
> ------------------------------------------------------------------------------
> >> This SF email is sponsosred by:
> >> Try Windows Azure free for 90 days Click Here
> >> http://p.sf.net/sfu/sfd2d-msazure
> >> _______________________________________________
> >> Get Mixxx, the #1 Free MP3 DJ Mixing software Today
> >> http://mixxx.org
> >>
> >>
> >> Mixxx-devel mailing list
> >> Mixxx-devel@lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/mixxx-devel
> >>
> >>
> >>
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > This SF email is sponsosred by:
> > Try Windows Azure free for 90 days Click Here
> > http://p.sf.net/sfu/sfd2d-msazure
> > _______________________________________________
> > Get Mixxx, the #1 Free MP3 DJ Mixing software Today
> > http://mixxx.org
> >
> >
> > Mixxx-devel mailing list
> > Mixxx-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/mixxx-devel
>
>
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> Get Mixxx, the #1 Free MP3 DJ Mixing software Today
> http://mixxx.org
>
>
> Mixxx-devel mailing list
> Mixxx-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/mixxx-devel
>
------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure
_______________________________________________
Get Mixxx, the #1 Free MP3 DJ Mixing software Today
http://mixxx.org
Mixxx-devel mailing list
Mixxx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mixxx-devel
