Why not just add a menu option "sync metadata" that disables playback
and warns you of the consequences of having files open in other
programs before writing the metadata?
Ewan.
On Apr 1, 2012 8:41 AM, "Ilkka Tuohela" <ilkka.tuoh...@gmail.com
<mailto:ilkka.tuoh...@gmail.com>> wrote:
Not even file access or locking, but modifying a file mapped with
mmap on the fly.
BTW this raises another issue: if any other program outside mixxx
modifies file tags on disk while the track is loaded to mixxx,
same problem with file mmap garbage can happen, if the mmap is
directly to on-disk file! There is no way to prevent other
programs from modifying a file while mixxx is playing.
Of course, if mixxx does not mmap the raw file directly, but
decodes the file to a buffer and plays from that, the mmap issue
with tags should not happen in the first place.
I think what some other programs do here is to:
- open original file for reading, feed it to decoder
- write/copy the decoded raw wav data to a static memory buffer
(one for each deck)
- close the original file
- play the original wav file completely from memory buffer,
ignoring any changes to file on disk
In deckadance this has raised another known bug, since they have
artificially limited the memory buffer to quite small value by
modern standards and can only load files less than about 12
minutes long (64MB buffer per track?). If we would go this way, I
would prefer a user customizable maximum size for track read
buffer, my laptops have 4-16GB of memory and I don't mind giving
the DJ program half of it so let me have 1GB read buffer for each
deck if I want to allocate the memory!
I completely understand why you would prefer mmap, but the issue
raised here can't be avoided while using mmap on the file on disk.
On 1 Apr 2012, at 06:00, Scott wrote:
> My mistake, I just looked at #728197 and if I understand
correctly it is
> a file access / lock issue.
>
> On Sat, 2012-03-31 at 22:46 -0400, RJ Ryan wrote:
>> We actually use TagLib for all of our metadata processing needs
-- we
>> didn't roll metadata writing ourselves or anything like that.
>>
>>
>> On Sat, Mar 31, 2012 at 10:18 PM, Scott
<nappingcrac...@gmail.com <mailto:nappingcrac...@gmail.com>>
>> wrote:
>> This sounds a bit like NIH (not invented here) syndrome.
>>
>> I suggest using id3lib to do the actual file
manipulation and
>> simply
>> hook Mixxx's front end into that API. This would work for at
>> least ID3v1
>> and ID3v2 tags.
>>
>> id3lib http://id3lib.sourceforge.net/ has been around for a
>> while and is
>> used by many tagger / player programs. EasyTAG uses it, and
>> I've used
>> that program without problems for years on tens of thousands
>> of files at
>> a time.
>>
>> This would leave the quality control to a group focused
>> specifically on
>> manipulating audio file tags and metadata.
>>
>> -Scott
>>
>> On Sat, 2012-03-31 at 22:40 +0100, Adam Davison wrote:
>>> Hi,
>>>
>>> I think Albert's general point was more that if you have any
>> memory
>>> corruption anywhere in Mixxx, then the results of our write
>> to the
>>> file are undefined. Even if we fix this specific memory
>> corruption bug
>>> then we have no way to guarantee there aren't more lurking.
>> In fact I
>>> could probably guarantee you that there are memory
>> corruption bugs
>>> lurking in code the size of Mixxx + all the libraries living
>> in our
>>> process space.
>>>
>>> One safe way to do it perhaps would be to have a very small
>> helper
>>> program which handled the writing in a different process.
>>>
>>> Adam
>>>
>>> On 31 March 2012 18:54, Ilkka Tuohela
>> <ilkka.tuoh...@gmail.com <mailto:ilkka.tuoh...@gmail.com>> wrote:
>>>>
>>>> Maybe I'm asking the obvious, but why wouldn't following
>> process work just fine:
>>>>
>>>> - Store new tags to database fields
>>>> - Make a copy of the file on disk to temporary file,
>> original being still mmaped or not doesn't matter
>>>> - Write tag changes to the temporary file
>>>> - If writing temporary file was successful (disk space was
>> enough etc), unlink old file and rename new file to original
>> name
>>>> - If writing temporary file was not successful, remove
>> temporary file and report error, maybe revert database
fields?
>>>>
>>>> At least on unix-like systems, the mmap will hold a
>> reference pointing to the original deleted file, and the
>> changes to tags can't affect it, preventing corruption. This
>> will use extra bytes from disk for all rewritten files
(mmaped
>> bytes + new file with new tags) as long as the mmap FD is
>> open, but since we don't hold hundreds of files open in
>> player, the extra disk space used should not be an issue and
>> is released anyway on fs level when the FD is closed by
mixxx
>> (song has been played).
>>>>
>>>> Additionally, if we wish to show the updated tags in
>> decks, I think the tags should be and are based on database
>> entries not file tags and should reflect the current info
>> correctly (as long as there is a slot to update the
details in
>> player on the fly).
>>>>
>>>> I have no idea if this works similarly on windows
>> filesystems though, but I think so.
>>>>
>>>> *hile*
>>>>
>>>> On 31 Mar 2012, at 18:13, Owen Williams wrote:
>>>>
>>>>> I did develop an initial crappy workaround that prevents
>> many of the
>>>>> memory-corruption situations. Basically, if Mixxx can
>> know for sure
>>>>> that the music file is not open, it's safe to update the
>> metadata. The
>>>>> trick is figuring out if in fact that file is open or
>> not.
>>>>>
>>>>> I don't believe there's any risk in corrupting the file
>> on disk. The
>>>>> corruption I experienced was only in terms of playback --
>> still a big
>>>>> problem because it would cause static during sets.
>>>>>
>>>>> Mixxx is a nice interface for tag editing, though, so it
>> might be worth
>>>>> pushing up the importance of that bug.
>>>>>
>>>>> (one solution might be queuing the metadata update in the
>> file reader
>>>>> object itself, and then writing the data in the
>> destructor. Is it safe
>>>>> to assume only one file object per file, though?)
>>>>>
>>>>> Owen
>>>>>
>>>>>
>>>>> On Sat, 2012-03-31 at 10:38 -0400, Albert Santoni wrote:
>>>>>> Hi Keith,
>>>>>>
>>>>>> I'm sorry you had a crappy experience with this - Our
>> intention was to
>>>>>> reduce the risk of data loss.... which is the opposite
>> of what
>>>>>> happened to you.
>>>>>>
>>>>>> Mixxx did not write write metadata to any music files
>> until 1.9
>>>>>> because we were concerned that we didn't have the
>> quality control in
>>>>>> place to ensure we would never ship a library-destroying
>> bug. Just
>>>>>> imagine if we had a bug that caused your MP3s to get
>> corrupted...
>>>>>>
>>>>>> And it turns out this fear wasn't totally unfounded.
>> After 1.9
>>>>>> shipped, Owen tracked down a subtle memory corruption
>> bug related to
>>>>>> Mixxx writing to MP3s, and although there were no
>> reports of corrupt
>>>>>> files, file writing and memory corruption are not a
>> great combination.
>>>>>> (You might be able to make an argument that Mixxx is
>> more susceptible
>>>>>> to heap corruption programming errors which may not
>> affect the actual
>>>>>> file writing itself if it's done on the stack. I'm sure
>> the
>>>>>> TrackInfoObjects are kept on the heap though, so maybe
>> worst case you
>>>>>> could just end up with corrupt metadata. But never say
>> never...)
>>>>>>
>>>>>> As Daniel pointed, it was due to bug #728197 that we
>> disabled metadata
>>>>>> writing for Mixxx 1.10.0 and made a quick 1.9.2 release.
>>>>>>
>>>>>> If you have any suggestions on how we can communicate
>> that metadata
>>>>>> isn't written or want to work on #728197, you know how
>> to get in
>>>>>> touch... :)
>>>>>>
>>>>>> Thanks,
>>>>>> Albert
>>>>>>
>>>>>>
>>>>>> On Sat, Mar 31, 2012 at 4:32 AM,
>> keithsalisb...@gmail.com <mailto:keithsalisb...@gmail.com>
>>>>>> <keithsalisb...@gmail.com
<mailto:keithsalisb...@gmail.com>> wrote:
>>>>>>> Sorry guys, normally I wouldn't say this, but something
>> happened today
>>>>>>> which I really don't understand why this choice has
>> been made, maybe
>>>>>>> someone can explain.
>>>>>>>
>>>>>>> So in the last week or so, I've spent many hours
>> tweaking the metadata
>>>>>>> in my dj library, sorting out track titles, artists,
>> labels, keys,
>>>>>>> bpms all sorts of goodness which makes a record box
>> more than just a
>>>>>>> bunch of files.
>>>>>>>
>>>>>>> It was my assumption all this goodness was being baked
>> into the files.
>>>>>>>
>>>>>>> Yesterday as part of building a new version I moved
>> my .mixxx config,
>>>>>>> and associated database etc out the way to create a new
>> one.
>>>>>>>
>>>>>>> And here it is, to my surprise and quite honestly my
>> shock, my files
>>>>>>> we ALL back to their tatty messed up disorganised state
>> - ALL my time
>>>>>>> and effort had been reverted.
>>>>>>>
>>>>>>> Now you'll notice I was smart enough to backup my data
>> before creating
>>>>>>> a new database, so I've not really lost anything,
>> however I AM upset
>>>>>>> that this happened. I would expect more from Mixxx. I
>> actually trusted
>>>>>>> Mixxx more than Rhythmbox or those other media browsers
>> to do the
>>>>>>> right thing by my files.
>>>>>>>
>>>>>>> So what's the deal here - I can understand a concern
>> allowing users to
>>>>>>> edit their files - sure - but there should be a
>> config/preferences to
>>>>>>> allow me to choose. And more over, I should be made
>> aware that all
>>>>>>> this goodness is not really in my files. Somehow.
>>>>>>>
>>>>>>> My point is, I'm a power user, but I'm sure many users
>> out there have
>>>>>>> edited their files believing they're updating the
>> files, and only when
>>>>>>> they change computers, or rebuild their computer or
>> whatever, that
>>>>>>> they find out all that data is lost.
>>>>>>>
>>>>>>> Thoughts?
>>>>>>>
>>>>>>> Keith
>>>>>>>
>>>>>>>
>>
------------------------------------------------------------------------------
>>>>>>> This SF email is sponsosred by:
>>>>>>> Try Windows Azure free for 90 days Click Here
>>>>>>> http://p.sf.net/sfu/sfd2d-msazure
>>>>>>> _______________________________________________
>>>>>>> Get Mixxx, the #1 Free MP3 DJ Mixing software Today
>>>>>>> http://mixxx.org
>>>>>>>
>>>>>>>
>>>>>>> Mixxx-devel mailing list
>>>>>>> Mixxx-devel@lists.sourceforge.net
<mailto:Mixxx-devel@lists.sourceforge.net>
>>>>>>>
>> https://lists.sourceforge.net/lists/listinfo/mixxx-devel
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>
------------------------------------------------------------------------------
>>>>> This SF email is sponsosred by:
>>>>> Try Windows Azure free for 90 days Click Here
>>>>> http://p.sf.net/sfu/sfd2d-msazure
>>>>> _______________________________________________
>>>>> Get Mixxx, the #1 Free MP3 DJ Mixing software Today
>>>>> http://mixxx.org
>>>>>
>>>>>
>>>>> Mixxx-devel mailing list
>>>>> Mixxx-devel@lists.sourceforge.net
<mailto:Mixxx-devel@lists.sourceforge.net>
>>>>> https://lists.sourceforge.net/lists/listinfo/mixxx-devel
>>>>
>>>>
>>>>
>>
------------------------------------------------------------------------------
>>>> This SF email is sponsosred by:
>>>> Try Windows Azure free for 90 days Click Here
>>>> http://p.sf.net/sfu/sfd2d-msazure
>>>> _______________________________________________
>>>> Get Mixxx, the #1 Free MP3 DJ Mixing software Today
>>>> http://mixxx.org
>>>>
>>>>
>>>> Mixxx-devel mailing list
>>>> Mixxx-devel@lists.sourceforge.net
<mailto:Mixxx-devel@lists.sourceforge.net>
>>>> https://lists.sourceforge.net/lists/listinfo/mixxx-devel
>>>
>>>
>>
------------------------------------------------------------------------------
>>> This SF email is sponsosred by:
>>> Try Windows Azure free for 90 days Click Here
>>> http://p.sf.net/sfu/sfd2d-msazure
>>> _______________________________________________
>>> Get Mixxx, the #1 Free MP3 DJ Mixing software Today
>>> http://mixxx.org
>>>
>>>
>>> Mixxx-devel mailing list
>>> Mixxx-devel@lists.sourceforge.net
<mailto:Mixxx-devel@lists.sourceforge.net>
>>> https://lists.sourceforge.net/lists/listinfo/mixxx-devel
>>
>>
>>
>>
------------------------------------------------------------------------------
>> This SF email is sponsosred by:
>> Try Windows Azure free for 90 days Click Here
>> http://p.sf.net/sfu/sfd2d-msazure
>> _______________________________________________
>> Get Mixxx, the #1 Free MP3 DJ Mixing software Today
>> http://mixxx.org
>>
>>
>> Mixxx-devel mailing list
>> Mixxx-devel@lists.sourceforge.net
<mailto:Mixxx-devel@lists.sourceforge.net>
>> https://lists.sourceforge.net/lists/listinfo/mixxx-devel
>>
>>
>>
>
>
>
>
------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> Get Mixxx, the #1 Free MP3 DJ Mixing software Today
> http://mixxx.org
>
>
> Mixxx-devel mailing list
> Mixxx-devel@lists.sourceforge.net
<mailto:Mixxx-devel@lists.sourceforge.net>
> https://lists.sourceforge.net/lists/listinfo/mixxx-devel
------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure
_______________________________________________
Get Mixxx, the #1 Free MP3 DJ Mixing software Today
http://mixxx.org
Mixxx-devel mailing list
Mixxx-devel@lists.sourceforge.net
<mailto:Mixxx-devel@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/mixxx-devel
------------------------------------------------------------------------------
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure
_______________________________________________
Get Mixxx, the #1 Free MP3 DJ Mixing software Today
http://mixxx.org
Mixxx-devel mailing list
Mixxx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mixxx-devel
