On Sun, Jul 19, 2015 at 11:47 AM, Samer Afach <[email protected]> wrote:
> Dear Werner: > > Thanks for the suggestions. I'll keep these suggestions as last resort > because my program is cross-platform, so I'm trying to avoid such low-level > functions. > > I'm thinking now of using an md5 hash of the file instead of the size. Do > you think that would suffice and work? The only problem with that is that > it could be a little expensive, but that's still OK if that's the only > problem because the files aren't really that big (a few megabytes). > If might be more efficient to use a file synchronization system, or maybe just run rsync periodically. > > All the best, > Samer > > On 19.07.2015 16:37, Werner Benger wrote: > > Hi Samer, > > > if those applications are linking dynamically with HDF5, you could > interface there by providing a "patched" version of HDF5 that is binary > compatible, but still performs some file locking, possibly via a modified > VFD. > > Just checking the file sizes won't be a sure check, in particular it would > not work for and application that does not append data but modifies > existing ones. You should be able to check the "last modified" date on the > file as well, and if the HDF5 file cannot be opened by another application, > then it's either corrupted or still open and data written to it. > > It should also be possible to find out which process has file handles to a > specific file open, both Linux and Windows seem to allow that, just in an > OS-specific way: > > > http://stackoverflow.com/questions/6583158/finding-open-file-descriptors-for-a-process-linux-c-code > > https://technet.microsoft.com/en-us/sysinternals/bb896653.aspx > > It may require your program to run under admin rights though. > > Cheers, > Werner > > > On 19.07.2015 12:32, Samer Afach wrote: > > Dear Werner: > > Unfortunately I don't have the source of the writing program. It could be > anything actually; labview, C, C++, etc... . I'm working on a collaborative > program for collecting data, and my software is supposed to upload the data > to the main storage server for backup and data analysis. Everyone is free > to use any tools they want for data acquisition. > > Is there any solution to my problem that doesn't involve me having > authority on the writing program? Currently I'm using a method that I > invented, where I measure the size of the file multiple times, and if the > size doesn't change for some time, then that decide that the file isn't > being written... that's the only way I could come up with, but this has > nothing to do with HDF5. > > All the best, > Samer > > On 19.07.2015 12:23, Werner Benger wrote: > > Hi Samer, > > do you have full source code access to both programs, the one writing the > file and the one uploading it? > > If so, would adding some file locking mechanism ( > <https://en.wikipedia.org/wiki/File_locking> > https://en.wikipedia.org/wiki/File_locking ) help your case? > > > Cheers, > Werner > > > On 19.07.2015 02:50, Samer Afach wrote: > > Dear Werner: > > Thank you for your response and your time. > > I tried modifying the source code of h5check, but it's very complicated > and returns strings instead of error codes. Besides, I tried to compile it > by including the three files in the main c file and it gave errors that > don't seem to be easily trackable. It'll take me lots of time to prepare > something reliable out of it with full testing. I wonder why the hdf group > didn't implement such a functionality in low language, I thought it exists > and I can't find it. :( > > About my second question, apparently I wasn't clear on what I really need > and I was misunderstood. What I need is to check whether the file I want to > read is being accessed by some other program. My problem is that the file I > wanna deal with is being written by some other program, and I want to > initiate an upload after it's done writing. So I would like to check > whether the other program is finished writing and closed it before I send > it to my upload queue. > > All the best, > Samer > > On 18.07.2015 23:11, Werner Benger wrote: > > Hi Samer, > > On 18.07.2015 16:23, Samer Afach wrote: > > Dear pros: > > I have two similar issues I would like to ask about, and I'd be grateful > if you could help me: > > 1- How can I check HDF5 file integrity in C/C++ (I prefer low level C)? I > know there's a software called h5check, but I need a function to do that, > not a software. In my application, it's not an option to execute h5check > from my system. > > Is there any objection against you taking the source code of h5check, > rename the main function to h5check_main() and then call it as C function > from your code? > > > 2- How can I do a quick check in C/C++ to see whether the file is being > written/modified/open for write? I looked in this link > <https://www.hdfgroup.org/HDF5/doc/H5.format.html#FileMetaData> which > discusses metadata, but there doesn't seem to be a clear way through the > C/C++ interface for me a to read the first two bits that give me that > information. Could you please help with that with a simple example? > > This sounds as if you want to know what the HDF5 library is currently > doing with a file? Well what you could do is to modify the virtual file > driver that you want to use, and let it keep track of that happens with the > file. It's the VFD that does the actual writing of a file content, so you > can trace its write() call to keep track what exactly happens when. It's > some effort of course. > > Cheers, > Werner > > > > > All the best, > Samer > > > _______________________________________________ > Hdf-forum is for HDF software users > [email protected]http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 > > > -- > ___________________________________________________________________________ > Dr. Werner Benger Visualization Research > Center for Computation & Technology at Louisiana State University (CCT/LSU) > 2019 Digital Media Center, Baton Rouge, Louisiana 70803 > Tel.: +1 225 578 4809 Fax.: +1 225 578-5362 > > > > _______________________________________________ > Hdf-forum is for HDF software users > [email protected]http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 > > > > > _______________________________________________ > Hdf-forum is for HDF software users > [email protected]http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 > > > -- > ___________________________________________________________________________ > Dr. Werner Benger Visualization Research > Center for Computation & Technology at Louisiana State University (CCT/LSU) > 2019 Digital Media Center, Baton Rouge, Louisiana 70803 > Tel.: +1 225 578 4809 Fax.: +1 225 578-5362 > > > > _______________________________________________ > Hdf-forum is for HDF software users > [email protected]http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 > > > > > _______________________________________________ > Hdf-forum is for HDF software users > [email protected]http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 > > > -- > ___________________________________________________________________________ > Dr. Werner Benger Visualization Research > Center for Computation & Technology at Louisiana State University (CCT/LSU) > 2019 Digital Media Center, Baton Rouge, Louisiana 70803 > Tel.: +1 225 578 4809 Fax.: +1 225 578-5362 > > > > _______________________________________________ > Hdf-forum is for HDF software users > [email protected]http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 > > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org > Twitter: https://twitter.com/hdf5 > -- George N. White III <[email protected]> Head of St. Margarets Bay, Nova Scotia
_______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org Twitter: https://twitter.com/hdf5
