I meant to respond sooner. To start, I am very confused in this statement: "What I was hoping to do was to split this so that data could go to two machines during the course of the experiment bearing in mind that the write CIFS folder is changing over the course of the two weeks continuously."
Please define: - Split the file (end after X bytes, create new file?) - "the write CIFS folder is changing" (is it changing the mapping/mount?) I'm curious what you actually "need" here -- just two systems reading the same file? Splitting the files across multiple systems? This is really a classic issue of the program and protocol, not any services. I run into this regularly with clustered nodes and file systems, they do not solve issues that can only be addressed at the program/client. I'm also curious if you're running up against 32-bit limitations in the SMB client (2GiB file size limitation), hence why you need to change the system (?). In the POSIX (UNIX/Linux) DAQ world, this is a classic case/use for buffers/pipes/sockets. Instead of writing to a file, you write to a special file, which is then acted upon or serviced by another program. So while one program is writing to what it thinks is a file, it's actually writing to a solution on the other end which can split the data up as it sees fit. This is very commonly used for continuous logging, but also for data when long batch jobs are the norm too. Although Microsoft raided SCO Xenix to add basic UNIX-like redirection and piping for MS-DOS 2.0, they did not bring over most of these features, much less NTFS and SMB do not support such. However, it might be interesting to see what would happen if you created such a file on the UNIX side that is shared out via SMB. I've never attempted such myself, as I don't know how SMB handles addressing blocks in such special inodes. -- Bryan P.S. I've done a lot of DAQ in my time, with Windows, Linux, VxWorks and a couple of UNIX flavors (including RT). I'm curious to know more about your instrument, drivers, etc... -- Bryan J Smith Professional, Technical Annoyance Linked Profile: http://www.linkedin.com/in/bjsmith ------------------------------------------------------------ "Now if you own an automatic ... sell it! You are totally missing out on the coolest part of driving" -- Johnny O'Connell ----- Original Message ---- From: Bryan Hepworth <[email protected]> To: "[email protected]" <[email protected]> Sent: Thu, December 30, 2010 8:11:17 PM Subject: [rhelv5-list] OT Question - Directory and File duplication Hi All I'm hoping someone may be able to point me to a specific utility for this problem. So far I haven't seen anything, so input is appreciated. I know this is totally off topic so just pointers is fine. Problem is as follows.. I have a machine that creates sequencing data over a typical 10 working days period of time. The machine writes to a data area and then uses this data over the course of the experiment to analyse and re-write files - we end up with a file around 0.5TB once it's all done. The instrument -Windows - needs a CIFS share which is fine, it writes to a samba share on a designated server. What I was hoping to do was to split this so that data could go to two machines during the course of the experiment bearing in mind that the write CIFS folder is changing over the course of the two weeks continuously. All suggestions would be most welcome. Bryan _______________________________________________ rhelv5-list mailing list [email protected] https://www.redhat.com/mailman/listinfo/rhelv5-list
