I'm curious what other OS/file systems have this capability natively...it's an interesting, though edge-case, optimization. A quick Google only turns up Lustre http://ieeexplore.ieee.org/iel5/4215348/4215349/04215390.pdf?isnumber=42 15349&prod=CNF&arnumber=4215390&arSt=267&ared=274&arAuthor=Yu%2C+Weikuan %3B+Vetter%2C+Jeffrey%3B+Canon%2C+R.+Shane%3B+Jiang%2C+Song as having "..an innovative file joining feature that joins files in place...".
That being said, implicit in your question is that the different clients are *already* writing to a single networked file system. Why not just have them all write to a single file to begin with? Since they are contiguous blocks in a known order, it'd seem you could get away with just having them start at different offsets. I believe that should all be doable with fairly simple managed or p/invoked code. Alternatively, if you can hook into the OS on the reading box - you can fake it without requiring source changes to the reading app. If you have that possibility, I'd start by looking at (probably in this order): 1. WinFUSE http://www.suchwerk.net/sodcms_FUSE_for_WINDOWS.htm which purports to have a userland file system in managed code 2. Windows File System Filter Drivers http://www.microsoft.com/whdc/driver/filterdrv/default.mspx 3. Windows Installable File System Kit http://www.microsoft.com/whdc/devtools/ifskit/default.mspx Your goal there would just be to intercept a request for "bigfile001.dat" and read "file001-1.dat", "file001-2.dat", etc. --Mark Brackett > -----Original Message----- > From: Discussion of advanced .NET topics. [mailto:ADVANCED- > [EMAIL PROTECTED] On Behalf Of Michael Sharpe > Sent: Tuesday, February 05, 2008 12:24 PM > To: ADVANCED-DOTNET@DISCUSS.DEVELOP.COM > Subject: Re: [ADVANCED-DOTNET] Join/Merge multiple files together > > Sadly, no. We have no control over the consumer application of the > required data file. It cannot accept the data in chunks or pieces. > It > cannot accept a data stream either so providing it data on the fly is > out > of the question. It can only accept it as a single data file. > Otherwise > we would have pushed for this a long time ago.... > > It seems like this is quite a difficult problem to address. Maybe > Windows > is not the best platform to be running for our needs. We have seen > other > O/S that can do this very easily. > > > > > > John Brett <[EMAIL PROTECTED]> > Sent by: "Discussion of advanced .NET topics." > <ADVANCED-DOTNET@DISCUSS.DEVELOP.COM> > 02/05/2008 11:02 AM > Please respond to > "Discussion of advanced .NET topics." > <ADVANCED-DOTNET@DISCUSS.DEVELOP.COM> > > > To > ADVANCED-DOTNET@DISCUSS.DEVELOP.COM > cc > > Subject > Re: Join/Merge multiple files together > > > > > > > > Is it at all possible to combine 2 (or more) data files together > without > > A) opening the data files to read and B) creating a "new" file out of > > them? > > Can you change the problem to make it easier to solve? > Depending upon what you need to do with the end-product, can you change > the reading application to use an index or other mechanism to indicate > the > set of files to read? Can you create a file-reader shim that > aggregates the files on the fly whilst reading? Can you change the > application that generates these files to append to a single file? > Just trying to understand why you have to have exactly one file to > work with, since that seems to be causing you difficulties. > > John > > =================================== > This list is hosted by DevelopMentor(r) http://www.develop.com > > View archives and manage your subscription(s) at > http://discuss.develop.com > > > =================================== > This list is hosted by DevelopMentor(r) http://www.develop.com > > View archives and manage your subscription(s) at > http://discuss.develop.com =================================== This list is hosted by DevelopMentor® http://www.develop.com View archives and manage your subscription(s) at http://discuss.develop.com
