Re: [Haskell-cafe] Re: Darcs and NFS Resolution
On Sat, 2009-09-12 at 23:24 +0100, Ganesh Sittampalam wrote: Darcs already has a WIN32-specific workaround for renaming going wrong when the new file exists, and my initial guess was that was what was going wrong here. BTW, this is not necessary afaik. Rename over an existing file works just fine on Windows. Cabal uses it in its implementation of writeFileAtomic and that certainly works for writing existing files, indeed that is its raison d'être. The only time it goes wrong is if another process has the target file open, and in that case removing the target will fail too. Duncan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Re: Darcs and NFS Resolution
[Due to circumstances beyond my control, I cannot CC the OP. Sorry.] Jason Dagit da...@codersbase.com writes: [Darcs] assumes that when your OS is [POSIX], all your mounted volumes will natively support POSIX. Some CIFS servers (namely, Samba) *do* implement POSIX semantics. I take it to OP is using a Microsoft CIFS server? Is this the same problem as http://wiki.darcs.net/sshfs ? renameFile :: FilePath - FilePath - IO () renameFile old new = -- [...] System.Cmd.rawSystem mv [old,new] -- [...] which ensures that when the operating system is not WIN32, that renaming of files will be performed by the OS shell. Ew. I'm not keen on calling mv(1) to handle each rename, let alone via sh (which WILL explode on some paths, and allow injection attacks). I'm also puzzled as to why this works -- surely mv(1) assumes POSIX semantics, too? I would be interested in seeing the exact error transcript, preferably as an issue on bugs.d.n. I'm not sure the problem has been diagnosed correctly. Shelling out is clearly a poor long term solution; a longer term solution would ideally introduce into the Haskell System.Directory library the ability to apply the correct functions transparently in accordance with the file system of the volume. I guess someone should file a bug with them (in GHC's trac?). Yes, and in the meantime we can implement something like this in workaround.hs. I definitely think this SHOULD NOT be enabled by default, unless you're going to ONLY enable it in the specific scenario of CIFS on Linux. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Darcs and NFS Resolution
On Sat, Sep 12, 2009 at 5:22 PM, Trent W. Buck trentb...@gmail.com wrote: Ew. I'm not keen on calling mv(1) to handle each rename, let alone via sh (which WILL explode on some paths, and allow injection attacks). rawSystem does not use sh (hence the raw) --Max ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Re: Darcs and NFS Resolution
trentb...@gmail.com (Trent W. Buck) writes: I'm also puzzled as to why this works -- surely mv(1) assumes POSIX semantics, too? I would be interested in seeing the exact error transcript, preferably as an issue on bugs.d.n. I'm not sure the problem has been diagnosed correctly. Well, not really: mv will try harder than just calling rename(2). E.g. mv also works across mount points, where it'll go as far as doing a copy and unlink. I definitely think this SHOULD NOT be enabled by default, unless you're going to ONLY enable it in the specific scenario of CIFS on Linux. Well, we can always catch the rename(2) error and try unlinking and renaming again (carefully, unlike current Workaround's renameFile which just does that on any error). Yours, Petr. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Darcs and NFS Resolution
On Sep 12, 2009, at 11:22 , Trent W. Buck wrote: Jason Dagit da...@codersbase.com writes: which ensures that when the operating system is not WIN32, that renaming of files will be performed by the OS shell. I'm also puzzled as to why this works -- surely mv(1) assumes POSIX semantics, too? I would be interested in seeing the exact error transcript, preferably as an issue on bugs.d.n. I'm not sure the problem has been diagnosed correctly. In order to handle the case where you're moving across filesystems, mv(1) gracefully degrades to cp + rm. rename(2) does not. This also happens to work around compatibility issues with native CIFS (and possibly older HP/UX, not that anyone likely cares). Yes, and in the meantime we can implement something like this in workaround.hs. I definitely think this SHOULD NOT be enabled by default, unless you're going to ONLY enable it in the specific scenario of CIFS on Linux. I definitely think you've made incorrect assumptions; the proposed workaround is reasonable even if not ideal. (I would fall back to the mv(1) method only if rename(2) fails.) -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allb...@kf8nh.com system administrator [openafs,heimdal,too many hats] allb...@ece.cmu.edu electrical and computer engineering, carnegie mellon universityKF8NH PGP.sig Description: This is a digitally signed message part ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Darcs and NFS Resolution
On Sat, 12 Sep 2009, Brandon S. Allbery KF8NH wrote: On Sep 12, 2009, at 11:22 , Trent W. Buck wrote: Jason Dagit da...@codersbase.com writes: which ensures that when the operating system is not WIN32, that renaming of files will be performed by the OS shell. I'm also puzzled as to why this works -- surely mv(1) assumes POSIX semantics, too? I would be interested in seeing the exact error transcript, preferably as an issue on bugs.d.n. I'm not sure the problem has been diagnosed correctly. In order to handle the case where you're moving across filesystems, mv(1) gracefully degrades to cp + rm. rename(2) does not. This also happens to work around compatibility issues with native CIFS (and possibly older HP/UX, not that anyone likely cares). I don't think that darcs is ever likely to want to do a move across filesystems - unless someone has actually put a mount point in the middle of their darcs repo (and perhaps not even then for the metadata operations such as the one that was failing here, as I think those are all inside a single directory). Darcs already has a WIN32-specific workaround for renaming going wrong when the new file exists, and my initial guess was that was what was going wrong here. The workaround just tries to remove the new file and retries the rename. The original poster doesn't make clear whether he tried my suggested fix enabling that workaround unconditionally before resorting to shelling out. Cheers, Ganesh ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Darcs and NFS Resolution
On Sep 12, 2009, at 18:24 , Ganesh Sittampalam wrote: On Sat, 12 Sep 2009, Brandon S. Allbery KF8NH wrote: In order to handle the case where you're moving across filesystems, mv(1) gracefully degrades to cp + rm. rename(2) does not. This also happens to work around compatibility issues with native CIFS (and possibly older HP/UX, not that anyone likely cares). I don't think that darcs is ever likely to want to do a move across filesystems - unless I'm not claiming it does, only explaining why mv(1) is *not* identical to rename(2) as claimed. It also happens to be true that it will do something instead of failing if the destination exists and is a directory (but possibly not what you want). -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allb...@kf8nh.com system administrator [openafs,heimdal,too many hats] allb...@ece.cmu.edu electrical and computer engineering, carnegie mellon universityKF8NH PGP.sig Description: This is a digitally signed message part ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] RE: Darcs and NFS Resolution
Thanks to everyone who pointed me in the right direction on this problem, I have been able to find a work around that allows me to push to an archive on a network file share. After some digging, I discovered the root of the problem. Briefly, Darcs uses the standard library System.Directory to perform file manipulations. This library make use of compiler pragmas to conditionally compile either the Windows or Linux specific functions based on the host operating system. This approach assumes that when your operating system is a Linux variant, all your mounted volumes will natively support POSIX. When you have mounted CIFS volumes, this will result in errors when calling those library functions to create, rename, copy and delete files or folders! Given the flexibility of the Linux operating system, A more versatile implementation for System.Directory might be able to detect the file system of each volume at runtime and choose the appropriate functions to call. But I digress... To workaround the inability to push to a CIFS volume in darcs, I modified the workaround.hs module to shell out to the OS's rename function rather than using the Haskell library's rename function. Specifically: I added the following code after the #else near line 80: renameFile :: FilePath - FilePath - IO () renameFile old new = do ecode - System.Cmd.rawSystem mv [old,new] case ecode of ExitSuccess - return () ExitFailure n - putStrLn (renameFile failed with exit code ++ show n) which ensures that when the operating system is not WIN32, that renaming of files will be performed by the OS shell. I then added the System.Cmd module to the list of imports by inserting the following code near line 21 import qualified System.Cmd(rawSystem) after a recompile I could push to a CIFS volume, for example: sudo darcs push /mnt/cifsvol this is an obvious hack, and does not address the inability to put to a CIFS share (put depends upon copyFile and would need to be hacked to shell-out as well). Archives on the CIFS share have to be created by navigating to that folder and initializing. Shelling out is clearly a poor long term solution; a longer term solution would ideally introduce into the Haskell System.Directory library the ability to apply the correct functions transparently in accordance with the file system of the volume. Cheers -Darrell Lewis-Sandy ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] RE: Darcs and NFS Resolution
Hi Darrell, This message definitely also belongs on the darcs-us...@darcs.net mailing list. You can find information about how to subscribe here: http://lists.osuosl.org/mailman/listinfo/darcs-users Please join our list! I'm adding the list to the CC list in my reply. On Fri, Sep 11, 2009 at 9:20 AM, Lewis-Sandy, Darrell darre...@amgen.comwrote: Thanks to everyone who pointed me in the right direction on this problem, I have been able to find a work around that allows me to push to an archive on a network file share. After some digging, I discovered the root of the problem. Briefly, Darcs uses the standard library System.Directory to perform file manipulations. This library make use of compiler pragmas to conditionally compile either the Windows or Linux specific functions based on the host operating system. This approach assumes that when your operating system is a Linux variant, all your mounted volumes will natively support POSIX. When you have mounted CIFS volumes, this will result in errors when calling those library functions to create, rename, copy and delete files or folders! Given the flexibility of the Linux operating system, A more versatile implementation for System.Directory might be able to detect the file system of each volume at runtime and choose the appropriate functions to call. But I digress... To workaround the inability to push to a CIFS volume in darcs, I modified the workaround.hs module to shell out to the OS's rename function rather than using the Haskell library's rename function. Specifically: Ah clever way to test this. I added the following code after the #else near line 80: renameFile :: FilePath - FilePath - IO () renameFile old new = do ecode - System.Cmd.rawSystem mv [old,new] case ecode of ExitSuccess - return () ExitFailure n - putStrLn (renameFile failed with exit code ++ show n) which ensures that when the operating system is not WIN32, that renaming of files will be performed by the OS shell. I then added the System.Cmd module to the list of imports by inserting the following code near line 21 import qualified System.Cmd(rawSystem) after a recompile I could push to a CIFS volume, for example: sudo darcs push /mnt/cifsvol this is an obvious hack, and does not address the inability to put to a CIFS share (put depends upon copyFile and would need to be hacked to shell-out as well). Archives on the CIFS share have to be created by navigating to that folder and initializing. Interesting. Shelling out is clearly a poor long term solution; a longer term solution would ideally introduce into the Haskell System.Directory library the ability to apply the correct functions transparently in accordance with the file system of the volume. Yes, and in the meantime we can implement something like this in workaround.hs. Thanks! Jason ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
