[perl #38146] [TODO] OS.pmc - file copy
The discussion in this ticket appears to have petered out in June 2006. The state of any efforts to work on it is unclear. This RT was mentioned by Whiteknight on his blog on July 17 2009: Likewise, Ticket #38146 discusses the creation of a file copying utility, although discussion there has since degraded into a general discussion about what's the best architecture to use for implementing these kinds of functions. Input on that discussion, or a solution to the problem, would both be appreciated (and moving the ticket from RT to Trac where it can be even more visible would be a big plus!) Since there may be people who participated in this thread who are not yet following Trac or parrot-dev, I'm posting this to the old list as well. Can anyone summarize where we stand? Thank you very much. kid51
Re: [perl #38146] [TODO] OS.pmc - file copy 38146
Leopold Toetsch wrote: On Jun 18, 2006, at 2:02, Vishal Soni via RT wrote: I am just wonedring if it would make sense to seperate out code for each supported operating system under a directory structure. At the time of build the specific code for target operating system is added to the source tree. Yep. Actually we already have and use the needed infrastructure for platform code. See also: - config/gen/platform* - src/platform.c (generated from above) Given that I started OS.pmc I was preparing to iterate it a little more soon. Meanwhile, I got a design doubt about creating different files for different operating systems. Basically, there are specific functions whose difference is just in the name of a function. Should we separate it anyway, or use a '#define' approach? I do not defent any of the approaches. Just want to know what to do. $ambs
Re: [perl #38146] [TODO] OS.pmc - file copy 38146
On Jun 18, 2006, at 2:02, Vishal Soni via RT wrote: I am just wonedring if it would make sense to seperate out code for each supported operating system under a directory structure. At the time of build the specific code for target operating system is added to the source tree. Yep. Actually we already have and use the needed infrastructure for platform code. See also: - config/gen/platform* - src/platform.c (generated from above) leo
Re: [perl #38146] [TODO] OS.pmc - file copy 38146
Hi Leo, So do we need to change os.pmc to leverage this infrastructure and get rid of the platform specific code( currently implemented via IFDEF) from os.pmc? -Vishal On 6/19/06, Leopold Toetsch [EMAIL PROTECTED] wrote: On Jun 18, 2006, at 2:02, Vishal Soni via RT wrote: I am just wonedring if it would make sense to seperate out code for each supported operating system under a directory structure. At the time of build the specific code for target operating system is added to the source tree. Yep. Actually we already have and use the needed infrastructure for platform code. See also: - config/gen/platform* - src/platform.c (generated from above) leo -- Thanks, Vishal
Re: [perl #38146] [TODO] OS.pmc - file copy 38146
On Jun 19, 2006, at 19:30, Vishal Soni wrote: So do we need to change os.pmc to leverage this infrastructure and get rid of the platform specific code( currently implemented via IFDEF) from os.pmc? I think that all platform-specific code should be factored out, i.e. the existing methods should call some platform functions. leo
Re: [perl #38146] [TODO] OS.pmc - file copy 38146
Hi Leo, That sounds great. One quick question would it make sense to start defining a generic platform interface that that all supported platforms need to implement. A quick example of a similar abstraction would be the Apache Portable Runtime (http://apr.apache.org/). Let me know what your thoughts are. -Vishal On 6/19/06, Leopold Toetsch [EMAIL PROTECTED] wrote: On Jun 19, 2006, at 19:30, Vishal Soni wrote: So do we need to change os.pmc to leverage this infrastructure and get rid of the platform specific code( currently implemented via IFDEF) from os.pmc? I think that all platform-specific code should be factored out, i.e. the existing methods should call some platform functions. leo -- Thanks, Vishal
[perl #38146] [TODO] OS.pmc - file copy 38146
Hi, I am trying implement #38146 todo item. While looking at the code for os.pmc there are IFDEF constructs defined for different operating systems (For e.g. WIN32 for now). I am just wonedring if it would make sense to seperate out code for each supported operating system under a directory structure. At the time of build the specific code for target operating system is added to the source tree. This very similar to the Linux source tree where the architecture specific code is implemented in a differen directory structure and correct directory structure is selected based on the architecture you are compiling for. There a couple of benifits that come to mind: 1. All the OS specific is maintained under seperated directory structure for each OS. 2. Defines a clean seperation b/w OS and Parrot Code. 3. Makes it easy to port Parrot to new platforms without breaking the implemented functionality for other platforms. Let me know what your thoguths are. Thanks, Vishal
[perl #38146] [TODO] OS.pmc - file copy 38146
Hi, I am trying implement #38146 todo item. While looking at the code for os.pmc there are IFDEF constructs defined for different operating systems (For e.g. WIN32 for now). I am just wonedring if it would make sense to seperate out code for each supported operating system under a directory structure. At the time of build the specific code for target operating system is added to the source tree. This very similar to the Linux source tree where the architecture specific code is implemented in a differen directory structure and correct directory structure is selected based on the architecture you are compiling for. There a couple of benifits that come to mind: 1. All the OS specific is maintained under seperated directory structure for each OS. 2. Defines a clean seperation b/w OS and Parrot Code. 3. Makes it easy to port Parrot to new platforms without breaking the implemented functionality for other platforms. Let me know what your thoguths are. Thanks, Vishal
Re: [perl #38146] [TODO] OS.pmc - file copy
I'm not implementing copy at the moment as I lack knowledge. I might
just write the default open/while(){write}/close method for cases when
everything else fails.
BTW, it will go for File.pmc accordingly with Leo.
Joshua Juran wrote:
On Jan 11, 2006, at 7:02 PM, Chip Salzenberg wrote:
On Wed, Jan 11, 2006 at 04:16:55PM -0500, Joshua Juran wrote:
Since before System 7 (approaching two decades ago), Mac OS has had a
system call that exchanges the contents of two files. The purpose of
this call is to implement a 'safe save' strategy ...
Is this still a system call in Mac OS X?
Yes, the original FSpExchangeFiles() call persists along with most of
the calls pertaining to FSSpecs. New code written only for Mac OS 9 and
above could also use the newer FSRef-based FSExchangeObjects() call
which subsumes it.
Josh
--
Alberto Simões - Departamento de Informática - Universidade do Minho
Campus de Gualtar - 4710-057 Braga - Portugal
Re: [perl #38146] [TODO] OS.pmc - file copy
On 1/12/06, Alberto Manuel Brandão Simões [EMAIL PROTECTED] wrote:
I'm not implementing copy at the moment as I lack knowledge. I might
just write the default open/while(){write}/close method for cases when
everything else fails.
BTW, it will go for File.pmc accordingly with Leo.
since there seem to be non-File-specific methods destined for the File
PMC, perhaps it could be renamed to something more appropriate, like
FileSystem. thoughts?
~jerry
Re: [perl #38146] [TODO] OS.pmc - file copy
At 18:03 03/01/2006 -0800, you wrote: # New Ticket Created by Will Coleda # Please include the string: [perl #38146] # in the subject line of all future correspondence about this issue. # URL: https://rt.perl.org/rt3/Ticket/Display.html?id=38146 OS.pmc should provide both a: and rename(oldname, newname) François. copy(source_file,target) And a copy(array_of_source_files,targetDir)
Re: [perl #38146] [TODO] OS.pmc - file copy
On Thu, Jan 12, 2006 at 06:27:11AM -0800, jerry gay wrote:
On 1/12/06, Alberto Manuel Brandão Simões [EMAIL PROTECTED] wrote:
I'm not implementing copy at the moment as I lack knowledge. I might
just write the default open/while(){write}/close method for cases when
everything else fails.
since there seem to be non-File-specific methods destined for the File
PMC, perhaps it could be renamed to something more appropriate, like
FileSystem. thoughts?
I think getting away from File is good. By analogy with OS, for
convenience, and in keeping with common usage, I like FS. Today,
anyway.
PS: If some class is named Filesystem someday, the S shouldn't be
capitalized: filesystem is an established single term in CS.
Filesys maybe as a short form?
--
Chip Salzenberg [EMAIL PROTECTED]
Re: [perl #38146] [TODO] OS.pmc - file copy
I'd vote for that being the default method that can be overridden on a
per platform basis with a more functional/efficient version.
-J
--
On Thu, Jan 12, 2006 at 12:07:33PM +, Alberto Manuel Brand?o Sim?es wrote:
I'm not implementing copy at the moment as I lack knowledge. I might
just write the default open/while(){write}/close method for cases when
everything else fails.
BTW, it will go for File.pmc accordingly with Leo.
Joshua Juran wrote:
On Jan 11, 2006, at 7:02 PM, Chip Salzenberg wrote:
On Wed, Jan 11, 2006 at 04:16:55PM -0500, Joshua Juran wrote:
Since before System 7 (approaching two decades ago), Mac OS has had a
system call that exchanges the contents of two files. The purpose of
this call is to implement a 'safe save' strategy ...
Is this still a system call in Mac OS X?
Yes, the original FSpExchangeFiles() call persists along with most of
the calls pertaining to FSSpecs. New code written only for Mac OS 9 and
above could also use the newer FSRef-based FSExchangeObjects() call
which subsumes it.
Josh
--
Alberto Sim?es - Departamento de Inform?tica - Universidade do Minho
Campus de Gualtar - 4710-057 Braga - Portugal
pgpiApFZRjbry.pgp
Description: PGP signature
Re: [perl #38146] [TODO] OS.pmc - file copy
An aside: I suspect that OS.pmc is going to grow beyond maintainability (and fast loading) in the manner of Perl's POSIX module. But for now, it's absolutely great that we're collecting each portability hack into a single implementation. On Sat, Jan 07, 2006 at 01:04:29PM -1000, Joshua Hoblitt wrote: I think it's prudent that OS.pmc's default copy() method behaves in the same way as File::Copy::copy() as this is likely the lowest possible common denominator across platforms (note that I said behavior, not implementation). This is sufficient for the majority of file copy tasks and will make prevent most people from having to worry about portability issues. More sophisticated behavior, like metadata replication, should be left to another method (perhaps syscopy()) that has platform specific behavior(s). My first thought was that if you're not going to copy any metadata at all, then you're not looking at an OS method. But then then I remembered that some operating systems might actually have high-efficiency fast tricks for copying (e.g. the way Linux is going with its splice() system call proposals). So OS is fine. However, instead of shying away from the metadata problem, we should embrace it. Define copying with metadata as a standard feature with a standard interface. Something like: metacopy($src, $dest, owner = 0, # optional to copy mtime = 1, # mandatory to copy atime = undef, # forbidden to copy ... ) This provides a portable way of expressing what you need and want. If the current operating system can't accomodate you, you'll get a failure or exception (not making any assumptions on that point). Hm. If you say 'contents = undef', you might get a copying of metadata _only_. Neat. Come to think of it, Perl 5 could use one of these. And somebody should check with the p6 guys to see what they think of it ... they might be in a mood to (re)specify filesystem manipulation. -- Chip Salzenberg [EMAIL PROTECTED]
Re: [perl #38146] [TODO] OS.pmc - file copy
On Jan 10, 2006, at 10:29 PM, Chip Salzenberg wrote: On Sat, Jan 07, 2006 at 01:04:29PM -1000, Joshua Hoblitt wrote: ... More sophisticated behavior, like metadata replication, should be left to another method (perhaps syscopy()) that has platform specific behavior(s). My first thought was that if you're not going to copy any metadata at all, then you're not looking at an OS method. But then then I remembered that some operating systems might actually have high-efficiency fast tricks for copying (e.g. the way Linux is going with its splice() system call proposals). So OS is fine. However, instead of shying away from the metadata problem, we should embrace it. Define copying with metadata as a standard feature with a standard interface. Something like: metacopy($src, $dest, owner = 0, # optional to copy mtime = 1, # mandatory to copy atime = undef, # forbidden to copy ... ) I suggest using -1 to mean forbidden, and any false value to mean optional. This provides a portable way of expressing what you need and want. If the current operating system can't accomodate you, you'll get a failure or exception (not making any assumptions on that point). Hm. If you say 'contents = undef', you might get a copying of metadata _only_. Neat. This reminds me of something that's been bothering me for a long time. Since before System 7 (approaching two decades ago), Mac OS has had a system call that exchanges the contents of two files. The purpose of this call is to implement a 'safe save' strategy: When the user saves a document, you create a temp file, write the new contents to it, exchange with the existing file, and delete the temp file (which now has the old contents). This is obviously better than just overwriting the original file, since there's no risk of crashing or running out of disk space leaving a truncated file. It's also better than using rename(), because in the end it's the same file -- except for the modification date, the metadata are preserved, including the file ID number (read: inode) -- with different contents. If you rename() instead of exchange/delete, you have a new file, and you have to set all the metadata yourself, and you may not have the necessary knowledge to do that. I suppose you could backup the original file and then overwrite it, but you lose atomicity. What bothers me is that I haven't seen exchange-files anywhere else. Then again, Mac OS doesn't have move-and-rename-simultaneously, so nobody's perfect. Josh
Re: [perl #38146] [TODO] OS.pmc - file copy
On Wed, Jan 11, 2006 at 04:16:55PM -0500, Joshua Juran wrote: Since before System 7 (approaching two decades ago), Mac OS has had a system call that exchanges the contents of two files. The purpose of this call is to implement a 'safe save' strategy ... Is this still a system call in Mac OS X? -- Chip Salzenberg [EMAIL PROTECTED]
Re: [perl #38146] [TODO] OS.pmc - file copy
On Jan 11, 2006, at 7:02 PM, Chip Salzenberg wrote: On Wed, Jan 11, 2006 at 04:16:55PM -0500, Joshua Juran wrote: Since before System 7 (approaching two decades ago), Mac OS has had a system call that exchanges the contents of two files. The purpose of this call is to implement a 'safe save' strategy ... Is this still a system call in Mac OS X? Yes, the original FSpExchangeFiles() call persists along with most of the calls pertaining to FSSpecs. New code written only for Mac OS 9 and above could also use the newer FSRef-based FSExchangeObjects() call which subsumes it. Josh
Re: [perl #38146] [TODO] OS.pmc - file copy
From: Joshua Isom [EMAIL PROTECTED] Date: Fri, 6 Jan 2006 23:53:51 -0600 On Jan 6, 2006, at 10:17 PM, Joshua Juran wrote: On Jan 6, 2006, at 4:11 PM, Alberto Simoes via RT wrote: This needs some more discussion. If we look to Perl, for instance, it doesn't have a built-in copy. You should use either a module, or open both files, copy contents, and close both files. You assume cp semantics. What about preserving metadata? What about multiple forks? What about per-file properties that are invented after you ship your code? And what if the source and destination are on the same file server? So it'd probably be best to put it in os.pmc with checks to know which operating system it's compiled for. That way, each language implementation that includes an operator to copy a file can do so more safely. Isn't part of the point of parrot to make writing a compiler not be system dependent? In fact, if perl had a copy operator instead of doing the open/print combo, the resource fork issue on Macs would be taken care of. Thinking in another direction, why is there only one OS class? Seems to me it would be cleaner to implement such things as file copy if there were a class hierarchy that reflected OS family relationships. Even more importantly, having multiple classes would make it much easier for language implementors to define OS-independent interfaces for language-specific features. This would then be a simple matter of writing multimethods that dispatch on the right OS subclasses, without worrying about which class actually gets instantiated. My $0.02, -- Bob Rogers http://rgrjr.dyndns.org/
Re: [perl #38146] [TODO] OS.pmc - file copy
Bob Rogers wrote: From: Joshua Isom [EMAIL PROTECTED] Date: Fri, 6 Jan 2006 23:53:51 -0600 On Jan 6, 2006, at 10:17 PM, Joshua Juran wrote: On Jan 6, 2006, at 4:11 PM, Alberto Simoes via RT wrote: This needs some more discussion. If we look to Perl, for instance, it doesn't have a built-in copy. You should use either a module, or open both files, copy contents, and close both files. You assume cp semantics. What about preserving metadata? What about multiple forks? What about per-file properties that are invented after you ship your code? And what if the source and destination are on the same file server? So it'd probably be best to put it in os.pmc with checks to know which operating system it's compiled for. That way, each language implementation that includes an operator to copy a file can do so more safely. Isn't part of the point of parrot to make writing a compiler not be system dependent? In fact, if perl had a copy operator instead of doing the open/print combo, the resource fork issue on Macs would be taken care of. Thinking in another direction, why is there only one OS class? Seems to me it would be cleaner to implement such things as file copy if there were a class hierarchy that reflected OS family relationships. Even more importantly, having multiple classes would make it much easier for language implementors to define OS-independent interfaces for language-specific features. This would then be a simple matter of writing multimethods that dispatch on the right OS subclasses, without worrying about which class actually gets instantiated. I think we might have a bigger problem than a copy per OS. I think we will have a copy per FileSystem if we get too detailed in its process. Alberto -- Alberto Simões - Departamento de Informática - Universidade do Minho Campus de Gualtar - 4710-057 Braga - Portugal
Re: [perl #38146] [TODO] OS.pmc - file copy
I think it's prudent that OS.pmc's default copy() method behaves in the same way as File::Copy::copy() as this is likely the lowest possible common denominator across platforms (note that I said behavior, not implementation). This is sufficient for the majority of file copy tasks and will make prevent most people from having to worry about portability issues. More sophisticated behavior, like metadata replication, should be left to another method (perhaps syscopy()) that has platform specific behavior(s). Cheers, -J -- pgpdddxZB0Vm1.pgp Description: PGP signature
[perl #38146] [TODO] OS.pmc - file copy
[coke - Tue Jan 03 18:03:31 2006]: OS.pmc should provide both a: copy(source_file,target) And a copy(array_of_source_files,targetDir) This needs some more discussion. If we look to Perl, for instance, it doesn't have a built-in copy. You should use either a module, or open both files, copy contents, and close both files. Thus, what I ask if we should have a copy method on OS.pmc. Basically, I think we have three hipothesis: 1) do not implement copy at all. 2) implement it under os.pmc, in C 3) implement it somewhere else in PIR Given that we have (I think) all the needed instructions in PIR, it will be a lot easier to make copy portable using the third choice... but I'm not sure if Parrot will ship with a PIR library... Please, share your ideas. Alberto
Re: [perl #38146] [TODO] OS.pmc - file copy
On Jan 6, 2006, at 4:11 PM, Alberto Simoes via RT wrote: [coke - Tue Jan 03 18:03:31 2006]: OS.pmc should provide both a: copy(source_file,target) And a copy(array_of_source_files,targetDir) This needs some more discussion. If we look to Perl, for instance, it doesn't have a built-in copy. You should use either a module, or open both files, copy contents, and close both files. You assume cp semantics. What about preserving metadata? What about multiple forks? What about per-file properties that are invented after you ship your code? And what if the source and destination are on the same file server? Thus, what I ask if we should have a copy method on OS.pmc. Basically, I think we have three hipothesis: 1) do not implement copy at all. You are in a maze of twisty little ad-hoc, informally-specified bug-ridden slow implementations of file copy, all different. 2) implement it under os.pmc, in C 3) implement it somewhere else in PIR Given that we have (I think) all the needed instructions in PIR, it will be a lot easier to make copy portable using the third choice... but I'm not sure if Parrot will ship with a PIR library... Please, share your ideas. I'd like to see every OS get a chance to provide its own file copier. This is important to Mac users, since resource forks may exist and the AppleShare protocol (AFP) allows for intra-server file copy. The same arguments also apply to other file systems. The Genie kernel (part of Lamp, a Unix-like environment on Mac OS) has a copyfile() system call for just this purpose. The knowledge of how to copy a file correctly in a Mac environment (which is non-trivial) is encapsulated in a single place, and usable by any program, either directly through the system call, through cp (for shell scripts), or through a module-loading interpreter. For more typical systems, supply copyfile() in a library as you would a missing kernel call. This usage assumes that copyfile() blocks until completion. Clearly there will be a desire for a concurrent processing mode as well. Josh
Re: [perl #38146] [TODO] OS.pmc - file copy
On Jan 6, 2006, at 10:17 PM, Joshua Juran wrote: On Jan 6, 2006, at 4:11 PM, Alberto Simoes via RT wrote: This needs some more discussion. If we look to Perl, for instance, it doesn't have a built-in copy. You should use either a module, or open both files, copy contents, and close both files. You assume cp semantics. What about preserving metadata? What about multiple forks? What about per-file properties that are invented after you ship your code? And what if the source and destination are on the same file server? So it'd probably be best to put it in os.pmc with checks to know which operating system it's compiled for. That way, each language implementation that includes an operator to copy a file can do so more safely. Isn't part of the point of parrot to make writing a compiler not be system dependent? In fact, if perl had a copy operator instead of doing the open/print combo, the resource fork issue on Macs would be taken care of. Thus, what I ask if we should have a copy method on OS.pmc. Basically, I think we have three hipothesis: 1) do not implement copy at all. You are in a maze of twisty little ad-hoc, informally-specified bug-ridden slow implementations of file copy, all different. 2) implement it under os.pmc, in C 3) implement it somewhere else in PIR Given that we have (I think) all the needed instructions in PIR, it will be a lot easier to make copy portable using the third choice... but I'm not sure if Parrot will ship with a PIR library... Please, share your ideas. I'd like to see every OS get a chance to provide its own file copier. This is important to Mac users, since resource forks may exist and the AppleShare protocol (AFP) allows for intra-server file copy. The same arguments also apply to other file systems. The Genie kernel (part of Lamp, a Unix-like environment on Mac OS) has a copyfile() system call for just this purpose. The knowledge of how to copy a file correctly in a Mac environment (which is non-trivial) is encapsulated in a single place, and usable by any program, either directly through the system call, through cp (for shell scripts), or through a module-loading interpreter. For more typical systems, supply copyfile() in a library as you would a missing kernel call. This usage assumes that copyfile() blocks until completion. Clearly there will be a desire for a concurrent processing mode as well. Josh Another thing is that moving a file across file systems isn't guaranteed to work(I don't know if it is on any platform), and to do so would require copying. Implementing a copy that supported host specific features in os.pmc would help ease that problem. If copy were implemented in pir, and rename were implemented in os.pmc, a problem would arise. The mv command takes care of this, and in my view, for convenience, so should any rename command in os.pmc.
[perl #38146] [TODO] OS.pmc - file copy
# New Ticket Created by Will Coleda # Please include the string: [perl #38146] # in the subject line of all future correspondence about this issue. # URL: https://rt.perl.org/rt3/Ticket/Display.html?id=38146 OS.pmc should provide both a: copy(source_file,target) And a copy(array_of_source_files,targetDir)
