Re: [patch] NioFileUtils, FileUtilsAdapter + factory (was Re: AW: Adding a methof to StringUtils)
> You do not gain by using nio when copying files to a local drive. As far as I know, you do gain a little, in the flexibility offered (memory mapped files, channels etc). The code is also (usually) much easier to maintain compared to standard IO. > > I do not know what in the implementation of nio allows to gain this > performance. When I first looked at it, I think it had something to do with the blocking behaviour of standard IO, NIO is actually Non-(blocking) IO in this respect. Sort of how Apache 1 (used blocking threads), -> Apache 2 (uses non-blocking model), at least I think that's how the performance is achieved. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [patch] NioFileUtils, FileUtilsAdapter + factory (was Re: AW: Adding a methof to StringUtils)
> I'm afraid I've never studied nio in much detail... do > its performance enhancements extend to non-file-based > IO? > > -Matt Hello Matt, I never studied nio in detail either. What I found out empirically in a Windows environment is that copying files to a share goes about 4 times faster when using nio compared to the traditional method. You do not gain by using nio when copying files to a local drive. I do not know what in the implementation of nio allows to gain this performance. Regards, Antoine - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [patch] NioFileUtils, FileUtilsAdapter + factory (was Re: AW: Adding a methof to StringUtils)
--- Martijn Kruithof <[EMAIL PROTECTED]> wrote: > Matt Benson schreef: > > >viewcvs(svn) shows I did it 6.5 months ago... > >apparently so. Speaking of which, Kev (I think) > >mentioned these are static in ResourceUtils, but as > >they are unreleased we could change them to > instance > >methods. The behavior there is fairly > >straightforward, though, so I'm not sure if > anything > >would be gained by this... > > > >-Matt > > > > > Whell it would mean that if we do it in FileUtils, > we wouldn't benfit > from it when Resources come into play. Futhermore, > if introduce the nio > copy under the responsibility of the ResourceUtils > all users of > FileUtils would benefit right away. I'm afraid I've never studied nio in much detail... do its performance enhancements extend to non-file-based IO? -Matt > > Martijn > > - > To unsubscribe, e-mail: > [EMAIL PROTECTED] > For additional commands, e-mail: > [EMAIL PROTECTED] > > __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [patch] NioFileUtils, FileUtilsAdapter + factory (was Re: AW: Adding a methof to StringUtils)
Matt Benson schreef: viewcvs(svn) shows I did it 6.5 months ago... apparently so. Speaking of which, Kev (I think) mentioned these are static in ResourceUtils, but as they are unreleased we could change them to instance methods. The behavior there is fairly straightforward, though, so I'm not sure if anything would be gained by this... -Matt Whell it would mean that if we do it in FileUtils, we wouldn't benfit from it when Resources come into play. Futhermore, if introduce the nio copy under the responsibility of the ResourceUtils all users of FileUtils would benefit right away. Martijn - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [patch] NioFileUtils, FileUtilsAdapter + factory (was Re: AW: Adding a methof to StringUtils)
--- Martijn Kruithof <[EMAIL PROTECTED]> wrote: > Matt Benson schreef: > > >Another thought about the factory--it should cache > an > >instance of each FileUtils type to minimize object > >creation. We can either just have it be known that > >FileUtilsAdapter impls should be stateless or have > a > >StatelessFileUtilsAdapter interface--if > implemented, > >cache, else don't. > > > >thoughts? > > > >-Matt > > > Well I am certainly in favour of keeping FileUtils > stateless, by the My point was only that we can't really enforce that FileUtilsAdapter (or whatever) impls are stateless. If we are making this factory-based we could allow users to (theoretically) provide their own implementation. If such an impl was stateful/non-threadsafe, singletons would misbehave. That's where my idea of a subinterface was coming from. Anyway... > way, didn't you move copying of files to > ResourceUtils some time ago? > viewcvs(svn) shows I did it 6.5 months ago... apparently so. Speaking of which, Kev (I think) mentioned these are static in ResourceUtils, but as they are unreleased we could change them to instance methods. The behavior there is fairly straightforward, though, so I'm not sure if anything would be gained by this... -Matt > Martijn > > - > To unsubscribe, e-mail: > [EMAIL PROTECTED] > For additional commands, e-mail: > [EMAIL PROTECTED] > > __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [patch] NioFileUtils, FileUtilsAdapter + factory (was Re: AW: Adding a methof to StringUtils)
Matt Benson schreef: Another thought about the factory--it should cache an instance of each FileUtils type to minimize object creation. We can either just have it be known that FileUtilsAdapter impls should be stateless or have a StatelessFileUtilsAdapter interface--if implemented, cache, else don't. thoughts? -Matt Well I am certainly in favour of keeping FileUtils stateless, by the way, didn't you move copying of files to ResourceUtils some time ago? Martijn - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [patch] NioFileUtils, FileUtilsAdapter + factory (was Re: AW: Adding a methof to StringUtils)
O by the way I am / was looking if FileUtils could be split into os dependend version this way, so I actually did some work on this as well. Martijn Kruithof schreef: I actually had a similar problem in my dtd work recently. We had a class that applied one single strategy to determine something important for telecommunication systems, had some static methods, some nonstatic methods, was used by different components of which some could not be updated. Backward compatibility was crucial, but at the same time it was important that every component would use the correct strategy. What I did was turn the constructor into an factory (however still staying a constructor, just creating the destination class), and turn the class into a proxy, towards several implementations, one of which would be the singleton actually used (one of which having the original implementation in nonstatic methods only). Maybe the same way can be used to replace the FileUtils as wel, make the FileUtils class a Proxy with built in Factory. Martijn - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [patch] NioFileUtils, FileUtilsAdapter + factory (was Re: AW: Adding a methof to StringUtils)
I actually had a similar problem in my dtd work recently. We had a class that applied one single strategy to determine something important for telecommunication systems, had some static methods, some nonstatic methods, was used by different components of which some could not be updated. Backward compatibility was crucial, but at the same time it was important that every component would use the correct strategy. What I did was turn the constructor into an factory (however still staying a constructor, just creating the destination class), and turn the class into a proxy, towards several implementations, one of which would be the singleton actually used (one of which having the original implementation in nonstatic methods only). Maybe the same way can be used to replace the FileUtils as wel, make the FileUtils class a Proxy with built in Factory. Martijn - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [patch] NioFileUtils, FileUtilsAdapter + factory (was Re: AW: Adding a methof to StringUtils)
Matt Benson wrote: Another thought about the factory--it should cache an instance of each FileUtils type to minimize object creation. We can either just have it be known that FileUtilsAdapter impls should be stateless or have a StatelessFileUtilsAdapter interface--if implemented, cache, else don't. I only know that the existing FileUtils implementations are stateless. Maybe we can simply assume that this will be the case in the future. Regards, Antoine thoughts? -Matt - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [patch] NioFileUtils, FileUtilsAdapter + factory (was Re: AW: Adding a methof to StringUtils)
Another thought about the factory--it should cache an instance of each FileUtils type to minimize object creation. We can either just have it be known that FileUtilsAdapter impls should be stateless or have a StatelessFileUtilsAdapter interface--if implemented, cache, else don't. thoughts? -Matt --- Matt Benson <[EMAIL PROTECTED]> wrote: > --- Kevin Jackson <[EMAIL PROTECTED]> wrote: > > > > I found it in Kev's (long :) mail: > > > > > > > Sorry it was rather long, but there were a few > files > > packed in there! > > parenthetical smiley above. > > > > Since the other impls will be conditionally > > compiled, > > > we should use Class.forInstance()... > > > > > > > ok, so we use dynamic class-loading to get the > > correct fileutils? Is > > there an example in the code already of how I > should > > do this, (like in > > the ComplilerAdapter code perhaps?) > > > That's the way I understand it. Antoine cited > Regexp > factory stuff (o.a.t.a.util.regexp); it looks like > the > biggest lesson to take from here would be the use of > ClasspathUtils: > > return (FileUtilsAdapter) > ClasspathUtils.newInstance(className, > FileUtilsAdapter.class.getClassLoader(), > FileUtilsAdapter.class); > > -Matt > > [SNIP] > > __ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam > protection around > http://mail.yahoo.com > > - > To unsubscribe, e-mail: > [EMAIL PROTECTED] > For additional commands, e-mail: > [EMAIL PROTECTED] > > __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [patch] NioFileUtils, FileUtilsAdapter + factory (was Re: AW: Adding a methof to StringUtils)
--- Kevin Jackson <[EMAIL PROTECTED]> wrote: > > I found it in Kev's (long :) mail: > > > > Sorry it was rather long, but there were a few files > packed in there! parenthetical smiley above. > > Since the other impls will be conditionally > compiled, > > we should use Class.forInstance()... > > > > ok, so we use dynamic class-loading to get the > correct fileutils? Is > there an example in the code already of how I should > do this, (like in > the ComplilerAdapter code perhaps?) > That's the way I understand it. Antoine cited Regexp factory stuff (o.a.t.a.util.regexp); it looks like the biggest lesson to take from here would be the use of ClasspathUtils: return (FileUtilsAdapter) ClasspathUtils.newInstance(className, FileUtilsAdapter.class.getClassLoader(), FileUtilsAdapter.class); -Matt [SNIP] __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [patch] NioFileUtils, FileUtilsAdapter + factory (was Re: AW: Adding a methof to StringUtils)
Kevin Jackson wrote:
I found it in Kev's (long :) mail:
Sorry it was rather long, but there were a few files packed in there!
No problem, I should have washed my glasses this morning, ...
Since the other impls will be conditionally compiled,
we should use Class.forInstance()...
ok, so we use dynamic class-loading to get the correct fileutils? Is
there an example in the code already of how I should do this, (like in
the ComplilerAdapter code perhaps?)
in the RegexpFactory there is an example.
What would the Project reference be used for?
in the RegexpFactory, a project reference is used so that properties can
be accessed, and that the user can direct ant to use a particular
implementation.
Kev: your mail showed:
public class NioFileUtils extends FileUtils implements
FileUtilsAdapter {...}
public class Java6FileUtils extends FileUtils
implements FileUtilsAdapter {...}
But shouldn't that actually be:
+public class FileUtils implements FileUtilsAdapter
{...}
public class NioFileUtils extends FileUtils {...}
public class Java6FileUtils extends NioFileUtils {...}
Probably yes, I think I cobbled it together in a hap-hazard fashion -
first the interface, then change the FileUtils etc, so I could
probably remove the implements FileUtilsAdapter from the subclasses, I
just wanted to make sure I wasn't 'polishing a turd', ie wasting time
getting every little thing right, if the overall direction of the code
was wrong.
A last thing : I think that the default buffer size (variable
TRANSFER_SIZE) should be reduced from 81920 to 32000 and should be
configurable.
I used this nio copy before, and larger transfer sizes were sometimes
creating exceptions (error messages coming from the operating system).
Thanks for comments
Kev
Antoine
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [patch] NioFileUtils, FileUtilsAdapter + factory (was Re: AW: Adding a methof to StringUtils)
> I found it in Kev's (long :) mail:
>
Sorry it was rather long, but there were a few files packed in there!
> Since the other impls will be conditionally compiled,
> we should use Class.forInstance()...
>
ok, so we use dynamic class-loading to get the correct fileutils? Is
there an example in the code already of how I should do this, (like in
the ComplilerAdapter code perhaps?)
>
> What would the Project reference be used for?
>
> Kev: your mail showed:
>
> public class NioFileUtils extends FileUtils implements
> FileUtilsAdapter {...}
>
> public class Java6FileUtils extends FileUtils
> implements FileUtilsAdapter {...}
>
>
> But shouldn't that actually be:
>
> +public class FileUtils implements FileUtilsAdapter
> {...}
>
> public class NioFileUtils extends FileUtils {...}
>
> public class Java6FileUtils extends NioFileUtils {...}
>
Probably yes, I think I cobbled it together in a hap-hazard fashion -
first the interface, then change the FileUtils etc, so I could
probably remove the implements FileUtilsAdapter from the subclasses, I
just wanted to make sure I wasn't 'polishing a turd', ie wasting time
getting every little thing right, if the overall direction of the code
was wrong.
Thanks for comments
Kev
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [patch] NioFileUtils, FileUtilsAdapter + factory (was Re: AW: Adding a methof to StringUtils)
--- Antoine Levy-Lambert <[EMAIL PROTECTED]> wrote:
> Hello Kevin,
>
> NioFileUtils should go into another package
> org.apache.tools.ant.util.java14 otherwise we are
> going to have build
> problems under java 1.2
> This should be entered in the build.xml (selector
> needs.jdk14+)
> Therefore NioFileUtils would be packaged in
> ant-nodeps.jar
>
I don't really care either way here, but couldn't we
explicitly exclude the file(s)? Why must they live in
another directory?
> We need also a FileUtilsFactory. The
> FileUtilsFactory would instantiate
> NioFileUtils if the java runtime is 1.4 or 1.5, the
> normal FileUtils
> otherwise.
> This should be developed along the lines of the
> RegexpFactory, which
> means no imports of optional classes such as
> NioFileUtils or
> Java16FileUtils.
I found it in Kev's (long :) mail:
package org.apache.tools.ant.util;
public class FileUtilsAdapterFactory {
private FileUtilsAdapterFactory() {}
public static FileUtilsAdapter getFileUtils(final
String type) {
if (type.equals("nio")) {
//just print out to see if it's actually
picking up the correct class
System.out.println("Using nio fileutils");
return new NioFileUtils();
} else if (type.equals("java6")) {
System.out.println("Using java6
fileutils");
return new Java6FileUtils();
} else {
System.out.println("Using classic
fileutils");
return new FileUtils();
}
}
public static FileUtilsAdapter getBestFileUtils()
{
final int v =
JavaEnvUtils.getJavaVersionNumber();
if (v >= 16) {
return getFileUtils("java6");
} else if (v >=14) {
return getFileUtils("nio");
} else {
return new FileUtils();
}
}
}
Since the other impls will be conditionally compiled,
we should use Class.forInstance()...
>
> Similarly, users of FileUtils would call
> FileUtilsFactory.newFileUtils(Project p) instead of
> FileUtils.newFileUtils() where a project member
> variable is available,
> otherwise FileUtilsFactory.newFileUtils().
What would the Project reference be used for?
Kev: your mail showed:
public class NioFileUtils extends FileUtils implements
FileUtilsAdapter {...}
public class Java6FileUtils extends FileUtils
implements FileUtilsAdapter {...}
But shouldn't that actually be:
+public class FileUtils implements FileUtilsAdapter
{...}
public class NioFileUtils extends FileUtils {...}
public class Java6FileUtils extends NioFileUtils {...}
?
br,
Matt
__
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: [patch] NioFileUtils, FileUtilsAdapter + factory (was Re: AW: Adding a methof to StringUtils)
Hello Kevin,
NioFileUtils should go into another package
org.apache.tools.ant.util.java14 otherwise we are going to have build
problems under java 1.2
This should be entered in the build.xml (selector needs.jdk14+)
Therefore NioFileUtils would be packaged in ant-nodeps.jar
We need also a FileUtilsFactory. The FileUtilsFactory would instantiate
NioFileUtils if the java runtime is 1.4 or 1.5, the normal FileUtils
otherwise.
This should be developed along the lines of the RegexpFactory, which
means no imports of optional classes such as NioFileUtils or
Java16FileUtils.
Similarly, users of FileUtils would call
FileUtilsFactory.newFileUtils(Project p) instead of
FileUtils.newFileUtils() where a project member variable is available,
otherwise FileUtilsFactory.newFileUtils().
FileUtilsFactory would be the place where the right FileUtils
implementation is picked.
Regards,
Antoine
Kev Jackson wrote:
> Hi,
>
> As promised, here's the code that I hacked together today at Antoine's
> suggestion regarding a patch in bugzilla [1].
>
> Feel free to tear it to pieces and point out obvious problems. I've
> patched my version of copy to use this code instead of FileUtils
> directly and so far there have been no problems with it.
>
> Thanks
> Kev
>
> [1]http://issues.apache.org/bugzilla/show_bug.cgi?id=30094
>
>
> Index:
> D:/java_projects/ant-core-trunk/src/main/org/apache/tools/ant/util/FileUtils.java
> ===
> ---
> D:/java_projects/ant-core-trunk/src/main/org/apache/tools/ant/util/FileUtils.java
> (revision 395527)
> +++
> D:/java_projects/ant-core-trunk/src/main/org/apache/tools/ant/util/FileUtils.java
> (working copy)
> @@ -46,7 +46,7 @@
> * their last modification time.
> *
> */
> -public class FileUtils {
> +public class FileUtils implements FileUtilsAdapter {
>
> private static final FileUtils PRIMARY_INSTANCE = new FileUtils();
>
> Index:
> D:/java_projects/ant-core-trunk/src/main/org/apache/tools/ant/util/FileUtilsAdapter.java
> ===
> /*
> * Copyright 2006 The Apache Software Foundation
> *
> * Licensed under the Apache License, Version 2.0 (the "License");
> * you may not use this file except in compliance with the License.
> * You may obtain a copy of the License at
> *
> * http://www.apache.org/licenses/LICENSE-2.0
> *
> * Unless required by applicable law or agreed to in writing, software
> * distributed under the License is distributed on an "AS IS" BASIS,
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> * See the License for the specific language governing permissions and
> * limitations under the License.
> *
> */
>
> package org.apache.tools.ant.util;
>
> import java.io.File;
> import java.io.IOException;
> import java.net.MalformedURLException;
> import java.net.URL;
> import java.util.Vector;
>
> import org.apache.tools.ant.Project;
> import org.apache.tools.ant.types.FilterSetCollection;
>
> /**
> * An interface for the different types of FileUtils (classic, Nio, Java6+)
> *
> */
> public interface FileUtilsAdapter {
>
> /**
> * The granularity of timestamps under FAT.
> */
> public static final long FAT_FILE_TIMESTAMP_GRANULARITY = 2000;
>
> /**
> * The granularity of timestamps under Unix.
> */
> public static final long UNIX_FILE_TIMESTAMP_GRANULARITY = 1000;
>
> /**
> * The granularity of timestamps under the NT File System.
> * NTFS has a granularity of 100 nanoseconds, which is less
> * than 1 millisecond, so we round this up to 1 millisecond.
> */
> public static final long NTFS_FILE_TIMESTAMP_GRANULARITY = 1;
>
> /**
> * Get the URL for a file taking into account # characters.
> *
> * @param file the file whose URL representation is required.
> * @return The FileURL value.
> * @throws MalformedURLException if the URL representation cannot be
> * formed.
> */
> public URL getFileURL(File file) throws MalformedURLException;
>
> /**
> * Convenience method to copy a file from a source to a destination.
> * No filtering is performed.
> *
> * @param sourceFile Name of file to copy from.
> * Must not be null.
> * @param destFile Name of file to copy to.
> * Must not be null.
> *
> * @throws IOException if the copying fails.
> */
> public void copyFile(String sourceFile, String destFile)
> throws IOException;
>
> /**
> * Convenience method to copy a file from a source to a destination
> * specifying if token filtering must be used.
> *
> * @param sourceFile Name of file to copy from.
> * Must not be null.
> * @param destFile Name of file to c
