[cp-patches] [generics] Patch: FYI: japi bug in java.util.zip
I'm checking this in on the generics branch. This fixes a java.util.zip buglet pointed out by japi. Tom 2005-10-05 Tom Tromey [EMAIL PROTECTED] * java/util/zip/ZipFile.java (entries): Fixed return type. Index: java/util/zip/ZipFile.java === RCS file: /cvsroot/classpath/classpath/java/util/zip/ZipFile.java,v retrieving revision 1.16.2.8 diff -u -r1.16.2.8 ZipFile.java --- java/util/zip/ZipFile.java 27 Sep 2005 16:52:59 - 1.16.2.8 +++ java/util/zip/ZipFile.java 6 Oct 2005 00:39:58 - @@ -368,7 +368,7 @@ * * @exception IllegalStateException when the ZipFile has already been closed */ - public EnumerationZipEntry entries() + public Enumeration? extends ZipEntry entries() { checkClosed(); ___ Classpath-patches mailing list Classpath-patches@gnu.org http://lists.gnu.org/mailman/listinfo/classpath-patches
[cp-patches] [generics] Patch: FYI: generics in java.util.zip
I'm checking this in on the generics branch. This genericizes some types in java.util.zip, as pointed out by the new japi. Tom Index: ChangeLog from Tom Tromey [EMAIL PROTECTED] * java/util/zip/ZipFile.java (entries): Updated return type. (ZipEntryEnumeration): Updated 'implements' type. (entries): Updated type. (ZipEntryEnumeration.elements): Likewise. (readEntries): Updated. (getEntries): Likewise. (getEntry): Likewise. (getInputStream): Likewise. Index: java/util/zip/ZipFile.java === RCS file: /cvsroot/classpath/classpath/java/util/zip/ZipFile.java,v retrieving revision 1.16.2.7 diff -u -r1.16.2.7 ZipFile.java --- java/util/zip/ZipFile.java 20 Sep 2005 18:46:30 - 1.16.2.7 +++ java/util/zip/ZipFile.java 27 Sep 2005 16:41:39 - @@ -84,7 +84,7 @@ private final RandomAccessFile raf; // The entries of this zip file when initialized and not yet closed. - private HashMap entries; + private HashMapString, ZipEntry entries; private boolean closed = false; @@ -268,7 +268,7 @@ throw new EOFException(name); int centralOffset = readLeInt(raf, ebs); -entries = new HashMap(count+count/2); +entries = new HashMapString, ZipEntry (count+count/2); raf.seek(centralOffset); byte[] buffer = new byte[16]; @@ -368,7 +368,7 @@ * * @exception IllegalStateException when the ZipFile has already been closed */ - public Enumeration entries() + public EnumerationZipEntry entries() { checkClosed(); @@ -388,7 +388,7 @@ * @exception IllegalStateException when the ZipFile has already been closed. * @exception IOException when the entries could not be read. */ - private HashMap getEntries() throws IOException + private HashMapString, ZipEntry getEntries() throws IOException { synchronized(raf) { @@ -416,11 +416,11 @@ try { - HashMap entries = getEntries(); - ZipEntry entry = (ZipEntry) entries.get(name); + HashMapString, ZipEntry entries = getEntries(); + ZipEntry entry = entries.get(name); // If we didn't find it, maybe it's a directory. if (entry == null !name.endsWith(/)) -entry = (ZipEntry) entries.get(name + '/'); + entry = entries.get(name + '/'); return entry != null ? new ZipEntry(entry, name) : null; } catch (IOException ioe) @@ -491,9 +491,9 @@ { checkClosed(); -HashMap entries = getEntries(); +HashMapString, ZipEntry entries = getEntries(); String name = entry.getName(); -ZipEntry zipEntry = (ZipEntry) entries.get(name); +ZipEntry zipEntry = entries.get(name); if (zipEntry == null) return null; @@ -539,11 +539,11 @@ } } - private static class ZipEntryEnumeration implements Enumeration + private static class ZipEntryEnumeration implements EnumerationZipEntry { -private final Iterator elements; +private final IteratorZipEntry elements; -public ZipEntryEnumeration(Iterator elements) +public ZipEntryEnumeration(IteratorZipEntry elements) { this.elements = elements; } @@ -553,12 +553,12 @@ return elements.hasNext(); } -public Object nextElement() +public ZipEntry nextElement() { /* We return a clone, just to be safe that the user doesn't * change the entry. */ - return ((ZipEntry)elements.next()).clone(); + return (ZipEntry) (elements.next().clone()); } } ___ Classpath-patches mailing list Classpath-patches@gnu.org http://lists.gnu.org/mailman/listinfo/classpath-patches
Re: Patches to java.util.zip by Christian Schlichtherle
On Thu, 2005-09-01 at 02:39 +0200, Robert Schuster wrote: http://www.javaworld.com/jw-10-1997/jw-10-sunsuit.html Interesting article, especially this one: And while Sun can exercise its legal rights over the use of the Java name, it can do little to stop Microsoft from implementing a clean room version of the Java virtual machine, says Morgenthal. :) Of course. And that is why we are pretty strict on our code contribution requirements as outlined in the hacking guide and on the developer wiki. http://www.gnu.org/software/classpath/docs/hacking.html#SEC2 http://developer.classpath.org/mediation/ClasspathFirstSteps Microsoft got into trouble because they deliberately used the Java(TM) trademark to describe and promote their own product, licensed (parts of) the software they included in their products from Sun and went into several other undisclosed contractual obligations. Cheers, Mark -- Escape the Java Trap with GNU Classpath! http://www.gnu.org/philosophy/java-trap.html Join the community at http://planet.classpath.org/ signature.asc Description: This is a digitally signed message part ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
[cp-patches] RE: Patches to java.util.zip by Christian Schlichtherle
Hi everyone, the changes from int to long are required as to the ZIP file format specification available online from PKZIP Inc. The specification says that all integers are 4 byte unsigned integers. Java's int type is 4 byte signed, thus the type long is required to hold a ZIP file integer. You can find this in other popular Java based ZIP file implementations as well (Apache ant, the JDK, etc.). I have introduced this fix because jazzlib 0.07 definitely breaks on ZIP files larger than 2 GB. With my patches however, jazzlib is compatible to the ZIP 32 spec which can hold up to 4 GB. Furthermore, the constructors I've introduced are provided to allow an application developer to have the choice of choosing an encoding: UTF-8 as with Sun's JDK which is only relevant if you need to exchange ZIP files with JDK created ZIPs or CP437 to exchange ZIP files created by the rest of the world (PKZIP, Winzip, infozip, MS Explorer, etc). In my opinion, the latter is more relevant for real world internationalized applications. Removing all these fixes/enhancements make my submitted patches pretty useless and do not solve important bugs/limitations of jazzlib 0.07. Anyway, as nothing happened on these topics for almost half a year I've moved away and rolled my own. The resulting ZIP package is available at http://truezip.dev.java.net and does much more than just providing classes which are backwards compatible to JDK's java.util.zip package. It provides an additional layer of abstraction so that an application can treat a ZIP file like an ordinary directory using classes which are backwards compatible to JDK's java.io package. Using this you can create for instance an entry nested in two ZIP files by simply writing to a new FileOutputStream(outer.zip/inner.zip/file.txt). With best regards, Christian Schlichtherle --- Schlichtherle IT Services Wittelsbacher Str. 10a 10707 Berlin Mobil: 0173 / 27 12 470 mailto:[EMAIL PROTECTED] http://www.schlichtherle.de -Original Message- From: Mark Wielaard [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 30, 2005 11:45 PM To: John Leuner Cc: classpath-patches@gnu.org; Christian Schlichtherle Subject: Re: Patches to java.util.zip by Christian Schlichtherle Hi (moved to classpath-patches) On Tue, 2005-08-30 at 18:02 +0200, Mark Wielaard wrote: I need to go through the rest of these patches. It would help if they could be rediffed against current CVS head and extra functionality would be moved to a new package. I reviewed the rest but dropped everything that would need new constructors. It would be best to rewrite that part so that it is an extension for another (gnu) package. While reviewing the classes I fixed some other small issues that I saw in the code. I choose to treat all strings explicitly as UTF-8 encoded although CP437 would probably be also a defensible choice. I did not use any of your int - long changes in the method signatures since I was not sure how and where that was actually needed. If there is a change in the zip format could you give a reference to that and explain how the patch changes the values that gets written/read? I committed the following: 2005-08-30 Mark Wielaard [EMAIL PROTECTED] Christian Schlichtherle [EMAIL PROTECTED] * java/util/zip/ZipEntry.java (setTime): Use Calendar.setTimeInMillis(). (getTime): First parse extra bytes. Use Calendar.getTimeInMillis(). (parseExtra): Don't return early to make sure that KNOWN_EXTRA is always set. * java/util/zip/ZipFile.java (readEntries): Parse name and comment as UTF-8 string. (close): Check that raf is not null. * java/util/zip/ZipInputStream.java (getNextEntry): Set name as UTF-8 bytes. * java/util/zip/ZipOutputStream.java (setComment): Set comment as UTF-8 bytes. (putNextEntry): Likewise for name. (finish): Likewise for both. Could you check whether I missed something essential from your original patch? Thanks, Mark smime.p7s Description: S/MIME cryptographic signature ___ Classpath-patches mailing list Classpath-patches@gnu.org http://lists.gnu.org/mailman/listinfo/classpath-patches
RE: Patches to java.util.zip by Christian Schlichtherle
Hi everyone, the changes from int to long are required as to the ZIP file format specification available online from PKZIP Inc. The specification says that all integers are 4 byte unsigned integers. Java's int type is 4 byte signed, thus the type long is required to hold a ZIP file integer. You can find this in other popular Java based ZIP file implementations as well (Apache ant, the JDK, etc.). I have introduced this fix because jazzlib 0.07 definitely breaks on ZIP files larger than 2 GB. With my patches however, jazzlib is compatible to the ZIP 32 spec which can hold up to 4 GB. Furthermore, the constructors I've introduced are provided to allow an application developer to have the choice of choosing an encoding: UTF-8 as with Sun's JDK which is only relevant if you need to exchange ZIP files with JDK created ZIPs or CP437 to exchange ZIP files created by the rest of the world (PKZIP, Winzip, infozip, MS Explorer, etc). In my opinion, the latter is more relevant for real world internationalized applications. Removing all these fixes/enhancements make my submitted patches pretty useless and do not solve important bugs/limitations of jazzlib 0.07. Anyway, as nothing happened on these topics for almost half a year I've moved away and rolled my own. The resulting ZIP package is available at http://truezip.dev.java.net and does much more than just providing classes which are backwards compatible to JDK's java.util.zip package. It provides an additional layer of abstraction so that an application can treat a ZIP file like an ordinary directory using classes which are backwards compatible to JDK's java.io package. Using this you can create for instance an entry nested in two ZIP files by simply writing to a new FileOutputStream(outer.zip/inner.zip/file.txt). With best regards, Christian Schlichtherle --- Schlichtherle IT Services Wittelsbacher Str. 10a 10707 Berlin Mobil: 0173 / 27 12 470 mailto:[EMAIL PROTECTED] http://www.schlichtherle.de -Original Message- From: Mark Wielaard [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 30, 2005 11:45 PM To: John Leuner Cc: classpath-patches@gnu.org; Christian Schlichtherle Subject: Re: Patches to java.util.zip by Christian Schlichtherle Hi (moved to classpath-patches) On Tue, 2005-08-30 at 18:02 +0200, Mark Wielaard wrote: I need to go through the rest of these patches. It would help if they could be rediffed against current CVS head and extra functionality would be moved to a new package. I reviewed the rest but dropped everything that would need new constructors. It would be best to rewrite that part so that it is an extension for another (gnu) package. While reviewing the classes I fixed some other small issues that I saw in the code. I choose to treat all strings explicitly as UTF-8 encoded although CP437 would probably be also a defensible choice. I did not use any of your int - long changes in the method signatures since I was not sure how and where that was actually needed. If there is a change in the zip format could you give a reference to that and explain how the patch changes the values that gets written/read? I committed the following: 2005-08-30 Mark Wielaard [EMAIL PROTECTED] Christian Schlichtherle [EMAIL PROTECTED] * java/util/zip/ZipEntry.java (setTime): Use Calendar.setTimeInMillis(). (getTime): First parse extra bytes. Use Calendar.getTimeInMillis(). (parseExtra): Don't return early to make sure that KNOWN_EXTRA is always set. * java/util/zip/ZipFile.java (readEntries): Parse name and comment as UTF-8 string. (close): Check that raf is not null. * java/util/zip/ZipInputStream.java (getNextEntry): Set name as UTF-8 bytes. * java/util/zip/ZipOutputStream.java (setComment): Set comment as UTF-8 bytes. (putNextEntry): Likewise for name. (finish): Likewise for both. Could you check whether I missed something essential from your original patch? Thanks, Mark smime.p7s Description: S/MIME cryptographic signature ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
RE: Patches to java.util.zip by Christian Schlichtherle
Christian Schlichtherle wrote: the changes from int to long are required as to the ZIP file format specification available online from PKZIP Inc. The specification says that all integers are 4 byte unsigned integers. Java's int type is 4 byte signed, thus the type long is required to hold a ZIP file integer. You can find this in other popular Java based ZIP file implementations as well (Apache ant, the JDK, etc.). Thanks for taking the time to explain this. We obviously should take in these fixes. Furthermore, the constructors I've introduced are provided to allow an application developer to have the choice of choosing an encoding: UTF-8 as with Sun's JDK which is only relevant if you need to exchange ZIP files with JDK created ZIPs or CP437 to exchange ZIP files created by the rest of the world (PKZIP, Winzip, infozip, MS Explorer, etc). In my opinion, the latter is more relevant for real world internationalized applications. Unfortunately, we cannot add additional public constructors, but I would suggest adding a system property to control the encoding used by our zip implementation. By default we should be compatible with the JDK, but this would allow applications and/or users to override the encoding to be compatible with the rest of the world. Thanks again. Regards, Jeroen ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
RE: Patches to java.util.zip by Christian Schlichtherle
Hi, On Wed, 2005-08-31 at 12:06 +0200, Jeroen Frijters wrote: Christian Schlichtherle wrote: the changes from int to long are required as to the ZIP file format specification available online from PKZIP Inc. The specification says that all integers are 4 byte unsigned integers. Java's int type is 4 byte signed, thus the type long is required to hold a ZIP file integer. You can find this in other popular Java based ZIP file implementations as well (Apache ant, the JDK, etc.). Thanks for taking the time to explain this. We obviously should take in these fixes. Yes, if someone can make a little testcase where we fail now and show how we are not properly converting the signed/unsigned ints at the moment that would be appreciated. Note that at the moment all public methods take and return longs already, we only store it as int internally and try treat it as unsigned already. It might be that we need to internally store everything in an long and not just treat the signed ints as unsigned ints as we do now in the code, but the patch contained so many unrelated changes that it was hard to make out which changes were explicitly for this issue. So if someone could extract those parts of the patch that would help a lot. Thanks, Mark signature.asc Description: This is a digitally signed message part ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
RE: Patches to java.util.zip by Christian Schlichtherle
Christian Schlichtherle wrote: Unfortunately, we cannot add additional public constructors, but I would suggest adding a system property to control the encoding used by our zip implementation. By default we should be compatible with the JDK, but this would allow applications and/or users to override the encoding to be compatible with the rest of the world. this would be an all-or-nothing-approach, i.e. you could have CP437 for either all ZIP* objects or none. The constructors however allow you to define this on a case by case basis, e.g. using CP437 for any file ending with a .zip suffix and UTF-8 for any file ending with a .jar suffix, which is the most reasonable general approach to deal with the encoding issue in my personal opinion (which is arguable however). Personally, I'm almost always in favor of compatibility with the JDK and in this case there is no doubt in my mind that we should use UTF-8 by default. I recognize that the system property is only a hack that solves a limited number of scenarios, but it's better than nothing. For my personal education: What's wrong about adding constructors? It is a violation of the Sun license included with the API specification -- you could argue about whether the license is valid or not, but that's not the point, and it would preclude us (or any JVM based on GNU Classpath) from ever passing the TCK. Regards, Jeroen ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
RE: Patches to java.util.zip by Christian Schlichtherle
Hi everyone, Unfortunately, we cannot add additional public constructors, but I would suggest adding a system property to control the encoding used by our zip implementation. By default we should be compatible with the JDK, but this would allow applications and/or users to override the encoding to be compatible with the rest of the world. this would be an all-or-nothing-approach, i.e. you could have CP437 for either all ZIP* objects or none. The constructors however allow you to define this on a case by case basis, e.g. using CP437 for any file ending with a .zip suffix and UTF-8 for any file ending with a .jar suffix, which is the most reasonable general approach to deal with the encoding issue in my personal opinion (which is arguable however). For my personal education: What's wrong about adding constructors? The result would still be backward compatible to the JDK source, so I think this would make up a good solution. This is also what people have often requested from Sun if you look at the bug tracker thread on this topic. With best regards, Christian smime.p7s Description: S/MIME cryptographic signature ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
RE: Patches to java.util.zip by Christian Schlichtherle
Hi everyone, Yes, if someone can make a little testcase where we fail now and show how we are not properly converting the signed/unsigned ints at the moment that would be appreciated. Note that at the moment all public methods take and return longs already, we only store it as int internally and try treat it as unsigned already. It might be that we need to internally store everything in an long and not just treat the signed ints as unsigned ints as we do now in the code, but the patch contained so many unrelated changes that it was hard to make out which changes were explicitly for this issue. So if someone could extract those parts of the patch that would help a lot. More specifically, the size and compressed size field in the ZipEntry class are causing the problems as some comparisons are happening on these. The issue is that once a big integer equal or greater than 2*1024^3 and smaller than 4*1024^3 is stored into a Java int, it is hard to use this int as if it were unsigned. You would have to do something like this on an int (untested): final static long NUM_FOUR_BYTE_INTS = 256L * 256L * 256L * 256L; // Number of 4 byte ints private static long int2long(int i) { return i = 0 ? (long)i : (long)NUM_FOUR_BYTE_INTS - (long)i; } private static void doIt() { int ok = 2 * 1024 * 1024 * 1024 - 1; // max signed int int tooBig = 3 * 1024 * 1024 * 1024; // Creates a negative integer // Compare ok and tooBig assert int2long(ok) int2long(tooBig) : This is ok!; assert ok tooBig : This fails!; } This is way too much computational overhead and it is way too easy to forget this on a comparison. Thus, the better way to deal with this issue is hold 4 byte signed ints in a Java long if you don't need to care for the memory overhead. With best regards, Christian smime.p7s Description: S/MIME cryptographic signature ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
Re: Patches to java.util.zip by Christian Schlichtherle
Christian Schlichtherle wrote: More specifically, the size and compressed size field in the ZipEntry class are causing the problems as some comparisons are happening on these. The issue is that once a big integer equal or greater than 2*1024^3 and smaller than 4*1024^3 is stored into a Java int, it is hard to use this int as if it were unsigned. You would have to do something like this on an int (untested): ... This is way too much computational overhead To compare two unsigned ints just invert their sign bits: I.e. (using C syntax): unsigned int32 i, j; signed int32 is = (signed int32) i; signed int32 js = (signed int32) j; Then (i COMP j) if and only if ((signed int32) (is ^ 0x80) COMP (signed int32) (js 0x8000)). Which is probably faster than: ((long) (is 0x) COMP (long) (js 0x)) -- --Per Bothner [EMAIL PROTECTED] http://per.bothner.com/ ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
Re: Patches to java.util.zip by Christian Schlichtherle
Jeroen Frijters wrote: Christian Schlichtherle wrote: For my personal education: What's wrong about adding constructors? It is a violation of the Sun license included with the API specification -- you could argue about whether the license is valid or not, but that's not the point, and it would preclude us (or any JVM based on GNU Classpath) from ever passing the TCK. Let us not beat about the bush. It would be bad for everyone (except Microsoft) if Java implementors were allowed to modify the APIs of the Java Class Libraries. Even if the API changes are upwards compatible (e.g. adding visible methods, constructors, fields or classes) the cause trouble. Specifically, if vendor XYZ has enhanced the APIs, then someone developing an application using vendor XYZ's Java platform may well find that their software does not run on other vendor's Java platform. Those with long memories will remember that Microsoft tried this trick. It was only the Sun license (and Sun's willingness to take Microsoft to court) that saved the Java community from the hell of vendor specific Java enhancements. -- Steve ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
RE: Patches to java.util.zip by Christian Schlichtherle
Hi Christian, On Wed, 2005-08-31 at 14:05 +0200, Christian Schlichtherle wrote: For my personal education: What's wrong about adding constructors? The result would still be backward compatible to the JDK source, so I think this would make up a good solution. This is also what people have often requested from Sun if you look at the bug tracker thread on this topic. In principle there is nothing wrong about it. But it isn't really what GNU Classpath is currently focusing on. Creating a Free Software implementation of the core class libraries is already a lot of work. Tracking extensions would currently be additional work that would distract from completion our set of core libraries. It is really better done in separate projects that don't have explicit compatibility as a goal. Your TrueZIP is a nice example of that. There you can do real innovation and create a really great new interface for people to use. For GNU Classpath we are focusing on providing the freedoms people except when using Free Software, better/smaller/faster/smarter implementations, good and complete documentation and creating opportunities for research and integration with other free software platforms (see native compilation in gcj, integration with #c and .net through ikvm, our gtk+ and qt peers for gnome and kde integration, all the research done http://www.gnu.org/software/classpath/stories.html etc). That doesn't mean we aren't looking for innovations, but we are looking for ways to make them as transparent as possible to our users. GNU Classpath should just feel/be better out of the box then other (proprietary) implementations without the user having to explicitly use extensions in their programs. Who knows what happens when we are entering the endgame and GNU Classpath is as complete as the proprietary non-free implementation. But we are not there yet, and that is certainly still one or two years ahead of us. But even then for a core class library implementation being conservative about extensions is a good thing. If you aren't careful you have to support a new way to use the library for years and then you will have to make really sure that it is worth it both for your users and to the developers that need to maintain backwards compatibility with it. Cheers, Mark signature.asc Description: This is a digitally signed message part ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
RE: Patches to java.util.zip by Christian Schlichtherle
on Wed, 31 Aug 2005, Christian Schlichtherle wrote: More specifically, the size and compressed size field in the ZipEntry class are causing the problems as some comparisons are happening on these. The issue is that once a big integer equal or greater than 2*1024^3 and smaller than 4*1024^3 is stored into a Java int, it is hard to use this int as if it were unsigned. You would have to do something like this on an int (untested): final static long NUM_FOUR_BYTE_INTS = 256L * 256L * 256L * 256L; // Number of 4 byte ints private static long int2long(int i) { return i = 0 ? (long)i : (long)NUM_FOUR_BYTE_INTS - (long)i; } private static void doIt() { int ok = 2 * 1024 * 1024 * 1024 - 1; // max signed int int tooBig = 3 * 1024 * 1024 * 1024; // Creates a negative integer // Compare ok and tooBig assert int2long(ok) int2long(tooBig) : This is ok!; assert ok tooBig : This fails!; } This is way too much computational overhead and it is way too easy to forget this on a comparison. Thus, the better way to deal with this issue is hold 4 byte signed ints in a Java long if you don't need to care for the memory overhead. Well, you could do that. But I find that: long x = (i 0xL); Converts an unsigned int (i) into a signed long (x) correctly just fine. /Sven ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
Re: Patches to java.util.zip by Christian Schlichtherle
Stephen Crawley wrote: Let us not beat about the bush. It would be bad for everyone (except Microsoft) if Java implementors were allowed to modify the APIs of the Java Class Libraries. If you are a Java(TM) licensee, I think you can modify the APIs through the respective JSRs at the JCP. Sun, IBM, and others do it all the time, afaict. The J2SE growth I see with each version has to come from somewhere :) If you are not a Java(TM) technology licensee, then you can, essentially, do whatever you want, modulo violating other people's rights, of course. Even if the API changes are upwards compatible (e.g. adding visible methods, constructors, fields or classes) the cause trouble. Specifically, if vendor XYZ has enhanced the APIs, then someone developing an application using vendor XYZ's Java platform may well find that their software does not run on other vendor's Java platform. Yeah, I fully agree. I see that sometimes, myself, too, and find it pretty annoying. Some people are apparently ocassionally developing their programs written in the Java programming language on some closed, proprietary implementation that contains APIs, which enhance the ones in GNU Classpath. So when I try to use their applications on a free runtime, like Kaffe, their applications just don't work. That's why educating developers about the Java Trap is important. Those with long memories will remember that Microsoft tried this trick. They even went so far as to drop RMI completely. And JNI. http://www.javaworld.com/jw-10-1997/jw-10-sunsuit.html It was only the Sun license (and Sun's willingness to take Microsoft to court) that saved the Java community from the hell of vendor specific Java enhancements. I only wish Sun had saved the Java community from com.sun.* and sun.*, tools.jar and similar Sun-specific, non-standarsised nosense, as well. I can't have it all, I guess :) cheers, dalibor topic ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
Re: Patches to java.util.zip by Christian Schlichtherle
Mark Wielaard wrote: But even then for a core class library implementation being conservative about extensions is a good thing. If you aren't careful you have to support a new way to use the library for years and then you will have to make really sure that it is worth it both for your users and to the developers that need to maintain backwards compatibility with it. I fully agree. I think the J2SE development over the last 10 years offers a strong argument in favour of conservativism. In 1996, the Java 1.0 Nutshell book came at whopping 460 pages, describing the language, and the APIs. In 1995, the Java in Nutshell book is now 1252 pages. And the language has not changed that much, to warrant such a blowup: it largely due to the JCP putting every API within their reach that could not run away fast enough into the J2SE. As a result, countless trees have been decimated for documenting all the backward compatibility cruft. ;) cheers, dalibor topic ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
Re: Patches to java.util.zip by Christian Schlichtherle
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 http://www.javaworld.com/jw-10-1997/jw-10-sunsuit.html Interesting article, especially this one: And while Sun can exercise its legal rights over the use of the Java name, it can do little to stop Microsoft from implementing a clean room version of the Java virtual machine, says Morgenthal. :) cu Robert -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFDFk2vG9cfwmwwEtoRAombAJ9UTVX6pPf1WLBog1fBIAOKkUCWrQCfQzGF 4iEmTksWnekMs9YRe8Aug1o= =9i55 -END PGP SIGNATURE- ___ Classpath mailing list Classpath@gnu.org http://lists.gnu.org/mailman/listinfo/classpath
Re: Patches to java.util.zip by Christian Schlichtherle
Hi, On Sun, 2005-07-24 at 12:29 +0700, John Leuner wrote: Unfortunately I haven't been able to give this task the attention it deserves and I want to give someone else the opportunity to deal with these patches. Below are some of the emails exchanged between Christian, Mark Wielaard and myself. I went through what I reviewed earlier and committed all uncontroversial stuff. I didn't actually review the big part of the patch. And I don't think we should add new public/protected constructors or methods. We should probably move the new functionality to a new gnu.java.util.zip package with extended functionality if possible. But I quickly went through some of the things that I could quickly format correctly and that I think can and should be submitted separately since they are good changes on their own anyway. Reformatted Adler32.java to follow out style guide and make the diff minimal and updated copyright year. This can probably be submitted separately with a little ChangeLog message plus a short explanation of the why of the changed (any benchmarks?) I haven't done any benchmarks myself. We probably should also have a little mauve tests for Adler32 to make sure we don't introduce any bugs. Cleaned up patch attached. Dropped the change to java/util/zip/CRC32.java which I didn't understand (what is CRC32If?): -public class CRC32 implements Checksum +public class CRC32 implements CRC32If Same for the changes to java/util/zip/Deflater.java (what is DeflaterIf?) Dropped the change to java/util/zip/DeflaterEngine.java private methods are always final: - private void updateHash() { + private final void updateHash() { Added copyright year to java/util/zip/DeflaterHuffman.java and changed order of modifiers to amtch coding styleguide. (Can also be submitted separately as obvious fix with ChangeLog entry.) Commited as: 2005-08-30 Christian Schlichtherle [EMAIL PROTECTED] * java/util/zip/DeflaterHuffman.java (bit4Reverse): Mark final. Removed the changes to java/util/zip/DeflaterOutputStream.java that involved DeflaterIf (which wasn't included and we cannot change the type and modifier of a protected field). Are you sure you want to comment out/remove the flush() method? (Can also be submitted separately with its own ChangeLog entry.) Dropped the removal of flush() but kept the increased buffer size. Committed as: 2005-08-30 Christian Schlichtherle [EMAIL PROTECTED] * java/util/zip/DeflaterOutputStream.java (DeflaterOutputStream(OutputStream)): Increase buffer size to 4096. (DeflaterOutputStream(OutputStream,Deflater)): Likewise. Updated the copyright year in java/util/zip/Inflater.java and removed the implements InflaterIf. Updated documentation is obviously correct and can be submitted separately with a ChangeLog entry. A similar fix was already applied earlier: 2005-07-13 David Gilbert [EMAIL PROTECTED] * java/util/zip/Inflater.java: minor API doc fixes. Remove the changes to java/util/zip/InflaterInputStream.java which all had to do with InflaterIf (not included). The change to java/util/zip/OutputWindow.java is unnecessary since private methods are already final: - private void slowRepeat(int rep_start, int len, int dist) + private final void slowRepeat(int rep_start, int len, int dist) java/util/zip/ZipException.java had as only change an update to the copyright years list. That is unnecessary. I need to go through the rest of these patches. It would help if they could be rediffed against current CVS head and extra functionality would be moved to a new package. Thanks, Mark Index: java/util/zip/Adler32.java === RCS file: /cvsroot/classpath/classpath/java/util/zip/Adler32.java,v retrieving revision 1.5 diff -u -r1.5 Adler32.java --- java/util/zip/Adler32.java 22 Jan 2002 22:27:02 - 1.5 +++ java/util/zip/Adler32.java 15 May 2005 19:46:45 - @@ -1,5 +1,5 @@ /* Adler32.java - Computes Adler32 data checksum of a data stream - Copyright (C) 1999, 2000, 2001 Free Software Foundation, Inc. + Copyright (C) 1999, 2000, 2001, 2005 Free Software Foundation, Inc. This file is part of GNU Classpath. @@ -153,6 +153,19 @@ } /** + * In update() we can defer the modulo operation: + * s1 maximally grows from 65521 (BASE) to 65521 + 255 * 3800 + * s2 maximally grows by 3800 * median(s1) = 2090079800 2^31 + * Calculated this value like this: + * + * int i = 0, a = BASE, b = a; + * for(; b 0; i++) + * b += (a += 255); + * System.out.println(max defer = + (i - 1)); + */ + private static final int DEFER = 3850; + + /** * Updates the checksum with the bytes taken from the array. * * @param buf an array of bytes @@ -161,20 +174,13 @@ */ public void update (byte[] buf, int off, int len) { -//(By Per Bothner) int s1 = checksum
Classpath licensing (Re: About the java.util.zip project)
I am copying this to the jazzlib and classpath MLs to get some other peoples' opinions on the Classpath license and how it should apply in a case like this. My name is Dongku, Kim. I am a Japanese developer. Hi My company makes commercial software. And, I work to make a module which makes a zip file from files by Java. So, I want to use your library. But I read the license at the homepage(http://jazzlib.sourceforge.net/). I think the license on your homepage means that I can use the library freely and I dont need to obey the GPL license No, the license says you MUST obey the GPL with one special exception ... if I use the library only by means of called the library to make a executable. If you combine jazzlib with independent modules to produce an executable you can copy and distribute the resulting executable under terms of your choice. So you can use jazzlib in your program without changing the license of your commercial software. For sure, if you can give me an answer for some questions, I am very happy for it. 1) If I use your library without change(source code) to make a commercial ziputility, should I open my ziputility source code? The license says you must combine with independent modules. If you write a commercial zip utility I am not sure that this other code will be an independent module. 2) If I should open my source code, is there other method not to do this? For example, we can pay for the license. No, I doubt this is possible. 3) If I use your library without change(source code) to make a commercial ziputility, what I should do not to violate your license? I am not sure how to answer that, perhaps another developer can answer this question? -- John Leuner [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Classpath licensing (Re: About the java.util.zip project)
Hi, On Sun, 2003-11-23 at 13:48, John Leuner wrote: I am copying this to the jazzlib and classpath MLs to get some other peoples' opinions on the Classpath license and how it should apply in a case like this. Best is to direct such questions to [EMAIL PROTECTED] The FSF is the official copyright holder of Classpath and sets the distribution terms. My company makes commercial software. And, I work to make a module which makes a zip file from files by Java. So, I want to use your library. But I read the license at the homepage(http://jazzlib.sourceforge.net/). I think the license on your homepage means that I can use the library freely and I dont need to obey the GPL license No, the license says you MUST obey the GPL with one special exception ... if I use the library only by means of called the library to make a executable. If you combine jazzlib with independent modules to produce an executable you can copy and distribute the resulting executable under terms of your choice. So you can use jazzlib in your program without changing the license of your commercial software. I believe John his interpretation is correct, but see below. For sure, if you can give me an answer for some questions, I am very happy for it. 1) If I use your library without change(source code) to make a commercial ziputility, should I open my ziputility source code? The license says you must combine with independent modules. If you write a commercial zip utility I am not sure that this other code will be an independent module. Best is to write with a clear example of how you plan do this and what the distribution terms of the larger work are to [EMAIL PROTECTED] so they can give advise on what is and isn't considered an independent module. 2) If I should open my source code, is there other method not to do this? For example, we can pay for the license. No, I doubt this is possible. The FSF does sometimes make exceptions as long as the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally can be guaranteed. You should contact [EMAIL PROTECTED] if you think you can satisfy those goals, but compliance with the current GNU Classpath (jazzlib) distribution terms is problematic. 3) If I use your library without change(source code) to make a commercial ziputility, what I should do not to violate your license? I am not sure how to answer that, perhaps another developer can answer this question? The FSF has a GPL-compliance lab which offers consulting services for companies who are working to develop products that incorporate Free Software so that they can do so in ways that comply with the terms of the GPL and other Free Software licenses. If you are interested in this service, please write a separate message to [EMAIL PROTECTED]. Cheers, Mark signature.asc Description: This is a digitally signed message part ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip speedup
I have created diff of my caching changes to java.util.zip versus current cvs (with merged ZipFile). If anybody will ever work on zip-decompression sensitive code and suffer performance problems, please try out if they help. http://nwn-j3d.sf.net/misc/zipcache.diff As for inclusion in main tree, changes are a bit ugly - very clear example of optimizing over design. But I wanted to put them somewhere on the net, so brave people could try out for themselves. I'm now leaving java.util.zip behind and will continue to play with Graphics2D. Artur ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip speedup
I have taken java version of java.util.zip from CVS and done few optimalizations. I have zipped entire classpath CVS tree (5MB packed). My test program goes through all entries and uncompress each to memory array (same every time, preallocated). I have tested validity of changed by adding write function to recreate all files - it works ok (after fixing one existing bug - current version in CVS has problems with unpacking files with 0 length). Did you do any profiling of the code? Classpath CVS version needs 3150ms. My optimized version manages to do it in 2125ms (results vary +-3ms in each case). I think this is quite reasonable speedup. But real fun comes with memory. Classpath CVS allocates around 130MB of garbage during processing, my version fits in 1MB (most of it being zip entries/names). I suppose that most of time difference comes from less gc work (but I have also made few optimalizations for reading entries from disk). That's quite an improvement. Are you saying that the current version generates 130MB of garbage, and yours creates 1MB? Could you send me a diff of your changes? I'm quite shocked to see how fast sun gc is - even if all difference comes from memory, we are talking about allocating and collecting 130MB in 1 second - this IS a lot. I suppose that with bigger/more complicated heap, gc time, and thus difference, would increase considerably. Anyway, on less advanced jvms, difference in amount of garbage should make greater impact[1]. Now, the ugly part. Few of optimalizations (algorithm and IO-related) are free. But biggest part (memory savings) require some caching. This means that around 35kb is allocated first time zip subsystem is used and will never get freed. There is also a need for quite close look at inflater sharing versus security. My question - is 35kb of memory taken 'forever' an acceptable price for a lot less garbage on runtime ? Bearing in mind that the java.util.zip implementation in Classpath is also available as a seperate library, we should consider the needs of people who might want to use this in an embedded system where space is at a premium. But obviously for modern workstations this first-time allocation is less important than the speed/memory benefits. John Leuner signature.asc Description: This is a digitally signed message part ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip speedup
John Leuner wrote: Did you do any profiling of the code? Yes, many times, but only under sun jre 1.4.1 client hotspot on windows. Both cpu and memory. For the cpu, I do not think there are any suprises - most of time is spend in Inflater.decodeHuffman, with InflaterHuffmanTree.getSymbol contributing the most there (most methods called from decodeHuffman get inlined probably). Next parts are Adler32.update and OutputWindow.repeat (last one can be an artifact due to System.arraycopy happening there, providing good point for gc break). I have done small optimalizations in few places, mostly in zip entry reading (it is now about 5 times faster) - but not in actual decompression routines. That's quite an improvement. Are you saying that the current version generates 130MB of garbage, and yours creates 1MB? Yes. This 1MB is not real 'garbage' - most of it are ZipEntry objects, which are actually used by test program. As for 130MB of garbage, most of it comes from OutputWindow allocation. I'm reading over 3000 files from zip, with 32kb byte[] allocated per file, it is already around 90MB. Next part comes from InflaterInputStream (4kb buffer), and quite a few from InflaterHuffmanTree, where temporary arrays are created each time. There is also an incredible amount of small arrays created in ZipEntry creation (for each read, small 2 or 4 byte array is created) - does not influence byte-count of garbage much, but it means that around 10 arrays are allocated per entry. I have done improvements in two major ways. First important one is changing way zip entries are read - it used to read header pieces word after word, I read entry at once - _very_ big improvement at first zip read. I have also removed most of temporary buffer allocations there. Second is caching Inflater. Unfortunately, I have had to hide this caching from world very deep, so there is no chance of somebody messing with cached Inflater in any way. This basically means that only InputStream created by ZipFile will gain this benefit - with ZipInputStream, GZIPInputStream and generic InflaterInputStream we have almost same situation as before. Inflater in turn cache OutputWindow and HuffmanTrees, which cache their temporary arrays. In reset methods I'm doing quite extensive cleaning - somebody with more zip experience probably will be able to remove some of these precautions (I was not sure if at some place there is not dependency on not-touched array element to be zero, so I clear almost all of them, except obvious cases). Cache works ok with multiple threads - just if you have more than one of InputStreams open at same time, second and following ones will produce garbage (I think that in majority of cases single thread is doing most of reading from zip). Could you send me a diff of your changes? Sure, I'm sending them to you in separate mail. Please note that work is not finished - no documentation is done, some in fact is outdated. I would probably also want to move factory methods to separate class. Bearing in mind that the java.util.zip implementation in Classpath is also available as a seperate library, we should consider the needs of people who might want to use this in an embedded system where space is at a premium. Three basic choices here. 1) In current state, there is zero problem with making it configurable. Everything boils down to two 'factory' free methods, which needs to be no-op for embedded case - so some kind of system or static variable will do the trick. 2) We can make Inflater cached at ZipFile instead of system level. When ZipFile is closed, Inflater would be freed - until then it would get reused for all streams from this ZipFile (as long as they are in single thread of course). Problem here is with the systems which would like to keep many ZipFile instances open, 'just in case' - they would pay multiple cost with not benefit. Variation of it is having reference counting for open ZipFiles - to have single static Inflater cache, but empty it when last ZipFile is closed (and recreate if new one is opened). But still you pay 36kb for having ZipFile open (in addition to possibly a lot for ZipEntries). Artur ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
java.util.zip speedup
I have taken java version of java.util.zip from CVS and done few optimalizations. I have zipped entire classpath CVS tree (5MB packed). My test program goes through all entries and uncompress each to memory array (same every time, preallocated). I have tested validity of changed by adding write function to recreate all files - it works ok (after fixing one existing bug - current version in CVS has problems with unpacking files with 0 length). I have measured time taken for processing using java3d high res timer, counting only part from opening to closing zip file - no pre and post computation, no jvm startup. I have started program many many times, to make sure everything is in memory cache. I have measured both first and average of 9 next iterations, but percentage speedup is similar. I have used sun jre 1.4.1 for testing. Classpath CVS version needs 3150ms. My optimized version manages to do it in 2125ms (results vary +-3ms in each case). I think this is quite reasonable speedup. But real fun comes with memory. Classpath CVS allocates around 130MB of garbage during processing, my version fits in 1MB (most of it being zip entries/names). I suppose that most of time difference comes from less gc work (but I have also made few optimalizations for reading entries from disk). I'm quite shocked to see how fast sun gc is - even if all difference comes from memory, we are talking about allocating and collecting 130MB in 1 second - this IS a lot. I suppose that with bigger/more complicated heap, gc time, and thus difference, would increase considerably. Anyway, on less advanced jvms, difference in amount of garbage should make greater impact[1]. Now, the ugly part. Few of optimalizations (algorithm and IO-related) are free. But biggest part (memory savings) require some caching. This means that around 35kb is allocated first time zip subsystem is used and will never get freed. There is also a need for quite close look at inflater sharing versus security. My question - is 35kb of memory taken 'forever' an acceptable price for a lot less garbage on runtime ? Artur [1] - On sun jvm with interpreter, difference is still around 1 second - but this time between 26s with classpath CVS versus 25s with my version... most time here is probably spend in actual decoding ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Bug in Java.Util.Zip
Hi I've found a bug in Java.Util.Zip : When you compress files with the zipstream with deflatefast (compression level 1-3) sometimes an exception is throwed. If I change in DeflaterHuffman the isFull method from : return last_lit == BUFSIZE; to return last_lit + 16 = BUFSIZE; the bug seems to have gone away, but I'm not sure that is the case for all cases. cya _ Join the worldÂ’s largest e-mail service with MSN Hotmail. http://www.hotmail.com ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Partial java.util.zip merge with libgcj
Hi, The following classes from java.util.zip were easy to merge with libgcj. 2001-10-05 Mark Wielaard [EMAIL PROTECTED] * java/util/zip/Adler32.java: Merge with libgcj * java/util/zip/CRC32.java: Merge with libgcj * java/util/zip/CheckedInputStream.java: New file from libgcj * java/util/zip/CheckedOutputStream.java: Ditto * java/util/zip/Checksum.java: Merge with libgcj * java/util/zip/DataFormatException.java: Ditto * java/util/zip/ZipException.java: Ditto * java/util/zip/Makefile.am: add new classes We can probably share some more classes which will make it easier to find and fix bugs. Does anybody know about a (g)zip test suite? Cheers, Mark -- Stuff to read: http://www.toad.com/gnu/whatswrong.html What's Wrong with Copy Protection, by John Gilmore ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Partial java.util.zip merge with Classpath
Hi, [Discussion on [EMAIL PROTECTED]] On Fri, Oct 05, 2001 at 09:20:56AM -0600, Tom Tromey wrote: Mark == Mark Wielaard [EMAIL PROTECTED] writes: Do we want to use all of jazzlib in libgcj or do we just want to continue using zlib? We could merge more classes then I have done now. Whether we want to replace zlib completely with java code is a seperate question. As Per Bothner already pointed out zlib is in use with a lot of programs and Jazzlib is only of interest to the pure java crowed and is fairly new. But only Deflater and Inflater use native code from zlib. We should really look at merging the other classes that are already written in java in both versions. I am CCing this to the Classpath list so people can comment on how much code can be merged. Switching to jazzlib doesn't mean we can delete zlib from the repository -- we will still need it for gcj. So there isn't a significant maintenance win there. But it does mean one less library to link against for an application compiled with gcj. But I don't know if that is such a great benefit. Even so I think there is a maintenance benefit to merging with Classpath. For one thing it means more potential bug-fixers for the code. That is the reason I am merging classes. Some differences will remain but there are lots of classes that don't have to be maintained seperately. Cheers, Mark -- Stuff to read: http://www.toad.com/gnu/whatswrong.html What's Wrong with Copy Protection, by John Gilmore ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: Partial java.util.zip merge with Classpath
On Saturday, October 6, 2001, at 08:48 AM, Mark Wielaard wrote: Do we want to use all of jazzlib in libgcj or do we just want to continue using zlib? We could merge more classes then I have done now. Whether we want to replace zlib completely with java code is a seperate question. As Per Bothner already pointed out zlib is in use with a lot of programs and Jazzlib is only of interest to the pure java crowed and is fairly new. We ought to do some performance tests. I'd certainly be in favour of switching to Jazzlib provided it doesn't turn out to be significantly slower. regards Bryce. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: jazzlib into java.util.zip
John == John Leuner [EMAIL PROTECTED] writes: John But what I really want to do is lock the CVS repository, so that John it is there for historical purposes only. One way is to set up your commitinfo file to reject every commit. I don't know whether SourceForge lets you edit commitinfo though. Another choice is to `cvs rm' everything and put a big README saying where it all migrated. That way the history will still be there but nobody will be confused by it. Tom ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: jazzlib into java.util.zip
John 2001-09-07 John Leuner [EMAIL PROTECTED] John* java/util/zip/*.java: imported the jazzlib code. Should Johnall be functional. John I compiled it, but haven't run it. It should all work however. Which repository is the master repository? FWIW I'd prefer Classpath. The header comments all mention Jazzlib and not Classpath. I would like to make Classpath the master repository. Is there a way to 'lock' the jazzlib one? The headers need to be changed to reflect Classpath, if someone has some free time, be my guest, otherwise I'll get round to it later. John Leuner ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: jazzlib into java.util.zip
On Sat, Sep 08, 2001 at 03:36:43PM +0100, John Leuner wrote: I would like to make Classpath the master repository. Is there a way to 'lock' the jazzlib one? In the past, I have been successful at killing an SF project, by explaining to SF staff that the project was migrating somewhere else. They are quite helpful, as long as you explain things clearly, and prove to them that you are not trying to make the Free/Opense Source code vanish for your private interests. So, if the whole jazzlib project is merged within Classpath, you can certainly ask them to remove the SF jazzlib project (and show them the Classpath cvsweb link with the jazzlib stuff). Etienne -- Etienne M. Gagnonhttp://www.info.uqam.ca/~egagnon/ SableVM: http://www.sablevm.org/ SableCC: http://www.sablecc.org/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: jazzlib into java.util.zip
On Sat, Sep 08, 2001 at 11:40:31AM -0400, Etienne M. Gagnon wrote: On Sat, Sep 08, 2001 at 03:36:43PM +0100, John Leuner wrote: I would like to make Classpath the master repository. Is there a way to 'lock' the jazzlib one? In the past, I have been successful at killing an SF project, by explaining to SF staff that the project was migrating somewhere else. They are quite helpful, as long as you explain things clearly, and prove to them that you are not trying to make the Free/Opense Source code vanish for your private interests. So, if the whole jazzlib project is merged within Classpath, you can certainly ask them to remove the SF jazzlib project (and show them the Classpath cvsweb link with the jazzlib stuff). That is an option. But what I really want to do is lock the CVS repository, so that it is there for historical purposes only. John Leuner ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: jazzlib into java.util.zip
John == John Leuner [EMAIL PROTECTED] writes: John 2001-09-07 John Leuner [EMAIL PROTECTED] John* java/util/zip/*.java: imported the jazzlib code. Should Johnall be functional. John I compiled it, but haven't run it. It should all work however. Which repository is the master repository? FWIW I'd prefer Classpath. The header comments all mention Jazzlib and not Classpath. Tom ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
java.util.zip
Hi, I'm trying to build the current cvs tree and can't due to dependencies between java.util.jar and java.util.zip. It seems there are some source files still to be checked in for java.util.zip. I can't e-mail the maintainer directly so I thought I'd ask here if anyone knows when the missing files will be added to java.util.zip? The web site says things are nearly complete. TIA, Ian Rogers ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
Hi, On Mon, Sep 03, 2001 at 10:34:26AM +, Ian Rogers wrote: I'm trying to build the current cvs tree and can't due to dependencies between java.util.jar and java.util.zip. Sorry, that is my fault. I checked in java/net/URLClassLoader.java which depends on the java.util.jar package which doesn't compile with plain Classpath because of the missing java.util.zip files. The attached patch to lib/standard.omit fixes the problem. I will commit it in little while. The reason that I didn't see the problem is because I configure classpath with: ./configure --with-jikes --with-japhar --with-javah=/usr/local/gcc-3.0/bin/gcjh --with-classlib=/usr/local/gcc-3.0/share/libgcj.jar And libgcj does contain all the java.util.zip files. It seems there are some source files still to be checked in for java.util.zip. I can't e-mail the maintainer directly so I thought I'd ask here if anyone knows when the missing files will be added to java.util.zip? The web site says things are nearly complete. The website is a bit out of date, but we were hoping to import jazzlib http://jazzlib.sourceforge.net/. I don't know if that is actually going to happen. Cheers, Mark -- Stuff to read: http://www.toad.com/gnu/whatswrong.html What's Wrong with Copy Protection, by John Gilmore Index: lib/standard.omit === RCS file: /cvs/classpath/lib/standard.omit,v retrieving revision 1.25 diff -u -u -r1.25 standard.omit --- lib/standard.omit 2001/06/25 04:43:56 1.25 +++ lib/standard.omit 2001/09/03 20:19:24 @@ -1,5 +1,6 @@ ../java/net/Authenticator.java$ ../java/net/JarURLConnection.java$ +../java/net/URLClassLoader.java$ ../java/lang/RuntimePermission.java$ ../java/text/MessageFormat.java$ ../gnu/test/.*$
Re: java.util.zip
Ian == Ian Rogers [EMAIL PROTECTED] writes: Ian I'm trying to build the current cvs tree and can't due to Ian dependencies between java.util.jar and java.util.zip. It seems Ian there are some source files still to be checked in for Ian java.util.zip. I can't e-mail the maintainer directly so I Ian thought I'd ask here if anyone knows when the missing files will Ian be added to java.util.zip? The web site says things are nearly Ian complete. I don't know the precise status of java.util.zip. There is a zip implementation in libgcj you could use. It relies on native code, though. Someone rewrote this code to use JNI a while back but I don't know what happened with that. Another choice is `jazzlib', which is a pure Java zlib implementation. I don't know the status of this either. Either way, some work is required. Lately I've been doing a fair amount of merge work, but java.util.zip is pretty low on my list (currently I'm working on java.text, but io, lang, security, and net are all ahead of zip). Tom ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
On Mon, Sep 03, 2001 at 02:45:41PM -0600, Tom Tromey wrote: Another choice is `jazzlib', which is a pure Java zlib implementation. I don't know the status of this either. I've checked the jazzlib web site. In the CVS repository, all source files bear the FSF copyright, and are licensed under the GPL+Classpath exception. I see no reason why this code could't be checked-in Classpath's CVS repository. Jazzlib has the advantage of being written in pure Java. I do not have CVS access. Maybe somebody who has could do it, or tell us why it can't be done (missing paper work, etc.)? Any volunteer? :) Etienne -- Etienne M. Gagnon [EMAIL PROTECTED] SableVM: http://www.sablevm.org/ SableCC: http://www.sablecc.org/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
Etienne M. Gagnon [EMAIL PROTECTED] writes: On Mon, Sep 03, 2001 at 02:45:41PM -0600, Tom Tromey wrote: Another choice is `jazzlib', which is a pure Java zlib implementation. I don't know the status of this either. I've checked the jazzlib web site. In the CVS repository, all source files bear the FSF copyright, and are licensed under the GPL+Classpath exception. John Leuner and Jochen Hoenicke should be able to do this when they are ready. -- Brian Jones [EMAIL PROTECTED] ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
jazzlib (java.util.zip implementation in java).
Hello, We (John Leuner and I) have assigned copyright for jazzlib to the FSF, so there shouldn't be any legal obstacle to including the code in classpath anymore. As far as I can see, there is no real java.util.zip implementation in classpath yet, so do you agree if I just commit the content of jazzlib to the classpath repository, overwriting the three small existing classes? The existing classes are just two exceptions and an interface. The other problem is the merge with libgcj as it already contains a working java.util.zip implementation based on CNI. However, the java.util.zip package wasn't merged yet, so there is no need to resolve this now. Jochen ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: jazzlib (java.util.zip implementation in java).
Jochen Hoenicke wrote: As far as I can see, there is no real java.util.zip implementation in classpath yet, so do you agree if I just commit the content of jazzlib to the classpath repository, overwriting the three small existing classes? That's fine. As you commit files, could you please go through them and clean up the formatting some to match the rest of Classpath? From a quick look over the source this morning, I'm referring to things like spaces between method names and the following parens, and spaces between keywords (if, etc) and their following parens. The other problem is the merge with libgcj as it already contains a working java.util.zip implementation based on CNI. However, the java.util.zip package wasn't merged yet, so there is no need to resolve this now. I agree. Paul ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: jazzlib (java.util.zip implementation in java).
On Jul 31, Paul Fisher wrote: Jochen Hoenicke wrote: As far as I can see, there is no real java.util.zip implementation in classpath yet, so do you agree if I just commit the content of jazzlib to the classpath repository, overwriting the three small existing classes? That's fine. As you commit files, could you please go through them and clean up the formatting some to match the rest of Classpath? From a quick look over the source this morning, I'm referring to things like spaces between method names and the following parens, and spaces between keywords (if, etc) and their following parens. Okay, I committed the space-changes to the jazzlib repository. There may be a few places left where spacing is not 100% correct, but I think I corrected most. On Jul 31, Aaron M. Renn wrote: Is Jazzlib 100% Java? If so, I would consider it superior to a native implementation, ceteris paribus. Yes it is 100% Java. Jochen ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
John Leuner wrote: Jochen has also completed a pure-java implementation of java.util.zip. I don't know if this will be the preferred implementation for classpath, but I suggest that it at least be a compile-time option. You can get the source and compiled classes at http://jazzlib.sourceforge.net/ Yeah! This is great! I would love to have this as the default implementation in Classpath. Would there be any reason to not make the pure java version the default? I don't see any. The code is functionally equivalent. Bryce McKinlay wrote: I agree, this is very cool. Have you run any tests to compare performance with the JDK's native zlib-based implementation? Reading compressed streams takes ca. 50% more time than with Sun's zlib based implementation. IIRC creating compressed streams was around factor 2 slower. This is with the IBM Java2-1.3 virtual machine and its JIT compiler. Some of this slow down is due to array bound checks in java. I don't know how good the JIT compiler is in inlining short functions, this may also explain some slow-down. Does it pass the ZipVerify test case? (see http://gcc.gnu.org/ml/java/2001-03/msg00374.html) Thanks for the link, I haven't seen this test before. There is a problem with ZipEntry.setExtra(). My version is more stringent at what kind of extra fields it allows. It doesn't allow a NullPointer and it only allows extra fields formed as described in appnotes.iz (zip file format spec). Sun's jdk doesn't care at all about the content but my implementation makes use of it to read the extended timestamp. When changing my implementation to ignore wrong parameters to setExtra() all tests pass. Jochen ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
java.util.zip
At http://www.gnu.org/software/classpath/status.html#java.util.zip you can read -- java.util.zip Maintainer: Loren Peace [EMAIL PROTECTED] Almost complete. -- But I haven't found a single class from java.util.zip in ftp://alpha.gnu.org/pub/gnu/classpath/classpath-0.02.tar.gz Where can I get the ones which are complete ? I've created java.util.zip.ZipConstants, how can I contribute it to Classpath? Regards, Elifarley C. Coelho [EMAIL PROTECTED] ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
Hi, On Mon, Mar 26, 2001 at 12:41:23PM -0300, Elifarley wrote: At http://www.gnu.org/software/classpath/status.html#java.util.zip Sorry, those pages are very old. We haven't heard from Loren Peace for a long time. The current plan is to import the java.util.zip implementation from libgcj http://gcc/gnu.org/java/. The only problem is that the native part is written in CNI and for Classpath we want (also) a JNI version in Classpath so VMs that only support JNI can also use it. Jochen Hoenicke has made a CNI to JNI conversion that can be found at http://www.informatik.uni-oldenburg.de/~delwi/classpath/JCL/. I still have not gotten around to testing and importing it in Classpath. I've created java.util.zip.ZipConstants, how can I contribute it to Classpath? Could you compare it to the version in libgcj? If you want to contribute large (more then 10/20 lines) pieces of code you do have to sign over your copyright to the FSF. That way all Classpath code can be legally defended by the FSF. Thanks, Mark -- Stuff to read: http://www.toad.com/gnu/whatswrong.html What's Wrong with Copy Protection, by John Gilmore ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
http://www.gnu.org/software/classpath/status.html#java.util.zip Sorry, those pages are very old. We haven't heard from Loren Peace for a long time. The current plan is to import the java.util.zip implementation from libgcj http://gcc/gnu.org/java/. The only problem is that the native part is written in CNI and for Classpath we want (also) a JNI version in Classpath so VMs that only support JNI can also use it. Jochen Hoenicke has made a CNI to JNI conversion that can be found at http://www.informatik.uni-oldenburg.de/~delwi/classpath/JCL/. I still have not gotten around to testing and importing it in Classpath. Jochen has also completed a pure-java implementation of java.util.zip. I don't know if this will be the preferred implementation for classpath, but I suggest that it at least be a compile-time option. You can get the source and compiled classes at http://jazzlib.sourceforge.net/ I've created java.util.zip.ZipConstants, how can I contribute it to Classpath? Could you compare it to the version in libgcj? If you want to contribute large (more then 10/20 lines) pieces of code you do have to sign over your copyright to the FSF. That way all Classpath code can be legally defended by the FSF. If the code mentioned above gets included in Classpath, then I will have to sign some documentation (even though I wrote only a fraction of the code). How do I go about doing this? John Leuner -- ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
Hi, On Mon, Mar 26, 2001 at 08:52:32PM +, John Leuner wrote: Jochen has also completed a pure-java implementation of java.util.zip. I don't know if this will be the preferred implementation for classpath, but I suggest that it at least be a compile-time option. You can get the source and compiled classes at http://jazzlib.sourceforge.net/ Yeah! This is great! I would love to have this as the default implementation in Classpath. Would there be any reason to not make the pure java version the default? (Most JVMs will probably link to zlib already but some might not.) If you want to contribute large (more then 10/20 lines) pieces of code you do have to sign over your copyright to the FSF. That way all Classpath code can be legally defended by the FSF. If the code mentioned above gets included in Classpath, then I will have to sign some documentation (even though I wrote only a fraction of the code). How do I go about doing this? Officially Paul Fisher mailto:[EMAIL PROTECTED] handles all FSF papers. Could you send him email to ask if he can send you the appropriate papers to sign. Some quick questions: Was any code shared with the jcraft people? Do you have a record of all people that contributed to the code? Is there a official ChangeLog file? (I can get most from CVS I suppose.) Thanks, Mark -- Stuff to read: http://www.toad.com/gnu/whatswrong.html What's Wrong with Copy Protection, by John Gilmore ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
Mark Wielaard wrote: Yeah! This is great! I would love to have this as the default implementation in Classpath. Would there be any reason to not make the pure java version the default? (Most JVMs will probably link to zlib already but some might not.) Only problem is that most jvms would like to read core lib from zipped/jarred archive... and to do this need zlib on native side. So it is basic bootstrapping problem - you need zip implementation to read zip implementation :) Of course one can possiblty store some basic classes in normal filesystem, with rest in zip. But it is quite tricky to get everything right. Artur ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
Yeah! This is great! I would love to have this as the default implementation in Classpath. Would there be any reason to not make the pure java version the default? (Most JVMs will probably link to zlib already but some might not.) Only problem is that most jvms would like to read core lib from zipped/jarred archive... and to do this need zlib on native side. So it is basic bootstrapping problem - you need zip implementation to read zip implementation :) Yeah, but that's the VM's problem. It wouldn't use java.util.zip anyway (native or non-native). Of course one can possiblty store some basic classes in normal filesystem, with rest in zip. But it is quite tricky to get everything right. Again it's up to the VM implementor. John Leuner -- ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
Jochen has also completed a pure-java implementation of java.util.zip. I don't know if this will be the preferred implementation for classpath, but I suggest that it at least be a compile-time option. You can get the source and compiled classes at http://jazzlib.sourceforge.net/ Yeah! This is great! I would love to have this as the default implementation in Classpath. Would there be any reason to not make the pure java version the default? I don't see any. The code is functionally equivalent. If you want to contribute large (more then 10/20 lines) pieces of code you do have to sign over your copyright to the FSF. That way all Classpath code can be legally defended by the FSF. If the code mentioned above gets included in Classpath, then I will have to sign some documentation (even though I wrote only a fraction of the code). How do I go about doing this? Officially Paul Fisher mailto:[EMAIL PROTECTED] handles all FSF papers. Could you send him email to ask if he can send you the appropriate papers to sign. Ok. Some quick questions: Was any code shared with the jcraft people? No Do you have a record of all people that contributed to the code? Jochen and myself. Some of the code was originally from libgcj (I saw Tom Tromey's name). Is there a official ChangeLog file? (I can get most from CVS I suppose.) I'm not sure. John Leuner -- ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
I should have made it more clear that this is a port to pure java code. My hidden agenda here is to use the code in a future java OS (see http://jos.org), which is why I didn't want to use zlib. IIRC, everything except Deflater and Inflater are already pure java in libgcj. Deflater and Inflater are just interfaces to zlib, so it makes sense for you to rewrite them from scratch. You should borrow some code from zlib, of course. And make sure you have RFC 1950-1952 handy. I'm working from the zlib C source and the rfc documents. I will have a look at the libgcj code and Jochen's code, to see what I can borrow / contribute in terms of code and javadoc. You can probably take all other classes in java.zip from libgcj without changes. Regarding javadoc I think there is very little in libgcj. Well the latest CVS seems to be pretty good, I'll try merge what docs I have and suggest some patches. John Leuner ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
I should have made it more clear that this is a port to pure java code. My hidden agenda here is to use the code in a future java OS (see http://jos.org), which is why I didn't want to use zlib. Nice. But doesn't JOS support JNI? How are other non-pure-java libraries suported? You could even consider implementing CNI support for JOS if you are concerned about the speed barrier of JNI. We have JNI support in one of the current alpha implementations, but again it's a political / philosophical issue, not one of practicality. I assume CNI requires C++ support? IIRC, everything except Deflater and Inflater are already pure java in libgcj. Deflater and Inflater are just interfaces to zlib, so it makes sense for you to rewrite them from scratch. You should borrow some code from zlib, of course. And make sure you have RFC 1950-1952 handy. I think having a pure java.util.zip is very sexy but keep in mind that the zlib library has been very well tested and by porting it to java you could introduce subtle bugs. So maybe only writing the JNI/CNI wrapper isn't such a bad choice. There is always that risk. I can test against the JNI implementation and other established utilities anyway, so there isn't such a great risk. I will have a look at the libgcj code and Jochen's code, to see what I can borrow / contribute in terms of code and javadoc. You can probably take all other classes in java.zip from libgcj without changes. Regarding javadoc I think there is very little in libgcj. I just comitted my Javadoc additions to the libgcj zip code and Tom Tromey just comitted a couple of small code aditions so make sure you get the very latest version from CVS. I'll do that. Thanks. John Leuner ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
"John" == John Leuner [EMAIL PROTECTED] writes: John I assume CNI requires C++ support? Yes. Tom ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
On Nov 16, John Leuner wrote: I should have made it more clear that this is a port to pure java code. My hidden agenda here is to use the code in a future java OS (see http://jos.org), which is why I didn't want to use zlib. IIRC, everything except Deflater and Inflater are already pure java in libgcj. Deflater and Inflater are just interfaces to zlib, so it makes sense for you to rewrite them from scratch. You should borrow some code from zlib, of course. And make sure you have RFC 1950-1952 handy. I will have a look at the libgcj code and Jochen's code, to see what I can borrow / contribute in terms of code and javadoc. You can probably take all other classes in java.zip from libgcj without changes. Regarding javadoc I think there is very little in libgcj. Jochen ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
Hi, On Fri, Nov 17, 2000 at 03:36:00PM +0100, Jochen Hoenicke wrote: On Nov 16, John Leuner wrote: I should have made it more clear that this is a port to pure java code. My hidden agenda here is to use the code in a future java OS (see http://jos.org), which is why I didn't want to use zlib. Nice. But doesn't JOS support JNI? How are other non-pure-java libraries suported? You could even consider implementing CNI support for JOS if you are concerned about the speed barrier of JNI. IIRC, everything except Deflater and Inflater are already pure java in libgcj. Deflater and Inflater are just interfaces to zlib, so it makes sense for you to rewrite them from scratch. You should borrow some code from zlib, of course. And make sure you have RFC 1950-1952 handy. I think having a pure java.util.zip is very sexy but keep in mind that the zlib library has been very well tested and by porting it to java you could introduce subtle bugs. So maybe only writing the JNI/CNI wrapper isn't such a bad choice. I will have a look at the libgcj code and Jochen's code, to see what I can borrow / contribute in terms of code and javadoc. You can probably take all other classes in java.zip from libgcj without changes. Regarding javadoc I think there is very little in libgcj. I just comitted my Javadoc additions to the libgcj zip code and Tom Tromey just comitted a couple of small code aditions so make sure you get the very latest version from CVS. Cheers, Mark ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
John Leuner [EMAIL PROTECTED] writes: I should have made it more clear that this is a port to pure java code. My hidden agenda here is to use the code in a future java OS (see http://jos.org), which is why I didn't want to use zlib. I will have a look at the libgcj code and Jochen's code, to see what I can borrow / contribute in terms of code and javadoc. That's great. It is what I thought you were doing. It's probably a good idea to borrow the javadoc and get the similar zlib type code working in Java. I definitely prefer a pure java implementation to the native zlib backed implementation. I've cc'd Paul F. on this so he can communicate with you about paper work. Any idea how long you think it will take to implement... possibly how the problem can be broken down into bits different people can work on? Brian -- Brian Jones [EMAIL PROTECTED] ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
java.util.zip
I've started a port of the zlib code to java. This is a non-trivial exercise and I don't want to promise that I'll finish it. So far I've written a lot of the inflation and huffman tree building code, but none of it works yet. If anyone wants to help out with the zlib engine stuff (or with the Java wrapper classes, java.util.jar in particular), feel free to contact me. I assume that if I do finish this I'll have to sign some paperwork, could someone give me advice on this? John Leuner ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
John Leuner ([EMAIL PROTECTED]) wrote: I've started a port of the zlib code to java. This is a non-trivial exercise and I don't want to promise that I'll finish it. So far I've written a lot of the inflation and huffman tree building code, but none of it works yet. If anyone wants to help out with the zlib engine stuff (or with the Java wrapper classes, java.util.jar in particular), feel free to contact me. I assume that if I do finish this I'll have to sign some paperwork, could someone give me advice on this? Hi. We've merged with libgcj over at Cygnus and they have already written this package. What we need to do is port it over to Classpath. The problem is that we still have some questions outstanding on how to handle native code, so this package is in a bit of limbo, but you should not write a new implementation from scratch. -- Aaron M. Renn ([EMAIL PROTECTED]) http://www.urbanophile.com/arenn/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
Aaron What we need to do is port it over to Classpath. The problem Aaron is that we still have some questions outstanding on how to Aaron handle native code, so this package is in a bit of limbo, but Aaron you should not write a new implementation from scratch. I thought someone had ported the java.util.zip native code from CNI to JNI already. We could just check that in as a temporary measure until the CNI/JNI integration problem is solved. I've been looking at the integration problem lately. I don't have anything ready to show yet. Tom ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip
Hi, On Wed, Nov 15, 2000 at 04:50:04PM -0700, Tom Tromey wrote: Mark I have been adding Javadoc to the java.util.zip java files from Mark libgcj but have not yet checked that in the libgcj tree. Mark (It is not completely finished.) Don't feel like you have to have it completely finished to do a commit. Any additions to the javadoc comments are worth committing. Yes you are right. Will do. Mark If you could look at the current libgcj implementation, Jopchen Mark his native code conversion and the jar implementation of Mark Classpath and compare that to what you have that would be Mark greatly appreciated. I definitely don't have time to do that :-( Sorry, wrong To: field. I was trying to encourage the original poster, John Leuner, to please help use with reviewing the current code even if that means that we won't use his implementation right away. Now that he has looked into zip, jar and zlib for his own implementation plans he would be great to have as a reviewer of the current code. Cheers, Mark ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: java.util.zip JNI/CNI
On Jun 14, Artur Biesiadowski wrote: Jochen Hoenicke wrote: I have removed the synchronization (though the perl script supports it), since it is unnecessary. The De/Inflater is by design not thread safe, since de/inflate and setInput are distinct functions. The comment I put in the header of the java files documents this. Please only doublecheck it is impossible to crash native code with malicious multithreading. It can fail with any exception, kill thread etc, but crashing a vm would be a security problem. Artur is right here, I didn't think of this. All methods, that call a zlib methods, except init should be synchronized. The native files should be read again with security in mind. We must also check if zstream is null and throw a NullPointerException otherwise. Mark, if you want I can correct this. Jochen
Re: java.util.zip JNI/CNI
Jochen Hoenicke wrote: Please only doublecheck it is impossible to crash native code with malicious multithreading. It can fail with any exception, kill thread etc, but crashing a vm would be a security problem. Artur is right here, I didn't think of this. All methods, that call a zlib methods, except init should be synchronized. The native files should be read again with security in mind. We must also check if zstream is null and throw a NullPointerException otherwise. Note that in CNI/libgcj, this is not an issue, because an unhandled sigsegv in native code will be automatically mapped into a NullPointerException. So, the synchronization in the CNI version may still be removed. regards [ bryce ]
Re: java.util.zip JNI/CNI
Bryce McKinlay wrote: Note that in CNI/libgcj, this is not an issue, because an unhandled sigsegv in native code will be automatically mapped into a NullPointerException. So, the synchronization in the CNI version may still be removed. If SIGSEGV will occur at arbitrary place it could be bad - serious memory leaks and maybe leaving zlib in inconsitent state so it won't be usable again. Just guessing, as I don't know zlib internals, but it is something to look at. On the other hand, if you think about CNI/libgcj such precautions are not needed for very different reason - you can just add a bit of C++ code that will crash/leak/anything. And as long as one is running only trusted code, there is no problem with requiring him to behave Deflater-friendly in code. Artur
Re: java.util.zip JNI/CNI
"Bryce" == Bryce McKinlay [EMAIL PROTECTED] writes: We must also check if zstream is null and throw a NullPointerException otherwise. Bryce Note that in CNI/libgcj, this is not an issue, because an Bryce unhandled sigsegv in native code will be automatically mapped Bryce into a NullPointerException. So, the synchronization in the CNI Bryce version may still be removed. I think we probably want explicit checks to handle the case of an MMU-less system. In this case we probably also want some kind of configuration point (aka "#if") so that we can disable the explicit check. Tom
Re: java.util.zip JNI/CNI
"Jochen" == Jochen Hoenicke [EMAIL PROTECTED] writes: With CNI a reference to gnu.gcj.RawData is used to hold data that the native code needs. With JNI a reference to a byte[] is used. It seems that a reference to RawData will not be touched by the Garbage collector. I am not sure why that is necessary in this case. Jochen This depends on the garbage collector. Some garbage collector Jochen may depend that all object fields point to real objects Jochen allocated with NewObject. I think many JVMs enforce that. Jochen With the byte[] I'm compatible but have a little bit more Jochen overhead. In some cases the efficiency of using RawData might be important. I don't have an example. In this particular case using byte[] is probably fine. Tom
Re: java.util.zip JNI/CNI
"Mark" == Mark Wielaard [EMAIL PROTECTED] writes: Mark The CNI version does all Nullpointer and array bounds checking Mark in native code. The JNI version does all that checking in Mark java. I am not sure why you would want to do that checking in Mark native code in this case. To avoid extra overhead. With CNI, having a single method is more efficient than having a Java part and a native part for each method. Mark The CNI version seems to synchronize all the native calls, the Mark JNI version does not. Yes. That represents a fairly conservative choice on my part. Tom
Re: java.util.zip JNI/CNI
Please only doublecheck it is impossible to crash native code with malicious multithreading. It can fail with any exception, kill thread etc, but crashing a vm would be a security problem. Artur is right here, I didn't think of this. All methods, that call a zlib methods, except init should be synchronized. The native files should be read again with security in mind. We must also check if zstream is null and throw a NullPointerException otherwise. Note that in CNI/libgcj, this is not an issue, because an unhandled sigsegv in native code will be automatically mapped into a NullPointerException. So, the synchronization in the CNI version may still be removed. But will "malicious multithreading" always cause a sigsegv? Isn't its behaviour more undefined? John
Non-native java.util.zip ?
I'm just wondering how hard it would be to produce non-native version of java.util.zip. Not a big gain as VM would need native zlib to bootstrap it from jar file, but it could be interesting to see how much it would suck speed-wise... Artur
Re: java.util.zip JNI/CNI
On Jun 12, Mark Wielaard wrote: Hi, I just looked at the differences between the java.util.zip implementation of libgcj with CNI and the version that Jochen Hoenicke made with JNI. http://www.Informatik.Uni-Oldenburg.DE/~delwi/classpath/JCL/ Since I know nothing of CNI or JNI I will need to learn a bit more about it. But I thought that it might be a good idea to tell you my first impressions so people can correct me when I am wrong. CNI is readable and JNI is not. This might be because the Perl script that Jochen made inserts a bit more casts and JNI functions then necessary. The JNI is really a bit unreadable. With the unnecessary casts you probably mean jobject to jbytearray, which are all typedefed to the same type in C. But for C++ they are necessary. With CNI a reference to gnu.gcj.RawData is used to hold data that the native code needs. With JNI a reference to a byte[] is used. It seems that a reference to RawData will not be touched by the Garbage collector. I am not sure why that is necessary in this case. This depends on the garbage collector. Some garbage collector may depend that all object fields point to real objects allocated with NewObject. I think many JVMs enforce that. With the byte[] I'm compatible but have a little bit more overhead. Classpath also contains its own native state library (NSA), which uses a hashtable to map objects' internal hashcode to data. This has other disadvantages: It has more runtime overhead, since it has to lookup the hashtable and there may only be one native state per object, so if the super class already uses it, you can't use it. The CNI version does all Nullpointer and array bounds checking in native code. The JNI version does all that checking in java. I am not sure why you would want to do that checking in native code in this case. I like the JNI version more since that does all the sanity checking in a java wrapper and then hands of the checked data to the native code. :-) The CNI version seems to synchronize all the native calls, the JNI version does not. That is how I interpret the first line of every CNI function: JvSynchronize sync (this); I have removed the synchronization (though the perl script supports it), since it is unnecessary. The De/Inflater is by design not thread safe, since de/inflate and setInput are distinct functions. The comment I put in the header of the java files documents this. Jochen
Re: Non-native java.util.zip ?
Artur Biesiadowski wrote: I'm just wondering how hard it would be to produce non-native version of java.util.zip. Not a big gain as VM would need native zlib to bootstrap it from jar file, but it could be interesting to see how much it would suck speed-wise... It probibly wouldn't actually be all that difficult. I don't think (m)any VMs actually use java.util.zip for bootstrap classloading. Kaffe has its own zip code for classloading, written in C (which does not depend on zlib, IIRC). libgcj doesn't need zip for bootstrap loading because core classes are compiled to native code. All you'd need to write (or port from C) is the actual inflate/deflate algorithm itself - dealing with the zipfile headers directory entries is already done in Java code. A good, pure-java zip implementation should actually perform pretty well, and on VMs with a high native call overhead, might actually be faster. regards [ bryce ]
Re: java.util.zip JNI/CNI
Mark Wielaard wrote: With CNI a reference to gnu.gcj.RawData is used to hold data that the native code needs. With JNI a reference to a byte[] is used. It seems that a reference to RawData will not be touched by the Garbage collector. I am not sure why that is necessary in this case. Yes, anything declared as RawData will not be marked or scanned by the GC (unless there is another "real" reference to the same object, of course). This is used to prevent data that looks like a pointer found in non-java objects from being misinterpreted, making the collector a little more precise. This is only necessary when the data pointed to by the RawData was allocated with malloc() or some other non-GC allocator, since you can call _Jv_AllocBytes() to get memory that will not be scanned for pointers. RawData can also be useful to implement weak references and such. The CNI version does all Nullpointer and array bounds checking in native code. The JNI version does all that checking in java. I am not sure why you would want to do that checking in native code in this case. I like the JNI version more since that does all the sanity checking in a java wrapper and then hands of the checked data to the native code. Its done in native code because its a bit cleaner to do it that way. If you did the bounds check in java, you'd need to make inflate(byte[], int, int) non-native, and then have an additional native method that gets called after the bounds check gets done. The actual bounds checking code looks almost exactly the same in CNI as in Java, so its just as readable to have it in the native code. The CNI version seems to synchronize all the native calls, the JNI version does not. That is how I interpret the first line of every CNI function: JvSynchronize sync (this); Correct. It is often hard to say whether something should be synchronized or not in situations where the API documentation does not specify it. In this case I think libgcj is taking a rather conservative approach - any code trying to use the same Inflater from different threads simultaneously sounds fairly broken to me, so the synchronization could probably be removed for efficiency. regards [ bryce ]
java.util.jar, java.util.zip Makefiles
Hi, I am cleaning up and finishing my java.util.jar implementation. But Classpath does not have a java.util.zip implementation yet. Would it be ok to imported the java.util.zip java files from libgcj into Classpath if I added documentation and the standard copyright header? There are already 3 files in java/util/zip but they don't have any javadoc and also not the standard copyright header. They refer to the LIBGCJ_LICENSE or LIBJAVA_LICENSE which are not in the Classpath cvs tree. Or can't this be done until there is a JNI implementation of the native code? What is the status of the Classpath/libgcj JNI/CNI issue? Jochen Hoenicke had converted the libgcj CNI files to JNI using a Perl script that he wrote. Could we add these files to classpath? See http://www.Informatik.Uni-Oldenburg.DE/~delwi/classpath/JCL/. Should I add a Makefile[.am] to the new java/util/jar directory? And if yes, what should be in that file and what does this do? Thanks, Mark
Re: java.util.jar, java.util.zip Makefiles
Mark Wielaard ([EMAIL PROTECTED]) wrote: I am cleaning up and finishing my java.util.jar implementation. But Classpath does not have a java.util.zip implementation yet. Would it be ok to imported the java.util.zip java files from libgcj into Classpath if I added documentation and the standard copyright header? There are already 3 files in java/util/zip but they don't have any javadoc and also not the standard copyright header. They refer to the LIBGCJ_LICENSE or LIBJAVA_LICENSE which are not in the Classpath cvs tree. Or can't this be done until there is a JNI implementation of the native code? What is the status of the Classpath/libgcj JNI/CNI issue? Jochen Hoenicke had converted the libgcj CNI files to JNI using a Perl script that he wrote. Could we add these files to classpath? See http://www.Informatik.Uni-Oldenburg.DE/~delwi/classpath/JCL/. Should I add a Makefile[.am] to the new java/util/jar directory? And if yes, what should be in that file and what does this do? java.util.zip is not merged completely due to the JNI/CNI issues you mentioned. Makefile.am is the automake file used to generate the ultimate Makefile. In our current implementation, I believe all the Makefile.am in java source directories do is make sure the files get included in a package build. Look at another directory for an example. You'll also need to add zlib to the configure.in. Maybe Brian Jones can help with that. -- Aaron M. Renn ([EMAIL PROTECTED]) http://www.urbanophile.com/arenn/
Re: java.util.zip (aka is Loren Peace still alive?)
Daniel Rall ([EMAIL PROTECTED]) wrote: Might want to check out FastJar. I think a search at freshmeat.net will reveal the web site for it. I believe fastjar is a C++ replacement for the Sun jar utility. It does not implement the java.util.jar package from the standard class library. It is GPL'd though, and is probably very useful for people like us. -- Aaron M. Renn ([EMAIL PROTECTED]) http://www.urbanophile.com/arenn/
RE: java.util.zip (aka is Loren Peace still alive?)
Might want to check out FastJar. I think a search at freshmeat.net will reveal the web site for it. Since java.util.jar is basically only manifest parsing, I plan to finish it by the end of May, but I'm getting ahead of myself. Anyway expect zip this week and I'm sorry to have held people up so long.
Re: java.util.zip (aka is Loren Peace still alive?)
Loren Peace ([EMAIL PROTECTED]) wrote: Everyone, sorry I have been a no-show for the last few months, things got pretty busy here, anyway I am 98-99% done, just ironing out the final issues. I should have a usable version, (i.e. still needs some work, but feature complete) up by the end of the weekend, I have only 4 methods still to go. Sounds great! The nicest thing about zip support in the short term is that it will enable us to run out of a zip/jar file (eg, glibj.zip). Since java.util.jar is basically only manifest parsing, I plan to finish it by the end of May, but I'm getting ahead of myself. Anyway expect zip this week and I'm sorry to have held people up so long. Thanks for your help on this. We still have a ton of work to do so it's not like we've all been sitting here just waiting for zip in order to make a complete product or anything. -- Aaron M. Renn ([EMAIL PROTECTED]) http://www.urbanophile.com/arenn/
java.util.zip
What happened to Loren Peace and java.util.zip? I don't recall him replying to the query Aaron put out a while back, but maybe he replied privately ... --John "All great truths begin as blasphemies." - George Bernard Shaw
Re: java.util.zip
John Keiser wrote: What happened to Loren Peace and java.util.zip? I don't recall him replying to the query Aaron put out a while back, but maybe he replied privately ... I was pretty sure, I replied to the group, but in case I didn't I am still alive and working on java.util.zip, there is really not too much left to do if i can ever get the chance to finish it. -- Loren Peace [EMAIL PROTECTED] === Vila Restal: I plan to live forever, or die trying!! (Blakes 7) --- Macintosh. We may not have done everything right, but at least we knew the century was going to end. ===
Re: java.util.zip
John Keiser wrote: Does anyone here have experience with the zip libraries? It would be nice to have java.util.zip in here, as that would make us that much closer to supporting the things that a compiler and JavaDeps would need to run. --John Keiser I am working on java.util.zip and I hope to have it finished, time permitting, by the end of next week. -- Loren Peace [EMAIL PROTECTED] === Vila Restal: I plan to live forever, or die trying!! (Blakes 7) --- Macintosh. We may not have done everything right, but at least we knew the century was going to end. ===
java.util.zip
Does anyone here have experience with the zip libraries? It would be nice to have java.util.zip in here, as that would make us that much closer to supporting the things that a compiler and JavaDeps would need to run. --John Keiser