Re: zip64 compatibility problems

[Martin Buchholz]

Submitted.


On Thu, Mar 21, 2013 at 8:05 AM, Xueming Shen wrote:

> Martin, 8007905 has been approved.
>
>
> On 2/10/13 10:30 PM, Xueming Shen wrote:
>
>> 8007905.
>>
>> But you will have to hold the push. I will send it through the CCC.
>>
>> -Sherman
>>
>> On 2/10/2013 7:59 PM, Martin Buchholz wrote:
>>
>>> On Sat, Feb 9, 2013 at 1:50 AM, Peter Levart
>>>  wrote:
>>>
>>>  Also, this does not disable ZIP64 - it only disables it when it is not
 needed (most other zip implementations can still read the output)
 "inhibit"
 seems better than "disable".


 disfavor ?


  I went for "inhibit".
>>>
>>> There's now a shiny new test case .
>>> Same Bat Place:
>>> http://cr.openjdk.java.net/~**martin/webrevs/openjdk8/**
>>> useZip64For64kEntries/
>>> I am now happy with this change and would like to submit.
>>> As always, I need a bug id.
>>>
>>
>

Re: zip64 compatibility problems

[Xueming Shen]

Martin, 8007905 has been approved.

On 2/10/13 10:30 PM, Xueming Shen wrote:

8007905.

But you will have to hold the push. I will send it through the CCC.

-Sherman

On 2/10/2013 7:59 PM, Martin Buchholz wrote:
On Sat, Feb 9, 2013 at 1:50 AM, Peter Levart  
wrote:



Also, this does not disable ZIP64 - it only disables it when it is not
needed (most other zip implementations can still read the output) 
"inhibit"

seems better than "disable".


disfavor ?



I went for "inhibit".

There's now a shiny new test case .
Same Bat Place:
http://cr.openjdk.java.net/~martin/webrevs/openjdk8/useZip64For64kEntries/ 


I am now happy with this change and would like to submit.
As always, I need a bug id.

Re: zip64 compatibility problems

[Martin Buchholz]

On Mon, Feb 11, 2013 at 6:53 AM, Alan Bateman wrote:

> One improvement would be to use try-with-resources.
>

I did that, despite the loss of portability.
Portable java programs might be able to start using these new-fangled
features starting around 2020.

Re: zip64 compatibility problems

[Alan Bateman]

On 11/02/2013 03:59, Martin Buchholz wrote:



On Sat, Feb 9, 2013 at 1:50 AM, Peter Levart > wrote:




Also, this does not disable ZIP64 - it only disables it when it is not
needed (most other zip implementations can still read the output) "inhibit"
seems better than "disable".


disfavor ?

I went for "inhibit".

There's now a shiny new test case .
Same Bat Place: 
http://cr.openjdk.java.net/~martin/webrevs/openjdk8/useZip64For64kEntries/ 


I am now happy with this change and would like to submit.
As always, I need a bug id.
The test looks fine, looks like it won't be too slow. One improvement 
would be to use try-with-resources.


-Alan.

Re: zip64 compatibility problems

[Xueming Shen]

8007905.

But you will have to hold the push. I will send it through the CCC.

-Sherman

On 2/10/2013 7:59 PM, Martin Buchholz wrote:

On Sat, Feb 9, 2013 at 1:50 AM, Peter Levart  wrote:


Also, this does not disable ZIP64 - it only disables it when it is not
needed (most other zip implementations can still read the output) "inhibit"
seems better than "disable".


disfavor ?



I went for "inhibit".

There's now a shiny new test case .
Same Bat Place:
http://cr.openjdk.java.net/~martin/webrevs/openjdk8/useZip64For64kEntries/
I am now happy with this change and would like to submit.
As always, I need a bug id.

Re: zip64 compatibility problems

[Martin Buchholz]

On Sat, Feb 9, 2013 at 1:50 AM, Peter Levart  wrote:

>
> Also, this does not disable ZIP64 - it only disables it when it is not
> needed (most other zip implementations can still read the output) "inhibit"
> seems better than "disable".
>
>
> disfavor ?
>
>
I went for "inhibit".

There's now a shiny new test case .
Same Bat Place:
http://cr.openjdk.java.net/~martin/webrevs/openjdk8/useZip64For64kEntries/
I am now happy with this change and would like to submit.
As always, I need a bug id.

Re: zip64 compatibility problems

[Peter Levart]

On 02/07/2013 06:51 PM, Martin Buchholz wrote:

On Thu, Feb 7, 2013 at 8:15 AM, Alan Bateman wrote:


On 07/02/2013 01:55, Martin Buchholz wrote:


Following up, please review the backward compatiblitiy support in:

http://cr.openjdk.java.net/~**martin/webrevs/openjdk8/**
useZip64For64kEntries/

(ideally users would have even more control via the API, but I ain't gonna
try to touch that)


No objection to adding a knob for this but do we need changes for reading
too? I'm just concerned that someone could use ZipOutputStream to create a
zip or JAR file that is not readable (in the same VM).



Such zip files have always been readable by the JDK's (and other folks')
zip implementations, so no changes should be needed (in theory).  I've
already fixed a case where the zip implementation in langtools regressed to
not be able to read such files.

That said, we could use more testing.



On the property name then we've started using jdk.* for JDK-specific
properties. Also as the default is to support ZIP64 when writing then maybe
this should have a disable* name so someone can specify
-Djdk.util.zip.disableZip64 on the command line without specifying a value.

Can you point me at exemplar code for reading such a system property?

Also, this does not disable ZIP64 - it only disables it when it is not
needed (most other zip implementations can still read the output) "inhibit"
seems better than "disable".


disfavor ?

Regards, Peter

Re: zip64 compatibility problems

[Martin Buchholz]

On Thu, Feb 7, 2013 at 8:15 AM, Alan Bateman wrote:

> On 07/02/2013 01:55, Martin Buchholz wrote:
>
>> Following up, please review the backward compatiblitiy support in:
>>
>> http://cr.openjdk.java.net/~**martin/webrevs/openjdk8/**
>> useZip64For64kEntries/
>>
>> (ideally users would have even more control via the API, but I ain't gonna
>> try to touch that)
>>
> No objection to adding a knob for this but do we need changes for reading
> too? I'm just concerned that someone could use ZipOutputStream to create a
> zip or JAR file that is not readable (in the same VM).
>
>
Such zip files have always been readable by the JDK's (and other folks')
zip implementations, so no changes should be needed (in theory).  I've
already fixed a case where the zip implementation in langtools regressed to
not be able to read such files.

That said, we could use more testing.


> On the property name then we've started using jdk.* for JDK-specific
> properties. Also as the default is to support ZIP64 when writing then maybe
> this should have a disable* name so someone can specify
> -Djdk.util.zip.disableZip64 on the command line without specifying a value.


Can you point me at exemplar code for reading such a system property?

Also, this does not disable ZIP64 - it only disables it when it is not
needed (most other zip implementations can still read the output) "inhibit"
seems better than "disable".

Re: zip64 compatibility problems

[Alan Bateman]

On 07/02/2013 01:55, Martin Buchholz wrote:

Following up, please review the backward compatiblitiy support in:

http://cr.openjdk.java.net/~martin/webrevs/openjdk8/useZip64For64kEntries/

(ideally users would have even more control via the API, but I ain't gonna
try to touch that)
No objection to adding a knob for this but do we need changes for 
reading too? I'm just concerned that someone could use ZipOutputStream 
to create a zip or JAR file that is not readable (in the same VM).


On the property name then we've started using jdk.* for JDK-specific 
properties. Also as the default is to support ZIP64 when writing then 
maybe this should have a disable* name so someone can specify 
-Djdk.util.zip.disableZip64 on the command line without specifying a value.


-Alan.

Re: zip64 compatibility problems

[Martin Buchholz]

Pushed into tl8.  Thanks for the reviews!

On Mon, Jan 28, 2013 at 3:36 AM, Alan Bateman wrote:

> On 26/01/2013 17:14, Martin Buchholz wrote:
>
>> :
>> Following up on this, I have a simple webrev:
>>
>> http://cr.openjdk.java.net/~**martin/webrevs/openjdk8/**LARGEFILE/
>>
>> with an "obviously correct" fix.  However:
>>
>> - we need a bug filed
>> - This change is completely untested.  I no longer have access to native
>> 32-bit systems where this bug might be manifested.  I have not tried to
>> actually provoke a failure, although it should not be too hard to create a
>> 3GB jar file with the contents of interest at the end, on a system where
>> off_t is signed 32-bit.
>> - As we discussed, it might be better to have a JLI_Open (or even better,
>> common C-level infrastructure for the whole project) but only you guys
>> have
>> access to the variety of systems to write and test such a thing, even if
>> it
>> is just a few lines of code.
>>
>> So next step here is up to you.
>>
> I've created a bug to track this first installation:
>
> 8006995: java launcher fails top en executable JAR > 2GB
>
> I think the proposed changes are okay, a no-brainer really. It would be
> nice if the open were moved to platform specific code, then we could use
> open64 and drop O_LARGEFILE flag. That might be something for future
> refactoring (I think JLI_Open was suggested in an earlier mail).
>
> Ideally we should have a test but we've had a lot of bad experience with
> files that need multi-GB zip files (slow, need lots of disk space) so I
> think it would be saner to leave it out this time.
>
> -Alan.
>

Re: zip64 compatibility problems

[Martin Buchholz]

Following up, please review the backward compatiblitiy support in:

http://cr.openjdk.java.net/~martin/webrevs/openjdk8/useZip64For64kEntries/

(ideally users would have even more control via the API, but I ain't gonna
try to touch that)

Re: zip64 compatibility problems

[Alan Bateman]

On 26/01/2013 17:14, Martin Buchholz wrote:

:
Following up on this, I have a simple webrev:

http://cr.openjdk.java.net/~martin/webrevs/openjdk8/LARGEFILE/

with an "obviously correct" fix.  However:

- we need a bug filed
- This change is completely untested.  I no longer have access to native
32-bit systems where this bug might be manifested.  I have not tried to
actually provoke a failure, although it should not be too hard to create a
3GB jar file with the contents of interest at the end, on a system where
off_t is signed 32-bit.
- As we discussed, it might be better to have a JLI_Open (or even better,
common C-level infrastructure for the whole project) but only you guys have
access to the variety of systems to write and test such a thing, even if it
is just a few lines of code.

So next step here is up to you.

I've created a bug to track this first installation:

8006995: java launcher fails top en executable JAR > 2GB

I think the proposed changes are okay, a no-brainer really. It would be 
nice if the open were moved to platform specific code, then we could use 
open64 and drop O_LARGEFILE flag. That might be something for future 
refactoring (I think JLI_Open was suggested in an earlier mail).


Ideally we should have a test but we've had a lot of bad experience with 
files that need multi-GB zip files (slow, need lots of disk space) so I 
think it would be saner to leave it out this time.


-Alan.

Re: zip64 compatibility problems

[Martin Buchholz]

On Sat, Jan 26, 2013 at 12:11 PM, Kumar Srinivasan
>
> So next step here is up to you.
>
> To be perfectly honest, I am swamped right now, I will try to fit this in.
> Sherman if you have cycles then go for it, I will help as needed.
>

Understood.  One possibility is to lower our standards a bit, and just
check this in as is without proof that it will fix a bug, because the
chance that it will is very much higher than the risk of injecting a new
undetected bug.

The real world use case (seen in the wild) is someone creating a non-zip64
3GB jar file, with the manifest at the end (yeah, they shouldn't do that)
and then running with java -jar on that.

Re: zip64 compatibility problems

[Kumar Srinivasan]

On 1/26/2013 9:14 AM, Martin Buchholz wrote:



On Tue, Jan 15, 2013 at 5:41 PM, Kumar Srinivasan 
mailto:kumar.x.sriniva...@oracle.com>> 
wrote:




You changed calls to open to do this:

 if ((fd = open(jarfile, O_RDONLY
#ifdef O_LARGEFILE
 | O_LARGEFILE /* large file mode on solaris */
#endif
#ifdef O_BINARY
 | O_BINARY /* use binary mode on windows */
#endif
 )) == -1)

But this is not done consistently - there are 2 other calls to
open in
the same file that didn't get the LARGEFILE treatment.  Why isn't
there a JLI_Open?

Maybe if you had reviewed my code changes, you would've caught
this. :)

I will look into it, maybe time for a JLI_Open as you suggested.


Following up on this, I have a simple webrev:

http://cr.openjdk.java.net/~martin/webrevs/openjdk8/LARGEFILE/ 



with an "obviously correct" fix.  However:

- we need a bug filed
- This change is completely untested.  I no longer have access to 
native 32-bit systems where this bug might be manifested.  I have not 
tried to actually provoke a failure, although it should not be too 
hard to create a 3GB jar file with the contents of interest at the 
end, on a system where off_t is signed 32-bit.
- As we discussed, it might be better to have a JLI_Open (or even 
better, common C-level infrastructure for the whole project) but only 
you guys have access to the variety of systems to write and test such 
a thing, even if it is just a few lines of code.


So next step here is up to you.

To be perfectly honest, I am swamped right now, I will try to fit this in.
Sherman if you have cycles then go for it, I will help as needed.

Kumar

Re: zip64 compatibility problems

[Martin Buchholz]

On Tue, Jan 15, 2013 at 5:41 PM, Kumar Srinivasan <
kumar.x.sriniva...@oracle.com> wrote:

>
>>
>> You changed calls to open to do this:
>>
>>  if ((fd = open(jarfile, O_RDONLY
>> #ifdef O_LARGEFILE
>>  | O_LARGEFILE /* large file mode on solaris */
>> #endif
>> #ifdef O_BINARY
>>  | O_BINARY /* use binary mode on windows */
>> #endif
>>  )) == -1)
>>
>> But this is not done consistently - there are 2 other calls to open in
>> the same file that didn't get the LARGEFILE treatment.  Why isn't
>> there a JLI_Open?
>>
> Maybe if you had reviewed my code changes, you would've caught this. :)
>
> I will look into it, maybe time for a JLI_Open as you suggested.
>

Following up on this, I have a simple webrev:

http://cr.openjdk.java.net/~martin/webrevs/openjdk8/LARGEFILE/

with an "obviously correct" fix.  However:

- we need a bug filed
- This change is completely untested.  I no longer have access to native
32-bit systems where this bug might be manifested.  I have not tried to
actually provoke a failure, although it should not be too hard to create a
3GB jar file with the contents of interest at the end, on a system where
off_t is signed 32-bit.
- As we discussed, it might be better to have a JLI_Open (or even better,
common C-level infrastructure for the whole project) but only you guys have
access to the variety of systems to write and test such a thing, even if it
is just a few lines of code.

So next step here is up to you.

Re: zip64 compatibility problems

[Martin Buchholz]

On Wed, Jan 23, 2013 at 9:17 PM, Xueming Shen  wrote:
> Hi Martin, sorry for the late response, was busy on other stuff these days.

Current deadline pressures are understood!

> I'm OK
> with the idea of introducing a system property to turn off zip64 support,
> but NOT
> by default. You can make it default in your internal build. You do the code,
> I do
> the internal process/paper work (need go CCC for it). Let's do it for 8
> first.

OK - I'll prepare some changes.

Re: zip64 compatibility problems

[Xueming Shen]

Hi Martin, sorry for the late response, was busy on other stuff these 
days. I'm OK
with the idea of introducing a system property to turn off zip64 
support, but NOT
by default. You can make it default in your internal build. You do the 
code, I do
the internal process/paper work (need go CCC for it). Let's do it for 8 
first.


-Sherman

On 1/23/13 4:10 PM, Martin Buchholz wrote:

Ping.

On Wed, Jan 16, 2013 at 12:24 PM, Martin Buchholz > wrote:


On Tue, Jan 15, 2013 at 5:33 PM, Martin Buchholz
mailto:marti...@google.com>> wrote:
> Actually, I think y'all should do 3 things:
> - backport Kumar's bug fix to jdk7
> - introduce a system property to turn off the zip64 support.
> I am working on a patch to introduce such a system property.
> We will probably default to disabling zip64 by default, but you will
> probably find that unacceptable.  But you can default the other way.
> - fix up some remaining largefile issues in parse_manifest
> I might contribute a bit of code here as well.

As I said, I'm willing to do more work on this mini-project myself,
but we need rough consensus on the direction we should go.

Re: zip64 compatibility problems

[Martin Buchholz]

Ping.

On Wed, Jan 16, 2013 at 12:24 PM, Martin Buchholz wrote:

> On Tue, Jan 15, 2013 at 5:33 PM, Martin Buchholz 
> wrote:
> > Actually, I think y'all should do 3 things:
> > - backport Kumar's bug fix to jdk7
> > - introduce a system property to turn off the zip64 support.
> > I am working on a patch to introduce such a system property.
> > We will probably default to disabling zip64 by default, but you will
> > probably find that unacceptable.  But you can default the other way.
> > - fix up some remaining largefile issues in parse_manifest
> > I might contribute a bit of code here as well.
>
> As I said, I'm willing to do more work on this mini-project myself,
> but we need rough consensus on the direction we should go.
>

Re: zip64 compatibility problems

[Martin Buchholz]

On Tue, Jan 15, 2013 at 5:33 PM, Martin Buchholz  wrote:
> Actually, I think y'all should do 3 things:
> - backport Kumar's bug fix to jdk7
> - introduce a system property to turn off the zip64 support.
> I am working on a patch to introduce such a system property.
> We will probably default to disabling zip64 by default, but you will
> probably find that unacceptable.  But you can default the other way.
> - fix up some remaining largefile issues in parse_manifest
> I might contribute a bit of code here as well.

As I said, I'm willing to do more work on this mini-project myself,
but we need rough consensus on the direction we should go.

Re: zip64 compatibility problems

[Kumar Srinivasan]

On 1/16/2013 4:20 AM, Alan Bateman wrote:

On 16/01/2013 03:40, Martin Buchholz wrote:

:

It's a different story for the groups of users just now adopting jdk7.
The situation is a bit grim for those using jar files with>64k entries.
jdk7 is not only creating jar files that are unreadable by jdk6 - they
are also unreadable by the jdk7 launcher!  With no visible benefit for
the user!

Just to be clear, do you mean just executable JAR files (java -jar)? I 
assume that library JAR files on the classpath or elsewhere with >64k 
are not an issue.
Only -jar would be affected since the launcher needs to read the 
manifest to extract the main-class name,
so the changeset which Martin refers to,  fixes the parse_manifest.c. 
Therefore using the construct

java -cp foo.jar Main   would/should work correctly via the classloader.

Kumar



-Alan

Re: zip64 compatibility problems

[Alan Bateman]

On 16/01/2013 03:40, Martin Buchholz wrote:

:

It's a different story for the groups of users just now adopting jdk7.
The situation is a bit grim for those using jar files with>64k entries.
jdk7 is not only creating jar files that are unreadable by jdk6 - they
are also unreadable by the jdk7 launcher!  With no visible benefit for
the user!

Just to be clear, do you mean just executable JAR files (java -jar)? I 
assume that library JAR files on the classpath or elsewhere with >64k 
are not an issue.


-Alan

Re: zip64 compatibility problems

[Martin Buchholz]

On Tue, Jan 15, 2013 at 6:24 PM, Xueming Shen  wrote:
> On 1/15/13 6:21 PM, Xueming Shen wrote:
>>
>>
>> zip3.0 has a flag to do the similar thing may be a compelling reason to
>> add such a system property, but make disabled default turns yourself
>> into the dark side:-)  have to explicitly turn the flag on will force the
>> people to think if this is something they really want to do.
>>
>> Given it has been in the major releases for years, I don't think it's
>> possible to disable it in openjdk.
>>
> I meant it's impossible to disable it by default..."

I agree with you that it's the right decision for OpenJDK proper.
You are consistent with zip 3.0 and also are obeying the zip spec

  4.4.1.4  If one of the fields in the end of central directory
  record is too small to hold required data, the field should be
  set to -1 (0x or 0x) and the ZIP64 format record
  should be created.

It's a different story for the groups of users just now adopting jdk7.
The situation is a bit grim for those using jar files with >64k entries.
jdk7 is not only creating jar files that are unreadable by jdk6 - they
are also unreadable by the jdk7 launcher!  With no visible benefit for
the user!

So temporarily turning off zip64 seems a better choice for us.

Re: zip64 compatibility problems

[Xueming Shen]

On 1/15/13 6:21 PM, Xueming Shen wrote:


zip3.0 has a flag to do the similar thing may be a compelling reason to
add such a system property, but make disabled default turns yourself
into the dark side:-)  have to explicitly turn the flag on will force the
people to think if this is something they really want to do.

Given it has been in the major releases for years, I don't think it's
possible to disable it in openjdk.


I meant it's impossible to disable it by default..."

Re: zip64 compatibility problems

[Xueming Shen]

zip3.0 has a flag to do the similar thing may be a compelling reason to
add such a system property, but make disabled default turns yourself
into the dark side:-)  have to explicitly turn the flag on will force the
people to think if this is something they really want to do.

Given it has been in the major releases for years, I don't think it's
possible to disable it in openjdk.

-Sherman

On 1/15/13 5:33 PM, Martin Buchholz wrote:

On Tue, Jan 15, 2013 at 5:23 PM, Xueming Shen  wrote:

Hi Martin,

I would assume you are not asking for a similar flag for jar to disable the
zip64 support (or even something at API level) to generate "old" style
jar/zip
file for > 64k entries, but the backport of this changeset to 7u, right?

Actually, I think y'all should do 3 things:
- backport Kumar's bug fix to jdk7
- introduce a system property to turn off the zip64 support.
I am working on a patch to introduce such a system property.
We will probably default to disabling zip64 by default, but you will
probably find that unacceptable.  But you can default the other way.
- fix up some remaining largefile issues in parse_manifest
I might contribute a bit of code here as well.

Re: zip64 compatibility problems

[Martin Buchholz]

You want code?  I got code.

@@ -75,6 +75,18 @@
 }

 /**
+ * Whether to use ZIP64 for zip files with more than 64k entries.
+ * When ZIP64 support in zip implementations is ubiquitous, this
+ * should default to true, but before then, defaulting to false is
+ * more portable, since most zip implementations tolerate
+ * incorrect total entry count fields.
+ */
+private static final boolean useZip64For64kEntries =
+java.security.AccessController.doPrivileged(
+new sun.security.action.GetBooleanAction(
+"java.util.zip.useZip64For64kEntries")).booleanValue();
+
+/**
  * Checks to make sure that this stream has not been closed.
  */
 private void ensureOpen() throws IOException {
@@ -534,8 +546,10 @@
 }
 int count = xentries.size();
 if (count >= ZIP64_MAGICCOUNT) {
-count = ZIP64_MAGICCOUNT;
-hasZip64 = true;
+hasZip64 |= useZip64For64kEntries;
+if (hasZip64) {
+count = ZIP64_MAGICCOUNT;
+}
 }
 if (hasZip64) {
 long off64 = written;


On Tue, Jan 15, 2013 at 5:33 PM, Martin Buchholz  wrote:
> On Tue, Jan 15, 2013 at 5:23 PM, Xueming Shen  wrote:
>> Hi Martin,
>>
>> I would assume you are not asking for a similar flag for jar to disable the
>> zip64 support (or even something at API level) to generate "old" style
>> jar/zip
>> file for > 64k entries, but the backport of this changeset to 7u, right?
>
> Actually, I think y'all should do 3 things:
> - backport Kumar's bug fix to jdk7
> - introduce a system property to turn off the zip64 support.
> I am working on a patch to introduce such a system property.
> We will probably default to disabling zip64 by default, but you will
> probably find that unacceptable.  But you can default the other way.
> - fix up some remaining largefile issues in parse_manifest
> I might contribute a bit of code here as well.

Re: zip64 compatibility problems

[Kumar Srinivasan]

On 1/15/2013 1:46 PM, Martin Buchholz wrote:

Hi zip64 team!

It's always a problem creating zip files using new zip spec features,
since older zip implementations will fail to read those files.

I see the jdk8 fix to allow the launcher to read zip64 files

changeset:   5812:f1838d040cc7
user:ksrini
date:2012-09-05 11:38 -0700
7194005: (launcher) needs to be enhanced for 64-bit jar file handling
Reviewed-by: darcy, sherman

but that has not been backported to jdk7, which seems like a serious
problem to me.

More generally, most zip implementations, including in the jdk, worked
just fine with >64k entries, since the entry count field was generally
just treated as a hint.  The change to use zip64 when there are >64k
entries makes zip files *less* portable in practice for the next few
years.  It would have been better to default to writing non-zip64 zip
files in such cases.

Note that zip 3.0 does write zip64 files for >64k entries, but also
provides a flag -fz- to change the default.

I'd like to see a backport of 7194005 and am also aware of some other
issues with our ugly friend parse_manifest.c.
Joe and I contemplated about this, in the end we decided not to, in 
order to allow for

some soak time in jdk8.



You changed calls to open to do this:

 if ((fd = open(jarfile, O_RDONLY
#ifdef O_LARGEFILE
 | O_LARGEFILE /* large file mode on solaris */
#endif
#ifdef O_BINARY
 | O_BINARY /* use binary mode on windows */
#endif
 )) == -1)

But this is not done consistently - there are 2 other calls to open in
the same file that didn't get the LARGEFILE treatment.  Why isn't
there a JLI_Open?

Maybe if you had reviewed my code changes, you would've caught this. :)

I will look into it, maybe time for a JLI_Open as you suggested.

Kumar



Comments?

Re: zip64 compatibility problems

[Martin Buchholz]

On Tue, Jan 15, 2013 at 5:23 PM, Xueming Shen  wrote:
> Hi Martin,
>
> I would assume you are not asking for a similar flag for jar to disable the
> zip64 support (or even something at API level) to generate "old" style
> jar/zip
> file for > 64k entries, but the backport of this changeset to 7u, right?

Actually, I think y'all should do 3 things:
- backport Kumar's bug fix to jdk7
- introduce a system property to turn off the zip64 support.
I am working on a patch to introduce such a system property.
We will probably default to disabling zip64 by default, but you will
probably find that unacceptable.  But you can default the other way.
- fix up some remaining largefile issues in parse_manifest
I might contribute a bit of code here as well.

Re: zip64 compatibility problems

[Xueming Shen]

Hi Martin,

I would assume you are not asking for a similar flag for jar to disable the
zip64 support (or even something at API level) to generate "old" style 
jar/zip

file for > 64k entries, but the backport of this changeset to 7u, right?

-Sherman


On 1/15/13 1:46 PM, Martin Buchholz wrote:

Hi zip64 team!

It's always a problem creating zip files using new zip spec features,
since older zip implementations will fail to read those files.

I see the jdk8 fix to allow the launcher to read zip64 files

changeset:   5812:f1838d040cc7
user:ksrini
date:2012-09-05 11:38 -0700
7194005: (launcher) needs to be enhanced for 64-bit jar file handling
Reviewed-by: darcy, sherman

but that has not been backported to jdk7, which seems like a serious
problem to me.

More generally, most zip implementations, including in the jdk, worked
just fine with >64k entries, since the entry count field was generally
just treated as a hint.  The change to use zip64 when there are >64k
entries makes zip files *less* portable in practice for the next few
years.  It would have been better to default to writing non-zip64 zip
files in such cases.

Note that zip 3.0 does write zip64 files for >64k entries, but also
provides a flag -fz- to change the default.

I'd like to see a backport of 7194005 and am also aware of some other
issues with our ugly friend parse_manifest.c.

You changed calls to open to do this:

 if ((fd = open(jarfile, O_RDONLY
#ifdef O_LARGEFILE
 | O_LARGEFILE /* large file mode on solaris */
#endif
#ifdef O_BINARY
 | O_BINARY /* use binary mode on windows */
#endif
 )) == -1)

But this is not done consistently - there are 2 other calls to open in
the same file that didn't get the LARGEFILE treatment.  Why isn't
there a JLI_Open?

Comments?