totalbytes_Read/Written are being tested though not the size > 4G. That
test mainly tests
the > 64K entries and total size > 4G (lots of > 32-bit offsets...), but
the total bytes read/
written are all < 4G (for each entry). An easy manual test I always do
after run that test is to
jar the resulting > 6g file and unjar it. That will actually check the
total bytes read/written
> 4G case.
If you keep using the same JRE which uses the system zlib, you might
still be able to jar and
unjar a single > 4G file. The total bytes read/written are used as one
of the validation in zip
format, ZipOutputStream writes these two number as size/csize fields of
the loc and cen
table, the ZipInputStream then verifies these numbers when inflating.
This round-trip happens
to work if the zos and zis both use the same JRE, in which both throw
away the high bits of
the total numbers (but their low bits are still match) But a simple jar
tvf tells you the size number
is incorrect. Also , if you jar some other files together with this > 4G
file, you probably will
get error when trying to extract individual entry file(s) out, as the
position info of those files
probably will get messed up.
Btw, the code to support > 4G total_in/out at java level is ready here,
need some more tests
and the bugid for this one and yours.
http://cr.openjdk.java.net/~sherman/undozip/webrev/
-Sherman
On 7/30/12 3:20 AM, Andrew Hughes wrote:
----- Original Message -----
On 07/27/2012 10:37 AM, Andrew Hughes wrote:
----- Original Message -----
The zip64 support (total_in/out) part probably can be done at Java
level
(ignore
the total_in/out in z_tream_s). Need to remove this dependency.
Will
take a look later.
Yes, it seems they still mention the size of total_in/out on the
website on the zlib site, and that they shouldn't be relied on:
http://www.zlib.net/zlib_faq.html#faq32
"Note however that the strm.total_in and strm_total_out counters
may be limited to 4 GB. These counters are provided as a
convenience and are not used internally by inflate() or deflate().
The application can easily set up its own counters updated after
each call of inflate() or deflate() to count beyond 4 GB"
"The word "may" appears several times above since there is a 4 GB
limit only if the compiler's long type is 32 bits. If the
compiler's long type is 64 bits, then the limit is 16 exabytes."
I notice a test went in with the 64-bit support, but I assume it
can't test these counters as the Deflater for a ZipStream is
protected. At least, they aren't failing on our builds with
system zlib.
That test is not configured to be run for "auto testing", it just
takes
too long to zip/unzip
a 4G+ file. I use it manually to test the ZIP64 support.
Yes, sorry, I noticed this after posting. I've since run it manually and it
passes
with a system zlib, both on the zip it creates and one I created containing just
a single 4.2gb file (a RHEL ISO image zipped). But I'm not sure if it's
testing total bytes
read for overflow.
I will give it a try to remove this dependency next week.
Thanks.
It would be
helpful if you can
help "migrate" the icetea patch. How does the icetea patch work now?
Always use the
system zlib, if it presents? any configurable option to switch on and
off?
It adds a switch and also support for setting the CFLAGS/LIBS:
USE_SYSTEM_ZLIB=true \
ZLIB_LIBS="$(pkg-config --libs zlib)" \
ZLIB_CFLAGS="$(pkg-config --cflags zlib)" \
This is the same nomenclature we use for the other libraries too
(lcms, jpeg, png, gif, cups, etc.)
I'll try and post the patch later today.
-Sherman
Are you actively working on this now or shall I take a look?
-Sherman
On 7/11/2012 12:47 AM, Alan Bateman wrote:
On 05/07/2012 17:11, Andrew Hughes wrote:
----- Original Message -----
Is there a way to get the native zlib libraries to get picked
up
instead of the hardcoded version within the JVM?
--
Azeem Jiva
@javawithjiva
We have this in IcedTea (USE_SYSTEM_ZLIB=true) and intend to get
it
upstream.
However, I don't see how this is related to HotSpot, as the zlib
usage
is in the jdk tree.
I think we need to (re)start the discussion on core-libs-dev with
a
view to eliminating the patches that the JDK has to zlib, see:
http://hg.openjdk.java.net/jdk8/jdk8/jdk/file/tip/src/share/native/java/util/zip/zlib-1.2.5/patches/ChangeLog_java
One of these changes relates to the zip64 support and I believe
there
are corner cases when inflating or deflating>2GB that won't work
if
using the system zlib. Sherman will likely recall the details.
Given
that the new build already supports using the system zlib (at
least
on
Linux) then it would be good to sort this out so that it just
works.
-Alan
