[
https://issues.apache.org/jira/browse/IVY-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12875545#action_12875545
]
Constantine Plotnikov commented on IVY-1194:
--------------------------------------------
Fix works for me for normal ivy operations. Note that go-ivy tutorial still
fails since ant "get" task still fetches compressed version of ivy.jar from
proxy. Maybe it make sense to reapply the fixes for this patch to ant "get"
task as well.
According to RFC2616 (http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html,
section 3.5) zlib variant should be used for "deflate" content encoding method.
So you need to support only the first method.
> Downloads from maven repository fail when using transparent HTTP proxies
> ------------------------------------------------------------------------
>
> Key: IVY-1194
> URL: https://issues.apache.org/jira/browse/IVY-1194
> Project: Ivy
> Issue Type: Bug
> Components: Core
> Affects Versions: 2.1.0
> Environment: Windows XP, Java 1.5.0.16, java 1.6.0.20
> Reporter: Constantine Plotnikov
> Assignee: Maarten Coene
> Priority: Blocker
> Fix For: 2.2.0-RC1
>
>
> The package without dependencies cannot fetch dependencies and samples in the
> package with dependencies do not work.
> Fetching dependencies fails with the following log:
> {quote}
> Buildfile: build.xml
> init-ivy:
> retrieve-all:
> [ivy:retrieve] :: Ivy 2.1.0 - 20090925235825 :: http://ant.apache.org/ivy/ ::
> [ivy:retrieve] :: loading settings :: url =
> jar:file:/C:/utils/apache/apache-ivy-2.1.0-wo/ivy.jar!/org/apache/ivy/core/settings/ivysettings.xml
> [ivy:retrieve] :: resolving dependencies :: org.apache.ivy#ivy;2.1.0
> [ivy:retrieve] confs: [core, httpclient, oro, vfs, sftp, standalone,
> ant, default, test, source]
> [ivy:retrieve] found commons-httpclient#commons-httpclient;3.0 in
> public
> [ivy:retrieve] found commons-codec#commons-codec;1.2 in public
> [ivy:retrieve] found oro#oro;2.0.8 in public
> [ivy:retrieve] found commons-vfs#commons-vfs;1.0 in public
> [ivy:retrieve] found com.jcraft#jsch;0.1.31 in public
> [ivy:retrieve] found ant#ant;1.6.2 in public
> [ivy:retrieve] found ant#ant-nodeps;1.6.2 in public
> [ivy:retrieve] found ant#ant-trax;1.6.2 in public
> [ivy:retrieve] found junit#junit;3.8.2 in public
> [ivy:retrieve] found commons-lang#commons-lang;2.4 in public
> [ivy:retrieve] found org.apache.ant#ant-testutil;1.7.0 in public
> [ivy:retrieve] found ant#ant-launcher;1.6.2 in public
> [ivy:retrieve] found xerces#xercesImpl;2.6.2 in public
> [ivy:retrieve] found xerces#xmlParserAPIs;2.6.2 in public
> [ivy:retrieve] downloading
> http://repo1.maven.org/maven2/junit/junit/3.8.2/junit-3.8.2.jar ...
> [ivy:retrieve] ............ (117kB)
> [ivy:retrieve] .. (0kB)
> [ivy:retrieve] :: resolution report :: resolve 1250ms :: artifacts dl 250ms
> [ivy:retrieve] :: evicted modules:
> [ivy:retrieve] commons-logging#commons-logging;1.0.3 by
> [commons-logging#commons-logging;1.0.4] in [default]
> ---------------------------------------------------------------------
> | | modules || artifacts |
> | conf | number| search|dwnlded|evicted|| number|dwnlded|
> ---------------------------------------------------------------------
> | core | 0 | 0 | 0 | 0 || 0 | 0 |
> | httpclient | 3 | 0 | 0 | 0 || 2 | 0 |
> | oro | 1 | 0 | 0 | 0 || 1 | 0 |
> | vfs | 2 | 0 | 0 | 0 || 1 | 0 |
> | sftp | 1 | 0 | 0 | 0 || 1 | 0 |
> | standalone | 0 | 0 | 0 | 0 || 0 | 0 |
> | ant | 1 | 0 | 0 | 0 || 1 | 0 |
> | default | 10 | 0 | 0 | 1 || 8 | 0 |
> | test | 6 | 0 | 0 | 0 || 6 | 0 |
> | source | 0 | 0 | 0 | 0 || 0 | 0 |
> ---------------------------------------------------------------------
> [ivy:retrieve] :: problems summary ::
> [ivy:retrieve] :::: WARNINGS
> [ivy:retrieve] problem while downloading module descriptor:
> http://repo1.maven.org/maven2/commons-logging/commons-logging/1.0.3/commons-logging-1.0.3.pom:
> invalid sha1: expected=<
> [ivy:retrieve] г computed=88c58ea4a562116ab15fb76c9097ee1f25cc750b (62ms)
> [ivy:retrieve] module not found:
> commons-logging#commons-logging;1.0.3
> [ivy:retrieve] ==== local: tried
> [ivy:retrieve] C:\Documents and
> Settings\Constantin.Plotnikov\.ivy2/local/commons-logging/commons-logging/1.0.3/ivys/ivy.xml
> [ivy:retrieve] -- artifact
> commons-logging#commons-logging;1.0.3!commons-logging.jar:
> [ivy:retrieve] C:\Documents and
> Settings\Constantin.Plotnikov\.ivy2/local/commons-logging/commons-logging/1.0.3/jars/commons-logging.jar
> [ivy:retrieve] ==== shared: tried
> [ivy:retrieve] C:\Documents and
> Settings\Constantin.Plotnikov\.ivy2/shared/commons-logging/commons-logging/1.0.3/ivys/ivy.xml
> [ivy:retrieve] -- artifact
> commons-logging#commons-logging;1.0.3!commons-logging.jar:
> [ivy:retrieve] C:\Documents and
> Settings\Constantin.Plotnikov\.ivy2/shared/commons-logging/commons-logging/1.0.3/jars/commons-logging.jar
> [ivy:retrieve] ==== public: tried
> [ivy:retrieve]
> http://repo1.maven.org/maven2/commons-logging/commons-logging/1.0.3/commons-logging-1.0.3.pom
> [ivy:retrieve] problem while downloading module descriptor:
> http://repo1.maven.org/maven2/commons-logging/commons-logging/1.0.4/commons-logging-1.0.4.pom:
> invalid sha1: expected=<
> [ivy:retrieve] module not found:
> commons-logging#commons-logging;1.0.4
> [ivy:retrieve] ==== local: tried
> [ivy:retrieve] C:\Documents and
> Settings\Constantin.Plotnikov\.ivy2/local/commons-logging/commons-logging/1.0.4/ivys/ivy.xml
> [ivy:retrieve] -- artifact
> commons-logging#commons-logging;1.0.4!commons-logging.jar:
> [ivy:retrieve] C:\Documents and
> Settings\Constantin.Plotnikov\.ivy2/local/commons-logging/commons-logging/1.0.4/jars/commons-logging.jar
> [ivy:retrieve] ==== shared: tried
> [ivy:retrieve] C:\Documents and
> Settings\Constantin.Plotnikov\.ivy2/shared/commons-logging/commons-logging/1.0.4/ivys/ivy.xml
> [ivy:retrieve] -- artifact
> commons-logging#commons-logging;1.0.4!commons-logging.jar:
> [ivy:retrieve] C:\Documents and
> Settings\Constantin.Plotnikov\.ivy2/shared/commons-logging/commons-logging/1.0.4/jars/commons-logging.jar
> [ivy:retrieve] ==== public: tried
> [ivy:retrieve]
> http://repo1.maven.org/maven2/commons-logging/commons-logging/1.0.4/commons-logging-1.0.4.pom
> [ivy:retrieve] [FAILED ] junit#junit;3.8.2!junit.jar:
> invalid sha1: expected=<
> [ivy:retrieve] [FAILED ] junit#junit;3.8.2!junit.jar:
> (0ms)
> [ivy:retrieve] ==== shared: tried
> [ivy:retrieve] C:\Documents and
> Settings\Constantin.Plotnikov\.ivy2/shared/junit/junit/3.8.2/jars/junit.jar
> [ivy:retrieve] ==== public: tried
> [ivy:retrieve]
> http://repo1.maven.org/maven2/junit/junit/3.8.2/junit-3.8.2.jar
> [ivy:retrieve] ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve] :: UNRESOLVED DEPENDENCIES ::
> [ivy:retrieve] ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve] :: commons-logging#commons-logging;1.0.3: not
> found
> [ivy:retrieve] :: commons-logging#commons-logging;1.0.4: not
> found
> [ivy:retrieve] ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve]
> [ivy:retrieve] ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve] :: FAILED DOWNLOADS ::
> [ivy:retrieve] :: ^ see resolution messages for details ^ ::
> [ivy:retrieve] ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve] :: junit#junit;3.8.2!junit.jar
> [ivy:retrieve] ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve]
> [ivy:retrieve]
> [ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
> {quote}
> The sample in package with dependencies fails with the following log:
> {quote}
> Buildfile: build.xml
> download-ivy:
> [mkdir] Created dir:
> C:\utils\apache\apache-ivy-2.1.0\src\example\go-ivy\ivy
> [echo] installing ivy...
> [get] Getting:
> http://repo1.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
> [get] To:
> C:\utils\apache\apache-ivy-2.1.0\src\example\go-ivy\ivy\ivy.jar
> install-ivy:
> [taskdef] java.util.zip.ZipException: error in opening zip file
> [taskdef] at java.util.zip.ZipFile.open(Native Method)
> [taskdef] at java.util.zip.ZipFile.<init>(ZipFile.java:203)
> [taskdef] at java.util.zip.ZipFile.<init>(ZipFile.java:234)
> [taskdef] at
> org.apache.tools.ant.AntClassLoader.getResourceURL(AntClassLoader.java:919)
> [taskdef] at
> org.apache.tools.ant.AntClassLoader$ResourceEnumeration.findNextResource(AntClassLoader.java:126)
> [taskdef] at
> org.apache.tools.ant.AntClassLoader$ResourceEnumeration.<init>(AntClassLoader.java:88)
> [taskdef] at
> org.apache.tools.ant.AntClassLoader.findResources(AntClassLoader.java:869)
> [taskdef] at java.lang.ClassLoader.getResources(ClassLoader.java:1015)
> [taskdef] at
> org.apache.tools.ant.taskdefs.Definer.resourceToURLs(Definer.java:267)
> [taskdef] at
> org.apache.tools.ant.taskdefs.Definer.execute(Definer.java:211)
> [taskdef] at
> org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:275)
> [taskdef] at org.apache.tools.ant.Task.perform(Task.java:364)
> [taskdef] at org.apache.tools.ant.Target.execute(Target.java:341)
> [taskdef] at org.apache.tools.ant.Target.performTasks(Target.java:369)
> [taskdef] at
> org.apache.tools.ant.Project.executeSortedTargets(Project.java:1216)
> [taskdef] at org.apache.tools.ant.Project.executeTarget(Project.java:1185)
> [taskdef] at
> org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:40)
> [taskdef] at
> org.apache.tools.ant.Project.executeTargets(Project.java:1068)
> [taskdef] at org.apache.tools.ant.Main.runBuild(Main.java:668)
> [taskdef] at org.apache.tools.ant.Main.startAnt(Main.java:187)
> [taskdef] at org.apache.tools.ant.launch.Launcher.run(Launcher.java:246)
> [taskdef] at org.apache.tools.ant.launch.Launcher.main(Launcher.java:67)
> [taskdef] Could not load definitions from resource
> org/apache/ivy/ant/antlib.xml. It could not be found.
> generate-src:
> go:
> [echo] using ivy to resolve commons-lang 2.1...
> BUILD FAILED
> C:\utils\apache\apache-ivy-2.1.0\src\example\go-ivy\build.xml:85: Could not
> create task or type of type: antlib:org.apache.ivy.ant:cachepath.
> Ant could not find the task or a class this task relies upon.
> This is common and has a number of causes; the usual
> solutions are to read the manual pages then download and
> install needed JAR files, or fix the build file:
> - You have misspelt 'antlib:org.apache.ivy.ant:cachepath'.
> Fix: check your spelling.
> - The task needs an external JAR file to execute
> and this is not found at the right place in the classpath.
> Fix: check the documentation for dependencies.
> Fix: declare the task.
> - The task is an Ant optional task and the JAR file and/or libraries
> implementing the functionality were not found at the time you
> yourself built your installation of Ant from the Ant sources.
> Fix: Look in the ANT_HOME/lib for the 'ant-' JAR corresponding to the
> task and make sure it contains more than merely a META-INF/MANIFEST.MF.
> If all it contains is the manifest, then rebuild Ant with the needed
> libraries present in ${ant.home}/lib/optional/ , or alternatively,
> download a pre-built release version from apache.org
> - The build file was written for a later version of Ant
> Fix: upgrade to at least the latest release version of Ant
> - The task is not an Ant core or optional task
> and needs to be declared using <taskdef>.
> - You are attempting to use a task defined using
> <presetdef> or <macrodef> but have spelt wrong or not
> defined it at the point of use
> Remember that for JAR files to be visible to Ant tasks implemented
> in ANT_HOME/lib, the files must be in the same directory or on the
> classpath
> Please neither file bug reports on this problem, nor email the
> Ant mailing lists, until all of these causes have been explored,
> as this is not an Ant bug.
> Total time: 1 second
> {quote}
> The reason for the bug is the squid. When request comes and it has a cached
> content with gzip content encoding, it return a content with encoded content,
> event if no encoding was specified in the request or "Accept-Encoding:
> identity" was specified. The below is a sample log that demonstrates the
> problem:
> {quote}
> DEBUG output created by Wget 1.11.4 on cygwin.
> --2010-06-02 12:41:55--
> http://repo1.maven.org/maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar
> Resolving repo1.maven.org... 38.97.124.18
> Caching repo1.maven.org => 38.97.124.18
> Connecting to repo1.maven.org|38.97.124.18|:80... connected.
> Created socket 4.
> Releasing 0x006d9100 (new refcount 1).
> ---request begin---
> GET /maven2/org/apache/ivy/ivy/2.1.0/ivy-2.1.0.jar HTTP/1.0
> User-Agent: Wget/1.11.4
> Accept: */*
> Host: repo1.maven.org
> Connection: Keep-Alive
> ---request end---
> HTTP request sent, awaiting response...
> ---response begin---
> HTTP/1.0 200 OK
> Server: nginx/0.7.62
> Date: Tue, 01 Jun 2010 13:09:36 GMT
> Content-Type: application/java-archive
> Last-Modified: Mon, 12 Oct 2009 21:19:02 GMT
> Content-Encoding: gzip
> X-Cache: MISS from squid_havp_node1
> X-Cache-Lookup: MISS from squid_havp_node1:3128
> Age: 70339
> Content-Length: 834513
> X-Cache: HIT from squid_havp_node1
> X-Cache-Lookup: HIT from squid_havp_node1:3128
> Connection: keep-alive
> ---response end---
> HTTP/1.0 200 OK
> Server: nginx/0.7.62
> Date: Tue, 01 Jun 2010 13:09:36 GMT
> Content-Type: application/java-archive
> Last-Modified: Mon, 12 Oct 2009 21:19:02 GMT
> Content-Encoding: gzip
> X-Cache: MISS from squid_havp_node1
> X-Cache-Lookup: MISS from squid_havp_node1:3128
> Age: 70339
> Content-Length: 834513
> X-Cache: HIT from squid_havp_node1
> X-Cache-Lookup: HIT from squid_havp_node1:3128
> Connection: keep-alive
> Registered socket 4 for persistent reuse.
> Length: 834513 (815K) [application/java-archive]
> Saving to: `ivy-2.1.0.jar'
> 0K .......... .......... .......... .......... .......... 6% 8.49M 0s
> 50K .......... .......... .......... .......... .......... 12% 12.0M 0s
> 100K .......... .......... .......... .......... .......... 18% 7.26M 0s
> 150K .......... .......... .......... .......... .......... 24% 9.28M 0s
> 200K .......... .......... .......... .......... .......... 30% 5.33M 0s
> 250K .......... .......... .......... .......... .......... 36% 9.31M 0s
> 300K .......... .......... .......... .......... .......... 42% 5.26M 0s
> 350K .......... .......... .......... .......... .......... 49% 9.15M 0s
> 400K .......... .......... .......... .......... .......... 55% 7.27M 0s
> 450K .......... .......... .......... .......... .......... 61% 7.42M 0s
> 500K .......... .......... .......... .......... .......... 67% 9.21M 0s
> 550K .......... .......... .......... .......... .......... 73% 6.11M 0s
> 600K .......... .......... .......... .......... .......... 79% 9.50M 0s
> 650K .......... .......... .......... .......... .......... 85% 8.84M 0s
> 700K .......... .......... .......... .......... .......... 92% 9.54M 0s
> 750K .......... .......... .......... .......... .......... 98% 6.04M 0s
> 800K .......... .... 100% 16.2K=1.0s
> 2010-06-02 12:41:56 (795 KB/s) - `ivy-2.1.0.jar' saved [834513/834513]
> {quote}
> In our environment, a bunch of files from repo1.maven.org are cached in the
> squid. So there were a lot of conflicts with content encoding.
> The possible workarounds:
> 1. Check returned content encoding from http client, if it is not identity,
> reissue the request with disabling the caches.
> 2. Ungzip files fetched with gzip content encoding.
> 3. Support gzip content encoding in common HTTP.
> 3. Use local maven proxies
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
