Jan,
On 2/19/2013 12:45 AM, Sievers, Jan wrote:
thanks Scott I think this will help to dig a little deeper.
I don't know how tycho handles the download...so I can't say exactly
what's happening here.
We delegate this to p2/ECF so it should work as if you would install/update
with your IDE.
All we do is configure the HTTP proxy via org.eclipse.core.net (and register
the default java.net.Authenticator, same as org.eclipse.core.net.ui does IIRC).
two follow-up questions:
1. is there any log/trace so we can check the fallback to the JRE proxy
provider is actually happening in this case?
Unfortunately, there is no tracing in the Urlconnection-based
provider. One thing you can do is turn on the httpclient 3.1 tracing
[1], and it should *not* be active in the case of NTLMv2 authentication
(where the urlconnection-based provider should be used).
Another thing you can do is explicitly disable the httpclient 3.1
provider (forcing the urlconnection provider to be used). See [2].
2. if the JRE proxy fallback happens, any chance different JDKs (OS/version)
may have differing/no support for NTLM proxy authentication?
Yes, there is a chance this is the case. I haven't kept up on the
different JDK's support for NTLMv2.
@Vincent another thing you can check to help us pin down the issue:
using the same HTTP proxy configurations in your IDE (in
Window>Prefs), can you successfully update/install any bundles e.g.
from eclipse.org ? just to make sure I got this right: you say the
build works fine on one machine but doesn't on another? what's the
difference between those 2 machines (OS, JDK) ?
If the httpclient v4 support is available but not consumed yet,
ultimately we may decide to bundle httpclient v4 with tycho if that
solves the issue. Regards, Jan
We are hoping that it does solve the issue once and for all...but based
upon my own history :), it is going to require testing to verify that
the httpclient4 NTLM support really works in the wild.
WRT bug [3] (consume ECF httpclient4 provider)...we/ECF recently had our
access to the eclipse.org signing accidentally disabled...meaning that
although we can produce integration builds right now, we cannot sign
them. We are working with Denis as we speak to address this, and hope
to have signing in place for next week's integration build.
Scott
[1] http://wiki.eclipse.org/Equinox_p2_Reporting_Problems
[2] http://wiki.eclipse.org/ECF_Filetransfer_Support_for_NTLMv2_Proxies
[3] https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449
-----Original Message----- From: [email protected]
[mailto:[email protected]] On Behalf Of Scott Lewis Sent:
Montag, 18. Februar 2013 23:50 To: [email protected] Subject: Re:
[p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy FYI, On 2/18/2013 12:56
PM, Sievers, Jan wrote:
- cross-posting to p2-dev -
tycho takes the maven HTTP proxy configuration from settings.xml and configures
p2 accordingly (via proxyservice API in org.eclipse.core.net)
if resolution and download of maven plugins works but p2 resolution does not,
this is a hint that either
a) tycho fails to transfer the proxy info to p2 or
b) p2 / org.eclipse.core.net does not support NTLM proxy authentication
since nobody reported a) yet I doubt this and suspect b)
I don't know how tycho handles the download...so I can't say exactly
what's happening here.
WRT b...p2 (via ECF filetransfer) does indeed support NTLMv2. There's a
long history to this...see [1]...but the upshot is that ECF filetransfer
has a workaround to not use the apache httpclient 3.1 provider, in the
event of NTLMv2 (since apache httpclient < 4.0 did not support NTLMv2,
and that's what ECF used until recently).
@p2-devs any hints as to b) above?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2 may be related.
@Vincent just in case using apache httpclient v4 would help, this is not up to
you; it's a change which would have to be done in p2 (or at least in tycho)
No...it's a change that's underway...intended for Kepler: i.e. see this
enhancement [2]. The desire here is that apache httpclient 4.x
reportedly has support for ntlmv2 proxies, where httpclient 3.1 did
not. We have a completed provider based upon httpclient 4.x, and would
appreciate some help with testing against various proxy configurations
in coming integration builds.
But again...since I don't know how/what tycho does WRT downloading, this
might not address your proxy issue.
Thanks,
Scott
[1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=252002
[2] https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449
Another thing you should try is to enable advanced HTTP tracing as per [1] and
post the traces
Regards,
Jan
[1] http://wiki.eclipse.org/Equinox_p2_Reporting_Problems
-----Original Message-----
From: [email protected] [mailto:[email protected]] On
Behalf Of Girard-reydet, Vincent
Sent: Montag, 18. Februar 2013 18:50
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
Hi Jan,
Thank you for your answer. No I'm not using encrypted passwords for now. I have
triple-checked the settings and it's using the proxy (I've tried removing the
section completely and the Jenkins fails to download the Maven plugins). Here
is the proxy section I'm using (passwords, logins and hosts modified):
<proxies>
<proxy>
<id>bp</id>
<active>true</active>
<protocol>http</protocol>
<username>MYUSERNAME</username>
<password>MYPASSWORD</password>
<host>my.proxy.host</host>
<port>808080</port>
<!--<nonProxyHosts>local.net|some.host.com</nonProxyHosts>-->
</proxy>
</proxies>
Note that I have dropped the wagon-http-lightweight-2.2.jar in the
${MAVEN_HOME}/lib/ext folder of my Maven installation as well as in the folder
of Jenkins, just to be sure.
But what I really don't understand is why it works in command line and not in
Jenkins. I don't see where the difference is, as I'm using the same version of
Maven and the same config file. I don't have enough knowledge of the platform
resolution process to understand: where does Maven stop, where does Tycho stop
and where does pure P2 code start in the platform resolution process, and how
does the chain of transmission of credentials work? I thought that TYCHO-279
had solved the problem, but from the fix I was not able to completely
understand the process.
To me it seems that Tycho should configure the proxy/credentials resolution in
p2 before invoking it to resolve the platform, so the bug would be in the
Tycho/Jenkins interaction, but I may be wrong.
To try the httpclient4, I have to consume the newest version of
org.eclipse.ecf.filetransfer.feature, that's right? What would be the
recommended way of doing (or is it what I have to ask to [email protected])?
Vincent
Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited, 20 Canada
Square, London, E14 5NJ
t: +44 20 7948 5122
BP Gas Marketing Limited, a company registered in England and Wales with the
company number 908982, VAT number GB 365 6789 95 and whose registered office is
Chertsey Road, Sunbury on Thames, Middlesex, TW16 7BP
This message may contain information that is privileged or confidential. If you
are not the intended recipient please delete it and inform the sender
immediately. Within the bounds of law, the UK based entities in the Trading arm
of the BP Group retain all emails and IMs and monitor them to ensure compliance
with their internal policies and for other legitimate business purposes.
-----Original Message-----
From: [email protected] [mailto:[email protected]] On
Behalf Of Sievers, Jan
Sent: 18 February 2013 16:28
To: Tycho user list
Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
are you using maven-encrypted passwords?
https://bugs.eclipse.org/bugs/show_bug.cgi?id=394023
are you sure you added proxy credentials in settings.xml [1] ?
other than that you may have hit an NTLM limitation in the outdated apache
httpclient 3.x still used by p2.
here is the bug to upgrade to httpclient v4 which seems to indicate it has
better support for NTLM:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2
in this case I suggest to ask on [email protected] for any hints.
Regards
Jan
[1] http://maven.apache.org/settings.html#Proxies
From: [email protected] [mailto:[email protected]] On
Behalf Of Girard-reydet, Vincent
Sent: Montag, 18. Februar 2013 16:07
To: [email protected]
Subject: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy
Hi folks,
I'm struggling to configure a Maven build using Tycho on Jenkins behind my
company's proxy. I have a set of modules that build perfectly well from the
console (I just rebuilt everything using a fresh m2 repo and the build is
completely successful). Now I' trying to automate it with Jenkins (see the full
env spec at the end of the email). I have configured a Maven build in Jenkins.
For the local and global settings, I'm pointing at the exact same settings file
I'm using in the console. This settings file contains the proxy definition,
with the username, password and everything (and working fine in console). The
build is using a private Maven repo.
For my build, I'm using a target platform file (NOT using p2 repositories in
the pom.xml), as per the below. This platform is resolved OK when building in
console.
The Jenkins build uses correctly the proxy information to download all Maven
plugins (including Tycho plugins). However, when Tycho starts resolving the
platform, it somehow fails to collect the proxy information (which it should
since TYCHO-279 is now closed). I tried all the workarounds described to
circumvene TYCHO-279 (using the ECF proxy properties), I tried also using a
Nexus repo, nothing worked. I'm a bit screwed new, it seems to be somehow
Jenkins-specific but I don't' have a clue what's going on.
Would someone be able to help me investigate the problem?
Environment spec:
- Windows 7
- NTLM2 proxy
- Maven 3.0.4
- JDK 1.6.39 x86_64
- Jenkins v1.501 (out of the box install, running as a Windows service)
- Tycho v0.16
Platform configuration in parent pom:
<plugin>
<groupId>org.eclipse.tycho</groupId>
<artifactId>target-platform-configuration</artifactId>
<version>${tycho.version}</version>
<configuration>
<target>
<artifact>
<groupId>FAKE_GROUP_ID</groupId>
<artifactId>target</artifactId>
<version>1.0.0</version>
<classifier>indigo</classifier>
</artifact>
</target>
<environments>
<environment>
<os>win32</os>
<ws>win32</ws>
<arch>x86</arch>
</environment>
<environment>
<os>win32</os>
<ws>win32</ws>
<arch>x86_64</arch>
</environment>
</environments>
</configuration>
</plugin>
Platform definition :
<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?pde version="3.6"?>
<target name="indigo" sequenceNumber="14"> <locations> <location includeAllPlatforms="false" includeMode="slicer"
includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.equinox.server.servletbridge.feature.group"
version="1.0.100.v20110502-42DF9oB5865K5D"/>
<repository location="http://download.eclipse.org/eclipse/updates/3.7"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit">
<unit id="org.eclipse.ajdt.feature.group" version="2.2.0.e37x-RELEASE-20120704-0900"/>
<unit id="org.eclipse.equinox.weaving.sdk.feature.group"
version="1.0.200.I20120427-0800"/>
<repository location="http://download.eclipse.org/tools/ajdt/37/update"/>
</location>
<location includeAllPlatforms="false" includeMode="slicer" includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.sdk.ide" version="3.7.2.M20120208-0800"/> <unit
id="org.eclipse.emf.ecore.feature.group" version="2.7.1.v20120127-1122"/> <unit id="org.eclipse.emf.sdk.feature.group" version="2.7.2.v20120130-0943"/> <unit
id="org.eclipse.emf.compare.feature.group" version="1.2.2.v20120214-0915"/> <unit id="org.eclipse.emf.query.feature.group" version="1.5.0.v20110502-1600-218Z7w311A142A1142"/>
<unit id="org.eclipse.draw2d.feature.group"
version="3.7.2.v20110927-2020-4617w3122212803131"/>
<unit id="org.eclipse.xsd.sdk.feature.group" version="2.7.1.v20120130-0943"/> <unit
id="org.eclipse.objectteams.otequinox.feature.group" version="2.0.2.201202051448"/> <unit
id="org.eclipse.equinox.sdk.feature.group" version="3.7.1.R37x_v20110907-7M7W8i8eNV4WsRkue-4Vq4J6pCyW"/>
<unit id="javax.transaction" version="1.1.1.v201105210645"/>
<unit id="org.eclipse.equinox.executable.feature.group"
version="3.5.0.v20110530-7P7NFUFFLWUl76mart"/>
<unit id="org.eclipse.jdt.feature.group"
version="3.7.1.r371_v20110810-0800-7z8gFcoFMLfTabvKsR5Qm9rBGEBK"/>
<unit id="org.eclipse.rcp.sdk.id" version="3.7.1.M20110909-1335"/>
<repository location="http://download.eclipse.org/releases/indigo"/>
</location>
</locations>
<environment>
<arch>x86</arch>
</environment>
</target>
Error log in Jenkins (no error when building from console):
Downloaded:
http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plugin/1.3/aspectj-maven-plugin-1.3.jar
(37 KB at 63.6 KB/sec) [INFO] Computing target platform for MavenProject:
<MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @
C:\jenkins_workspace\test\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 12:30:08 org.apache.commons.httpclient.auth.AuthChallengeProcessor
selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 12:30:08 org.apache.commons.httpclient.HttpMethodDirector
processProxyAuthChallenge
INFO: No credentials available for NTLM <any realm>@iaccess.bp.com:80
mavenExecutionResult exceptions not empty message : Internal error:
java.lang.RuntimeException: Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
cause : Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
Stack trace :
org.apache.maven.InternalErrorException: Internal error:
java.lang.RuntimeException: Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:168)
at
org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
at
org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158)
at hudson.maven.Maven3Builder.call(Maven3Builder.java:100)
at hudson.maven.Maven3Builder.call(Maven3Builder.java:66)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:326)
at
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: Failed to resolve target definition
C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileContentToTargetPlatform(P2TargetPlatformResolver.java:338)
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.computeTargetPlatform(P2TargetPlatformResolver.java:207)
at
org.eclipse.tycho.core.resolver.DefaultTychoDependencyResolver.resolveProject(DefaultTychoDependencyResolver.java:94)
at
org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProjectsRead(TychoMavenLifecycleParticipant.java:82)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:274)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
... 19 more
Caused by:
org.eclipse.tycho.p2.target.facade.TargetDefinitionResolutionException: Failed
to load metadata repository from location
http://download.eclipse.org/eclipse/updates/3.7
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(TargetDefinitionResolver.java:190)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.resolveContent(TargetDefinitionResolver.java:109)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolverService.resolveFromArguments(TargetDefinitionResolverService.java:64)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolverService.getTargetDefinitionContent(TargetDefinitionResolverService.java:54)
at
org.eclipse.tycho.p2.target.TargetPlatformBuilderImpl.addTargetDefinition(TargetPlatformBuilderImpl.java:289)
at
org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileContentToTargetPlatform(P2TargetPlatformResolver.java:331)
... 24 more
Caused by: org.eclipse.equinox.p2.core.ProvisionException: HTTP Proxy
Authentication Required:
http://download.eclipse.org/eclipse/updates/3.7/content.xml
at
org.eclipse.equinox.internal.p2.repository.CacheManager.createCache(CacheManager.java:192)
at
org.eclipse.tycho.p2.remote.TychoP2RepositoryCacheManager.createCache(TychoP2RepositoryCacheManager.java:62)
at
org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.getLocalFile(SimpleMetadataRepositoryFactory.java:66)
at
org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.load(SimpleMetadataRepositoryFactory.java:88)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.factoryLoad(MetadataRepositoryManager.java:57)
at
org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:758)
at
org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:651)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:96)
at
org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:92)
at
org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadRepository(RemoteMetadataRepositoryManager.java:82)
at
org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(TargetDefinitionResolver.java:188)
... 29 more
Caused by: org.eclipse.ecf.filetransfer.BrowseFileTransferException: Proxy auth
required
at
org.eclipse.ecf.provider.filetransfer.browse.URLFileSystemBrowser.runRequest(URLFileSystemBrowser.java:129)
at
org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser$DirectoryJob.run(AbstractFileSystemBrowser.java:69)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53)
channel stopped
Output from the console (OK):
Downloaded:
http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plugin/1.3/aspectj-maven-plugin-1.3.jar
(37 KB at 63.6 KB/sec) [INFO] Scanning for projects...
[INFO] Computing target platform for MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT
@ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
18-Feb-2013 14:58:28 org.apache.commons.httpclient.auth.AuthChallengeProcessor
selectAuthScheme
INFO: ntlm authentication scheme selected
18-Feb-2013 14:58:28 org.apache.commons.httpclient.HttpMethodDirector
processProxyAuthChallenge
INFO: No credentials available for NTLM <any realm>@iaccess.bp.com:80 [WARNING]
De-selecting bundles in a target definition file is not supported. See
http://wiki.eclipse.org/Tycho_Messages_Explained#Target_File_Include_Bundles for
alternatives.
[INFO] Resolving dependencies of MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @
C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] The following locally built units have been used to resolve project
dependencies:
<list of dependencies to local modules>
[INFO] Resolving class path of MavenProject: <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @
C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml
[WARNING] Dependency from C:\git\ok\<MODULE_ID_HIDDEN> to nested classpath entry
C:\git\ok\<OTHER_MODULE>\icons can not be represented in Maven model and will not be
visible to non-OSGi aware Maven plugins
Thank,
Vincent
Vincent Girard-Reydet - BP Gas Marketing Limited, 20 Canada Square, London, E14
5NJ BP Gas Marketing Limited, a company registered in England and Wales with
the company number 908982, VAT number GB 365 6789 95 and whose registered
office is Chertsey Road, Sunbury on Thames, Middlesex, TW16 7BP This message
may contain information that is privileged or confidential. If you are not the
intended recipient please delete it and inform the sender immediately. Within
the bounds of law, the UK based entities in the Trading arm of the BP Group
retain all emails and IMs and monitor them to ensure compliance with their
internal policies and for other legitimate business purposes.
_______________________________________________
tycho-user mailing list
[email protected]
https://dev.eclipse.org/mailman/listinfo/tycho-user
_______________________________________________
tycho-user mailing list
[email protected]
https://dev.eclipse.org/mailman/listinfo/tycho-user
_______________________________________________
p2-dev mailing list
[email protected]
https://dev.eclipse.org/mailman/listinfo/p2-dev
_______________________________________________
p2-dev mailing list
[email protected]
https://dev.eclipse.org/mailman/listinfo/p2-dev
_______________________________________________
p2-dev mailing list
[email protected]
https://dev.eclipse.org/mailman/listinfo/p2-dev
_______________________________________________
p2-dev mailing list
[email protected]
https://dev.eclipse.org/mailman/listinfo/p2-dev
