thanks Scott I think this will help to dig a little deeper. >I don't know how tycho handles the download...so I can't say exactly >what's happening here.
We delegate this to p2/ECF so it should work as if you would install/update with your IDE. All we do is configure the HTTP proxy via org.eclipse.core.net (and register the default java.net.Authenticator, same as org.eclipse.core.net.ui does IIRC). two follow-up questions: 1. is there any log/trace so we can check the fallback to the JRE proxy provider is actually happening in this case? 2. if the JRE proxy fallback happens, any chance different JDKs (OS/version) may have differing/no support for NTLM proxy authentication? @Vincent another thing you can check to help us pin down the issue: using the same HTTP proxy configurations in your IDE (in Window>Prefs), can you successfully update/install any bundles e.g. from eclipse.org ? just to make sure I got this right: you say the build works fine on one machine but doesn't on another? what's the difference between those 2 machines (OS, JDK) ? If the httpclient v4 support is available but not consumed yet, ultimately we may decide to bundle httpclient v4 with tycho if that solves the issue. Regards, Jan -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Scott Lewis Sent: Montag, 18. Februar 2013 23:50 To: [email protected] Subject: Re: [p2-dev] Tycho 0.16 + Jenkins + NTLM2 proxy FYI, On 2/18/2013 12:56 PM, Sievers, Jan wrote: > - cross-posting to p2-dev - > > tycho takes the maven HTTP proxy configuration from settings.xml and > configures p2 accordingly (via proxyservice API in org.eclipse.core.net) > if resolution and download of maven plugins works but p2 resolution does not, > this is a hint that either > > a) tycho fails to transfer the proxy info to p2 or > b) p2 / org.eclipse.core.net does not support NTLM proxy authentication > > since nobody reported a) yet I doubt this and suspect b) I don't know how tycho handles the download...so I can't say exactly what's happening here. WRT b...p2 (via ECF filetransfer) does indeed support NTLMv2. There's a long history to this...see [1]...but the upshot is that ECF filetransfer has a workaround to not use the apache httpclient 3.1 provider, in the event of NTLMv2 (since apache httpclient < 4.0 did not support NTLMv2, and that's what ECF used until recently). > > @p2-devs any hints as to b) above? > https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2 may be related. > > @Vincent just in case using apache httpclient v4 would help, this is not up > to you; it's a change which would have to be done in p2 (or at least in tycho) No...it's a change that's underway...intended for Kepler: i.e. see this enhancement [2]. The desire here is that apache httpclient 4.x reportedly has support for ntlmv2 proxies, where httpclient 3.1 did not. We have a completed provider based upon httpclient 4.x, and would appreciate some help with testing against various proxy configurations in coming integration builds. But again...since I don't know how/what tycho does WRT downloading, this might not address your proxy issue. Thanks, Scott [1] https://bugs.eclipse.org/bugs/show_bug.cgi?id=252002 [2] https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449 > Another thing you should try is to enable advanced HTTP tracing as per [1] > and post the traces > > Regards, > Jan > > [1] http://wiki.eclipse.org/Equinox_p2_Reporting_Problems > > > -----Original Message----- > From: [email protected] [mailto:[email protected]] > On Behalf Of Girard-reydet, Vincent > Sent: Montag, 18. Februar 2013 18:50 > To: Tycho user list > Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy > > Hi Jan, > > Thank you for your answer. No I'm not using encrypted passwords for now. I > have triple-checked the settings and it's using the proxy (I've tried > removing the section completely and the Jenkins fails to download the Maven > plugins). Here is the proxy section I'm using (passwords, logins and hosts > modified): > > <proxies> > <proxy> > <id>bp</id> > <active>true</active> > <protocol>http</protocol> > <username>MYUSERNAME</username> > <password>MYPASSWORD</password> > <host>my.proxy.host</host> > <port>808080</port> > <!--<nonProxyHosts>local.net|some.host.com</nonProxyHosts>--> > </proxy> > </proxies> > > Note that I have dropped the wagon-http-lightweight-2.2.jar in the > ${MAVEN_HOME}/lib/ext folder of my Maven installation as well as in the > folder of Jenkins, just to be sure. > > But what I really don't understand is why it works in command line and not in > Jenkins. I don't see where the difference is, as I'm using the same version > of Maven and the same config file. I don't have enough knowledge of the > platform resolution process to understand: where does Maven stop, where does > Tycho stop and where does pure P2 code start in the platform resolution > process, and how does the chain of transmission of credentials work? I > thought that TYCHO-279 had solved the problem, but from the fix I was not > able to completely understand the process. > > To me it seems that Tycho should configure the proxy/credentials resolution > in p2 before invoking it to resolve the platform, so the bug would be in the > Tycho/Jenkins interaction, but I may be wrong. > > To try the httpclient4, I have to consume the newest version of > org.eclipse.ecf.filetransfer.feature, that's right? What would be the > recommended way of doing (or is it what I have to ask to [email protected])? > > Vincent > > Vincent Girard-Reydet - LNG Optimisation - BP Gas Marketing Limited, 20 > Canada Square, London, E14 5NJ > t: +44 20 7948 5122 > BP Gas Marketing Limited, a company registered in England and Wales with the > company number 908982, VAT number GB 365 6789 95 and whose registered office > is Chertsey Road, Sunbury on Thames, Middlesex, TW16 7BP > This message may contain information that is privileged or confidential. If > you are not the intended recipient please delete it and inform the sender > immediately. Within the bounds of law, the UK based entities in the Trading > arm of the BP Group retain all emails and IMs and monitor them to ensure > compliance with their internal policies and for other legitimate business > purposes. > > > -----Original Message----- > From: [email protected] [mailto:[email protected]] > On Behalf Of Sievers, Jan > Sent: 18 February 2013 16:28 > To: Tycho user list > Subject: Re: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy > > are you using maven-encrypted passwords? > https://bugs.eclipse.org/bugs/show_bug.cgi?id=394023 > > are you sure you added proxy credentials in settings.xml [1] ? > > other than that you may have hit an NTLM limitation in the outdated apache > httpclient 3.x still used by p2. > here is the bug to upgrade to httpclient v4 which seems to indicate it has > better support for NTLM: > https://bugs.eclipse.org/bugs/show_bug.cgi?id=337449#c2 > > in this case I suggest to ask on [email protected] for any hints. > > Regards > Jan > > [1] http://maven.apache.org/settings.html#Proxies > > > > From: [email protected] [mailto:[email protected]] > On Behalf Of Girard-reydet, Vincent > Sent: Montag, 18. Februar 2013 16:07 > To: [email protected] > Subject: [tycho-user] Tycho 0.16 + Jenkins + NTLM2 proxy > > Hi folks, > > I'm struggling to configure a Maven build using Tycho on Jenkins behind my > company's proxy. I have a set of modules that build perfectly well from the > console (I just rebuilt everything using a fresh m2 repo and the build is > completely successful). Now I' trying to automate it with Jenkins (see the > full env spec at the end of the email). I have configured a Maven build in > Jenkins. For the local and global settings, I'm pointing at the exact same > settings file I'm using in the console. This settings file contains the proxy > definition, with the username, password and everything (and working fine in > console). The build is using a private Maven repo. > For my build, I'm using a target platform file (NOT using p2 repositories in > the pom.xml), as per the below. This platform is resolved OK when building in > console. > > The Jenkins build uses correctly the proxy information to download all Maven > plugins (including Tycho plugins). However, when Tycho starts resolving the > platform, it somehow fails to collect the proxy information (which it should > since TYCHO-279 is now closed). I tried all the workarounds described to > circumvene TYCHO-279 (using the ECF proxy properties), I tried also using a > Nexus repo, nothing worked. I'm a bit screwed new, it seems to be somehow > Jenkins-specific but I don't' have a clue what's going on. > > > Would someone be able to help me investigate the problem? > > Environment spec: > - Windows 7 > - NTLM2 proxy > - Maven 3.0.4 > - JDK 1.6.39 x86_64 > - Jenkins v1.501 (out of the box install, running as a Windows service) > - Tycho v0.16 > > Platform configuration in parent pom: > > <plugin> > <groupId>org.eclipse.tycho</groupId> > <artifactId>target-platform-configuration</artifactId> > <version>${tycho.version}</version> > <configuration> > <target> > <artifact> > <groupId>FAKE_GROUP_ID</groupId> > <artifactId>target</artifactId> > <version>1.0.0</version> > <classifier>indigo</classifier> > </artifact> > </target> > <environments> > <environment> > <os>win32</os> > <ws>win32</ws> > <arch>x86</arch> > </environment> > <environment> > <os>win32</os> > <ws>win32</ws> > <arch>x86_64</arch> > </environment> > </environments> > </configuration> > </plugin> > > Platform definition : > <?xml version="1.0" encoding="UTF-8" standalone="no"?> <?pde version="3.6"?> > > <target name="indigo" sequenceNumber="14"> <locations> <location > includeAllPlatforms="false" includeMode="slicer" includeSource="false" > type="InstallableUnit"> <unit > id="org.eclipse.equinox.server.servletbridge.feature.group" > version="1.0.100.v20110502-42DF9oB5865K5D"/> > <repository location="http://download.eclipse.org/eclipse/updates/3.7"/> > </location> > <location includeAllPlatforms="false" includeMode="slicer" > includeSource="false" type="InstallableUnit"> <unit > id="org.eclipse.ajdt.feature.group" > version="2.2.0.e37x-RELEASE-20120704-0900"/> > <unit id="org.eclipse.equinox.weaving.sdk.feature.group" > version="1.0.200.I20120427-0800"/> > <repository location="http://download.eclipse.org/tools/ajdt/37/update"/> > </location> > <location includeAllPlatforms="false" includeMode="slicer" > includeSource="false" type="InstallableUnit"> <unit id="org.eclipse.sdk.ide" > version="3.7.2.M20120208-0800"/> <unit > id="org.eclipse.emf.ecore.feature.group" version="2.7.1.v20120127-1122"/> > <unit id="org.eclipse.emf.sdk.feature.group" version="2.7.2.v20120130-0943"/> > <unit id="org.eclipse.emf.compare.feature.group" > version="1.2.2.v20120214-0915"/> <unit > id="org.eclipse.emf.query.feature.group" > version="1.5.0.v20110502-1600-218Z7w311A142A1142"/> > <unit id="org.eclipse.draw2d.feature.group" > version="3.7.2.v20110927-2020-4617w3122212803131"/> > <unit id="org.eclipse.xsd.sdk.feature.group" version="2.7.1.v20120130-0943"/> > <unit id="org.eclipse.objectteams.otequinox.feature.group" > version="2.0.2.201202051448"/> <unit > id="org.eclipse.equinox.sdk.feature.group" > version="3.7.1.R37x_v20110907-7M7W8i8eNV4WsRkue-4Vq4J6pCyW"/> > <unit id="javax.transaction" version="1.1.1.v201105210645"/> > > <unit id="org.eclipse.equinox.executable.feature.group" > version="3.5.0.v20110530-7P7NFUFFLWUl76mart"/> > <unit id="org.eclipse.jdt.feature.group" > version="3.7.1.r371_v20110810-0800-7z8gFcoFMLfTabvKsR5Qm9rBGEBK"/> > <unit id="org.eclipse.rcp.sdk.id" version="3.7.1.M20110909-1335"/> > > <repository location="http://download.eclipse.org/releases/indigo"/> > </location> > </locations> > <environment> > <arch>x86</arch> > </environment> > </target> > > Error log in Jenkins (no error when building from console): > > Downloaded: > http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plugin/1.3/aspectj-maven-plugin-1.3.jar > (37 KB at 63.6 KB/sec) [INFO] Computing target platform for MavenProject: > <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ > C:\jenkins_workspace\test\<MODULE_ID_HIDDEN>\pom.xml > 18-Feb-2013 12:30:08 > org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme > INFO: ntlm authentication scheme selected > 18-Feb-2013 12:30:08 org.apache.commons.httpclient.HttpMethodDirector > processProxyAuthChallenge > INFO: No credentials available for NTLM <any realm>@iaccess.bp.com:80 > mavenExecutionResult exceptions not empty message : Internal error: > java.lang.RuntimeException: Failed to resolve target definition > C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target > > cause : Failed to resolve target definition > C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target > Stack trace : > org.apache.maven.InternalErrorException: Internal error: > java.lang.RuntimeException: Failed to resolve target definition > C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target > at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:168) > at > org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java:79) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329) > at > org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239) > at > org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158) > at hudson.maven.Maven3Builder.call(Maven3Builder.java:100) > at hudson.maven.Maven3Builder.call(Maven3Builder.java:66) > at hudson.remoting.UserRequest.perform(UserRequest.java:118) > at hudson.remoting.UserRequest.perform(UserRequest.java:48) > at hudson.remoting.Request$2.run(Request.java:326) > at > hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) > at > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > at java.lang.Thread.run(Thread.java:662) > Caused by: java.lang.RuntimeException: Failed to resolve target definition > C:\jenkins_workspace\test\FAKE_GROUP_ID\indigo.target > at > org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileContentToTargetPlatform(P2TargetPlatformResolver.java:338) > at > org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.computeTargetPlatform(P2TargetPlatformResolver.java:207) > at > org.eclipse.tycho.core.resolver.DefaultTychoDependencyResolver.resolveProject(DefaultTychoDependencyResolver.java:94) > at > org.eclipse.tycho.core.maven.TychoMavenLifecycleParticipant.afterProjectsRead(TychoMavenLifecycleParticipant.java:82) > at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:274) > at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156) > ... 19 more > Caused by: > org.eclipse.tycho.p2.target.facade.TargetDefinitionResolutionException: > Failed to load metadata repository from location > http://download.eclipse.org/eclipse/updates/3.7 > at > org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(TargetDefinitionResolver.java:190) > at > org.eclipse.tycho.p2.target.TargetDefinitionResolver.resolveContent(TargetDefinitionResolver.java:109) > at > org.eclipse.tycho.p2.target.TargetDefinitionResolverService.resolveFromArguments(TargetDefinitionResolverService.java:64) > at > org.eclipse.tycho.p2.target.TargetDefinitionResolverService.getTargetDefinitionContent(TargetDefinitionResolverService.java:54) > at > org.eclipse.tycho.p2.target.TargetPlatformBuilderImpl.addTargetDefinition(TargetPlatformBuilderImpl.java:289) > at > org.eclipse.tycho.p2.resolver.P2TargetPlatformResolver.addTargetFileContentToTargetPlatform(P2TargetPlatformResolver.java:331) > ... 24 more > Caused by: org.eclipse.equinox.p2.core.ProvisionException: HTTP Proxy > Authentication Required: > http://download.eclipse.org/eclipse/updates/3.7/content.xml > at > org.eclipse.equinox.internal.p2.repository.CacheManager.createCache(CacheManager.java:192) > at > org.eclipse.tycho.p2.remote.TychoP2RepositoryCacheManager.createCache(TychoP2RepositoryCacheManager.java:62) > at > org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.getLocalFile(SimpleMetadataRepositoryFactory.java:66) > at > org.eclipse.equinox.internal.p2.metadata.repository.SimpleMetadataRepositoryFactory.load(SimpleMetadataRepositoryFactory.java:88) > at > org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.factoryLoad(MetadataRepositoryManager.java:57) > at > org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:758) > at > org.eclipse.equinox.internal.p2.repository.helpers.AbstractRepositoryManager.loadRepository(AbstractRepositoryManager.java:651) > at > org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:96) > at > org.eclipse.equinox.internal.p2.metadata.repository.MetadataRepositoryManager.loadRepository(MetadataRepositoryManager.java:92) > at > org.eclipse.tycho.p2.remote.RemoteMetadataRepositoryManager.loadRepository(RemoteMetadataRepositoryManager.java:82) > at > org.eclipse.tycho.p2.target.TargetDefinitionResolver.loadRepository(TargetDefinitionResolver.java:188) > ... 29 more > Caused by: org.eclipse.ecf.filetransfer.BrowseFileTransferException: Proxy > auth required > at > org.eclipse.ecf.provider.filetransfer.browse.URLFileSystemBrowser.runRequest(URLFileSystemBrowser.java:129) > at > org.eclipse.ecf.provider.filetransfer.browse.AbstractFileSystemBrowser$DirectoryJob.run(AbstractFileSystemBrowser.java:69) > at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53) > channel stopped > > Output from the console (OK): > > Downloaded: > http://repo.maven.apache.org/maven2/org/codehaus/mojo/aspectj-maven-plugin/1.3/aspectj-maven-plugin-1.3.jar > (37 KB at 63.6 KB/sec) [INFO] Scanning for projects... > [INFO] Computing target platform for MavenProject: > <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml > 18-Feb-2013 14:58:28 > org.apache.commons.httpclient.auth.AuthChallengeProcessor selectAuthScheme > INFO: ntlm authentication scheme selected > 18-Feb-2013 14:58:28 org.apache.commons.httpclient.HttpMethodDirector > processProxyAuthChallenge > INFO: No credentials available for NTLM <any realm>@iaccess.bp.com:80 > [WARNING] De-selecting bundles in a target definition file is not supported. > See > http://wiki.eclipse.org/Tycho_Messages_Explained#Target_File_Include_Bundles > for alternatives. > [INFO] Resolving dependencies of MavenProject: > <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml > [WARNING] The following locally built units have been used to resolve project > dependencies: > <list of dependencies to local modules> > [INFO] Resolving class path of MavenProject: > <MODULE_ID_HIDDEN>:1.0.0-SNAPSHOT @ C:\git\ok\<MODULE_ID_HIDDEN>\pom.xml > [WARNING] Dependency from C:\git\ok\<MODULE_ID_HIDDEN> to nested classpath > entry C:\git\ok\<OTHER_MODULE>\icons can not be represented in Maven model > and will not be visible to non-OSGi aware Maven plugins > > Thank, > Vincent > Vincent Girard-Reydet - BP Gas Marketing Limited, 20 Canada Square, London, > E14 5NJ BP Gas Marketing Limited, a company registered in England and Wales > with the company number 908982, VAT number GB 365 6789 95 and whose > registered office is Chertsey Road, Sunbury on Thames, Middlesex, TW16 7BP > This message may contain information that is privileged or confidential. If > you are not the intended recipient please delete it and inform the sender > immediately. Within the bounds of law, the UK based entities in the Trading > arm of the BP Group retain all emails and IMs and monitor them to ensure > compliance with their internal policies and for other legitimate business > purposes. > > _______________________________________________ > tycho-user mailing list > [email protected] > https://dev.eclipse.org/mailman/listinfo/tycho-user > _______________________________________________ > tycho-user mailing list > [email protected] > https://dev.eclipse.org/mailman/listinfo/tycho-user > _______________________________________________ > p2-dev mailing list > [email protected] > https://dev.eclipse.org/mailman/listinfo/p2-dev _______________________________________________ p2-dev mailing list [email protected] https://dev.eclipse.org/mailman/listinfo/p2-dev _______________________________________________ p2-dev mailing list [email protected] https://dev.eclipse.org/mailman/listinfo/p2-dev
