Re: IVY-1577 - how to activate automatic usage of Maven profiles
Hi Jaikiran, thanks for recommendation. We'll first upgrade to 2.5.0-rc2 and then decide how to continue with the cache issue on Jenkins. Everything is tracked on NUTCH-2669. Thanks, Sebastian On 2018/10/26 13:24:02, Jaikiran Pai wrote: > Hi Sebastian, > > > On 24/10/18 9:27 PM, Sebastian Nagel wrote: > > > > Also you have to make sure that no other project using an older ivy version > > fills the cache before. What are your recommendations to make sure that > > the local ivy cache stays correct on the Apache Jenkins cluster where we > > have little control which builds run on which machine: use a > > project-specific > > cache or no cache at all? Or is there a way to get also the benefits from a > > shared cache? > > Normally, using the shared cache should be fine since released > artifacts/metadata doesn't usually change so can be shared across > projects. However, for issues like the one you ran into, like you see, > it can cause odd problems. I would suggest using a (Nutch) project > specific cache location for you Jenkins jobs, so that the cache is > "known" and just shared by only your Nutch jobs and your jobs are in > much more control of them. Something like ~/.ivy2/nutch/cache/ as the > location. > > -Jaikiran > > >
Re: IVY-1577 - how to activate automatic usage of Maven profiles
Hi Sebastian, On 24/10/18 9:27 PM, Sebastian Nagel wrote: > > Also you have to make sure that no other project using an older ivy version > fills the cache before. What are your recommendations to make sure that > the local ivy cache stays correct on the Apache Jenkins cluster where we > have little control which builds run on which machine: use a project-specific > cache or no cache at all? Or is there a way to get also the benefits from a > shared cache? Normally, using the shared cache should be fine since released artifacts/metadata doesn't usually change so can be shared across projects. However, for issues like the one you ran into, like you see, it can cause odd problems. I would suggest using a (Nutch) project specific cache location for you Jenkins jobs, so that the cache is "known" and just shared by only your Nutch jobs and your jobs are in much more control of them. Something like ~/.ivy2/nutch/cache/ as the location. -Jaikiran
Re: IVY-1577 - how to activate automatic usage of Maven profiles
Hi Jaikiran,
ok, I can confirm that it works when the local ivy cache is wiped out,
in our case (and on Linux) by
rm -rf ~/.ivy2/cache/javax.ws.rs/
the Nutch build succeeds using ivy 2.5.0-rc1.
Also you have to make sure that no other project using an older ivy version
fills the cache before. What are your recommendations to make sure that
the local ivy cache stays correct on the Apache Jenkins cluster where we
have little control which builds run on which machine: use a project-specific
cache or no cache at all? Or is there a way to get also the benefits from a
shared cache?
Thanks,
Sebastian
On 2018/10/24 14:45:06, Jaikiran Pai wrote:
> Hi Sebastian,
>
> There shouldn't be anything explicit to enable this in 2.5.0-rc1. When
> you tried this with 2.5.0-rc1 did you clean up your local cache to make
> sure the it's fetching the module descriptor afresh from the Maven repo
> instead of local cache?
>
> FWIW, I just gave this a try locally. What I did was use this simple
> ivy.xml:
>
> http://ant.apache.org/ivy/osgi;>
>
>
>
>
>
>
>
>
> Then:
>
>
> java -jar apache-ivy-2.5.0-rc1/ivy-2.5.0-rc1.jar -cache ./temp-cache
>
>
> This resolved the dependencies correctly with 2.5.0-rc1 whereas when
> used with 2.4.0 it fails.
>
> If it still fails with a fresh cache with 2.5.0-rc1 with the Nutch
> project, let us know and I'll dig a bit deeper. If you have some steps
> for us to try and reproduce this easily with the setup/project you are
> using, let us know.
>
> -Jaikiran
>
>
> On 24/10/18 6:07 PM, Sebastian Nagel wrote:
> > Hi,
> >
> > the Apache Nutch community faces an dependency issue with the property
> > "packaging.type" defined in the Maven profiles. More details about the issue
> > are found in https://issues.apache.org/jira/browse/NUTCH-2669
> > Until now we have neither a reliable work-around nor a solution. Upgrading
> > to ivy 2.5.0-rc1 or a nightly build rc2 jar seems not not work. Maybe some
> > additional
> > steps are necessary.
> >
> > Below the ant/ivy logs and more. A description of all steps to upgrade the
> > ivy lib
> > is found in NUTCH-2669.
> >
> > Thanks,
> > Sebastian
> >
> > [ivy:resolve] loadData of javax.ws.rs#javax.ws.rs-api;2.1 of
> > rootConf=default
> > [ivy:resolve] using default to resolve javax.ws.rs#javax.ws.rs-api;2.1
> > [ivy:resolve] default: Checking cache for: dependency:
> > javax.ws.rs#javax.ws.rs-api;2.1 {compile=[compile(*), master(*)],
> > runtime=[runtime(*)]}
> > [ivy:resolve] No entry is found in the ModuleDescriptorCache :
> > /home/ubuntu/.ivy2/cache/javax.ws.rs/javax.ws.rs-api/ivy-2.1.xml
> > [ivy:resolve] post 1.3 ivy file: using exact as default matcher
> > [ivy:resolve] found ivy file in cache for javax.ws.rs#javax.ws.rs-api;2.1
> > (resolved by maven2):
> > /home/ubuntu/.ivy2/cache/javax.ws.rs/javax.ws.rs-api/ivy-2.1.xml
> > [ivy:resolve] default: module revision found in cache:
> > javax.ws.rs#javax.ws.rs-api;2.1
> > [ivy:resolve] found javax.ws.rs#javax.ws.rs-api;2.1 in maven2
> > ..
> > [ivy:resolve] Sort dependencies of : javax.ws.rs#javax.ws.rs-api;2.1 /
> > Number of dependencies = 1
> > [ivy:resolve] Sort done for : javax.ws.rs#javax.ws.rs-api;2.1
> > ..
> > [ivy:resolve] Module descriptor is processed :
> > javax.ws.rs#javax.ws.rs-api;2.1
> > ..
> > [ivy:resolve]trying
> > /home/ubuntu/.ivy2/local/javax.ws.rs/javax.ws.rs-api/2.1/${packaging.type}s/javax.ws.rs-api.${packaging.type}
> > [ivy:resolve] tried
> > /home/ubuntu/.ivy2/local/javax.ws.rs/javax.ws.rs-api/2.1/${packaging.type}s/javax.ws.rs-api.${packaging.type}
> > [ivy:resolve] local: resource not reachable for
> > javax.ws.rs#javax.ws.rs-api;2.1:
> > res=/home/ubuntu/.ivy2/local/javax.ws.rs/javax.ws.rs-api/2.1/${packaging.type}s/javax.ws.rs-api.${packaging.type}
> > [ivy:resolve]trying
> > http://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.${packaging.type}
> > [ivy:resolve] tried
> > http://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.${packaging.type}
> > [ivy:resolve] HTTP response status: 404
> > url=http://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.$%7Bpackaging.type%7D
> > [ivy:resolve] CLIENT ERROR: Not Found
> > url=http://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.$%7Bpackaging.type%7D
> > [ivy:resolve] maven2: resource not reachable for
> > javax/ws/rs#javax.ws.rs-api;2.1:
> > res=http://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.${packaging.type}
> > [ivy:resolve]trying
> > https://repository.apache.org/content/repositories/snapshots/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.${packaging.type}
> > [ivy:resolve] tried
> > https://repository.apache.org/content/repositories/snapshots/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.${packaging.type}
> > [ivy:resolve] HTTP response status:
Re: IVY-1577 - how to activate automatic usage of Maven profiles
Hi Sebastian,
There shouldn't be anything explicit to enable this in 2.5.0-rc1. When
you tried this with 2.5.0-rc1 did you clean up your local cache to make
sure the it's fetching the module descriptor afresh from the Maven repo
instead of local cache?
FWIW, I just gave this a try locally. What I did was use this simple
ivy.xml:
http://ant.apache.org/ivy/osgi;>
Then:
java -jar apache-ivy-2.5.0-rc1/ivy-2.5.0-rc1.jar -cache ./temp-cache
This resolved the dependencies correctly with 2.5.0-rc1 whereas when
used with 2.4.0 it fails.
If it still fails with a fresh cache with 2.5.0-rc1 with the Nutch
project, let us know and I'll dig a bit deeper. If you have some steps
for us to try and reproduce this easily with the setup/project you are
using, let us know.
-Jaikiran
On 24/10/18 6:07 PM, Sebastian Nagel wrote:
> Hi,
>
> the Apache Nutch community faces an dependency issue with the property
> "packaging.type" defined in the Maven profiles. More details about the issue
> are found in https://issues.apache.org/jira/browse/NUTCH-2669
> Until now we have neither a reliable work-around nor a solution. Upgrading
> to ivy 2.5.0-rc1 or a nightly build rc2 jar seems not not work. Maybe some
> additional
> steps are necessary.
>
> Below the ant/ivy logs and more. A description of all steps to upgrade the
> ivy lib
> is found in NUTCH-2669.
>
> Thanks,
> Sebastian
>
> [ivy:resolve] loadData of javax.ws.rs#javax.ws.rs-api;2.1 of rootConf=default
> [ivy:resolve] using default to resolve javax.ws.rs#javax.ws.rs-api;2.1
> [ivy:resolve] default: Checking cache for: dependency:
> javax.ws.rs#javax.ws.rs-api;2.1 {compile=[compile(*), master(*)],
> runtime=[runtime(*)]}
> [ivy:resolve] No entry is found in the ModuleDescriptorCache :
> /home/ubuntu/.ivy2/cache/javax.ws.rs/javax.ws.rs-api/ivy-2.1.xml
> [ivy:resolve] post 1.3 ivy file: using exact as default matcher
> [ivy:resolve] found ivy file in cache for javax.ws.rs#javax.ws.rs-api;2.1
> (resolved by maven2):
> /home/ubuntu/.ivy2/cache/javax.ws.rs/javax.ws.rs-api/ivy-2.1.xml
> [ivy:resolve] default: module revision found in cache:
> javax.ws.rs#javax.ws.rs-api;2.1
> [ivy:resolve] found javax.ws.rs#javax.ws.rs-api;2.1 in maven2
> ..
> [ivy:resolve] Sort dependencies of : javax.ws.rs#javax.ws.rs-api;2.1 / Number
> of dependencies = 1
> [ivy:resolve] Sort done for : javax.ws.rs#javax.ws.rs-api;2.1
> ..
> [ivy:resolve] Module descriptor is processed : javax.ws.rs#javax.ws.rs-api;2.1
> ..
> [ivy:resolve]trying
> /home/ubuntu/.ivy2/local/javax.ws.rs/javax.ws.rs-api/2.1/${packaging.type}s/javax.ws.rs-api.${packaging.type}
> [ivy:resolve] tried
> /home/ubuntu/.ivy2/local/javax.ws.rs/javax.ws.rs-api/2.1/${packaging.type}s/javax.ws.rs-api.${packaging.type}
> [ivy:resolve] local: resource not reachable for
> javax.ws.rs#javax.ws.rs-api;2.1:
> res=/home/ubuntu/.ivy2/local/javax.ws.rs/javax.ws.rs-api/2.1/${packaging.type}s/javax.ws.rs-api.${packaging.type}
> [ivy:resolve]trying
> http://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.${packaging.type}
> [ivy:resolve] tried
> http://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.${packaging.type}
> [ivy:resolve] HTTP response status: 404
> url=http://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.$%7Bpackaging.type%7D
> [ivy:resolve] CLIENT ERROR: Not Found
> url=http://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.$%7Bpackaging.type%7D
> [ivy:resolve] maven2: resource not reachable for
> javax/ws/rs#javax.ws.rs-api;2.1:
> res=http://repo1.maven.org/maven2/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.${packaging.type}
> [ivy:resolve]trying
> https://repository.apache.org/content/repositories/snapshots/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.${packaging.type}
> [ivy:resolve] tried
> https://repository.apache.org/content/repositories/snapshots/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.${packaging.type}
> [ivy:resolve] HTTP response status: 404
> url=https://repository.apache.org/content/repositories/snapshots/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.$%7Bpackaging.type%7D
> [ivy:resolve] CLIENT ERROR: Not Found
> url=https://repository.apache.org/content/repositories/snapshots/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.$%7Bpackaging.type%7D
> [ivy:resolve] apache-snapshot: resource not reachable for
> javax/ws/rs#javax.ws.rs-api;2.1:
> res=https://repository.apache.org/content/repositories/snapshots/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.${packaging.type}
> [ivy:resolve]trying
> http://oss.sonatype.org/content/repositories/releases/javax/ws/rs/javax.ws.rs-api/2.1/javax.ws.rs-api-2.1.${packaging.type}
> [ivy:resolve] tried
>
