[jira] Commented: (IVY-1159) Resolved Ivy Properties written to cache during ivy:resolve incorrectly represents transitive evictions
[
https://issues.apache.org/jira/browse/IVY-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12877218#action_12877218
]
Maarten Coene commented on IVY-1159:
I think such a change would be too big to make it into the next release.
So I've fixed it like this:
- the replaceDynamicRev=true setting will replace dynamic revisions (not
static ones) with the resolved revision
- a new replaceForcedRev=true setting will update static/dynamic revisions
with the forced revision (defaults to 'false' for backward compatibility)
the replacement of static evicted revisions as I suggested earlier isn't
possible because this depends on the configurations being resolved.
So in your case, you should call ivy:deliver like this:
{code}
ivy:deliver replaceDynamicRev=true replaceForcedRev=true ... /
{code}
Hope this solves your problem.
Maarten
Resolved Ivy Properties written to cache during ivy:resolve incorrectly
represents transitive evictions
Key: IVY-1159
URL: https://issues.apache.org/jira/browse/IVY-1159
Project: Ivy
Issue Type: Bug
Components: Core
Affects Versions: trunk
Reporter: Ed Burcher
Fix For: 2.2.0-RC1
Attachments: IVY1159.tar.gz
In ResolveEngine.resolve the code that writes the properties file appears to
be incorrect.
When the dependencies collection includes two or more entries for the same
dependency (one from the root ivy file and the others being transitives), the
properties file will always only ever be populated with the information from
the IvyNode that belongs to the root ivy file (effectively the comment states
this, so it certainly appears intentional).
This produces the following bugs / undesirable effects:
* incorrect delivered descriptor in some situations (where a dependency is
both directly and transitively imported)
* order of declaration of dependencies alters behaviour
These are pretty major problems because (as demonstrated below) a
delivered/published ivy descriptor can identify completely bogus revisions
for the dynamic-static replacements, which are not to the actual revisions
used when compiling etc
Set up
module *one* has no dependencies and has been published (as rev = 1, say)
module *two* has a dependency on module one (only) and has been published (as
rev = 1, also)
module *three* has dependencies on both modules one and two and *lists them
in the order two,one*
In the module three descriptor the revisions mentioned for two and one do not
actually exist (two=0 one=0 say - repository only contains rev=1 of both)
Ivy settings (attached) has a resolver that refers to a local repo, and has
force=true and local=true set.
Problem Case - use case: publishing module 3
1) ivy:resolve on module three, using refresh=true, transitive=true.
Otherwise nothing special here.
then
2) ivy:deliver (status=reelase, pubdate=now, deliverpattern supplied,
pubrevision supplied (rev=1))
3) (lastly, the ivy:publish step woud happen here, but is not relevant to the
problem)
Expected outcome:
Because the primary resolver has force=true, rev 1 of both module
dependencies of 'three' should be selected [Because rev=1 publications are
the only ones present in the repository] when resolving three's declared deps
(both of which declare a dep on rev=0).
Actual outcome:
Delivered descriptor correctly names two as being resolved to rev=1.
Incorrectly names module one as being resolved to rev=0 (which doesn't exist
- and never has!)
Workaround:
Reverse the order of the declared dependencies in three's ivy descriptor (i.e
new order is one, two) - problem does not occur.
Diagnosis:
Logs for resolve appear to show everything is fine (the eviction is noted,
the generated report xml shows the declared direct dependency (one, rev=0)
being evicted by the transitive dependency (one, rev=1 as declared in two's
published ivy descriptor)
However, debugging DeliverEngine.java and ResolveEngine.java - it is apparent
that the 'resolved ivy properties file' is used to drive the replacement of
dynamic revisions with static ones during deliver. In the error case, the
properties file shows this:
+revision\:\...@\#\:+0\:\...@\#\:+module\:\...@\#\:+one\: snip :=0 ?
+revision\:\...@\#\:+0\:\...@\#\:+module\:\...@\#\:+two\: snip :=1 release
The '?' is put there because the IvyNode that represented the direct
dependency has no descriptor (correctly, because it has been evicted).
Because the real selected revision is not put into the properties file, the
ivy:deliver task is doomed to produce wrong results.
The correct behaviour in this case would be for the EvictionInfo stored
against the evicted IvyNode to be inspected to find the
[jira] Commented: (IVY-1159) Resolved Ivy Properties written to cache during ivy:resolve incorrectly represents transitive evictions
[ https://issues.apache.org/jira/browse/IVY-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12870387#action_12870387 ] Ed Burcher commented on IVY-1159: - Hi Maarten - you raise some interesting points. The question of whether a revision is treated as being static or dynamic would therefore appear to be a combination of the Revision Spec (1.0 vs 1.0+ etc), and the resolver properties. Secondly, let's assert that the delivery engine needs to understand whether something was treated as static or dynamic to effect the correct delivery behaviour (dynamic being something that should be replaced with the resolved version). Because of this, I'd argue that the static/dynamic determination that was 'current' whilst obtaining each selected revision should be retained as part of the retained resolve state, so that it can be used by the delivery engine. In other words, the resolver that was used to obtain the ultimately selected revision should contribute state to the delivery engine specifically because the static/dynamic treatment can only be fully understood with reference to a particular resolver. -- Your second bullet is again very interesting and I have to say that personally I _would_ make the same argument for static revisions: Presumably the situation described will only occur with static revisions where transitivity is enabled during resolve. Is it possible that the original vision of a distinction between static and dynamic revisions was made substantially less clear by the introduction of transitive resolution in 1.4+ ? I'd argue this because it's clear that when transitive resolution is not used, behaviour is basically consistent. Static revs are not replaced in deliver, whereas dynamic ones are. As soon as you go with transitive resolution, evictions become possible both for 'static' and 'dynamic' revisions. (Moreover different ivy descriptors in the overall set may have different revision specs - some of which may be static and some dynamic). Whereas pre-1.4 ivy offers a delivered descriptor that is consistent with the resolve (for any combination of static/dynamic), your bullet illustrates that this is not the case anymore. Personally I am not convinced the introduction of transitive resolution should have had any bearing on whether deliver is supposed to _in principle_ produce a file that allows an identical, static, repeatable resolution in future. In practice I believe that using transitive resolution makes and and all revisions evictable and hence dynamic. I therefore propose the following pseudo logic: IF resolve mode is TRANSITIVE treat all dependencies as dynamic when emitting delivered ivy file ELSE for each dependency: IF the specification is dynamic OR if the resolver that selected the revision is forced-dynamic, then treat as dynamic in the delivery stage ELSE treat as static -- Regarding your suggestion - I think a new flag could be a pragmatic way to solve this in the short term. However I am not convinced it will work as stated - the properties file that drives the replacement in ivy:deliver is actually written in the ivy:resolve stage (when the bug occurs, this file contains bogus data). Since the resolve task does not have knowledge of the flags that will be passed to the deliver task, the written data would still be bad. This is basically because it treats a rev as static when it should have been treated dynamically given that it existed within a transitive, forced-resolve context. Resolved Ivy Properties written to cache during ivy:resolve incorrectly represents transitive evictions Key: IVY-1159 URL: https://issues.apache.org/jira/browse/IVY-1159 Project: Ivy Issue Type: Bug Components: Core Affects Versions: trunk Reporter: Ed Burcher Fix For: 2.2.0-RC1 Attachments: IVY1159.tar.gz In ResolveEngine.resolve the code that writes the properties file appears to be incorrect. When the dependencies collection includes two or more entries for the same dependency (one from the root ivy file and the others being transitives), the properties file will always only ever be populated with the information from the IvyNode that belongs to the root ivy file (effectively the comment states this, so it certainly appears intentional). This produces the following bugs / undesirable effects: * incorrect delivered descriptor in some situations (where a dependency is both directly and transitively imported) * order of declaration of dependencies alters behaviour These are pretty major problems because (as demonstrated below) a delivered/published ivy descriptor can identify completely bogus revisions for the dynamic-static
[jira] Commented: (IVY-1159) Resolved Ivy Properties written to cache during ivy:resolve incorrectly represents transitive evictions
[
https://issues.apache.org/jira/browse/IVY-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12868857#action_12868857
]
Maarten Coene commented on IVY-1159:
Ed, I understand your concerns.
- Regarding the force/dynamic style behaviour: this is only true within the
context of the resolver in force mode. If this resolver doesn't find the module
for instance, the next resolver is used but the revision is no longer
'latest.integration'. The same argument can be made when these properties are
written out: this happens after the resolve process, no longer within the scope
of the resolver in forced mode, which means Ivy will consider the dependency
having a static revision again.
- Regarding the deliverd ivy.xml representing the resolution results. You can
make the same argument for evicted revisions. Static evicted revisions in the
delivered ivy.xml will also not get replaced by the resolved revisions. Maybe
we could add an extra attribute to the deliver Ant task indicating that static
revisions should get replaced as well? Something like:
{noformat}
ivy:deliver replaceDynamicRev=true replaceStaticRev=true /
{noformat}
The default value of this 'replaceStaticRev' attribute would be false to keep
backwards compatibility. What do you think?
Resolved Ivy Properties written to cache during ivy:resolve incorrectly
represents transitive evictions
Key: IVY-1159
URL: https://issues.apache.org/jira/browse/IVY-1159
Project: Ivy
Issue Type: Bug
Components: Core
Affects Versions: trunk
Reporter: Ed Burcher
Fix For: 2.2.0-RC1
Attachments: IVY1159.tar.gz
In ResolveEngine.resolve the code that writes the properties file appears to
be incorrect.
When the dependencies collection includes two or more entries for the same
dependency (one from the root ivy file and the others being transitives), the
properties file will always only ever be populated with the information from
the IvyNode that belongs to the root ivy file (effectively the comment states
this, so it certainly appears intentional).
This produces the following bugs / undesirable effects:
* incorrect delivered descriptor in some situations (where a dependency is
both directly and transitively imported)
* order of declaration of dependencies alters behaviour
These are pretty major problems because (as demonstrated below) a
delivered/published ivy descriptor can identify completely bogus revisions
for the dynamic-static replacements, which are not to the actual revisions
used when compiling etc
Set up
module *one* has no dependencies and has been published (as rev = 1, say)
module *two* has a dependency on module one (only) and has been published (as
rev = 1, also)
module *three* has dependencies on both modules one and two and *lists them
in the order two,one*
In the module three descriptor the revisions mentioned for two and one do not
actually exist (two=0 one=0 say - repository only contains rev=1 of both)
Ivy settings (attached) has a resolver that refers to a local repo, and has
force=true and local=true set.
Problem Case - use case: publishing module 3
1) ivy:resolve on module three, using refresh=true, transitive=true.
Otherwise nothing special here.
then
2) ivy:deliver (status=reelase, pubdate=now, deliverpattern supplied,
pubrevision supplied (rev=1))
3) (lastly, the ivy:publish step woud happen here, but is not relevant to the
problem)
Expected outcome:
Because the primary resolver has force=true, rev 1 of both module
dependencies of 'three' should be selected [Because rev=1 publications are
the only ones present in the repository] when resolving three's declared deps
(both of which declare a dep on rev=0).
Actual outcome:
Delivered descriptor correctly names two as being resolved to rev=1.
Incorrectly names module one as being resolved to rev=0 (which doesn't exist
- and never has!)
Workaround:
Reverse the order of the declared dependencies in three's ivy descriptor (i.e
new order is one, two) - problem does not occur.
Diagnosis:
Logs for resolve appear to show everything is fine (the eviction is noted,
the generated report xml shows the declared direct dependency (one, rev=0)
being evicted by the transitive dependency (one, rev=1 as declared in two's
published ivy descriptor)
However, debugging DeliverEngine.java and ResolveEngine.java - it is apparent
that the 'resolved ivy properties file' is used to drive the replacement of
dynamic revisions with static ones during deliver. In the error case, the
properties file shows this:
+revision\:\...@\#\:+0\:\...@\#\:+module\:\...@\#\:+one\: snip :=0 ?
[jira] Commented: (IVY-1159) Resolved Ivy Properties written to cache during ivy:resolve incorrectly represents transitive evictions
[
https://issues.apache.org/jira/browse/IVY-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12867915#action_12867915
]
Maarten Coene commented on IVY-1159:
I don't think that the delivered ivy file should contain rev=1, but it should
still contain rev=0 for both your dependencies.
The reason is that only dynamic revisions should get replaced, not static ones
(even if the are evicted by another one).
I'm working on a fix so that in the case of a forced revision it doesn't get
replaced.
Resolved Ivy Properties written to cache during ivy:resolve incorrectly
represents transitive evictions
Key: IVY-1159
URL: https://issues.apache.org/jira/browse/IVY-1159
Project: Ivy
Issue Type: Bug
Components: Core
Affects Versions: trunk
Reporter: Ed Burcher
Fix For: 2.2.0-RC1
Attachments: IVY1159.tar.gz
In ResolveEngine.resolve the code that writes the properties file appears to
be incorrect.
When the dependencies collection includes two or more entries for the same
dependency (one from the root ivy file and the others being transitives), the
properties file will always only ever be populated with the information from
the IvyNode that belongs to the root ivy file (effectively the comment states
this, so it certainly appears intentional).
This produces the following bugs / undesirable effects:
* incorrect delivered descriptor in some situations (where a dependency is
both directly and transitively imported)
* order of declaration of dependencies alters behaviour
These are pretty major problems because (as demonstrated below) a
delivered/published ivy descriptor can identify completely bogus revisions
for the dynamic-static replacements, which are not to the actual revisions
used when compiling etc
Set up
module *one* has no dependencies and has been published (as rev = 1, say)
module *two* has a dependency on module one (only) and has been published (as
rev = 1, also)
module *three* has dependencies on both modules one and two and *lists them
in the order two,one*
In the module three descriptor the revisions mentioned for two and one do not
actually exist (two=0 one=0 say - repository only contains rev=1 of both)
Ivy settings (attached) has a resolver that refers to a local repo, and has
force=true and local=true set.
Problem Case - use case: publishing module 3
1) ivy:resolve on module three, using refresh=true, transitive=true.
Otherwise nothing special here.
then
2) ivy:deliver (status=reelase, pubdate=now, deliverpattern supplied,
pubrevision supplied (rev=1))
3) (lastly, the ivy:publish step woud happen here, but is not relevant to the
problem)
Expected outcome:
Because the primary resolver has force=true, rev 1 of both module
dependencies of 'three' should be selected [Because rev=1 publications are
the only ones present in the repository] when resolving three's declared deps
(both of which declare a dep on rev=0).
Actual outcome:
Delivered descriptor correctly names two as being resolved to rev=1.
Incorrectly names module one as being resolved to rev=0 (which doesn't exist
- and never has!)
Workaround:
Reverse the order of the declared dependencies in three's ivy descriptor (i.e
new order is one, two) - problem does not occur.
Diagnosis:
Logs for resolve appear to show everything is fine (the eviction is noted,
the generated report xml shows the declared direct dependency (one, rev=0)
being evicted by the transitive dependency (one, rev=1 as declared in two's
published ivy descriptor)
However, debugging DeliverEngine.java and ResolveEngine.java - it is apparent
that the 'resolved ivy properties file' is used to drive the replacement of
dynamic revisions with static ones during deliver. In the error case, the
properties file shows this:
+revision\:\...@\#\:+0\:\...@\#\:+module\:\...@\#\:+one\: snip :=0 ?
+revision\:\...@\#\:+0\:\...@\#\:+module\:\...@\#\:+two\: snip :=1 release
The '?' is put there because the IvyNode that represented the direct
dependency has no descriptor (correctly, because it has been evicted).
Because the real selected revision is not put into the properties file, the
ivy:deliver task is doomed to produce wrong results.
The correct behaviour in this case would be for the EvictionInfo stored
against the evicted IvyNode to be inspected to find the appropriate revision
information. I have attached a pretty ugly example of how this could be
achieved.
Just before the comment (The evicted modules have no description, so we
can't put their status)
{code}
if (depDescriptor == null) {
EvictionData ed = null;
[jira] Commented: (IVY-1159) Resolved Ivy Properties written to cache during ivy:resolve incorrectly represents transitive evictions
[ https://issues.apache.org/jira/browse/IVY-1159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12867934#action_12867934 ] Ed Burcher commented on IVY-1159: - Thanks for taking a look. However your assessment puzzles me. My understanding of a resolver having force=true is that every revision in the ivy file is treated as dynamic. That's certainly my reading of this extract from http://ant.apache.org/ivy/history/trunk/settings/resolvers.html#common (under heading) Force Any standard resolver can be used in force mode, which is used mainly to handle local development builds. In force mode, the resolver attempts to find a dependency whatever the requested revision is (internally it replace the requested revision by 'latest.integration'), and if it finds one, it forces this revision to be returned, even when used in a chain with returnFirst=false. As such I'd expect to see dynamic-style behaviour on deliver in this case. I'd also make the same argument by suggesting that it 'always makes sense' that the deliver task writes a published ivy file that represents the resolution that just took place. After all, the point about dynamic and static ivy files is that the dynamic ones become static at the point where you want to fix the revision identities (Because you want repeatable, unchanging, predictable resolution behaviour for published artifacts). It would seem strange to me for the resolve step to force resolution to some acceptable artifacts, but for ivy not to allow a repeatable, static, resolve of the same in future by not supporting generation of a delivered ivy file to represent what was resolved. Even weirder would be if the deliver task produced an ivy file containing static revisions that *may not even exist* (as in my test case) and certainly were not the ones actually resolved. Resolved Ivy Properties written to cache during ivy:resolve incorrectly represents transitive evictions Key: IVY-1159 URL: https://issues.apache.org/jira/browse/IVY-1159 Project: Ivy Issue Type: Bug Components: Core Affects Versions: trunk Reporter: Ed Burcher Fix For: 2.2.0-RC1 Attachments: IVY1159.tar.gz In ResolveEngine.resolve the code that writes the properties file appears to be incorrect. When the dependencies collection includes two or more entries for the same dependency (one from the root ivy file and the others being transitives), the properties file will always only ever be populated with the information from the IvyNode that belongs to the root ivy file (effectively the comment states this, so it certainly appears intentional). This produces the following bugs / undesirable effects: * incorrect delivered descriptor in some situations (where a dependency is both directly and transitively imported) * order of declaration of dependencies alters behaviour These are pretty major problems because (as demonstrated below) a delivered/published ivy descriptor can identify completely bogus revisions for the dynamic-static replacements, which are not to the actual revisions used when compiling etc Set up module *one* has no dependencies and has been published (as rev = 1, say) module *two* has a dependency on module one (only) and has been published (as rev = 1, also) module *three* has dependencies on both modules one and two and *lists them in the order two,one* In the module three descriptor the revisions mentioned for two and one do not actually exist (two=0 one=0 say - repository only contains rev=1 of both) Ivy settings (attached) has a resolver that refers to a local repo, and has force=true and local=true set. Problem Case - use case: publishing module 3 1) ivy:resolve on module three, using refresh=true, transitive=true. Otherwise nothing special here. then 2) ivy:deliver (status=reelase, pubdate=now, deliverpattern supplied, pubrevision supplied (rev=1)) 3) (lastly, the ivy:publish step woud happen here, but is not relevant to the problem) Expected outcome: Because the primary resolver has force=true, rev 1 of both module dependencies of 'three' should be selected [Because rev=1 publications are the only ones present in the repository] when resolving three's declared deps (both of which declare a dep on rev=0). Actual outcome: Delivered descriptor correctly names two as being resolved to rev=1. Incorrectly names module one as being resolved to rev=0 (which doesn't exist - and never has!) Workaround: Reverse the order of the declared dependencies in three's ivy descriptor (i.e new order is one, two) - problem does not occur. Diagnosis: Logs for resolve appear to show everything is fine (the eviction is noted, the
