Hi Daniel and Cheolsoo,
No problem, thanks for the clarification. I filed PIG-4027 to fix this
issue.
Thanks,
Lorand
On 06/19/2014 08:39 PM, Daniel Dai wrote:
Sorry to break your build, Lorand. We will try to stick to a tez
release (0.5.0 maybe) once tez becomes stable. Now we don't have a
choice since we need tez bug fixes which only exist in tez trunk.
Before that, we can do whatever to make other developer's life easier.
I don't think the issues you listed is a problem, just leave a Jira
open, so by the time we release Pig 0.14.0, we don't forget to remove
the flag.
Thanks,
Daniel
On Thu, Jun 19, 2014 at 11:01 AM, Cheolsoo Park <piaozhe...@gmail.com> wrote:
Hi Lorand,
Sorry for the inconvenience. True that Tez snapshot jars are updated
frequently. I haven't run into this problem only because I closely follow
the Pig on Tez development.
+1 to set 'changing="true"' until we pin Tez to a stable release.
Thanks,
Cheolsoo
On Thu, Jun 19, 2014 at 6:50 AM, Lorand Bendig <lben...@gmail.com> wrote:
Hi,
After updating to the latest revision from trunk I couldn't build Pig due
to the issue described in PIG-4019.
The parameter list of
org.apache.tez.dag.api.VertexManagerPluginContext#setVertexParallelism
has changed and however Daniel's patch fixed it, my build still failed.
I found, that it's because of the Tez SNAPSHOT dependencies. The ivy
resolver for apache-snapshots sets checkModified="true" which updates the
metadata of Tez dependencies if there are modifications, but since the
intermediary Tez builds are published under the same revision
(0.5.0-incubating-SNAPSHOT) the local cache will only be populated for the
first time with the jars but won't get updated afterwards. So API changes
at Tez side which affects Pig, will result in build error.
Am I the only one with this issue? Or am I missing something?
A solution would be to set 'changing="true"' for the Tez dependencies in
ivy.xml.
Thus ivy would always look up the repository to check whether there are
changes and would download the most recent timestamped 0.5.0 snapshot jars.
There are, however couple of issues with this:
- Resolution of Tez jars will fail during offline builds, when one sets
usecacheonly="true" for ivy:resolve. Since by
default it's not the case, I wouldn't worry about this.
- This attribute definitely needs to be removed once Tez dependencies
evolve from SNAPSHOT
Please let me know what you think and I'll file a JIRA if necessary.
Thanks,
Lorand
