[jira] [Commented] (YARN-5071) address HBase compatibility issues with trunk
[ https://issues.apache.org/jira/browse/YARN-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905968#comment-15905968 ] Sangjin Lee commented on YARN-5071: --- I believe this is largely addressed by your work (thanks!). I'm fine with closing this. [~vrushalic], let me know what you think. > address HBase compatibility issues with trunk > - > > Key: YARN-5071 > URL: https://issues.apache.org/jira/browse/YARN-5071 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Sangjin Lee >Priority: Critical > Labels: YARN-5355 > > The trunk is now adding or planning to add more and more > backward-incompatible changes. Some examples include > - remove v.1 metrics classes (HADOOP-12504) > - update jersey version (HADOOP-9613) > - target java 8 by default (HADOOP-11858) > This poses big challenges for the timeline service v.2 as we have a > dependency on hbase which depends on an older version of hadoop. > We need to find a way to solve/contain/manage these risks before it is too > late. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5071) address HBase compatibility issues with trunk
[ https://issues.apache.org/jira/browse/YARN-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15905915#comment-15905915 ] Haibo Chen commented on YARN-5071: -- [~sjlee0], [~vrushalic] this is filed a while ago, do you still see some existing compatibilities issues? bq. Regarding #1, we have made some effort to isolate at least the unit tests (YARN-5045), but there is code for coprocessor and the schema creator. We probably need to isolate them (and its dependencies) so that they are managed against the old version of hadoop. I believe we have decided not to isolate them in YARN-4985. bq. #2 might not arise that often (hbase client mostly depends only on hadoop-common), it is also harder to fix once it happens. We probably have to keep an eye on hadoop-common incompatibility changes that can break hbase-client and ask HBase to shade when the incompatible change is unavoidable ? > address HBase compatibility issues with trunk > - > > Key: YARN-5071 > URL: https://issues.apache.org/jira/browse/YARN-5071 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Sangjin Lee >Priority: Critical > Labels: YARN-5355 > > The trunk is now adding or planning to add more and more > backward-incompatible changes. Some examples include > - remove v.1 metrics classes (HADOOP-12504) > - update jersey version (HADOOP-9613) > - target java 8 by default (HADOOP-11858) > This poses big challenges for the timeline service v.2 as we have a > dependency on hbase which depends on an older version of hadoop. > We need to find a way to solve/contain/manage these risks before it is too > late. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5071) address HBase compatibility issues with trunk
[ https://issues.apache.org/jira/browse/YARN-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15317325#comment-15317325 ] Sangjin Lee commented on YARN-5071: --- After the HBase upgrade (YARN-5070) and the last rebase, we will assess whether we still have an issue. If not, we can either close this issue for now or at least remove the merge blocker label. > address HBase compatibility issues with trunk > - > > Key: YARN-5071 > URL: https://issues.apache.org/jira/browse/YARN-5071 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Sangjin Lee >Priority: Critical > Labels: yarn-2928-1st-milestone > > The trunk is now adding or planning to add more and more > backward-incompatible changes. Some examples include > - remove v.1 metrics classes (HADOOP-12504) > - update jersey version (HADOOP-9613) > - target java 8 by default (HADOOP-11858) > This poses big challenges for the timeline service v.2 as we have a > dependency on hbase which depends on an older version of hadoop. > We need to find a way to solve/contain/manage these risks before it is too > late. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5071) address HBase compatibility issues with trunk
[ https://issues.apache.org/jira/browse/YARN-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297501#comment-15297501 ] Sangjin Lee commented on YARN-5071: --- I tried rebasing our branch with the current trunk, and I was able to build it cleanly and all timeline service unit tests including the hbase ones pass FWIW. Thus, it appears we are still doing OK with the current setup. FYI. > address HBase compatibility issues with trunk > - > > Key: YARN-5071 > URL: https://issues.apache.org/jira/browse/YARN-5071 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Sangjin Lee >Priority: Critical > Labels: yarn-2928-1st-milestone > > The trunk is now adding or planning to add more and more > backward-incompatible changes. Some examples include > - remove v.1 metrics classes (HADOOP-12504) > - update jersey version (HADOOP-9613) > - target java 8 by default (HADOOP-11858) > This poses big challenges for the timeline service v.2 as we have a > dependency on hbase which depends on an older version of hadoop. > We need to find a way to solve/contain/manage these risks before it is too > late. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5071) address HBase compatibility issues with trunk
[ https://issues.apache.org/jira/browse/YARN-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278780#comment-15278780 ] Sangjin Lee commented on YARN-5071: --- Filed HBASE-15812. > address HBase compatibility issues with trunk > - > > Key: YARN-5071 > URL: https://issues.apache.org/jira/browse/YARN-5071 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Sangjin Lee >Priority: Critical > > The trunk is now adding or planning to add more and more > backward-incompatible changes. Some examples include > - remove v.1 metrics classes (HADOOP-12504) > - update jersey version (HADOOP-9613) > - target java 8 by default (HADOOP-11858) > This poses big challenges for the timeline service v.2 as we have a > dependency on hbase which depends on an older version of hadoop. > We need to find a way to solve/contain/manage these risks before it is too > late. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5071) address HBase compatibility issues with trunk
[ https://issues.apache.org/jira/browse/YARN-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278773#comment-15278773 ] Sangjin Lee commented on YARN-5071: --- No. Let me open one. > address HBase compatibility issues with trunk > - > > Key: YARN-5071 > URL: https://issues.apache.org/jira/browse/YARN-5071 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Sangjin Lee >Priority: Critical > > The trunk is now adding or planning to add more and more > backward-incompatible changes. Some examples include > - remove v.1 metrics classes (HADOOP-12504) > - update jersey version (HADOOP-9613) > - target java 8 by default (HADOOP-11858) > This poses big challenges for the timeline service v.2 as we have a > dependency on hbase which depends on an older version of hadoop. > We need to find a way to solve/contain/manage these risks before it is too > late. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5071) address HBase compatibility issues with trunk
[ https://issues.apache.org/jira/browse/YARN-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278772#comment-15278772 ] Sangjin Lee commented on YARN-5071: --- There may be more incompatible changes coming along in the trunk in the future. The motivation for this JIRA is to see what we can do to safeguard ourselves on our end, given the special dependency on HBase. Also, it would be GREAT if some of the changes you can do to make HBase work against the trunk can be made into the 1.2.x line. Thanks! > address HBase compatibility issues with trunk > - > > Key: YARN-5071 > URL: https://issues.apache.org/jira/browse/YARN-5071 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Sangjin Lee >Priority: Critical > > The trunk is now adding or planning to add more and more > backward-incompatible changes. Some examples include > - remove v.1 metrics classes (HADOOP-12504) > - update jersey version (HADOOP-9613) > - target java 8 by default (HADOOP-11858) > This poses big challenges for the timeline service v.2 as we have a > dependency on hbase which depends on an older version of hadoop. > We need to find a way to solve/contain/manage these risks before it is too > late. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5071) address HBase compatibility issues with trunk
[ https://issues.apache.org/jira/browse/YARN-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278770#comment-15278770 ] stack commented on YARN-5071: - Should be an easy fix. Is there an hbase issue to address? > address HBase compatibility issues with trunk > - > > Key: YARN-5071 > URL: https://issues.apache.org/jira/browse/YARN-5071 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Sangjin Lee >Priority: Critical > > The trunk is now adding or planning to add more and more > backward-incompatible changes. Some examples include > - remove v.1 metrics classes (HADOOP-12504) > - update jersey version (HADOOP-9613) > - target java 8 by default (HADOOP-11858) > This poses big challenges for the timeline service v.2 as we have a > dependency on hbase which depends on an older version of hadoop. > We need to find a way to solve/contain/manage these risks before it is too > late. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5071) address HBase compatibility issues with trunk
[
https://issues.apache.org/jira/browse/YARN-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278763#comment-15278763
]
Sangjin Lee commented on YARN-5071:
---
The main issue we saw was that in the trunk all metrics v.1 classes have been
removed as part of HADOOP-12504. Specifically it removed
{{org.apache.hadoop.metrics.MetricsServlet}}. HBase's HttpServer depends on
this servlet
(https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java#L679).
> address HBase compatibility issues with trunk
> -
>
> Key: YARN-5071
> URL: https://issues.apache.org/jira/browse/YARN-5071
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
>Priority: Critical
>
> The trunk is now adding or planning to add more and more
> backward-incompatible changes. Some examples include
> - remove v.1 metrics classes (HADOOP-12504)
> - update jersey version (HADOOP-9613)
> - target java 8 by default (HADOOP-11858)
> This poses big challenges for the timeline service v.2 as we have a
> dependency on hbase which depends on an older version of hadoop.
> We need to find a way to solve/contain/manage these risks before it is too
> late.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5071) address HBase compatibility issues with trunk
[ https://issues.apache.org/jira/browse/YARN-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278704#comment-15278704 ] stack commented on YARN-5071: - bq. what do you think for HBase's supporting trunk code? We will have too sooner or later. The three items listed here seem fine. We don't do metrics v1. Jersey could be tricky but we'll have to deal -- shade ours or just update. HBase 2.0 will be jdk8. branch-1 is tested against jdk8. What else you seeing? > address HBase compatibility issues with trunk > - > > Key: YARN-5071 > URL: https://issues.apache.org/jira/browse/YARN-5071 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Sangjin Lee >Priority: Critical > > The trunk is now adding or planning to add more and more > backward-incompatible changes. Some examples include > - remove v.1 metrics classes (HADOOP-12504) > - update jersey version (HADOOP-9613) > - target java 8 by default (HADOOP-11858) > This poses big challenges for the timeline service v.2 as we have a > dependency on hbase which depends on an older version of hadoop. > We need to find a way to solve/contain/manage these risks before it is too > late. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5071) address HBase compatibility issues with trunk
[
https://issues.apache.org/jira/browse/YARN-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278663#comment-15278663
]
Tsuyoshi Ozawa commented on YARN-5071:
--
{quote}
In principle, I don't think this is really a HBase problem at the moment as
3.0.0 has not been released yet.
{quote}
I agree with you. I meant that we should focus on how Hadoop ecosystem,
including HBase, can migrate branch-2 based code to trunk easily and smoothly.
I think we should get feedback from users of Hadoop, including HBase
developers, to avoid critical problems. In other words, this is a good time to
recheck Hadoop Compatibility guide.
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html
> address HBase compatibility issues with trunk
> -
>
> Key: YARN-5071
> URL: https://issues.apache.org/jira/browse/YARN-5071
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Affects Versions: YARN-2928
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
>Priority: Critical
>
> The trunk is now adding or planning to add more and more
> backward-incompatible changes. Some examples include
> - remove v.1 metrics classes (HADOOP-12504)
> - update jersey version (HADOOP-9613)
> - target java 8 by default (HADOOP-11858)
> This poses big challenges for the timeline service v.2 as we have a
> dependency on hbase which depends on an older version of hadoop.
> We need to find a way to solve/contain/manage these risks before it is too
> late.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5071) address HBase compatibility issues with trunk
[ https://issues.apache.org/jira/browse/YARN-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278640#comment-15278640 ] Sangjin Lee commented on YARN-5071: --- Regarding #2, I don't think java 8 would be an issue, but things like updated dependencies or changed hadoop-common APIs would be a problem. In principle, I don't think this is really a HBase problem at the moment as 3.0.0 has not been released yet. > address HBase compatibility issues with trunk > - > > Key: YARN-5071 > URL: https://issues.apache.org/jira/browse/YARN-5071 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Sangjin Lee >Priority: Critical > > The trunk is now adding or planning to add more and more > backward-incompatible changes. Some examples include > - remove v.1 metrics classes (HADOOP-12504) > - update jersey version (HADOOP-9613) > - target java 8 by default (HADOOP-11858) > This poses big challenges for the timeline service v.2 as we have a > dependency on hbase which depends on an older version of hadoop. > We need to find a way to solve/contain/manage these risks before it is too > late. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5071) address HBase compatibility issues with trunk
[ https://issues.apache.org/jira/browse/YARN-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278626#comment-15278626 ] Tsuyoshi Ozawa commented on YARN-5071: -- I think we should have a discussion with HBase guys. [~stack] [~iwasakims] what do you think for HBase's supporting trunk code? Can we have a help or work to do so at Hadoop side? We'd like to know the barrier and problems of running hbase client under the trunk environment. > address HBase compatibility issues with trunk > - > > Key: YARN-5071 > URL: https://issues.apache.org/jira/browse/YARN-5071 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Sangjin Lee >Priority: Critical > > The trunk is now adding or planning to add more and more > backward-incompatible changes. Some examples include > - remove v.1 metrics classes (HADOOP-12504) > - update jersey version (HADOOP-9613) > - target java 8 by default (HADOOP-11858) > This poses big challenges for the timeline service v.2 as we have a > dependency on hbase which depends on an older version of hadoop. > We need to find a way to solve/contain/manage these risks before it is too > late. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-5071) address HBase compatibility issues with trunk
[ https://issues.apache.org/jira/browse/YARN-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278587#comment-15278587 ] Sangjin Lee commented on YARN-5071: --- We have 2 areas of concern: # running our timeline service server side code under an older environment (coprocessor, schema creator, and the unit tests) # running hbase client under the trunk environment Regarding #1, we have made some effort to isolate at least the unit tests (YARN-5045), but there is code for coprocessor and the schema creator. We probably need to isolate them (and its dependencies) so that they are managed against the old version of hadoop. While issue #2 might not arise that often (hbase client mostly depends only on hadoop-common), it is also harder to fix once it happens. Any thoughts on how we can handle this would be welcome. > address HBase compatibility issues with trunk > - > > Key: YARN-5071 > URL: https://issues.apache.org/jira/browse/YARN-5071 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Affects Versions: YARN-2928 >Reporter: Sangjin Lee >Assignee: Sangjin Lee >Priority: Critical > > The trunk is now adding or planning to add more and more > backward-incompatible changes. Some examples include > - remove v.1 metrics classes (HADOOP-12504) > - update jersey version (HADOOP-9613) > - target java 8 by default (HADOOP-11858) > This poses big challenges for the timeline service v.2 as we have a > dependency on hbase which depends on an older version of hadoop. > We need to find a way to solve/contain/manage these risks before it is too > late. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
