[jira] [Updated] (TEZ-4101) Eliminate some guava dependencies by Java8+ features - Preconditions
[ https://issues.apache.org/jira/browse/TEZ-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated TEZ-4101: -- Attachment: TEZ-4101.02.patch > Eliminate some guava dependencies by Java8+ features - Preconditions > > > Key: TEZ-4101 > URL: https://issues.apache.org/jira/browse/TEZ-4101 > Project: Apache Tez > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: TEZ-4101.01.patch, TEZ-4101.02.patch > > > 1. Preconditions.checkNotNull -> Objects.requireNonNull > 2. wrap others into a tez common class -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (TEZ-4114) Upgrade to Jetty 9.4
[ https://issues.apache.org/jira/browse/TEZ-4114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated TEZ-4114: -- Description: Hadoop already did it in: https://issues.apache.org/jira/browse/HADOOP-16152 Hive: "in progress", only abandoned jiras https://issues.apache.org/jira/browse/HIVE-21961 https://issues.apache.org/jira/browse/HIVE-21211 was: Hadoop already did it in: https://issues.apache.org/jira/browse/HADOOP-16152 Hive: "in progress" this one seems abandoned: https://issues.apache.org/jira/browse/HIVE-21961 > Upgrade to Jetty 9.4 > > > Key: TEZ-4114 > URL: https://issues.apache.org/jira/browse/TEZ-4114 > Project: Apache Tez > Issue Type: Bug >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: TEZ-4114.01.patch > > > Hadoop already did it in: https://issues.apache.org/jira/browse/HADOOP-16152 > Hive: "in progress", only abandoned jiras > https://issues.apache.org/jira/browse/HIVE-21961 > https://issues.apache.org/jira/browse/HIVE-21211 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TEZ-4101) Eliminate some guava dependencies by Java8+ features - Preconditions
[ https://issues.apache.org/jira/browse/TEZ-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17020232#comment-17020232 ] László Bodor commented on TEZ-4101: --- [~jeagles]: what do you think about this kind of separation of Preconditions? I feel the pain of handling guava versions in tez (+tez on hadoop), but at the same time I don't see the benefit of them...having multiple tez tickets which are blocked on guava (especially Preconditions), just because Precondition can throw a formatted exception if a boolean is evaluated to false... maybe I can even "copy" the implementation instead of wrapping the calls to guava (it might be something similar what you did with Stopwatch in TEZ-3256) > Eliminate some guava dependencies by Java8+ features - Preconditions > > > Key: TEZ-4101 > URL: https://issues.apache.org/jira/browse/TEZ-4101 > Project: Apache Tez > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: TEZ-4101.01.patch, TEZ-4101.02.patch > > > 1. Preconditions.checkNotNull -> Objects.requireNonNull > 2. wrap others into a tez common class -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (TEZ-4101) Eliminate some guava dependencies by Java8+ features - Preconditions
[ https://issues.apache.org/jira/browse/TEZ-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17020232#comment-17020232 ] László Bodor edited comment on TEZ-4101 at 1/21/20 1:30 PM: [~jeagles]: what do you think about this kind of separation of Preconditions? I feel the pain of handling guava versions in tez (+tez on hadoop), but at the same time I don't see the benefit of them...having multiple tez tickets which are blocked on guava (especially Preconditions), just because Precondition can throw a formatted exception if a boolean is evaluated to false... maybe I can even "copy" the implementation instead of wrapping the calls to guava (it might be something similar what you did with Stopwatch in TEZ-3256) without Preconditions, I see more chance to proceed with hadoop upgrades (e.g. TEZ-4100) was (Author: abstractdog): [~jeagles]: what do you think about this kind of separation of Preconditions? I feel the pain of handling guava versions in tez (+tez on hadoop), but at the same time I don't see the benefit of them...having multiple tez tickets which are blocked on guava (especially Preconditions), just because Precondition can throw a formatted exception if a boolean is evaluated to false... maybe I can even "copy" the implementation instead of wrapping the calls to guava (it might be something similar what you did with Stopwatch in TEZ-3256) > Eliminate some guava dependencies by Java8+ features - Preconditions > > > Key: TEZ-4101 > URL: https://issues.apache.org/jira/browse/TEZ-4101 > Project: Apache Tez > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Attachments: TEZ-4101.01.patch, TEZ-4101.02.patch > > > 1. Preconditions.checkNotNull -> Objects.requireNonNull > 2. wrap others into a tez common class -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (TEZ-4113) TezUtils::createByteStringFromConf should use snappy instead of DeflaterOutputStream
[
https://issues.apache.org/jira/browse/TEZ-4113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17020493#comment-17020493
]
Ashutosh Chauhan commented on TEZ-4113:
---
+1
> TezUtils::createByteStringFromConf should use snappy instead of
> DeflaterOutputStream
>
>
> Key: TEZ-4113
> URL: https://issues.apache.org/jira/browse/TEZ-4113
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Trivial
> Attachments: Screenshot 2020-01-10 at 6.32.31 AM.png, TEZ-4113.1.patch
>
>
> Under concurrent workload, where lots of short running DAGs were submitted in
> Hive, HS2 spikes up heavily on CPU due to
> {{TezUtils::createByteStringFromConf}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (TEZ-3391) MR split file validation should be done in the AM
[ https://issues.apache.org/jira/browse/TEZ-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein reassigned TEZ-3391: -- Assignee: Ahmed Hussein (was: Nishant Dash) > MR split file validation should be done in the AM > - > > Key: TEZ-3391 > URL: https://issues.apache.org/jira/browse/TEZ-3391 > Project: Apache Tez > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Ahmed Hussein >Priority: Major > > We had a case where Split metadata size exceeded 1000. Instead of job > failing from validation during initialization in AM like mapreduce, each of > the tasks failed doing that validation during initialization. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (TEZ-3391) MR split file validation should be done in the AM
[ https://issues.apache.org/jira/browse/TEZ-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein updated TEZ-3391: --- Attachment: TEZ-3391.001.patch > MR split file validation should be done in the AM > - > > Key: TEZ-3391 > URL: https://issues.apache.org/jira/browse/TEZ-3391 > Project: Apache Tez > Issue Type: Bug >Reporter: Rohini Palaniswamy >Assignee: Ahmed Hussein >Priority: Major > Attachments: TEZ-3391.001.patch > > > We had a case where Split metadata size exceeded 1000. Instead of job > failing from validation during initialization in AM like mapreduce, each of > the tasks failed doing that validation during initialization. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
