[jira] [Commented] (HIVE-9574) Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14309648#comment-14309648
]
Jimmy Xiang commented on HIVE-9574:
---
Patch 1 is on RB: https://reviews.apache.org/r/30739/
> Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark
> Branch]
>
>
> Key: HIVE-9574
> URL: https://issues.apache.org/jira/browse/HIVE-9574
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Rui Li
>Assignee: Jimmy Xiang
> Fix For: spark-branch
>
> Attachments: HIVE-9574.1-spark.patch
>
>
> {{RowContainer.first}} may call {{InputFormat.getSplits}}, which is
> expensive. If we switch {{container}} and {{backupContainer}} frequently in
> {{HiveKVResultCache}}, it will downgrade performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9574) Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14309835#comment-14309835
]
Hive QA commented on HIVE-9574:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12697083/HIVE-9574.1-spark.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 7469 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_join_with_different_encryption_keys
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/713/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/713/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-713/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12697083 - PreCommit-HIVE-SPARK-Build
> Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark
> Branch]
>
>
> Key: HIVE-9574
> URL: https://issues.apache.org/jira/browse/HIVE-9574
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Rui Li
>Assignee: Jimmy Xiang
> Fix For: spark-branch
>
> Attachments: HIVE-9574.1-spark.patch
>
>
> {{RowContainer.first}} may call {{InputFormat.getSplits}}, which is
> expensive. If we switch {{container}} and {{backupContainer}} frequently in
> {{HiveKVResultCache}}, it will downgrade performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9574) Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310307#comment-14310307
]
Hive QA commented on HIVE-9574:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12697155/HIVE-9574.2-spark.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 7469 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_join_with_different_encryption_keys
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/714/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/714/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-714/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12697155 - PreCommit-HIVE-SPARK-Build
> Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark
> Branch]
>
>
> Key: HIVE-9574
> URL: https://issues.apache.org/jira/browse/HIVE-9574
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Rui Li
>Assignee: Jimmy Xiang
> Fix For: spark-branch
>
> Attachments: HIVE-9574.1-spark.patch, HIVE-9574.2-spark.patch,
> HIVE-9574.3-spark.patch
>
>
> {{RowContainer.first}} may call {{InputFormat.getSplits}}, which is
> expensive. If we switch {{container}} and {{backupContainer}} frequently in
> {{HiveKVResultCache}}, it will downgrade performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9574) Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310432#comment-14310432
]
Hive QA commented on HIVE-9574:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12697181/HIVE-9574.3-spark.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 7469 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_join_with_different_encryption_keys
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/715/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/715/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-715/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12697181 - PreCommit-HIVE-SPARK-Build
> Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark
> Branch]
>
>
> Key: HIVE-9574
> URL: https://issues.apache.org/jira/browse/HIVE-9574
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Rui Li
>Assignee: Jimmy Xiang
> Fix For: spark-branch
>
> Attachments: HIVE-9574.1-spark.patch, HIVE-9574.2-spark.patch,
> HIVE-9574.3-spark.patch
>
>
> {{RowContainer.first}} may call {{InputFormat.getSplits}}, which is
> expensive. If we switch {{container}} and {{backupContainer}} frequently in
> {{HiveKVResultCache}}, it will downgrade performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9574) Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310560#comment-14310560
]
Hive QA commented on HIVE-9574:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12697218/HIVE-9574.4-spark.patch
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 7467 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_join_with_different_encryption_keys
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_transform_ppr1
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/716/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/716/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-716/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12697218 - PreCommit-HIVE-SPARK-Build
> Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark
> Branch]
>
>
> Key: HIVE-9574
> URL: https://issues.apache.org/jira/browse/HIVE-9574
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Rui Li
>Assignee: Jimmy Xiang
> Fix For: spark-branch
>
> Attachments: HIVE-9574.1-spark.patch, HIVE-9574.2-spark.patch,
> HIVE-9574.3-spark.patch, HIVE-9574.4-spark.patch
>
>
> {{RowContainer.first}} may call {{InputFormat.getSplits}}, which is
> expensive. If we switch {{container}} and {{backupContainer}} frequently in
> {{HiveKVResultCache}}, it will downgrade performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9574) Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312898#comment-14312898
]
Hive QA commented on HIVE-9574:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12697551/HIVE-9574.5-spark.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 7471 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_join_with_different_encryption_keys
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_cbo_gby_empty
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/718/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/718/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-718/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12697551 - PreCommit-HIVE-SPARK-Build
> Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark
> Branch]
>
>
> Key: HIVE-9574
> URL: https://issues.apache.org/jira/browse/HIVE-9574
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Rui Li
>Assignee: Jimmy Xiang
> Fix For: spark-branch
>
> Attachments: HIVE-9574.1-spark.patch, HIVE-9574.2-spark.patch,
> HIVE-9574.3-spark.patch, HIVE-9574.4-spark.patch, HIVE-9574.5-spark.patch
>
>
> {{RowContainer.first}} may call {{InputFormat.getSplits}}, which is
> expensive. If we switch {{container}} and {{backupContainer}} frequently in
> {{HiveKVResultCache}}, it will downgrade performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9574) Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14312987#comment-14312987
]
Chao commented on HIVE-9574:
Regarding cbo_gby_empty.q, I found this error in hive.log:
{noformat}
...
NestedThrowablesStackTrace:
Method/Identifier expected at character 37 in "tableName == t1 && dbName == t2
&& ()"
org.datanucleus.store.query.QueryCompilerSyntaxException: Method/Identifier
expected at character 37 in "tableName == t1 && dbName == t2 && ()"
at
org.datanucleus.query.compiler.JDOQLParser.processPrimary(JDOQLParser.java:810)
at
org.datanucleus.query.compiler.JDOQLParser.processUnaryExpression(JDOQLParser.java:656)
at
org.datanucleus.query.compiler.JDOQLParser.processMultiplicativeExpression(JDOQLParser.java:582)
at
org.datanucleus.query.compiler.JDOQLParser.processAdditiveExpression(JDOQLParser.java:553)
at
org.datanucleus.query.compiler.JDOQLParser.processRelationalExpression(JDOQLParser.java:467)
at
org.datanucleus.query.compiler.JDOQLParser.processAndExpression(JDOQLParser.java:450)
at
org.datanucleus.query.compiler.JDOQLParser.processExclusiveOrExpression(JDOQLParser.java:436)
at
org.datanucleus.query.compiler.JDOQLParser.processInclusiveOrExpression(JDOQLParser.java:422)
at
org.datanucleus.query.compiler.JDOQLParser.processConditionalAndExpression(JDOQLParser.java:408)
at
org.datanucleus.query.compiler.JDOQLParser.processConditionalOrExpression(JDOQLParser.java:389)
at
org.datanucleus.query.compiler.JDOQLParser.processExpression(JDOQLParser.java:378)
at
org.datanucleus.query.compiler.JDOQLParser.processPrimary(JDOQLParser.java:785)
at
org.datanucleus.query.compiler.JDOQLParser.processUnaryExpression(JDOQLParser.java:656)
at
org.datanucleus.query.compiler.JDOQLParser.processMultiplicativeExpression(JDOQLParser.java:582)
at
org.datanucleus.query.compiler.JDOQLParser.processAdditiveExpression(JDOQLParser.java:553)
at
org.datanucleus.query.compiler.JDOQLParser.processRelationalExpression(JDOQLParser.java:467)
at
org.datanucleus.query.compiler.JDOQLParser.processAndExpression(JDOQLParser.java:450)
at
org.datanucleus.query.compiler.JDOQLParser.processExclusiveOrExpression(JDOQLParser.java:436)
at
org.datanucleus.query.compiler.JDOQLParser.processInclusiveOrExpression(JDOQLParser.java:422)
at
org.datanucleus.query.compiler.JDOQLParser.processConditionalAndExpression(JDOQLParser.java:412)
at
org.datanucleus.query.compiler.JDOQLParser.processConditionalOrExpression(JDOQLParser.java:389)
at
org.datanucleus.query.compiler.JDOQLParser.processExpression(JDOQLParser.java:378)
at org.datanucleus.query.compiler.JDOQLParser.parse(JDOQLParser.java:99)
at
org.datanucleus.query.compiler.JavaQueryCompiler.compileFilter(JavaQueryCompiler.java:467)
at
org.datanucleus.query.compiler.JDOQLCompiler.compile(JDOQLCompiler.java:113)
at
org.datanucleus.store.query.AbstractJDOQLQuery.compileInternal(AbstractJDOQLQuery.java:367)
at
org.datanucleus.store.rdbms.query.JDOQLQuery.compileInternal(JDOQLQuery.java:240)
at org.datanucleus.store.query.Query.executeQuery(Query.java:1744)
at org.datanucleus.store.query.Query.executeWithArray(Query.java:1672)
at org.datanucleus.api.jdo.JDOQuery.executeWithArray(JDOQuery.java:312)
at
org.apache.hadoop.hive.metastore.ObjectStore.getMTableColumnStatistics(ObjectStore.java:5969)
at
org.apache.hadoop.hive.metastore.ObjectStore.access$1000(ObjectStore.java:159)
at
org.apache.hadoop.hive.metastore.ObjectStore$6.getJdoResult(ObjectStore.java:6025)
at
org.apache.hadoop.hive.metastore.ObjectStore$6.getJdoResult(ObjectStore.java:6017)
{noformat}
I'm not sure what this mean.
> Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark
> Branch]
>
>
> Key: HIVE-9574
> URL: https://issues.apache.org/jira/browse/HIVE-9574
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Rui Li
>Assignee: Jimmy Xiang
> Fix For: spark-branch
>
> Attachments: HIVE-9574.1-spark.patch, HIVE-9574.2-spark.patch,
> HIVE-9574.3-spark.patch, HIVE-9574.4-spark.patch, HIVE-9574.5-spark.patch
>
>
> {{RowContainer.first}} may call {{InputFormat.getSplits}}, which is
> expensive. If we switch {{container}} and {{backupContainer}} frequently in
> {{HiveKVResultCache}}, it will downgrade performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9574) Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14313671#comment-14313671
]
Rui Li commented on HIVE-9574:
--
Just verified my test passed with the v5 patch.
> Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark
> Branch]
>
>
> Key: HIVE-9574
> URL: https://issues.apache.org/jira/browse/HIVE-9574
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Rui Li
>Assignee: Jimmy Xiang
> Fix For: spark-branch
>
> Attachments: HIVE-9574.1-spark.patch, HIVE-9574.2-spark.patch,
> HIVE-9574.3-spark.patch, HIVE-9574.4-spark.patch, HIVE-9574.5-spark.patch
>
>
> {{RowContainer.first}} may call {{InputFormat.getSplits}}, which is
> expensive. If we switch {{container}} and {{backupContainer}} frequently in
> {{HiveKVResultCache}}, it will downgrade performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9574) Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314491#comment-14314491
]
Jimmy Xiang commented on HIVE-9574:
---
Cool, thanks. Attached v6 that addressed more minor review comments.
> Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark
> Branch]
>
>
> Key: HIVE-9574
> URL: https://issues.apache.org/jira/browse/HIVE-9574
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Rui Li
>Assignee: Jimmy Xiang
> Fix For: spark-branch
>
> Attachments: HIVE-9574.1-spark.patch, HIVE-9574.2-spark.patch,
> HIVE-9574.3-spark.patch, HIVE-9574.4-spark.patch, HIVE-9574.5-spark.patch,
> HIVE-9574.6-spark.patch
>
>
> {{RowContainer.first}} may call {{InputFormat.getSplits}}, which is
> expensive. If we switch {{container}} and {{backupContainer}} frequently in
> {{HiveKVResultCache}}, it will downgrade performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9574) Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314697#comment-14314697
]
Hive QA commented on HIVE-9574:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12697814/HIVE-9574.6-spark.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 7471 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_index_auto_mult_tables
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udaf_percentile_approx_23
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_join_with_different_encryption_keys
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_cbo_gby_empty
org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchEmptyCommit
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/720/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/720/console
Test logs:
http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-720/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12697814 - PreCommit-HIVE-SPARK-Build
> Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark
> Branch]
>
>
> Key: HIVE-9574
> URL: https://issues.apache.org/jira/browse/HIVE-9574
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Rui Li
>Assignee: Jimmy Xiang
> Fix For: spark-branch
>
> Attachments: HIVE-9574.1-spark.patch, HIVE-9574.2-spark.patch,
> HIVE-9574.3-spark.patch, HIVE-9574.4-spark.patch, HIVE-9574.5-spark.patch,
> HIVE-9574.6-spark.patch
>
>
> {{RowContainer.first}} may call {{InputFormat.getSplits}}, which is
> expensive. If we switch {{container}} and {{backupContainer}} frequently in
> {{HiveKVResultCache}}, it will downgrade performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9574) Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314722#comment-14314722
]
Jimmy Xiang commented on HIVE-9574:
---
Test index_auto_mult_tables is ok for me on my box.
> Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark
> Branch]
>
>
> Key: HIVE-9574
> URL: https://issues.apache.org/jira/browse/HIVE-9574
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Rui Li
>Assignee: Jimmy Xiang
> Fix For: spark-branch
>
> Attachments: HIVE-9574.1-spark.patch, HIVE-9574.2-spark.patch,
> HIVE-9574.3-spark.patch, HIVE-9574.4-spark.patch, HIVE-9574.5-spark.patch,
> HIVE-9574.6-spark.patch
>
>
> {{RowContainer.first}} may call {{InputFormat.getSplits}}, which is
> expensive. If we switch {{container}} and {{backupContainer}} frequently in
> {{HiveKVResultCache}}, it will downgrade performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9574) Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314742#comment-14314742
]
Xuefu Zhang commented on HIVE-9574:
---
+1
> Lazy computing in HiveBaseFunctionResultList may hurt performance [Spark
> Branch]
>
>
> Key: HIVE-9574
> URL: https://issues.apache.org/jira/browse/HIVE-9574
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
>Reporter: Rui Li
>Assignee: Jimmy Xiang
> Fix For: spark-branch
>
> Attachments: HIVE-9574.1-spark.patch, HIVE-9574.2-spark.patch,
> HIVE-9574.3-spark.patch, HIVE-9574.4-spark.patch, HIVE-9574.5-spark.patch,
> HIVE-9574.6-spark.patch
>
>
> {{RowContainer.first}} may call {{InputFormat.getSplits}}, which is
> expensive. If we switch {{container}} and {{backupContainer}} frequently in
> {{HiveKVResultCache}}, it will downgrade performance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
