[jira] [Commented] (HIVE-21694) Hive driver wait time is fixed for task getting executed in parallel.
[ https://issues.apache.org/jira/browse/HIVE-21694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834973#comment-16834973 ] Sankar Hariappan commented on HIVE-21694: - +1 > Hive driver wait time is fixed for task getting executed in parallel. > - > > Key: HIVE-21694 > URL: https://issues.apache.org/jira/browse/HIVE-21694 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21694.01.patch, HIVE-21694.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > During a command execution hive driver executes the task in a separate thread > if the task to be executed is set as parallel. After starting the task, > driver checks if the task has finished execution or not. If the task > execution is not finished it waits for 2 seconds before waking up again to > check the task status. In case of task with execution time in milliseconds, > this wait time can induce substantial overhead. So the execution thread can > call notify to interrupt the wait call. This will make sure that the next > thread execution can start immediately. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21694) Hive driver wait time is fixed for task getting executed in parallel.
[
https://issues.apache.org/jira/browse/HIVE-21694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834789#comment-16834789
]
Hive QA commented on HIVE-21694:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12968048/HIVE-21694.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15982 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17153/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17153/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17153/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12968048 - PreCommit-HIVE-Build
> Hive driver wait time is fixed for task getting executed in parallel.
> -
>
> Key: HIVE-21694
> URL: https://issues.apache.org/jira/browse/HIVE-21694
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21694.01.patch, HIVE-21694.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> During a command execution hive driver executes the task in a separate thread
> if the task to be executed is set as parallel. After starting the task,
> driver checks if the task has finished execution or not. If the task
> execution is not finished it waits for 2 seconds before waking up again to
> check the task status. In case of task with execution time in milliseconds,
> this wait time can induce substantial overhead. So instead of fixed wait
> time, exponential backedup sleep time can be used to reduce the sleep
> overhead. The sleep time can start with 100ms and can increase up to 2
> seconds doubling on each iteration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21694) Hive driver wait time is fixed for task getting executed in parallel.
[
https://issues.apache.org/jira/browse/HIVE-21694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834750#comment-16834750
]
Hive QA commented on HIVE-21694:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
14s{color} | {color:red} ql generated 1 new + 2253 unchanged - 0 fixed = 2254
total (was 2253) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 58s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Naked notify in org.apache.hadoop.hive.ql.DriverContext.releaseRunnable()
At DriverContext.java:At DriverContext.java:[line 104] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17153/dev-support/hive-personality.sh
|
| git revision | master / ff98efa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17153/yetus/new-findbugs-ql.html
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17153/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Hive driver wait time is fixed for task getting executed in parallel.
> -
>
> Key: HIVE-21694
> URL: https://issues.apache.org/jira/browse/HIVE-21694
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21694.01.patch, HIVE-21694.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> During a command execution hive driver executes the task in a separate thread
> if the task to be executed is set as parallel. After starting the task,
> driver checks if the task has finished execution or not. If the task
[jira] [Commented] (HIVE-21694) Hive driver wait time is fixed for task getting executed in parallel.
[
https://issues.apache.org/jira/browse/HIVE-21694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834618#comment-16834618
]
Hive QA commented on HIVE-21694:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12968016/HIVE-21694.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15920 tests
executed
*Failed tests:*
{noformat}
TestCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=28)
[acid_vectorization_missing_cols.q,cte_7.q,cli_print_escape_crlf.q,alter_partition_coltype.q,orc_diff_part_cols2.q,cbo_rp_auto_join17.q,temp_table_truncate.q,float_equality.q,masking_6.q,skewjoinopt4.q,count.q,schema_evol_orc_vec_part_all_primitive_llap_io.q,parallel.q,alter_partition_change_col.q,groupby_grouping_sets2.q,truncate_column_buckets.q,infer_bucket_sort_list_bucket.q,vectorization_10.q,dbtxnmgr_query5.q,interval_udf.q,udaf_ngrams.q,spark_dynamic_partition_pruning.q,partition_condition_remover.q,ppd2.q,hybridgrace_hashjoin_2.q,vector_varchar_mapjoin1.q,llap_nullscan.q,vectorized_insert_into_bucketed_table.q,list_bucket_dml_13.q,vector_bucket.q]
TestCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=37)
[constantPropWhen.q,druid_timestamptz2.q,udf_floor.q,join20.q,authorization_view_3.q,cross_prod_4.q,orc_ppd_schema_evol_1b.q,exec_parallel_column_stats.q,nested_column_pruning.q,merge_dynamic_partition4.q,cte_3.q,extrapolate_part_stats_full.q,udaf_percentile_approx_23.q,nullability_transitive_inference.q,vector_ptf_part_simple.q,orc_merge6.q,vector_non_string_partition.q,udf_case.q,columnStatsUpdateForStatsOptimizer_1.q,parquet_array_map_emptynullvals.q,udf_length.q,orc_ppd_exception.q,sharedworkext.q,groupby_bigdata.q,concatenate_inherit_table_location.q,parallel_colstats.q,timestamp_4.q,udf7.q,innerjoin.q,parquet_vectorization_pushdown.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=124)
[bucketsortoptimize_insert_4.q,skewjoinopt8.q,vectorization_10.q,auto_join18_multi_distinct.q,custom_input_output_format.q,insert_into3.q,vectorization_6.q,skewjoinopt4.q,count.q,stats9.q,auto_join7.q,parallel.q,timestamp_1.q,union33.q,union_lateralview.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17151/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17151/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17151/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12968016 - PreCommit-HIVE-Build
> Hive driver wait time is fixed for task getting executed in parallel.
> -
>
> Key: HIVE-21694
> URL: https://issues.apache.org/jira/browse/HIVE-21694
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21694.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> During a command execution hive driver executes the task in a separate thread
> if the task to be executed is set as parallel. After starting the task,
> driver checks if the task has finished execution or not. If the task
> execution is not finished it waits for 2 seconds before waking up again to
> check the task status. In case of task with execution time in milliseconds,
> this wait time can induce substantial overhead. So instead of fixed wait
> time, exponential backedup sleep time can be used to reduce the sleep
> overhead. The sleep time can start with 100ms and can increase up to 2
> seconds doubling on each iteration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21694) Hive driver wait time is fixed for task getting executed in parallel.
[
https://issues.apache.org/jira/browse/HIVE-21694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834580#comment-16834580
]
Hive QA commented on HIVE-21694:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
53s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
58s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17151/dev-support/hive-personality.sh
|
| git revision | master / ff98efa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17151/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Hive driver wait time is fixed for task getting executed in parallel.
> -
>
> Key: HIVE-21694
> URL: https://issues.apache.org/jira/browse/HIVE-21694
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21694.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> During a command execution hive driver executes the task in a separate thread
> if the task to be executed is set as parallel. After starting the task,
> driver checks if the task has finished execution or not. If the task
> execution is not finished it waits for 2 seconds before waking up again to
> check the task status. In case of task with execution time in milliseconds,
> this wait time can induce substantial overhead. So instead of fixed wait
> time, exponential backedup sleep time can be used to reduce the sleep
> overhead. The sleep time can start with 100ms
[jira] [Commented] (HIVE-21694) Hive driver wait time is fixed for task getting executed in parallel.
[
https://issues.apache.org/jira/browse/HIVE-21694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834400#comment-16834400
]
Hive QA commented on HIVE-21694:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967957/HIVE-21694.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15918 tests
executed
*Failed tests:*
{noformat}
TestCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=28)
[acid_vectorization_missing_cols.q,cte_7.q,cli_print_escape_crlf.q,alter_partition_coltype.q,orc_diff_part_cols2.q,cbo_rp_auto_join17.q,temp_table_truncate.q,float_equality.q,masking_6.q,skewjoinopt4.q,count.q,schema_evol_orc_vec_part_all_primitive_llap_io.q,parallel.q,alter_partition_change_col.q,groupby_grouping_sets2.q,truncate_column_buckets.q,infer_bucket_sort_list_bucket.q,vectorization_10.q,dbtxnmgr_query5.q,interval_udf.q,udaf_ngrams.q,spark_dynamic_partition_pruning.q,partition_condition_remover.q,ppd2.q,hybridgrace_hashjoin_2.q,vector_varchar_mapjoin1.q,llap_nullscan.q,vectorized_insert_into_bucketed_table.q,list_bucket_dml_13.q,vector_bucket.q]
TestCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=37)
[constantPropWhen.q,druid_timestamptz2.q,udf_floor.q,join20.q,authorization_view_3.q,cross_prod_4.q,orc_ppd_schema_evol_1b.q,exec_parallel_column_stats.q,nested_column_pruning.q,merge_dynamic_partition4.q,cte_3.q,extrapolate_part_stats_full.q,udaf_percentile_approx_23.q,nullability_transitive_inference.q,vector_ptf_part_simple.q,orc_merge6.q,vector_non_string_partition.q,udf_case.q,columnStatsUpdateForStatsOptimizer_1.q,parquet_array_map_emptynullvals.q,udf_length.q,orc_ppd_exception.q,sharedworkext.q,groupby_bigdata.q,concatenate_inherit_table_location.q,parallel_colstats.q,timestamp_4.q,udf7.q,innerjoin.q,parquet_vectorization_pushdown.q]
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=124)
[bucketsortoptimize_insert_4.q,skewjoinopt8.q,vectorization_10.q,auto_join18_multi_distinct.q,custom_input_output_format.q,insert_into3.q,vectorization_6.q,skewjoinopt4.q,count.q,stats9.q,auto_join7.q,parallel.q,timestamp_1.q,union33.q,union_lateralview.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17144/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17144/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17144/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967957 - PreCommit-HIVE-Build
> Hive driver wait time is fixed for task getting executed in parallel.
> -
>
> Key: HIVE-21694
> URL: https://issues.apache.org/jira/browse/HIVE-21694
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21694.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> During a command execution hive driver executes the task in a separate thread
> if the task to be executed is set as parallel. After starting the task,
> driver checks if the task has finished execution or not. If the task
> execution is not finished it waits for 2 seconds before waking up again to
> check the task status. In case of task with execution time in milliseconds,
> this wait time can induce substantial overhead. So instead of fixed wait
> time, exponential backedup sleep time can be used to reduce the sleep
> overhead. The sleep time can start with 100ms and can increase up to 2
> seconds doubling on each iteration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21694) Hive driver wait time is fixed for task getting executed in parallel.
[
https://issues.apache.org/jira/browse/HIVE-21694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16834379#comment-16834379
]
Hive QA commented on HIVE-21694:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17144/dev-support/hive-personality.sh
|
| git revision | master / c5e6608 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17144/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Hive driver wait time is fixed for task getting executed in parallel.
> -
>
> Key: HIVE-21694
> URL: https://issues.apache.org/jira/browse/HIVE-21694
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21694.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> During a command execution hive driver executes the task in a separate thread
> if the task to be executed is set as parallel. After starting the task,
> driver checks if the task has finished execution or not. If the task
> execution is not finished it waits for 2 seconds before waking up again to
> check the task status. In case of task with execution time in milliseconds,
> this wait time can induce substantial overhead. So instead of fixed wait
> time, exponential backedup sleep time can be used to reduce the sleep
> overhead. The sleep time can start with 100ms
[jira] [Commented] (HIVE-21694) Hive driver wait time is fixed for task getting executed in parallel.
[ https://issues.apache.org/jira/browse/HIVE-21694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16833550#comment-16833550 ] mahesh kumar behera commented on HIVE-21694: [~gopalv] No its not related to HIVE-21646. The fix is to reduce the execution time of commands having many small tasks. If the tasks are set to run in parallel, then the minimum execution time is 2 seconds per task because of the sleep. So the idea is to start the sleep at 50ms or so and keep increasing exponentially until it reaches 2 seconds. I think wait based on a counter is a better idea. Will check the fix done for HIVE-20989. > Hive driver wait time is fixed for task getting executed in parallel. > - > > Key: HIVE-21694 > URL: https://issues.apache.org/jira/browse/HIVE-21694 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 4.0.0 > > > During a command execution hive driver executes the task in a separate thread > if the task to be executed is set as parallel. After starting the task, > driver checks if the task has finished execution or not. If the task > execution is not finished it waits for 2 seconds before waking up again to > check the task status. In case of task with execution time in milliseconds, > this wait time can induce substantial overhead. So instead of fixed wait > time, exponential backedup sleep time can be used to reduce the sleep > overhead. The sleep time can start with 100ms and can increase up to 2 > seconds doubling on each iteration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21694) Hive driver wait time is fixed for task getting executed in parallel.
[ https://issues.apache.org/jira/browse/HIVE-21694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16833512#comment-16833512 ] Gopal V commented on HIVE-21694: Is this related to - HIVE-21646? Sleep times are a bad idea in general, instead use the condition waits (see the fix in HIVE-20989, for example) > Hive driver wait time is fixed for task getting executed in parallel. > - > > Key: HIVE-21694 > URL: https://issues.apache.org/jira/browse/HIVE-21694 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 4.0.0 > > > During a command execution hive driver executes the task in a separate thread > if the task to be executed is set as parallel. After starting the task, > driver checks if the task has finished execution or not. If the task > execution is not finished it waits for 2 seconds before waking up again to > check the task status. In case of task with execution time in milliseconds, > this wait time can induce substantial overhead. So instead of fixed wait > time, exponential backedup sleep time can be used to reduce the sleep > overhead. The sleep time can start with 100ms and can increase up to 2 > seconds doubling on each iteration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
