[jira] [Commented] (HIVE-15273) Http Client not configured correctly

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15702862#comment-15702862
 ] 

Hive QA commented on HIVE-15273:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12840684/HIVE-15273.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10733 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample2] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample4] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample6] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample7] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample9] (batchId=38)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
 (batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2] 
(batchId=133)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2300/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2300/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2300/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12840684 - PreCommit-HIVE-Build

> Http Client not configured correctly
> 
>
> Key: HIVE-15273
> URL: https://issues.apache.org/jira/browse/HIVE-15273
> Project: Hive
>  Issue Type: Bug
>  Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Minor
> Attachments: 0001-adding-confing-to-http-client.patch, 
> HIVE-15273.patch
>
>
> Current used http client by the druid-hive record reader is constructed with 
> default values. Default values of numConnection and ReadTimeout are very 
> small which can lead to following exception " ERROR 
> [2ee34a2b-c8a5-4748-ab91-db3621d2aa5c main] CliDriver: Failed with exception 
> java.io.IOException:java.io.IOException: java.io.IOException: org.apache.h
> ive.druid.org.jboss.netty.channel.ChannelException: Channel disconnected"
> Full stack can be found 
> here.https://gist.github.com/b-slim/384ca6a96698f5b51ad9b171cff556a2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-15273) Http Client not configured correctly

[slim bouguerra (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15702667#comment-15702667
 ] 

slim bouguerra commented on HIVE-15273:
---

added PR https://github.com/apache/hive/pull/119

> Http Client not configured correctly
> 
>
> Key: HIVE-15273
> URL: https://issues.apache.org/jira/browse/HIVE-15273
> Project: Hive
>  Issue Type: Bug
>  Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Minor
> Attachments: 0001-adding-confing-to-http-client.patch, 
> HIVE-15273.patch
>
>
> Current used http client by the druid-hive record reader is constructed with 
> default values. Default values of numConnection and ReadTimeout are very 
> small which can lead to following exception " ERROR 
> [2ee34a2b-c8a5-4748-ab91-db3621d2aa5c main] CliDriver: Failed with exception 
> java.io.IOException:java.io.IOException: java.io.IOException: org.apache.h
> ive.druid.org.jboss.netty.channel.ChannelException: Channel disconnected"
> Full stack can be found 
> here.https://gist.github.com/b-slim/384ca6a96698f5b51ad9b171cff556a2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-15273) Http Client not configured correctly

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15702666#comment-15702666
 ] 

ASF GitHub Bot commented on HIVE-15273:
---

GitHub user b-slim opened a pull request:

https://github.com/apache/hive/pull/119

HIVE-15273 adding confing to http client



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/b-slim/hive fix_http_client

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/119.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #119


commit ae7c217f6937d8ec818a61df4a1579de0d11d36e
Author: Slim Bouguerra 
Date:   2016-11-23T22:49:07Z

adding confing to http client




> Http Client not configured correctly
> 
>
> Key: HIVE-15273
> URL: https://issues.apache.org/jira/browse/HIVE-15273
> Project: Hive
>  Issue Type: Bug
>  Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Minor
> Attachments: 0001-adding-confing-to-http-client.patch, 
> HIVE-15273.patch
>
>
> Current used http client by the druid-hive record reader is constructed with 
> default values. Default values of numConnection and ReadTimeout are very 
> small which can lead to following exception " ERROR 
> [2ee34a2b-c8a5-4748-ab91-db3621d2aa5c main] CliDriver: Failed with exception 
> java.io.IOException:java.io.IOException: java.io.IOException: org.apache.h
> ive.druid.org.jboss.netty.channel.ChannelException: Channel disconnected"
> Full stack can be found 
> here.https://gist.github.com/b-slim/384ca6a96698f5b51ad9b171cff556a2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-15273) Http Client not configured correctly

[slim bouguerra (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15702627#comment-15702627
 ] 

slim bouguerra commented on HIVE-15273:
---

[~leftylev] thanks for the comments ! i have uploaded a new patch.
[~jcamachorodriguez] thanks for testing it please checkout the new patch.

> Http Client not configured correctly
> 
>
> Key: HIVE-15273
> URL: https://issues.apache.org/jira/browse/HIVE-15273
> Project: Hive
>  Issue Type: Bug
>  Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Minor
> Attachments: 0001-adding-confing-to-http-client.patch, 
> HIVE-15273.patch
>
>
> Current used http client by the druid-hive record reader is constructed with 
> default values. Default values of numConnection and ReadTimeout are very 
> small which can lead to following exception " ERROR 
> [2ee34a2b-c8a5-4748-ab91-db3621d2aa5c main] CliDriver: Failed with exception 
> java.io.IOException:java.io.IOException: java.io.IOException: org.apache.h
> ive.druid.org.jboss.netty.channel.ChannelException: Channel disconnected"
> Full stack can be found 
> here.https://gist.github.com/b-slim/384ca6a96698f5b51ad9b171cff556a2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-15273) Http Client not configured correctly

[Jesus Camacho Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15702016#comment-15702016
 ] 

Jesus Camacho Rodriguez commented on HIVE-15273:


[~bslim], I have just checked the patch and overall looks good.

In addition to addressing [~leftylev]'s comments, could you remove the 
initialization for _ numConnection_ and _readTimeout_ with hardcoded values in 
DruidSerDe? Since we will pass through the _initialization_ method and default 
values already exist in HiveConf.java, we do not need to hardcode the value for 
these properties. Thanks

> Http Client not configured correctly
> 
>
> Key: HIVE-15273
> URL: https://issues.apache.org/jira/browse/HIVE-15273
> Project: Hive
>  Issue Type: Bug
>  Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Minor
> Attachments: 0001-adding-confing-to-http-client.patch
>
>
> Current used http client by the druid-hive record reader is constructed with 
> default values. Default values of numConnection and ReadTimeout are very 
> small which can lead to following exception " ERROR 
> [2ee34a2b-c8a5-4748-ab91-db3621d2aa5c main] CliDriver: Failed with exception 
> java.io.IOException:java.io.IOException: java.io.IOException: org.apache.h
> ive.druid.org.jboss.netty.channel.ChannelException: Channel disconnected"
> Full stack can be found 
> here.https://gist.github.com/b-slim/384ca6a96698f5b51ad9b171cff556a2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-15273) Http Client not configured correctly

[Lefty Leverenz (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15692910#comment-15692910
 ] 

Lefty Leverenz commented on HIVE-15273:
---

Review of configuration parameters:  Please make the parameter descriptions 
more formal.

For example, for *hive.druid.http.numConnection* you could change "num 
connection used by the http client" to "Number of connections used by the HTTP 
client."  For *hive.druid.http.read.timeout* you could change "read timeout 
period for the http client" to something like "Read timeout period for the HTTP 
client, in ."

Also, what does "PT1M" mean (as the default for 
*hive.druid.http.read.timeout*)?  Usually a time interval is specified with a 
TimeValidator, for example:

{code}
  HIVES_AUTO_PROGRESS_TIMEOUT("hive.auto.progress.timeout", "0s",
new TimeValidator(TimeUnit.SECONDS),
{code}

> Http Client not configured correctly
> 
>
> Key: HIVE-15273
> URL: https://issues.apache.org/jira/browse/HIVE-15273
> Project: Hive
>  Issue Type: Bug
>  Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Minor
> Attachments: 0001-adding-confing-to-http-client.patch
>
>
> Current used http client by the druid-hive record reader is constructed with 
> default values. Default values of numConnection and ReadTimeout are very 
> small which can lead to following exception " ERROR 
> [2ee34a2b-c8a5-4748-ab91-db3621d2aa5c main] CliDriver: Failed with exception 
> java.io.IOException:java.io.IOException: java.io.IOException: org.apache.h
> ive.druid.org.jboss.netty.channel.ChannelException: Channel disconnected"
> Full stack can be found 
> here.https://gist.github.com/b-slim/384ca6a96698f5b51ad9b171cff556a2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-15273) Http Client not configured correctly

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15691936#comment-15691936
 ] 

Hive QA commented on HIVE-15273:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12840337/0001-adding-confing-to-http-client.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10733 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2] 
(batchId=43)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2] 
(batchId=133)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
 (batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
 (batchId=145)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] 
(batchId=91)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2273/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2273/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2273/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12840337 - PreCommit-HIVE-Build

> Http Client not configured correctly
> 
>
> Key: HIVE-15273
> URL: https://issues.apache.org/jira/browse/HIVE-15273
> Project: Hive
>  Issue Type: Bug
>  Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Minor
> Attachments: 0001-adding-confing-to-http-client.patch
>
>
> Current used http client by the druid-hive record reader is constructed with 
> default values. Default values of numConnection and ReadTimeout are very 
> small which can lead to following exception " ERROR 
> [2ee34a2b-c8a5-4748-ab91-db3621d2aa5c main] CliDriver: Failed with exception 
> java.io.IOException:java.io.IOException: java.io.IOException: org.apache.h
> ive.druid.org.jboss.netty.channel.ChannelException: Channel disconnected"
> Full stack can be found 
> here.https://gist.github.com/b-slim/384ca6a96698f5b51ad9b171cff556a2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-15273) Http Client not configured correctly

[slim bouguerra (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15691629#comment-15691629
 ] 

slim bouguerra commented on HIVE-15273:
---

FYI the way lifecycle is created need to be fixed will send that in a follow up.

> Http Client not configured correctly
> 
>
> Key: HIVE-15273
> URL: https://issues.apache.org/jira/browse/HIVE-15273
> Project: Hive
>  Issue Type: Bug
>  Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Minor
> Attachments: 0001-adding-confing-to-http-client.patch
>
>
> Current used http client by the druid-hive record reader is constructed with 
> default values. Default values of numConnection and ReadTimeout are very 
> small which can lead to following exception " ERROR 
> [2ee34a2b-c8a5-4748-ab91-db3621d2aa5c main] CliDriver: Failed with exception 
> java.io.IOException:java.io.IOException: java.io.IOException: org.apache.h
> ive.druid.org.jboss.netty.channel.ChannelException: Channel disconnected"
> Full stack can be found 
> here.https://gist.github.com/b-slim/384ca6a96698f5b51ad9b171cff556a2



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)