[jira] [Commented] (HIVE-17034) The spark tar for itests is downloaded every time if md5sum is not installed
[
https://issues.apache.org/jira/browse/HIVE-17034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095759#comment-16095759
]
Rui Li commented on HIVE-17034:
---
Hi [~kgyrtkirk], the change here only affects the "re-download" logic:
{code}
if [[ ! -f $DOWNLOAD_DIR/$tarName ]]
then
curl -Sso $DOWNLOAD_DIR/$tarName $url
else
local md5File="$tarName".md5sum
curl -Sso $DOWNLOAD_DIR/$md5File "$url".md5sum
cd $DOWNLOAD_DIR
if type md5sum >/dev/null ! md5sum -c $md5File; then
curl -Sso $DOWNLOAD_DIR/$tarName $url || return 1
fi
{code}
If the tar doesn't exist in the first place, it'll be downloaded anyway.
For "re-download", if the developer really cares about updating the spark tar,
I assume he/she will be aware that md5sum is needed. Does that make sense?
> The spark tar for itests is downloaded every time if md5sum is not installed
>
>
> Key: HIVE-17034
> URL: https://issues.apache.org/jira/browse/HIVE-17034
> Project: Hive
> Issue Type: Test
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Trivial
> Fix For: 3.0.0
>
> Attachments: HIVE-17034.1.patch
>
>
> I think we should either skip verifying md5, or fail the build to let
> developer know md5sum is required.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17034) The spark tar for itests is downloaded every time if md5sum is not installed
[ https://issues.apache.org/jira/browse/HIVE-17034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16094726#comment-16094726 ] Zoltan Haindrich commented on HIVE-17034: - [~lirui] I think that the new solution will never download anything in case md5sum is not installed...I think it would be better to fail the build in case md5sum is not installed - I guess currently if someone doesn't have md5sum accessible in the path; will end up with cryptic messages about spark being not in the vicinity... > The spark tar for itests is downloaded every time if md5sum is not installed > > > Key: HIVE-17034 > URL: https://issues.apache.org/jira/browse/HIVE-17034 > Project: Hive > Issue Type: Test > Components: Spark >Reporter: Rui Li >Assignee: Rui Li >Priority: Trivial > Fix For: 3.0.0 > > Attachments: HIVE-17034.1.patch > > > I think we should either skip verifying md5, or fail the build to let > developer know md5sum is required. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17034) The spark tar for itests is downloaded every time if md5sum is not installed
[ https://issues.apache.org/jira/browse/HIVE-17034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16076784#comment-16076784 ] Ashutosh Chauhan commented on HIVE-17034: - +1 > The spark tar for itests is downloaded every time if md5sum is not installed > > > Key: HIVE-17034 > URL: https://issues.apache.org/jira/browse/HIVE-17034 > Project: Hive > Issue Type: Test >Reporter: Rui Li >Assignee: Rui Li > Attachments: HIVE-17034.1.patch > > > I think we should either skip verifying md5, or fail the build to let > developer know md5sum is required. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17034) The spark tar for itests is downloaded every time if md5sum is not installed
[
https://issues.apache.org/jira/browse/HIVE-17034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16074447#comment-16074447
]
Hive QA commented on HIVE-17034:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12875713/HIVE-17034.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10832 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[insert_overwrite_local_directory_1]
(batchId=237)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=140)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=232)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=232)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
(batchId=177)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=177)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
(batchId=177)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5888/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5888/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5888/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12875713 - PreCommit-HIVE-Build
> The spark tar for itests is downloaded every time if md5sum is not installed
>
>
> Key: HIVE-17034
> URL: https://issues.apache.org/jira/browse/HIVE-17034
> Project: Hive
> Issue Type: Test
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-17034.1.patch
>
>
> I think we should either skip verifying md5, or fail the build to let
> developer know md5sum is required.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
