[Impala-ASF-CR] IMPALA-7579: fix test query profile contains all events on S3
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: fix test_query_profile_contains_all_events on S3 .. IMPALA-7579: fix test_query_profile_contains_all_events on S3 This bug was introduced by IMPALA-6568 which added the test_query_profile_contains_all_events test. This test creates a file in the filesystem so that it can be used by 'load data inpath'. The test was using the hdfs_client object to do file system operations, but this client only works with hdfs. Switch to using the filesystem_client object which can work on other filesystems, including s3. Also create the file that will be moved by 'load data inpath' under the unique database directory; this means the file can be created without having to check if it exists already and must be deleted. Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Reviewed-on: http://gerrit.cloudera.org:8080/11461 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M tests/query_test/test_observability.py 1 file changed, 2 insertions(+), 3 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 4 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac
[Impala-ASF-CR] IMPALA-7579: fix test query profile contains all events on S3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: fix test_query_profile_contains_all_events on S3 .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 3 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 19 Sep 2018 22:07:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7579: fix test query profile contains all events on S3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: fix test_query_profile_contains_all_events on S3 .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3178/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 3 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 19 Sep 2018 18:41:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7579: fix test query profile contains all events on S3
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: fix test_query_profile_contains_all_events on S3 .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 19 Sep 2018 18:41:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7579: fix test query profile contains all events on S3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: fix test_query_profile_contains_all_events on S3 .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 3 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 19 Sep 2018 18:41:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7579: fix test query profile contains all events on S3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: fix test_query_profile_contains_all_events on S3 .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 19 Sep 2018 03:59:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7579: fix test query profile contains all events on S3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: fix test_query_profile_contains_all_events on S3 .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/706/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 19 Sep 2018 01:04:54 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7579: fix test query profile contains all events on S3
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: fix test_query_profile_contains_all_events on S3 .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3174/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 19 Sep 2018 00:34:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7579: fix test query profile contains all events on S3
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: fix test_query_profile_contains_all_events on S3 .. Patch Set 1: (5 comments) http://gerrit.cloudera.org:8080/#/c/11461/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/11461/1//COMMIT_MSG@7 PS1, Line 7: IMPALA-7579: use the generic filesystem_client to delete and create files in test_query_profile_contains_all_events. > nit: keep commit message lines to 72 characters (and don't wrap the title l Done http://gerrit.cloudera.org:8080/#/c/11461/1//COMMIT_MSG@7 PS1, Line 7: IMPALA-7579: use the generic filesystem_client to delete and create files in test_query_profile_contains_all_events. > make this line less verbose... something like "fix S3 file ops in test_quer Done http://gerrit.cloudera.org:8080/#/c/11461/1//COMMIT_MSG@11 PS1, Line 11: is > nit: it Done http://gerrit.cloudera.org:8080/#/c/11461/1//COMMIT_MSG@11 PS1, Line 11: > nit: get rid of extra ws Done http://gerrit.cloudera.org:8080/#/c/11461/1/tests/query_test/test_observability.py File tests/query_test/test_observability.py: http://gerrit.cloudera.org:8080/#/c/11461/1/tests/query_test/test_observability.py@224 PS1, Line 224: tmp > is this where the db is created? I thought it was under 'test-warehouse' Thanks for the suggestion -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac Gerrit-Comment-Date: Wed, 19 Sep 2018 00:33:13 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7579: fix test query profile contains all events on S3
Andrew Sherman has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/11461 ) Change subject: IMPALA-7579: fix test_query_profile_contains_all_events on S3 .. IMPALA-7579: fix test_query_profile_contains_all_events on S3 This bug was introduced by IMPALA-6568 which added the test_query_profile_contains_all_events test. This test creates a file in the filesystem so that it can be used by 'load data inpath'. The test was using the hdfs_client object to do file system operations, but this client only works with hdfs. Switch to using the filesystem_client object which can work on other filesystems, including s3. Also create the file that will be moved by 'load data inpath' under the unique database directory; this means the file can be created without having to check if it exists already and must be deleted. Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 --- M tests/query_test/test_observability.py 1 file changed, 2 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/11461/2 -- To view, visit http://gerrit.cloudera.org:8080/11461 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7c6e0899455dd4e12636b959fab4bde79f02fb95 Gerrit-Change-Number: 11461 Gerrit-PatchSet: 2 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Vuk Ercegovac
