[Impala-ASF-CR] IMPALA-7368: Add initial support for DATE type
Todd Lipcon has posted comments on this change. (
http://gerrit.cloudera.org:8080/12481 )
Change subject: IMPALA-7368: Add initial support for DATE type
..
Patch Set 11:
(32 comments)
nice work, pretty impressive test coverage. Mostly small items here.
http://gerrit.cloudera.org:8080/#/c/12481/11//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/12481/11//COMMIT_MSG@29
PS11, Line 29: - from STRING to DATE if the source string value is used in a
: context where a DATE value is expected.
it's interesting that this differs from the behavior of TIMESTAMP today, at
least for timestamp arithmetic expressions: SELECT '2019-01-01 12;12:12' +
interval 1 day does not do an implicit conversion. (interestingly it does in
MySQL).
Does the ANSI standard give rules for this?
http://gerrit.cloudera.org:8080/#/c/12481/11//COMMIT_MSG@38
PS11, Line 38: E.g: year('2019-02-15') must resolve to
:year(TIMESTAMP) instead of year(DATE). Note, that
year(DATE) is
:not implemented yet, so this is not an issue at the
moment but
:it will be in the future.
wouldn't the result be the same either way for this example?
http://gerrit.cloudera.org:8080/#/c/12481/11//COMMIT_MSG@42
PS11, Line 42: better fit
how is this defined, specifically?
http://gerrit.cloudera.org:8080/#/c/12481/11//COMMIT_MSG@70
PS11, Line 70: complete DATE type implementation
should we consider turning it off by default and only enabling it once we're
sure it's stable and correct?
http://gerrit.cloudera.org:8080/#/c/12481/11//COMMIT_MSG@72
PS11, Line 72: - Add date support to the random query generator.
should we transition TPC-DS test tables over to 'date' instead of 'string' for
the appropriate columns?
http://gerrit.cloudera.org:8080/#/c/12481/11/be/src/codegen/codegen-anyval.cc
File be/src/codegen/codegen-anyval.cc:
http://gerrit.cloudera.org:8080/#/c/12481/11/be/src/codegen/codegen-anyval.cc@68
PS11, Line 68: return cg->i64_type();
why do we lower to i64 instead of i32 given the slot size is only 4 bytes?
seems like we're using int32 below
http://gerrit.cloudera.org:8080/#/c/12481/11/be/src/exec/hdfs-table-sink.cc
File be/src/exec/hdfs-table-sink.cc:
http://gerrit.cloudera.org:8080/#/c/12481/11/be/src/exec/hdfs-table-sink.cc@500
PS11, Line 500: stringstream error_msg;
: error_msg << "Cannot write DATE column to a PARQUET
table.";
: return Status(error_msg.str());
nit: why not just pass that string directly?
Could we make this better by including the column name from the table desc?
http://gerrit.cloudera.org:8080/#/c/12481/11/be/src/exprs/aggregate-functions-ir.cc
File be/src/exprs/aggregate-functions-ir.cc:
http://gerrit.cloudera.org:8080/#/c/12481/11/be/src/exprs/aggregate-functions-ir.cc@381
PS11, Line 381: val_struct->sum / val_struct->count
Is this correct behavior when the dates are negative? Dividing as signed
integers means we'll "truncate towards zero" which means that average of
(1969-01-01, 1969-01-02) would be 1969-01-02 whereas average of 1971-01-01 and
1971-01-02 would be 1971-01-01. I think that's surprising because it exposes
the epoch reference point in the semantics of the operation.
Does the ANSI standard have anything to say here?
Checking briefly on sqlfiddle:
- postgres 9.6 doesn't support AVG(date) at all. Neither does Oracle 11g R2 nor
SQL Server 2017.
- mysql 5.6 implicitly casts the dates to an DECIMIAL and return something
weird like 19690101.5. Not sure what's going on under the covers.
- hive 3.1 doesn't support AVG(date) either
Maybe we should remove this functionality for now and consider adding it back
based on user demand?
http://gerrit.cloudera.org:8080/#/c/12481/11/be/src/exprs/conditional-functions-ir.cc
File be/src/exprs/conditional-functions-ir.cc:
http://gerrit.cloudera.org:8080/#/c/12481/11/be/src/exprs/conditional-functions-ir.cc@37
PS11, Line 37: ITERATE_OVER_SEQ
per comment elsewhere, I found it clearer in the old version since I didn't
need to understand boost magic
http://gerrit.cloudera.org:8080/#/c/12481/11/be/src/exprs/expr-test.cc
File be/src/exprs/expr-test.cc:
http://gerrit.cloudera.org:8080/#/c/12481/11/be/src/exprs/expr-test.cc@3386
PS11, Line 3386:
: TEST_F(ExprTest, CastDateExprs) {
add some tests for invalid dates like feb 30
http://gerrit.cloudera.org:8080/#/c/12481/11/be/src/exprs/expr-test.cc@3475
PS11, Line 3475: TestDateValue("cast(cast('1400-01-01 00:00:00' as timestamp)
as date)",
can you test a before-the-epoch timestamp that includes a time portion? eg
'1960-01-01 23:59' and make sure it truncates "down" and not truncate "towards
the epoch"?
http://gerrit.cloudera.org:8080/#/c/12481/11/be/src/runtime/date-value.h
File be/src/runtime/date-value.h:
[Impala-ASF-CR] [DOCS] Event based HMS sync works for Metadata V1 and V2
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12745 ) Change subject: [DOCS] Event based HMS sync works for Metadata V1 and V2 .. Patch Set 3: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/276/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/12745 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I366add1016d8dad876f100a0b27ba3051e2cafbb Gerrit-Change-Number: 12745 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 14 Mar 2019 02:34:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8303: [DOCS] Impala 3.2 release notes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12752 ) Change subject: IMPALA-8303: [DOCS] Impala 3.2 release notes .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/275/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/12752 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id50811c68ec29c218358a87288f08ad2bae16b55 Gerrit-Change-Number: 12752 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Mar 2019 02:32:07 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Event based HMS sync works for Metadata V1 and V2
Hello Bharath Vissapragada, Vihang Karajgaonkar, Todd Lipcon, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12745 to look at the new patch set (#3). Change subject: [DOCS] Event based HMS sync works for Metadata V1 and V2 .. [DOCS] Event based HMS sync works for Metadata V1 and V2 - Move the Metadata V2 note so that it does not apply to event based sync. Change-Id: I366add1016d8dad876f100a0b27ba3051e2cafbb --- M docs/topics/impala_metadata.xml 1 file changed, 3 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/12745/3 -- To view, visit http://gerrit.cloudera.org:8080/12745 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I366add1016d8dad876f100a0b27ba3051e2cafbb Gerrit-Change-Number: 12745 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] [DOCS] Event based HMS sync works for Metadata V1 and V2
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12745 ) Change subject: [DOCS] Event based HMS sync works for Metadata V1 and V2 .. Patch Set 3: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/276/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/12745 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I366add1016d8dad876f100a0b27ba3051e2cafbb Gerrit-Change-Number: 12745 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 14 Mar 2019 02:32:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8303: [DOCS] Impala 3.2 release notes
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12752 Change subject: IMPALA-8303: [DOCS] Impala 3.2 release notes .. IMPALA-8303: [DOCS] Impala 3.2 release notes Change-Id: Id50811c68ec29c218358a87288f08ad2bae16b55 --- M docs/impala_keydefs.ditamap M docs/topics/impala_fixed_issues.xml M docs/topics/impala_new_features.xml 3 files changed, 20 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/52/12752/1 -- To view, visit http://gerrit.cloudera.org:8080/12752 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id50811c68ec29c218358a87288f08ad2bae16b55 Gerrit-Change-Number: 12752 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-8303: [DOCS] Impala 3.2 release notes
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12752 ) Change subject: IMPALA-8303: [DOCS] Impala 3.2 release notes .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/275/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/12752 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id50811c68ec29c218358a87288f08ad2bae16b55 Gerrit-Change-Number: 12752 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Mar 2019 02:12:56 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Event based HMS sync works for Metadata V1 and V2
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12745 ) Change subject: [DOCS] Event based HMS sync works for Metadata V1 and V2 .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/274/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/12745 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I366add1016d8dad876f100a0b27ba3051e2cafbb Gerrit-Change-Number: 12745 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Thu, 14 Mar 2019 02:07:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8298: [DOCS] Update docs for complex types support in ORC
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/12744 ) Change subject: IMPALA-8298: [DOCS] Update docs for complex types support in ORC .. Patch Set 3: Thank you for drafting the doc, Quanlong! -- To view, visit http://gerrit.cloudera.org:8080/12744 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I261d41f536d653033cc1083873edf8c83508fdb7 Gerrit-Change-Number: 12744 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Mar 2019 02:03:43 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Event based HMS sync works for Metadata V1 and V2
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12745 ) Change subject: [DOCS] Event based HMS sync works for Metadata V1 and V2 .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/274/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/12745 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I366add1016d8dad876f100a0b27ba3051e2cafbb Gerrit-Change-Number: 12745 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Mar 2019 02:02:06 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Event based HMS sync works for Metadata V1 and V2
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12745 to look at the new patch set (#2). Change subject: [DOCS] Event based HMS sync works for Metadata V1 and V2 .. [DOCS] Event based HMS sync works for Metadata V1 and V2 - Move the Metadata V2 note so that it does not apply to event based sync. Change-Id: I366add1016d8dad876f100a0b27ba3051e2cafbb --- M docs/topics/impala_metadata.xml 1 file changed, 1 insertion(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/12745/2 -- To view, visit http://gerrit.cloudera.org:8080/12745 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I366add1016d8dad876f100a0b27ba3051e2cafbb Gerrit-Change-Number: 12745 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8298: [DOCS] Update docs for complex types support in ORC
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/12744 ) Change subject: IMPALA-8298: [DOCS] Update docs for complex types support in ORC .. Patch Set 3: Thank Alex for your review! -- To view, visit http://gerrit.cloudera.org:8080/12744 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I261d41f536d653033cc1083873edf8c83508fdb7 Gerrit-Change-Number: 12744 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Mar 2019 01:33:21 + Gerrit-HasComments: No
[Impala-ASF-CR] Move DEBIAN FRONTEND=noninteractive variable to the ubuntu section of the script
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/12723 ) Change subject: Move DEBIAN_FRONTEND=noninteractive variable to the ubuntu section of the script .. Patch Set 1: > > (1 comment) > > > > Let me know what you think. If we go with always using > > DEBIAN_FRONTEND=noninteractive, then please note that in the > commit > > message. > > I can confirm that with the current package listing, on an ubuntu > 16.04 docker image the script behaves the same way before and after > this patch (no interactive prompts on apt-get). > > Personally, I prefer to export DEBIAN_FRONTEND=noninteractive in > the interactive terminal case since > a) we're already assuming that this script can be used by automated > scripts (ex. docker/entrypoint.sh) > b) we already perform system-wide configuration changes (postgres, > ntp, etc.) so presumably if in the future we add a dependency that > does require user interaction we can modify the configuration files > as we do now (or revisit this patch altogether :-). Sounds good, we'll use DEBIAN_FRONTEND=noninteractive always. Can you add one line to the commit message documenting this. Something like: "This also changes the behavior to always use DEBIAN_FRONTEND=noninteractive even for interactive terminals." With that change, I'm ready to +2. -- To view, visit http://gerrit.cloudera.org:8080/12723 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifab2f56d3e119e4e73685c3cbd5249760ceb33ac Gerrit-Change-Number: 12723 Gerrit-PatchSet: 1 Gerrit-Owner: Hector Acosta Gerrit-Reviewer: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Thu, 14 Mar 2019 01:03:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8305: Generate JUnitXML for DCHECK failures
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12751 ) Change subject: IMPALA-8305: Generate JUnitXML for DCHECK failures .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2430/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12751 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic45d01b6a87b1d81967f1e10028af96a7cc3071a Gerrit-Change-Number: 12751 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Thu, 14 Mar 2019 00:54:26 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8299: Fix crash in GroupingAggregator on uninited hash table
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12746 ) Change subject: IMPALA-8299: Fix crash in GroupingAggregator on uninited hash table .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2429/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I381657c43f93eb4871b93d54532cb886198fd0b2 Gerrit-Change-Number: 12746 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Thu, 14 Mar 2019 00:44:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8305: Generate JUnitXML for DCHECK failures
Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12751 Change subject: IMPALA-8305: Generate JUnitXML for DCHECK failures .. IMPALA-8305: Generate JUnitXML for DCHECK failures DCHECKs log at the FATAL level and produce a statement starting with "Check failed:". This generates a JUnitXML when it sees a FATAL log file with "Check failed:". Some backend death tests specifically test hitting certain DCHECKs, so the be_tests directory is ignored. Something that hits a DCHECK would also generate a minidump, so this is just additional information beyond the existing JUnitXML for the minidump. Testing: - Tested on logs from a test that hit a DCHECK in impalad - Ran a normal error-free run Change-Id: Ic45d01b6a87b1d81967f1e10028af96a7cc3071a --- M bin/jenkins/finalize.sh 1 file changed, 15 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/51/12751/1 -- To view, visit http://gerrit.cloudera.org:8080/12751 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic45d01b6a87b1d81967f1e10028af96a7cc3071a Gerrit-Change-Number: 12751 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell
[Impala-ASF-CR] IMPALA-7917 (Part 3): Decouple Sentry from Impala
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/12684 ) Change subject: IMPALA-7917 (Part 3): Decouple Sentry from Impala .. Patch Set 9: (26 comments) http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java File fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java: http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java@66 PS9, Line 66: be may *become* stale, right? http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java File fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java: http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java@41 PS9, Line 41: boolean isAdmin(User user) throws ImpalaException; why is admin a special thing rather than just a separate privilege verified by the authorization checker interface? Just curious, doesn't need to change now. http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java@56 PS9, Line 56:* Gets all roles; typo http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java@85 PS9, Line 85:* Gets all privileges. can you clarify, this is all privileges granted to anyone anywhere? Or just on this server/service? http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java@93 PS9, Line 93: void updateDatabaseOwnerPrivilege(String serverName, String databaseName, can you document any expected canonicalization of the names? should they be lower cased? Are they case sensitive? http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java@95 PS9, Line 95: PrincipalType newOwnerType, TDdlExecResponse response) throws ImpalaException; I think it's worth expanding the javadoc slightly to explain why the _old_ values are passed in. Can multiple users be an owner of a database? http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java@100 PS9, Line 100:*/ same http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java File fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java: http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java@77 PS9, Line 77: public void createRole(User requestingUser, TCreateDropRoleParams params, should these functions throw UnsupportedOperationException so that if somehow it gets configured, it won't silently ignore requests? http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java File fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java: http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java@52 PS9, Line 52: Sentry for Catalogd parsing this sentence is a bit difficult - I read this as "sentry for catalogd". Maybe "for catalogd that uses Sentry" makes more sense? http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java@57 PS9, Line 57: Impalads for operations, such as SHOW ROLES and SHOW GRANT I think just saying "broadcasted to all Impalads" is enough, since it's used for anything that requires authorization metadata, right? http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java@127 PS9, Line 127: String roleName = params.getRole_names().get(0); For this and other calls below, can we verify that getRole_names().size() == 1 if we expect exactly one? (same with getGroup_names()) Can probably use Iterables.getOnlyElement(params.getRole_names()) http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java@261 PS9, Line 261: catalog_.getSentryProxy() == null || this would be a bug, right? perhaps we can call verifySentryServiceEnabled() here? http://gerrit.cloudera.org:8080/#/c/12684/9/fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java@263 PS9, Line 263: return; if object ownership is not enabled, doesnt it seem like we should throw an error back to the user instead of no-opping this?
[Impala-ASF-CR] IMPALA-8299: Fix crash in GroupingAggregator on uninited hash table
Hello Michael Ho, Tim Armstrong, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12746 to look at the new patch set (#2). Change subject: IMPALA-8299: Fix crash in GroupingAggregator on uninited hash table .. IMPALA-8299: Fix crash in GroupingAggregator on uninited hash table If HashTable::Init() fails, in some cases we may try to call HashTable::Close() on the table in GroupingAggregator::Close(), which can cause a crash. The solution is to set the hash table to a nullptr if Init() fails. The patch also fixes a potential issue where HashTable::Init() doesn't set its out parameter if Suballocator::Allocate fails. I also verified that the other use of HashTable, PhjBuilder, handles this case correctly. Testing: - Manually verified that the crash no longer occurs if HashTable::Init fails. Change-Id: I381657c43f93eb4871b93d54532cb886198fd0b2 --- M be/src/exec/grouping-aggregator-partition.cc M be/src/exec/hash-table.cc 2 files changed, 7 insertions(+), 2 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/12746/2 -- To view, visit http://gerrit.cloudera.org:8080/12746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I381657c43f93eb4871b93d54532cb886198fd0b2 Gerrit-Change-Number: 12746 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] Use `curl http://169.254.169.254/` to determine if we're running in aws
Hector Acosta has posted comments on this change. ( http://gerrit.cloudera.org:8080/12727 ) Change subject: Use `curl http://169.254.169.254/` to determine if we're running in aws .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/12727/1/bin/bootstrap_system.sh File bin/bootstrap_system.sh: http://gerrit.cloudera.org:8080/#/c/12727/1/bin/bootstrap_system.sh@240 PS1, Line 240: http://169.254.169.254 > On second thought: we may need a stricter test for AWS if the goal is to de I'm not sure I would be able to properly test other non-ec2 cases. But that's outside the scope of what I'm trying to accomplish here. I'm happy to either: a) Re-implement using wget and timeout values. b) Abandon this for now and create a more robust version based on testing whether ntp endpoints are reachable I'm leaning towards a, lmk if you agree. -- To view, visit http://gerrit.cloudera.org:8080/12727 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iddb2574dbcb3f97cf697095d1777e51ce463b205 Gerrit-Change-Number: 12727 Gerrit-PatchSet: 1 Gerrit-Owner: Hector Acosta Gerrit-Reviewer: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Wed, 13 Mar 2019 23:26:24 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8303: Generate JUnitXML for ASAN failures
Joe McDonnell has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12733 ) Change subject: IMPALA-8303: Generate JUnitXML for ASAN failures .. IMPALA-8303: Generate JUnitXML for ASAN failures This modifies bin/jenkins/finalize.sh to detect ASAN error messages. ASAN messages appear in the ERROR logs for impalad and in logs/be_tests/LastTest.log for backend tests. This checks both locations for "ERROR: AddressSanitizer:" messages. Testing: - Ran a Jenkins job with a known backend ASAN problem - Ran finalize.sh on logs from an impalad ASAN crash Change-Id: I5434703a02617d62dc0ed401b8cc1bf7a198a84d Reviewed-on: http://gerrit.cloudera.org:8080/12733 Reviewed-by: David Knupp Tested-by: Impala Public Jenkins --- M bin/jenkins/finalize.sh 1 file changed, 27 insertions(+), 0 deletions(-) Approvals: David Knupp: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/12733 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I5434703a02617d62dc0ed401b8cc1bf7a198a84d Gerrit-Change-Number: 12733 Gerrit-PatchSet: 2 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] IMPALA-8303: Generate JUnitXML for ASAN failures
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12733 ) Change subject: IMPALA-8303: Generate JUnitXML for ASAN failures .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12733 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5434703a02617d62dc0ed401b8cc1bf7a198a84d Gerrit-Change-Number: 12733 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 22:54:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8293 (WIP): Support for Ranger cache invalidation
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12748 ) Change subject: IMPALA-8293 (WIP): Support for Ranger cache invalidation .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2428/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12748 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I524ee93d09077dd4ff3d18fe517739b7776d01d7 Gerrit-Change-Number: 12748 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 22:44:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7982: Add host network usage to profile
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12747 ) Change subject: IMPALA-7982: Add host network usage to profile .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2427/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12747 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2cc74f87374080a74a13b7fb6e4da44a11d828ef Gerrit-Change-Number: 12747 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 22:42:37 + Gerrit-HasComments: No
[Impala-ASF-CR] Move DEBIAN FRONTEND=noninteractive variable to the ubuntu section of the script
Hector Acosta has posted comments on this change. ( http://gerrit.cloudera.org:8080/12723 ) Change subject: Move DEBIAN_FRONTEND=noninteractive variable to the ubuntu section of the script .. Patch Set 1: > (1 comment) > > Let me know what you think. If we go with always using > DEBIAN_FRONTEND=noninteractive, then please note that in the commit > message. I can confirm that with the current package listing, on an ubuntu 16.04 docker image the script behaves the same way before and after this patch (no interactive prompts on apt-get). Personally, I prefer to export DEBIAN_FRONTEND=noninteractive in the interactive terminal case since a) we're already assuming that this script can be used by automated scripts (ex. docker/entrypoint.sh) b) we already perform system-wide configuration changes (postgres, ntp, etc.) so presumably if in the future we add a dependency that does require user interaction we can modify the configuration files as we do now (or revisit this patch altogether :-). -- To view, visit http://gerrit.cloudera.org:8080/12723 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifab2f56d3e119e4e73685c3cbd5249760ceb33ac Gerrit-Change-Number: 12723 Gerrit-PatchSet: 1 Gerrit-Owner: Hector Acosta Gerrit-Reviewer: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 13 Mar 2019 22:42:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8100: Add initial support for Ranger
Todd Lipcon has posted comments on this change. (
http://gerrit.cloudera.org:8080/12632 )
Change subject: IMPALA-8100: Add initial support for Ranger
..
Patch Set 9:
(25 comments)
http://gerrit.cloudera.org:8080/#/c/12632/9//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/12632/9//COMMIT_MSG@30
PS9, Line 30: - Ran all FE tests
: - Ran all E2E authorization tests
did you run these with ranger configured? I'm surprised they would pass since
best I can tell this patch doesn't attempt to support GRANT/REVOKE statements
etc, right?
http://gerrit.cloudera.org:8080/#/c/12632/9/be/src/service/frontend.cc
File be/src/service/frontend.cc:
http://gerrit.cloudera.org:8080/#/c/12632/9/be/src/service/frontend.cc@41
PS9, Line 41:
"org.apache.impala.authorization.sentry.SentryAuthorizationFactory",
I wonder if we should switch this to be a more human-friendly flag instead of a
class name, before we ship it?
http://gerrit.cloudera.org:8080/#/c/12632/9/be/src/service/frontend.cc@46
PS9, Line 46: "identify the application that communicates with Ranger.");
Worth noting whether this is required, considering it has an empty default
http://gerrit.cloudera.org:8080/#/c/12632/9/be/src/service/frontend.cc@47
PS9, Line 47: DEFINE_string(server_name, "", "The name to use for securing this
impalad "
we probably can't rename the existing sentry-specific flags (for compat
reasons) but perhaps we can amend the help text for each of these flags to say
something like "only relevant if sentry is enabled" or something?
It might be worth hard-coding some checks to make sure
GetCommandLineFlagInfoOrDie("server_name").is_default for these flags if ranger
is set, and vice-versa for ranger-specific flags, so that people don't
accidentally cross-pollinate their configs
http://gerrit.cloudera.org:8080/#/c/12632/9/fe/pom.xml
File fe/pom.xml:
http://gerrit.cloudera.org:8080/#/c/12632/9/fe/pom.xml@112
PS9, Line 112:
it seems that ranger-plugins-common has a pom dependency on
mysql-connector-java. Is that really necessary? I thought ranger used a
"service" model rather than the plugin directly talking to any database?
http://gerrit.cloudera.org:8080/#/c/12632/9/fe/src/main/java/org/apache/impala/analysis/DescribeTableStmt.java
File fe/src/main/java/org/apache/impala/analysis/DescribeTableStmt.java:
http://gerrit.cloudera.org:8080/#/c/12632/9/fe/src/main/java/org/apache/impala/analysis/DescribeTableStmt.java@126
PS9, Line 126: Privilege.ANY);
I'm not 100% sure about this part of the change. If I don't have the
'VIEW_METADATA' privilege but I do have permissions on one or more columns,
should I still be able to use 'DESCRIBE' in all cases? For example, 'DESCRIBE
EXTENDED' on a view will show the view text and other metadata, but maybe I'm
not supposed to have that capability if I don't have VIEW_METADATA privileges?
Do we have some good documentation on the intended semantics here?
http://gerrit.cloudera.org:8080/#/c/12632/9/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java
File
fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java:
http://gerrit.cloudera.org:8080/#/c/12632/9/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@77
PS9, Line 77: // Any column access is special in Ranger. For example if
we want to check if
: // we have access to any column on a particular table, we
need to build a resource
: // without the column resource.
: // For example:
: // access type: RangerPolicyEngine.ANY_ACCESS
: // resources: [server=server1, database=foo, table=bar]
I'm not understanding this. When you are saying "Any column access" do you mean
"authorization checks for wildcard columns" or something? I think expanding
upon the example would be useful.
http://gerrit.cloudera.org:8080/#/c/12632/9/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@106
PS9, Line 106: if (request.getPrivilege() == Privilege.VIEW_METADATA) {
does this need to be special cased? It seems all the other privileges already
have an "implied" set which is equal to just the privilege itself. (ANY
probably does need to be special cased still)
http://gerrit.cloudera.org:8080/#/c/12632/9/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@117
PS9, Line 117: if (authorize(plugin, resource, user, privilege)) {
does this result in multiple audit events, one per implied privilege?
http://gerrit.cloudera.org:8080/#/c/12632/9/fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java@131
PS9, Line 131: new HashSet
can you use Collections.emptySet?
Also is this a TODO? Should we be doing group resolution
[Impala-ASF-CR] IMPALA-7982: Add host network usage to profile
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12747 ) Change subject: IMPALA-7982: Add host network usage to profile .. Patch Set 1: (15 comments) http://gerrit.cloudera.org:8080/#/c/12747/1/be/src/util/system-state-info-test.cc File be/src/util/system-state-info-test.cc: http://gerrit.cloudera.org:8080/#/c/12747/1/be/src/util/system-state-info-test.cc@79 PS1, Line 79: string dev_net = R"(Inter-| Receive | Transmit line too long (98 > 90) http://gerrit.cloudera.org:8080/#/c/12747/1/be/src/util/system-state-info-test.cc@80 PS1, Line 80:face |bytespackets errs drop fifo frame compressed multicast|bytespackets errs drop fifo colls carrier compressed line too long (124 > 90) http://gerrit.cloudera.org:8080/#/c/12747/1/be/src/util/system-state-info-test.cc@81 PS1, Line 81:lo: 12178099975318 24167151765000 0 0 0 12178099975318 24167151765000 0 0 0 line too long (146 > 90) http://gerrit.cloudera.org:8080/#/c/12747/1/be/src/util/system-state-info-test.cc@82 PS1, Line 82:br-c4d9b4cafca2: 1025814 409000 0 0 0 70616330 1638104000 0 0 0 line too long (138 > 90) http://gerrit.cloudera.org:8080/#/c/12747/1/be/src/util/system-state-info-test.cc@83 PS1, Line 83: eth0: 366039609986 388580561020 0 0 62231368 95492744135 174535524000 0 0 0)"; line too long (144 > 90) http://gerrit.cloudera.org:8080/#/c/12747/1/be/src/util/system-state-info-test.cc@134 PS1, Line 134: string dev_net_1 = R"(Inter-| Receive | Transmit line too long (100 > 90) http://gerrit.cloudera.org:8080/#/c/12747/1/be/src/util/system-state-info-test.cc@135 PS1, Line 135:face |bytespackets errs drop fifo frame compressed multicast|bytespackets errs drop fifo colls carrier compressed line too long (124 > 90) http://gerrit.cloudera.org:8080/#/c/12747/1/be/src/util/system-state-info-test.cc@136 PS1, Line 136:lo: 1000 2000000 0 0 0 3000 4000000 0 0 0 line too long (112 > 90) http://gerrit.cloudera.org:8080/#/c/12747/1/be/src/util/system-state-info-test.cc@137 PS1, Line 137:br-c4d9b4cafca2: 5000 409000 0 0 0 6000 7000000 0 0 0 line too long (128 > 90) http://gerrit.cloudera.org:8080/#/c/12747/1/be/src/util/system-state-info-test.cc@138 PS1, Line 138: eth0: 8000 9000020 0 0 1 11000 12000000 0 0 0)"; line too long (118 > 90) http://gerrit.cloudera.org:8080/#/c/12747/1/be/src/util/system-state-info-test.cc@139 PS1, Line 139: string dev_net_2 = R"(Inter-| Receive | Transmit line too long (100 > 90) http://gerrit.cloudera.org:8080/#/c/12747/1/be/src/util/system-state-info-test.cc@140 PS1, Line 140:face |bytespackets errs drop fifo frame compressed multicast|bytespackets errs drop fifo colls carrier compressed line too long (124 > 90) http://gerrit.cloudera.org:8080/#/c/12747/1/be/src/util/system-state-info-test.cc@141 PS1, Line 141:lo: 2000 4000000 0 0 0 6000 8000000 0 0 0 line too long (112 > 90) http://gerrit.cloudera.org:8080/#/c/12747/1/be/src/util/system-state-info-test.cc@142 PS1, Line 142:br-c4d9b4cafca2: 7000 609000 0 0 0 12000 14000000 0 0 0 line too long (130 > 90) http://gerrit.cloudera.org:8080/#/c/12747/1/be/src/util/system-state-info-test.cc@143 PS1, Line 143: eth0: 1 11000020 0 0 1 11000 12000000 0 0 0)"; line too long (120 > 90) -- To view, visit http://gerrit.cloudera.org:8080/12747 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2cc74f87374080a74a13b7fb6e4da44a11d828ef Gerrit-Change-Number: 12747 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 22:12:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8299: Fix crash in GroupingAggregator on uninited hash table
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12746 ) Change subject: IMPALA-8299: Fix crash in GroupingAggregator on uninited hash table .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2426/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I381657c43f93eb4871b93d54532cb886198fd0b2 Gerrit-Change-Number: 12746 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 13 Mar 2019 22:25:29 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-337: Add support for POST to the webserver
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12730 ) Change subject: IMPALA-337: Add support for POST to the webserver .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2425/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3bd3ecfab53263c13a4509c278569302c9901abb Gerrit-Change-Number: 12730 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 13 Mar 2019 22:24:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-337: Add support for POST to the webserver
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12730 ) Change subject: IMPALA-337: Add support for POST to the webserver .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2424/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3bd3ecfab53263c13a4509c278569302c9901abb Gerrit-Change-Number: 12730 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 13 Mar 2019 22:22:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8293 (WIP): Support for Ranger cache invalidation
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/12748 ) Change subject: IMPALA-8293 (WIP): Support for Ranger cache invalidation .. Patch Set 1: A sneak peak to force Ranger cache invalidation. REFRESH AUTHORIZATION works but INVALIDATE METADATA is still broken. Need figure out what to do with local catalog. -- To view, visit http://gerrit.cloudera.org:8080/12748 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I524ee93d09077dd4ff3d18fe517739b7776d01d7 Gerrit-Change-Number: 12748 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 22:16:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8293 (WIP): Support for Ranger cache invalidation
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12748 ) Change subject: IMPALA-8293 (WIP): Support for Ranger cache invalidation .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/12748/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java: http://gerrit.cloudera.org:8080/#/c/12748/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@47 PS1, Line 47: import org.apache.impala.authorization.NoneAuthorizationFactory.NoneAuthorizationRefresher; line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/12748/1/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java File fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java: http://gerrit.cloudera.org:8080/#/c/12748/1/fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java@55 PS1, Line 55: import org.apache.impala.authorization.NoneAuthorizationFactory.NoneAuthorizationRefresher; line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/12748/1/fe/src/test/java/org/apache/impala/testutil/PlannerTestCaseLoader.java File fe/src/test/java/org/apache/impala/testutil/PlannerTestCaseLoader.java: http://gerrit.cloudera.org:8080/#/c/12748/1/fe/src/test/java/org/apache/impala/testutil/PlannerTestCaseLoader.java@21 PS1, Line 21: import org.apache.impala.authorization.NoneAuthorizationFactory.NoneAuthorizationRefresher; line too long (91 > 90) -- To view, visit http://gerrit.cloudera.org:8080/12748 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I524ee93d09077dd4ff3d18fe517739b7776d01d7 Gerrit-Change-Number: 12748 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 22:15:54 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8293 (WIP): Support for Ranger cache invalidation
Fredy Wijaya has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12748 Change subject: IMPALA-8293 (WIP): Support for Ranger cache invalidation .. IMPALA-8293 (WIP): Support for Ranger cache invalidation This patch addes support for Ranger cache invalidation via INVALIDATE METADATA and REFRESH AUTHORIZATION. This patch uses a hacked way of invalidating Ranger caches and it will be fixed as soon the API in RANGER-2349 is available. Change-Id: I524ee93d09077dd4ff3d18fe517739b7776d01d7 --- M common/thrift/CatalogObjects.thrift M fe/src/main/java/org/apache/impala/authorization/AuthorizationChecker.java A fe/src/main/java/org/apache/impala/authorization/AuthorizationDelta.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/AuthorizationManager.java A fe/src/main/java/org/apache/impala/authorization/AuthorizationRefresher.java M fe/src/main/java/org/apache/impala/authorization/NoneAuthorizationFactory.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationFactory.java A fe/src/main/java/org/apache/impala/authorization/ranger/RangerAuthorizationRefresher.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationChecker.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationFactory.java A fe/src/main/java/org/apache/impala/authorization/sentry/SentryAuthorizationRefresher.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryCatalogdAuthorizationManager.java M fe/src/main/java/org/apache/impala/authorization/sentry/SentryProxy.java A fe/src/main/java/org/apache/impala/catalog/AuthzCache.java M fe/src/main/java/org/apache/impala/catalog/Catalog.java M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/FeCatalogManager.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M fe/src/test/java/org/apache/impala/common/FrontendTestBase.java M fe/src/test/java/org/apache/impala/testutil/CatalogServiceTestCatalog.java M fe/src/test/java/org/apache/impala/testutil/ImpaladTestCatalog.java M fe/src/test/java/org/apache/impala/testutil/PlannerTestCaseLoader.java 28 files changed, 535 insertions(+), 115 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/12748/1 -- To view, visit http://gerrit.cloudera.org:8080/12748 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I524ee93d09077dd4ff3d18fe517739b7776d01d7 Gerrit-Change-Number: 12748 Gerrit-PatchSet: 1 Gerrit-Owner: Fredy Wijaya
[Impala-ASF-CR] IMPALA-7982: Add host network usage to profile
Lars Volker has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12747 Change subject: IMPALA-7982: Add host network usage to profile .. IMPALA-7982: Add host network usage to profile This change adds host network usage to profiles. For each host that participates in the query execution it adds the incoming and outgoing bandwidth across all interfaces excluding the local loopback interface. This includes all data transmitted by the host as part of the execution of a query, other queries, and other processes running on the same system. The change adds tests for the added functionality to the backend and end to end tests. Change-Id: I2cc74f87374080a74a13b7fb6e4da44a11d828ef --- M be/src/runtime/query-state.cc M be/src/util/system-state-info-test.cc M be/src/util/system-state-info.cc M be/src/util/system-state-info.h M tests/query_test/test_observability.py 5 files changed, 253 insertions(+), 38 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/47/12747/1 -- To view, visit http://gerrit.cloudera.org:8080/12747 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I2cc74f87374080a74a13b7fb6e4da44a11d828ef Gerrit-Change-Number: 12747 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker
[Impala-ASF-CR] [DOCS] Event based HMS sync works for Metadata V1 and V2
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12745 ) Change subject: [DOCS] Event based HMS sync works for Metadata V1 and V2 .. Patch Set 1: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/273/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/12745 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I366add1016d8dad876f100a0b27ba3051e2cafbb Gerrit-Change-Number: 12745 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 21:56:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8299: Fix crash in GroupingAggregator on uninited hash table
Thomas Marshall has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12746 Change subject: IMPALA-8299: Fix crash in GroupingAggregator on uninited hash table .. IMPALA-8299: Fix crash in GroupingAggregator on uninited hash table If HashTable::Init() fails, in some cases we may try to call HashTable::Close() on the table in GroupingAggregator::Close(), which can cause a crash. The solution is to set the hash table to a nullptr if Init() fails. I also verified that the other use of HashTable, PhjBuilder, handles this case correctly. Testing: - Manually verified that the crash no longer occurs if HashTable::Init fails. Change-Id: I381657c43f93eb4871b93d54532cb886198fd0b2 --- M be/src/exec/grouping-aggregator-partition.cc 1 file changed, 6 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/12746/1 -- To view, visit http://gerrit.cloudera.org:8080/12746 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I381657c43f93eb4871b93d54532cb886198fd0b2 Gerrit-Change-Number: 12746 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall
[Impala-ASF-CR] Prototype for a remote read byte cache.
Anonymous Coward (474) has posted comments on this change. ( http://gerrit.cloudera.org:8080/12683 ) Change subject: Prototype for a remote read byte cache. .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/12683/1/be/src/util/cache.cc File be/src/util/cache.cc: http://gerrit.cloudera.org:8080/#/c/12683/1/be/src/util/cache.cc@81 PS1, Line 81: madvise(addr, buffer_size_, MADV_DONTDUMP) Should we also expose some configuration flags that allow the user to specify madvise options related to read ahead (MADV_NORMAL vs MADV_SEQUENTIAL vs MADV_RANDOM vs MADV_WILLNEED)? Even if we don't want to expose this config publicly in the final version, it might be useful in a prototype to allow experimenting with different kernel readahead policies. -- To view, visit http://gerrit.cloudera.org:8080/12683 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic312b0f7ac7875e00a3855ef21dce5b8a9aa67c5 Gerrit-Change-Number: 12683 Gerrit-PatchSet: 1 Gerrit-Owner: Philip Zeyliger Gerrit-Reviewer: Anonymous Coward (474) Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Philip Zeyliger Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 13 Mar 2019 21:46:39 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-337: Add support for POST to the webserver
Joe McDonnell has posted comments on this change. (
http://gerrit.cloudera.org:8080/12730 )
Change subject: IMPALA-337: Add support for POST to the webserver
..
Patch Set 2:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/12730/2/be/src/util/webserver.cc
File be/src/util/webserver.cc:
http://gerrit.cloudera.org:8080/#/c/12730/2/be/src/util/webserver.cc@450
PS2, Line 450: // TODO(wdb): for this and other HTTP requests, we should
log the
> What does wdb mean here?
Good point, this comment, which was copied from Kudu, doesn't make sense here.
wdb is Will Berkeley, a developer on Kudu. I will remove that part of the
message.
http://gerrit.cloudera.org:8080/#/c/12730/2/tests/custom_cluster/test_web_pages.py
File tests/custom_cluster/test_web_pages.py:
http://gerrit.cloudera.org:8080/#/c/12730/2/tests/custom_cluster/test_web_pages.py@49
PS2, Line 49: response =
requests.post("http://localhost:25000/pprof/symbol;, post_content)
> Is it possible at this stage to have a test that shows a successful POST?
Added a test that has a POST within the limits. However, none of our callbacks
make use of the request type yet. It treats a POST just like a GET.
--
To view, visit http://gerrit.cloudera.org:8080/12730
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3bd3ecfab53263c13a4509c278569302c9901abb
Gerrit-Change-Number: 12730
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell
Gerrit-Reviewer: Andrew Sherman
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Comment-Date: Wed, 13 Mar 2019 21:37:11 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-337: Add support for POST to the webserver
Hello Andrew Sherman, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12730 to look at the new patch set (#4). Change subject: IMPALA-337: Add support for POST to the webserver .. IMPALA-337: Add support for POST to the webserver In order to support the TraceEvent path handlers from Kudu (and other functionality like pprof symbolization), the webserver needs to support HTTP POST. This changes the signature for HTTP handler callbacks to take a 'WebRequest' struct instead of a simple map of arguments. This struct exposes the request type, POST data, and unparsed query strings, enabling the callbacks to handle POSTs. The struct itself is imported from Kudu's webserver code, so we can start to share utility HTTP handlers (eg pprof or tracing) more easily between Kudu and Impala. The maximum length of a POST request is limited by the new webserver_max_post_length_bytes parameter, which defaults to 1MB. This patch was written by Todd Lipcon, based on code from Kudu. Testing: - Added a test for too big POSTs to custom_cluster/test_web_pages.py - Verifies the existing core tests pass Change-Id: I3bd3ecfab53263c13a4509c278569302c9901abb --- M be/src/catalog/catalog-server.cc M be/src/catalog/catalog-server.h M be/src/rpc/rpc-trace.cc M be/src/service/impala-http-handler.cc M be/src/service/impala-http-handler.h M be/src/statestore/statestore.cc M be/src/statestore/statestore.h M be/src/util/default-path-handlers.cc M be/src/util/default-path-handlers.h M be/src/util/logging-support.cc M be/src/util/metrics.cc M be/src/util/metrics.h M be/src/util/pprof-path-handlers.cc M be/src/util/thread.cc M be/src/util/webserver-test.cc M be/src/util/webserver.cc M be/src/util/webserver.h M tests/custom_cluster/test_web_pages.py 18 files changed, 251 insertions(+), 147 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/12730/4 -- To view, visit http://gerrit.cloudera.org:8080/12730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3bd3ecfab53263c13a4509c278569302c9901abb Gerrit-Change-Number: 12730 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] IMPALA-337: Add support for POST to the webserver
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/12730 ) Change subject: IMPALA-337: Add support for POST to the webserver .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/12730/3/tests/custom_cluster/test_web_pages.py File tests/custom_cluster/test_web_pages.py: http://gerrit.cloudera.org:8080/#/c/12730/3/tests/custom_cluster/test_web_pages.py@56 PS3, Line 56: ; > flake8: E703 statement ends with a semicolon Done -- To view, visit http://gerrit.cloudera.org:8080/12730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3bd3ecfab53263c13a4509c278569302c9901abb Gerrit-Change-Number: 12730 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 13 Mar 2019 21:40:03 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-337: Add support for POST to the webserver
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12730 ) Change subject: IMPALA-337: Add support for POST to the webserver .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/12730/3/tests/custom_cluster/test_web_pages.py File tests/custom_cluster/test_web_pages.py: http://gerrit.cloudera.org:8080/#/c/12730/3/tests/custom_cluster/test_web_pages.py@56 PS3, Line 56: ; flake8: E703 statement ends with a semicolon -- To view, visit http://gerrit.cloudera.org:8080/12730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3bd3ecfab53263c13a4509c278569302c9901abb Gerrit-Change-Number: 12730 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 13 Mar 2019 21:38:43 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-337: Add support for POST to the webserver
Hello Andrew Sherman, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12730 to look at the new patch set (#3). Change subject: IMPALA-337: Add support for POST to the webserver .. IMPALA-337: Add support for POST to the webserver In order to support the TraceEvent path handlers from Kudu (and other functionality like pprof symbolization), the webserver needs to support HTTP POST. This changes the signature for HTTP handler callbacks to take a 'WebRequest' struct instead of a simple map of arguments. This struct exposes the request type, POST data, and unparsed query strings, enabling the callbacks to handle POSTs. The struct itself is imported from Kudu's webserver code, so we can start to share utility HTTP handlers (eg pprof or tracing) more easily between Kudu and Impala. The maximum length of a POST request is limited by the new webserver_max_post_length_bytes parameter, which defaults to 1MB. This patch was written by Todd Lipcon, based on code from Kudu. Testing: - Added a test for too big POSTs to custom_cluster/test_web_pages.py - Verifies the existing core tests pass Change-Id: I3bd3ecfab53263c13a4509c278569302c9901abb --- M be/src/catalog/catalog-server.cc M be/src/catalog/catalog-server.h M be/src/rpc/rpc-trace.cc M be/src/service/impala-http-handler.cc M be/src/service/impala-http-handler.h M be/src/statestore/statestore.cc M be/src/statestore/statestore.h M be/src/util/default-path-handlers.cc M be/src/util/default-path-handlers.h M be/src/util/logging-support.cc M be/src/util/metrics.cc M be/src/util/metrics.h M be/src/util/pprof-path-handlers.cc M be/src/util/thread.cc M be/src/util/webserver-test.cc M be/src/util/webserver.cc M be/src/util/webserver.h M tests/custom_cluster/test_web_pages.py 18 files changed, 251 insertions(+), 147 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/30/12730/3 -- To view, visit http://gerrit.cloudera.org:8080/12730 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3bd3ecfab53263c13a4509c278569302c9901abb Gerrit-Change-Number: 12730 Gerrit-PatchSet: 3 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] [DOCS] Event based HMS sync works for Metadata V1 and V2
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12745 ) Change subject: [DOCS] Event based HMS sync works for Metadata V1 and V2 .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/273/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/12745 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I366add1016d8dad876f100a0b27ba3051e2cafbb Gerrit-Change-Number: 12745 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 21:34:53 + Gerrit-HasComments: No
[Impala-ASF-CR] [DOCS] Event based HMS sync works for Metadata V1 and V2
Alex Rodoni has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12745 Change subject: [DOCS] Event based HMS sync works for Metadata V1 and V2 .. [DOCS] Event based HMS sync works for Metadata V1 and V2 - Move the Metadata V2 note so that it does not apply to event based sync. Change-Id: I366add1016d8dad876f100a0b27ba3051e2cafbb --- M docs/topics/impala_metadata.xml 1 file changed, 8 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/45/12745/1 -- To view, visit http://gerrit.cloudera.org:8080/12745 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I366add1016d8dad876f100a0b27ba3051e2cafbb Gerrit-Change-Number: 12745 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni
[Impala-ASF-CR] IMPALA-7718: [DOCS] Additional info in the extended EXPLAIN output
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12732 ) Change subject: IMPALA-7718: [DOCS] Additional info in the extended EXPLAIN output .. IMPALA-7718: [DOCS] Additional info in the extended EXPLAIN output Change-Id: I0ad5794d8a9b62cc7d01d023f56e700dc018f24b Reviewed-on: http://gerrit.cloudera.org:8080/12732 Tested-by: Impala Public Jenkins Reviewed-by: Paul Rogers --- M docs/topics/impala_explain.xml M docs/topics/impala_explain_level.xml 2 files changed, 81 insertions(+), 103 deletions(-) Approvals: Impala Public Jenkins: Verified Paul Rogers: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/12732 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I0ad5794d8a9b62cc7d01d023f56e700dc018f24b Gerrit-Change-Number: 12732 Gerrit-PatchSet: 5 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers
[Impala-ASF-CR] IMPALA-7718: [DOCS] Additional info in the extended EXPLAIN output
Paul Rogers has posted comments on this change. ( http://gerrit.cloudera.org:8080/12732 ) Change subject: IMPALA-7718: [DOCS] Additional info in the extended EXPLAIN output .. Patch Set 4: Code-Review+2 LGTM -- To view, visit http://gerrit.cloudera.org:8080/12732 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I0ad5794d8a9b62cc7d01d023f56e700dc018f24b Gerrit-Change-Number: 12732 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Paul Rogers Gerrit-Comment-Date: Wed, 13 Mar 2019 21:25:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/12734 ) Change subject: IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync .. Patch Set 3: Code-Review+1 Thanks for making the suggested changes. -- To view, visit http://gerrit.cloudera.org:8080/12734 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2091d8b6afe1e75d78ce542e76351310562d81ac Gerrit-Change-Number: 12734 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 13 Mar 2019 20:54:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12734 ) Change subject: IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync .. IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync Change-Id: I2091d8b6afe1e75d78ce542e76351310562d81ac Reviewed-on: http://gerrit.cloudera.org:8080/12734 Tested-by: Impala Public Jenkins Reviewed-by: Vihang Karajgaonkar Reviewed-by: Alex Rodoni --- M docs/topics/impala_metadata.xml 1 file changed, 24 insertions(+), 0 deletions(-) Approvals: Impala Public Jenkins: Verified Vihang Karajgaonkar: Looks good to me, but someone else must approve Alex Rodoni: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/12734 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I2091d8b6afe1e75d78ce542e76351310562d81ac Gerrit-Change-Number: 12734 Gerrit-PatchSet: 4 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/12734 ) Change subject: IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12734 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2091d8b6afe1e75d78ce542e76351310562d81ac Gerrit-Change-Number: 12734 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 13 Mar 2019 21:01:56 + Gerrit-HasComments: No
[Impala-ASF-CR] Use test -x to check for ntp-wait
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/12726 ) Change subject: Use test -x to check for ntp-wait .. Patch Set 1: > > This looks good to me. Quick question: Is this a cosmetic issue > or > > is there an actual failure case? I'm wondering if we want a JIRA > > for this. If it is cosmetic, then I think we can skip the JIRA. > > There's a failure case: > > ntp-wait runs `/usr/sbin/ntpq -c "rv 0"` regardless of whether > --help is used or not.. Its exit status depends on the output of > ntpq. > > So for example: > [root@8280aa22322d /]# ntp-wait --help ; echo $? > /usr/sbin/ntp-wait version [unknown] calling Getopt::Std::getopts > (version 1.07 [paranoid]), > running under Perl version 5.16.3. > > Usage: ntp-wait [-OPTIONS [-MORE_OPTIONS]] [--] [PROGRAM_ARG1 ...] > > The following single-character options are accepted: > With arguments: -n -s > Boolean (without arguments): -v > > Options may be merged together. -- stops processing of options. > Space is not required between options and their arguments. > [Now continuing due to backward compatibility and excessive > paranoia. > See 'perldoc Getopt::Std' about $Getopt::Std::STANDARD_HELP_VERSION.] > 1 > [root@8280aa22322d /]# ntpd > [root@8280aa22322d /]# ntp-wait --help ; echo $? > /usr/sbin/ntp-wait version [unknown] calling Getopt::Std::getopts > (version 1.07 [paranoid]), > running under Perl version 5.16.3. > > Usage: ntp-wait [-OPTIONS [-MORE_OPTIONS]] [--] [PROGRAM_ARG1 ...] > > The following single-character options are accepted: > With arguments: -n -s > Boolean (without arguments): -v > > Options may be merged together. -- stops processing of options. > Space is not required between options and their arguments. > [Now continuing due to backward compatibility and excessive > paranoia. > See 'perldoc Getopt::Std' about $Getopt::Std::STANDARD_HELP_VERSION.] > 0 > > > It also looks like --help is simply ignored. Thanks! Given there is a failure case, can you file an IMPALA jira that ntp-wait --help does not always return 0? Then, update the commit message to reference that JIRA. Please include in the commit message your conclusion that ntp-wait --help still runs ntpq and can have a non-zero return code. Good catch! -- To view, visit http://gerrit.cloudera.org:8080/12726 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I53c63dfa651ac242050171da70540d24c4caf32c Gerrit-Change-Number: 12726 Gerrit-PatchSet: 1 Gerrit-Owner: Hector Acosta Gerrit-Reviewer: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 13 Mar 2019 20:53:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12734 ) Change subject: IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync .. Patch Set 3: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/272/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/12734 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2091d8b6afe1e75d78ce542e76351310562d81ac Gerrit-Change-Number: 12734 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 13 Mar 2019 20:48:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/12734 ) Change subject: IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/12734/2/docs/topics/impala_metadata.xml File docs/topics/impala_metadata.xml: http://gerrit.cloudera.org:8080/#/c/12734/2/docs/topics/impala_metadata.xml@207 PS2, Line 207: Hive metastore service file. > nit: change to "of Hive metastore service". i think the word "file" is red Done -- To view, visit http://gerrit.cloudera.org:8080/12734 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2091d8b6afe1e75d78ce542e76351310562d81ac Gerrit-Change-Number: 12734 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 13 Mar 2019 20:27:44 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12734 ) Change subject: IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync .. Patch Set 3: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/272/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/12734 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2091d8b6afe1e75d78ce542e76351310562d81ac Gerrit-Change-Number: 12734 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 13 Mar 2019 20:27:43 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync
Hello Bharath Vissapragada, Vihang Karajgaonkar, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12734 to look at the new patch set (#3). Change subject: IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync .. IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync Change-Id: I2091d8b6afe1e75d78ce542e76351310562d81ac --- M docs/topics/impala_metadata.xml 1 file changed, 24 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/12734/3 -- To view, visit http://gerrit.cloudera.org:8080/12734 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2091d8b6afe1e75d78ce542e76351310562d81ac Gerrit-Change-Number: 12734 Gerrit-PatchSet: 3 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] Use test -x to check for ntp-wait
Hector Acosta has posted comments on this change. ( http://gerrit.cloudera.org:8080/12726 ) Change subject: Use test -x to check for ntp-wait .. Patch Set 1: > This looks good to me. Quick question: Is this a cosmetic issue or > is there an actual failure case? I'm wondering if we want a JIRA > for this. If it is cosmetic, then I think we can skip the JIRA. There's a failure case: ntp-wait runs `/usr/sbin/ntpq -c "rv 0"` regardless of whether --help is used or not.. Its exit status depends on the output of ntpq. So for example: [root@8280aa22322d /]# ntp-wait --help ; echo $? /usr/sbin/ntp-wait version [unknown] calling Getopt::Std::getopts (version 1.07 [paranoid]), running under Perl version 5.16.3. Usage: ntp-wait [-OPTIONS [-MORE_OPTIONS]] [--] [PROGRAM_ARG1 ...] The following single-character options are accepted: With arguments: -n -s Boolean (without arguments): -v Options may be merged together. -- stops processing of options. Space is not required between options and their arguments. [Now continuing due to backward compatibility and excessive paranoia. See 'perldoc Getopt::Std' about $Getopt::Std::STANDARD_HELP_VERSION.] 1 [root@8280aa22322d /]# ntpd [root@8280aa22322d /]# ntp-wait --help ; echo $? /usr/sbin/ntp-wait version [unknown] calling Getopt::Std::getopts (version 1.07 [paranoid]), running under Perl version 5.16.3. Usage: ntp-wait [-OPTIONS [-MORE_OPTIONS]] [--] [PROGRAM_ARG1 ...] The following single-character options are accepted: With arguments: -n -s Boolean (without arguments): -v Options may be merged together. -- stops processing of options. Space is not required between options and their arguments. [Now continuing due to backward compatibility and excessive paranoia. See 'perldoc Getopt::Std' about $Getopt::Std::STANDARD_HELP_VERSION.] 0 It also looks like --help is simply ignored. -- To view, visit http://gerrit.cloudera.org:8080/12726 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I53c63dfa651ac242050171da70540d24c4caf32c Gerrit-Change-Number: 12726 Gerrit-PatchSet: 1 Gerrit-Owner: Hector Acosta Gerrit-Reviewer: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 13 Mar 2019 20:05:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12734 ) Change subject: IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/271/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/12734 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2091d8b6afe1e75d78ce542e76351310562d81ac Gerrit-Change-Number: 12734 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 13 Mar 2019 18:57:42 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/12734 ) Change subject: IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync .. Patch Set 2: (1 comment) Rest looks good to me. Just one minor correction below. http://gerrit.cloudera.org:8080/#/c/12734/2/docs/topics/impala_metadata.xml File docs/topics/impala_metadata.xml: http://gerrit.cloudera.org:8080/#/c/12734/2/docs/topics/impala_metadata.xml@207 PS2, Line 207: Hive metastore service file. nit: change to "of Hive metastore service". i think the word "file" is redundant and can be removed. -- To view, visit http://gerrit.cloudera.org:8080/12734 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2091d8b6afe1e75d78ce542e76351310562d81ac Gerrit-Change-Number: 12734 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 13 Mar 2019 18:48:24 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12734 ) Change subject: IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/271/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/12734 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2091d8b6afe1e75d78ce542e76351310562d81ac Gerrit-Change-Number: 12734 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 13 Mar 2019 18:41:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/12734 ) Change subject: IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/12734/1/docs/topics/impala_metadata.xml File docs/topics/impala_metadata.xml: http://gerrit.cloudera.org:8080/#/c/12734/1/docs/topics/impala_metadata.xml@207 PS1, Line 207: file. > suggest you to add "of Hive metastore service" here to make it clearer sinc Done -- To view, visit http://gerrit.cloudera.org:8080/12734 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2091d8b6afe1e75d78ce542e76351310562d81ac Gerrit-Change-Number: 12734 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 13 Mar 2019 18:41:22 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync
Hello Bharath Vissapragada, Vihang Karajgaonkar, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/12734 to look at the new patch set (#2). Change subject: IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync .. IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync Change-Id: I2091d8b6afe1e75d78ce542e76351310562d81ac --- M docs/topics/impala_metadata.xml 1 file changed, 24 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/34/12734/2 -- To view, visit http://gerrit.cloudera.org:8080/12734 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I2091d8b6afe1e75d78ce542e76351310562d81ac Gerrit-Change-Number: 12734 Gerrit-PatchSet: 2 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar
[Impala-ASF-CR] IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync
Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/12734 ) Change subject: IMPALA-7974: [DOCS] Phase 2: Configure Hive for event-based HMS sync .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/12734/1/docs/topics/impala_metadata.xml File docs/topics/impala_metadata.xml: http://gerrit.cloudera.org:8080/#/c/12734/1/docs/topics/impala_metadata.xml@207 PS1, Line 207: file. suggest you to add "of Hive metastore service" here to make it clearer since there are can be many hive-site.xml files http://gerrit.cloudera.org:8080/#/c/12734/1/docs/topics/impala_metadata.xml@213 PS1, Line 213: /property> > Question for Vihang: The above to configurations are only available in CDH Hive. We plan to make those changes to upstream but it will be available in Hive 3.2 onwards. Hence I think its okay to skip it as long as we document these configurations to the CDH documentation. If users are using Apache Hive 2.1.1 (open-source version of Apache Hive) the only configuration which is needed is hive.metastore.transactional.event.listeners. -- To view, visit http://gerrit.cloudera.org:8080/12734 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2091d8b6afe1e75d78ce542e76351310562d81ac Gerrit-Change-Number: 12734 Gerrit-PatchSet: 1 Gerrit-Owner: Alex Rodoni Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Wed, 13 Mar 2019 18:35:11 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8303: Generate JUnitXML for ASAN failures
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12733 ) Change subject: IMPALA-8303: Generate JUnitXML for ASAN failures .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3905/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/12733 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5434703a02617d62dc0ed401b8cc1bf7a198a84d Gerrit-Change-Number: 12733 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 18:28:50 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8303: Generate JUnitXML for ASAN failures
David Knupp has posted comments on this change. ( http://gerrit.cloudera.org:8080/12733 ) Change subject: IMPALA-8303: Generate JUnitXML for ASAN failures .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12733 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5434703a02617d62dc0ed401b8cc1bf7a198a84d Gerrit-Change-Number: 12733 Gerrit-PatchSet: 1 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: David Knupp Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 18:25:04 + Gerrit-HasComments: No
[Impala-ASF-CR] Move DEBIAN FRONTEND=noninteractive variable to the ubuntu section of the script
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/12723 ) Change subject: Move DEBIAN_FRONTEND=noninteractive variable to the ubuntu section of the script .. Patch Set 1: (1 comment) Let me know what you think. If we go with always using DEBIAN_FRONTEND=noninteractive, then please note that in the commit message. http://gerrit.cloudera.org:8080/#/c/12723/1/bin/bootstrap_system.sh File bin/bootstrap_system.sh: http://gerrit.cloudera.org:8080/#/c/12723/1/bin/bootstrap_system.sh@101 PS1, Line 101: export DEBIAN_FRONTEND=noninteractive This is not quite identical. Previously, we were setting this if we are not on an interactive terminal. Now we are always setting it. I think this is probably a good thing to use DEBIAN_FRONTEND=noninteractive always. I haven't run this recently, but I think maybe kerberos asks about certain settings. I think it is better for this script to avoid needing manual interaction beyond the first prompt. If we wanted to maintain the old way, we could check "if [[ ! -t 1 ]]" or whatnot. -- To view, visit http://gerrit.cloudera.org:8080/12723 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ifab2f56d3e119e4e73685c3cbd5249760ceb33ac Gerrit-Change-Number: 12723 Gerrit-PatchSet: 1 Gerrit-Owner: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 13 Mar 2019 18:15:52 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Set correct permissions for ~/.ssh files
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/12725 ) Change subject: Set correct permissions for ~/.ssh files .. Patch Set 1: (2 comments) Thanks for working on this! For this one, can you file a JIRA? This sounds like something that other people could hit. http://gerrit.cloudera.org:8080/#/c/12725/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12725/1//COMMIT_MSG@9 PS1, Line 9: ssdh sshd? http://gerrit.cloudera.org:8080/#/c/12725/1//COMMIT_MSG@10 PS1, Line 10: Authentication refused: bad ownership or modes for file $HOME/.ssh/authorized_keys Can you include what operation would fail? Is this happening when you are connecting to the development machine using a public key? What works with this change that didn't before? -- To view, visit http://gerrit.cloudera.org:8080/12725 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3dd6b12318dd99c67127bff10a89adb931f44087 Gerrit-Change-Number: 12725 Gerrit-PatchSet: 1 Gerrit-Owner: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 13 Mar 2019 18:05:30 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8298: [DOCS] Update docs for complex types support in ORC
Alex Rodoni has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12744 ) Change subject: IMPALA-8298: [DOCS] Update docs for complex types support in ORC .. IMPALA-8298: [DOCS] Update docs for complex types support in ORC Change-Id: I261d41f536d653033cc1083873edf8c83508fdb7 Reviewed-on: http://gerrit.cloudera.org:8080/12744 Tested-by: Impala Public Jenkins Reviewed-by: Alex Rodoni --- M docs/shared/impala_common.xml M docs/topics/impala_complex_types.xml M docs/topics/impala_orc.xml M docs/topics/impala_parquet.xml 4 files changed, 68 insertions(+), 88 deletions(-) Approvals: Impala Public Jenkins: Verified Alex Rodoni: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/12744 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I261d41f536d653033cc1083873edf8c83508fdb7 Gerrit-Change-Number: 12744 Gerrit-PatchSet: 3 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8298: [DOCS] Update docs for complex types support in ORC
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/12744 ) Change subject: IMPALA-8298: [DOCS] Update docs for complex types support in ORC .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12744 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I261d41f536d653033cc1083873edf8c83508fdb7 Gerrit-Change-Number: 12744 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 18:04:14 + Gerrit-HasComments: No
[Impala-ASF-CR] Use test -x to check for ntp-wait
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/12726 ) Change subject: Use test -x to check for ntp-wait .. Patch Set 1: Code-Review+1 This looks good to me. Quick question: Is this a cosmetic issue or is there an actual failure case? I'm wondering if we want a JIRA for this. If it is cosmetic, then I think we can skip the JIRA. -- To view, visit http://gerrit.cloudera.org:8080/12726 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I53c63dfa651ac242050171da70540d24c4caf32c Gerrit-Change-Number: 12726 Gerrit-PatchSet: 1 Gerrit-Owner: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Wed, 13 Mar 2019 17:48:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8298: [DOCS] Update docs for complex types support in ORC
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12744 ) Change subject: IMPALA-8298: [DOCS] Update docs for complex types support in ORC .. Patch Set 2: Verified+1 Build Successful https://jenkins.impala.io/job/gerrit-docs-auto-test/270/ : Doc tests passed. -- To view, visit http://gerrit.cloudera.org:8080/12744 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I261d41f536d653033cc1083873edf8c83508fdb7 Gerrit-Change-Number: 12744 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 17:51:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8298: [DOCS] Update docs of complex types about ORC support
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/12744 ) Change subject: IMPALA-8298: [DOCS] Update docs of complex types about ORC support .. Patch Set 1: (4 comments) http://gerrit.cloudera.org:8080/#/c/12744/1/docs/topics/impala_complex_types.xml File docs/topics/impala_complex_types.xml: http://gerrit.cloudera.org:8080/#/c/12744/1/docs/topics/impala_complex_types.xml@347 PS1, Line 347: or and http://gerrit.cloudera.org:8080/#/c/12744/1/docs/topics/impala_complex_types.xml@1671 PS1, Line 1671: data > or ORC data Ignore this comment http://gerrit.cloudera.org:8080/#/c/12744/1/docs/topics/impala_complex_types.xml@1674 PS1, Line 1674: the Parquet or ORC http://gerrit.cloudera.org:8080/#/c/12744/1/docs/topics/impala_complex_types.xml@1675 PS1, Line 1675: The same for ORC file format. > Remove this after adding ORC at line 1671. Ignore this comment -- To view, visit http://gerrit.cloudera.org:8080/12744 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I261d41f536d653033cc1083873edf8c83508fdb7 Gerrit-Change-Number: 12744 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 17:31:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8298: [DOCS] Update docs for complex types support in ORC
Alex Rodoni has uploaded a new patch set (#2) to the change originally created by Quanlong Huang. ( http://gerrit.cloudera.org:8080/12744 ) Change subject: IMPALA-8298: [DOCS] Update docs for complex types support in ORC .. IMPALA-8298: [DOCS] Update docs for complex types support in ORC Change-Id: I261d41f536d653033cc1083873edf8c83508fdb7 --- M docs/shared/impala_common.xml M docs/topics/impala_complex_types.xml M docs/topics/impala_orc.xml M docs/topics/impala_parquet.xml 4 files changed, 68 insertions(+), 88 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/44/12744/2 -- To view, visit http://gerrit.cloudera.org:8080/12744 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I261d41f536d653033cc1083873edf8c83508fdb7 Gerrit-Change-Number: 12744 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8298: [DOCS] Update docs for complex types support in ORC
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12744 ) Change subject: IMPALA-8298: [DOCS] Update docs for complex types support in ORC .. Patch Set 2: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/270/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/12744 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I261d41f536d653033cc1083873edf8c83508fdb7 Gerrit-Change-Number: 12744 Gerrit-PatchSet: 2 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 17:34:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8298: [DOCS] Update docs of complex types about ORC support
Alex Rodoni has posted comments on this change. ( http://gerrit.cloudera.org:8080/12744 ) Change subject: IMPALA-8298: [DOCS] Update docs of complex types about ORC support .. Patch Set 1: (5 comments) http://gerrit.cloudera.org:8080/#/c/12744/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/12744/1//COMMIT_MSG@7 PS1, Line 7: of complex types about ORC support for complex type support in ORC http://gerrit.cloudera.org:8080/#/c/12744/1/docs/shared/impala_common.xml File docs/shared/impala_common.xml: http://gerrit.cloudera.org:8080/#/c/12744/1/docs/shared/impala_common.xml@2181 PS1, Line 2181: impala_complex_types.xml ../topics/impala_complex_types.xml http://gerrit.cloudera.org:8080/#/c/12744/1/docs/topics/impala_complex_types.xml File docs/topics/impala_complex_types.xml: http://gerrit.cloudera.org:8080/#/c/12744/1/docs/topics/impala_complex_types.xml@347 PS1, Line 347: format formats http://gerrit.cloudera.org:8080/#/c/12744/1/docs/topics/impala_complex_types.xml@1671 PS1, Line 1671: data or ORC data http://gerrit.cloudera.org:8080/#/c/12744/1/docs/topics/impala_complex_types.xml@1675 PS1, Line 1675: The same for ORC file format. Remove this after adding ORC at line 1671. -- To view, visit http://gerrit.cloudera.org:8080/12744 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I261d41f536d653033cc1083873edf8c83508fdb7 Gerrit-Change-Number: 12744 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 17:00:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Add missing sudo call to `service postgresql start`
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/12724 ) Change subject: Add missing sudo call to `service postgresql start` .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/12724 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie3859ad1308b7cd1d483d043b9f71d9f95abdf75 Gerrit-Change-Number: 12724 Gerrit-PatchSet: 1 Gerrit-Owner: Hector Acosta Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Laszlo Gaal Gerrit-Comment-Date: Wed, 13 Mar 2019 16:08:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8298: [DOCS] Update docs of complex types about ORC support
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12744 ) Change subject: IMPALA-8298: [DOCS] Update docs of complex types about ORC support .. Patch Set 1: Verified-1 Build Failed https://jenkins.impala.io/job/gerrit-docs-auto-test/269/ : Doc tests failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/12744 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I261d41f536d653033cc1083873edf8c83508fdb7 Gerrit-Change-Number: 12744 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Alex Rodoni Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 15:32:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8298: [DOCS] Update docs of complex types about ORC support
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12744 ) Change subject: IMPALA-8298: [DOCS] Update docs of complex types about ORC support .. Patch Set 1: Build Started https://jenkins.impala.io/job/gerrit-docs-auto-test/269/ Testing docs change - this change appears to modify docs/ and no code. This is experimental - please report any issues to tarmstr...@cloudera.com or on this JIRA: IMPALA-7317 -- To view, visit http://gerrit.cloudera.org:8080/12744 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I261d41f536d653033cc1083873edf8c83508fdb7 Gerrit-Change-Number: 12744 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 13 Mar 2019 15:14:31 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8298: [DOCS] Update docs of complex types about ORC support
Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/12744 Change subject: IMPALA-8298: [DOCS] Update docs of complex types about ORC support .. IMPALA-8298: [DOCS] Update docs of complex types about ORC support Change-Id: I261d41f536d653033cc1083873edf8c83508fdb7 --- M docs/shared/impala_common.xml M docs/topics/impala_complex_types.xml M docs/topics/impala_orc.xml M docs/topics/impala_parquet.xml 4 files changed, 44 insertions(+), 71 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/44/12744/1 -- To view, visit http://gerrit.cloudera.org:8080/12744 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I261d41f536d653033cc1083873edf8c83508fdb7 Gerrit-Change-Number: 12744 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang
[Impala-ASF-CR] IMPALA-7368: Add initial support for DATE type
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12481 ) Change subject: IMPALA-7368: Add initial support for DATE type .. Patch Set 11: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/2423/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12481 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iea8155ef09557e0afa2f8b2d0b2dc9d0896dc30f Gerrit-Change-Number: 12481 Gerrit-PatchSet: 11 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 13 Mar 2019 13:53:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7368: Add initial support for DATE type
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/12481 ) Change subject: IMPALA-7368: Add initial support for DATE type .. Patch Set 11: > Uploaded patch set 11. Rebased it. -- To view, visit http://gerrit.cloudera.org:8080/12481 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iea8155ef09557e0afa2f8b2d0b2dc9d0896dc30f Gerrit-Change-Number: 12481 Gerrit-PatchSet: 11 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 13 Mar 2019 13:07:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-7368: Add initial support for DATE type
Attila Jeges has uploaded a new patch set (#11). (
http://gerrit.cloudera.org:8080/12481 )
Change subject: IMPALA-7368: Add initial support for DATE type
..
IMPALA-7368: Add initial support for DATE type
DATE values describe a particular year/month/day in the form
-MM-dd. For example: DATE '2019-02-15'. DATE values do not have a
time of day component. The range of values supported for the DATE type
is -01-01 to -12-31.
This initial DATE type support covers TEXT and HBASE fileformats only.
'DateValue' is used as the internal type to represent DATE values.
The changes are as follows:
- Support for DATE literal syntax.
- Explicit casting between DATE and other types:
- from STRING to DATE. The string value must be formatted as
-MM-dd.
- from DATE to STRING. The resulting string value is formatted as
-MM-dd.
- from TIMESTAMP to DATE. The source timestamp's time of day
component is ignored.
- from DATE to TIMESTAMP. The target timestamp's time of day
component is set to 00:00:00.
- Implicit casting between DATE and other types:
- from STRING to DATE if the source string value is used in a
context where a DATE value is expected.
- from DATE to TIMESTAMP if the source date value is used in a
context where a TIMESTAMP value is expected.
- Since both STRING -> DATE and STRING -> TIMESTAMP implicit
conversions are possible, the function resolution logic was changed
to select the right version of overloaded functions:
- If both implicit conversions are applicable equally well,
STRING -> TIMESTAMP is preferred for backward compatibility
reasons. E.g: year('2019-02-15') must resolve to
year(TIMESTAMP) instead of year(DATE). Note, that year(DATE) is
not implemented yet, so this is not an issue at the moment but
it will be in the future.
- If one implicit conversion is a better fit over the other, we
must choose the better one. E.g:
if(false, '2011-01-01', DATE '1499-02-02') must resolve to
if(BOOLEAN, DATE, DATE) instead of
if(BOOLEAN, TIMESTAMP, TIMESTAMP).
- Codegen infrastructure changes for expression evaluation.
- 'IS [NOT] NULL' and '[NOT] IN' predicates.
- Common comparison operators (including the 'BETWEEN' operator).
- Infrastructure changes for built-in functions.
- Some built-in functions: conditional, aggregate, analytical and
math functions.
- C++ UDF/UDA support.
- Support partitioning and grouping by DATE.
- Beeswax, HiveServer2 support.
These items are tightly coupled and it makes sense to implement them
in one change-set.
Testing:
- A new partitioned TEXT table 'functional.date_tbl' (and the
corresponding HBASE table 'functional_hbase.date_tbl') was
introduced for DATE-related tests.
- BE and FE tests were extended to cover DATE type.
- E2E tests:
- since DATE type is supported for TEXT and HBASE fileformats
only, most DATE tests were implemented separately in
tests/query_test/test_date_queries.py.
Note, that this change-set is not a complete DATE type implementation,
but it lays the foundation for future work:
- Add date support to the random query generator.
- Implement a complete set of built-in functions.
- Add Parquet support.
- Add Kudu support.
- Optionally support Avro and ORC.
For further details, see IMPALA-6169.
Change-Id: Iea8155ef09557e0afa2f8b2d0b2dc9d0896dc30f
---
M be/src/codegen/codegen-anyval.cc
M be/src/codegen/codegen-anyval.h
M be/src/codegen/gen_ir_descriptions.py
M be/src/codegen/llvm-codegen.cc
M be/src/exec/aggregator.cc
M be/src/exec/data-source-scan-node.cc
M be/src/exec/hash-table.cc
M be/src/exec/hdfs-scanner-ir.cc
M be/src/exec/hdfs-table-sink.cc
M be/src/exec/text-converter.cc
M be/src/exec/text-converter.inline.h
M be/src/exprs/agg-fn-evaluator.cc
M be/src/exprs/aggregate-functions-ir.cc
M be/src/exprs/aggregate-functions.h
M be/src/exprs/anyval-util.cc
M be/src/exprs/anyval-util.h
M be/src/exprs/case-expr.cc
M be/src/exprs/case-expr.h
M be/src/exprs/cast-functions-ir.cc
M be/src/exprs/cast-functions.h
M be/src/exprs/conditional-functions-ir.cc
M be/src/exprs/conditional-functions.h
M be/src/exprs/expr-test.cc
M be/src/exprs/expr-value.h
M be/src/exprs/hive-udf-call.cc
M be/src/exprs/hive-udf-call.h
M be/src/exprs/in-predicate-ir.cc
M be/src/exprs/in-predicate.h
M be/src/exprs/is-null-predicate-ir.cc
M be/src/exprs/literal.cc
M be/src/exprs/literal.h
M be/src/exprs/math-functions-ir.cc
M be/src/exprs/math-functions.h
M be/src/exprs/null-literal.cc
M be/src/exprs/null-literal.h
M be/src/exprs/operators-ir.cc
M be/src/exprs/operators.h
M be/src/exprs/scalar-expr-evaluator.cc
M be/src/exprs/scalar-expr-evaluator.h
M be/src/exprs/scalar-expr-ir.cc
M be/src/exprs/scalar-expr.cc
M be/src/exprs/scalar-expr.h
M be/src/exprs/scalar-fn-call.cc
M be/src/exprs/scalar-fn-call.h
M be/src/exprs/slot-ref.cc
M be/src/exprs/slot-ref.h
M
[Impala-ASF-CR](2.x) IMPALA-6625: Skip computing parquet conjuncts for non-Parquet scans
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12741 ) Change subject: IMPALA-6625: Skip computing parquet conjuncts for non-Parquet scans .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/12741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: comment Gerrit-Change-Id: I9d6c26d42db090c8a15c602f6419ad6399c329e7 Gerrit-Change-Number: 12741 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 13 Mar 2019 10:17:31 + Gerrit-HasComments: No
[Impala-ASF-CR](2.x) IMPALA-6625: Skip computing parquet conjuncts for non-Parquet scans
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12741 ) Change subject: IMPALA-6625: Skip computing parquet conjuncts for non-Parquet scans .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/3904/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/12741 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: 2.x Gerrit-MessageType: comment Gerrit-Change-Id: I9d6c26d42db090c8a15c602f6419ad6399c329e7 Gerrit-Change-Number: 12741 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Pooja Nilangekar Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Wed, 13 Mar 2019 06:28:19 + Gerrit-HasComments: No
