[ https://issues.apache.org/jira/browse/IMPALA-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joe McDonnell resolved IMPALA-4534. ----------------------------------- Fix Version/s: Not Applicable Resolution: Won't Fix > Not all of the data load files follow the accepted format when partitioning > test data tables > -------------------------------------------------------------------------------------------- > > Key: IMPALA-4534 > URL: https://issues.apache.org/jira/browse/IMPALA-4534 > Project: IMPALA > Issue Type: Task > Components: Infrastructure > Affects Versions: Impala 2.8.0 > Reporter: David Knupp > Priority: Major > Labels: test-infra > Fix For: Not Applicable > > > When loading Impala test data, we "generally" partition tables in our data > load process by adding PARTITION_COLUMNS and ALTER sections to the schema > template files, e.g. from functional_schema_template.sql: > {noformat} > ---- DATASET > functional > ---- BASE_TABLE_NAME > alltypessmall > ---- COLUMNS > id int > bool_col boolean > tinyint_col tinyint > smallint_col smallint > int_col int > bigint_col bigint > float_col float > double_col double > date_string_col string > string_col string > timestamp_col timestamp > ---- PARTITION_COLUMNS > year int > month int > ---- ALTER > ALTER TABLE {table_name} ADD IF NOT EXISTS PARTITION(year=2009, month=1); > ALTER TABLE {table_name} ADD IF NOT EXISTS PARTITION(year=2009, month=2); > ALTER TABLE {table_name} ADD IF NOT EXISTS PARTITION(year=2009, month=3); > ALTER TABLE {table_name} ADD IF NOT EXISTS PARTITION(year=2009, month=4); > {noformat} > However, some tables forego this, and combine the PARTITION BY clause with > the CREATE TABLE clause, and they may or may not include an ALTER section. > This sidesteps logic in generate-schema-statements.py that specifically > branches based upon whether the PARTITION_COLUMNS and/or ALTER sections have > been defined. > We should investigate what effect the omission of these sections has on our > data load process for those tables. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org