[jira] [Resolved] (IMPALA-10879) Add parquet stats to iceberg manifest
[ https://issues.apache.org/jira/browse/IMPALA-10879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges resolved IMPALA-10879. --- Resolution: Implemented > Add parquet stats to iceberg manifest > - > > Key: IMPALA-10879 > URL: https://issues.apache.org/jira/browse/IMPALA-10879 > Project: IMPALA > Issue Type: Improvement > Components: Backend, Frontend >Affects Versions: Impala 4.0.0 >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > Labels: impala-iceberg > > Parquet stats should be written to iceberg manifest as per-datafile metrics. > This task is specifically about the following metrics: > - column_sizes : Map from column id to the total size on disk of all regions > that store the column. Does not include bytes necessary to read other > columns, like footers. Leave null for row-oriented formats > - null_value_counts : Map from column id to number of null values in the > column. > - lower_bounds : Map from column id to lower bound in the column serialized > as binary. Each value must be less than or equal to all non-null, non-NaN > values in the column for the file. > - upper_bounds : Map from column id to upper bound in the column serialized > as binary. Each value must be greater than or equal to all non-null, non-Nan > values in the column for the file. > Iceberg manifest doc: > https://iceberg.apache.org/spec/#manifests > lower_bounds and upper_bounds values should be Single-value serialized to > binary: > https://iceberg.apache.org/spec/#appendix-d-single-value-serialization -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10879) Add parquet stats to iceberg manifest
[ https://issues.apache.org/jira/browse/IMPALA-10879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404526#comment-17404526 ] Attila Jeges commented on IMPALA-10879: --- CR: https://gerrit.cloudera.org/#/c/17806/ > Add parquet stats to iceberg manifest > - > > Key: IMPALA-10879 > URL: https://issues.apache.org/jira/browse/IMPALA-10879 > Project: IMPALA > Issue Type: Improvement > Components: Backend, Frontend >Affects Versions: Impala 4.0.0 >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > Labels: impala-iceberg > > Parquet stats should be written to iceberg manifest as per-datafile metrics. > This task is specifically about the following metrics: > - column_sizes : Map from column id to the total size on disk of all regions > that store the column. Does not include bytes necessary to read other > columns, like footers. Leave null for row-oriented formats > - null_value_counts : Map from column id to number of null values in the > column. > - lower_bounds : Map from column id to lower bound in the column serialized > as binary. Each value must be less than or equal to all non-null, non-NaN > values in the column for the file. > - upper_bounds : Map from column id to upper bound in the column serialized > as binary. Each value must be greater than or equal to all non-null, non-Nan > values in the column for the file. > Iceberg manifest doc: > https://iceberg.apache.org/spec/#manifests > lower_bounds and upper_bounds values should be Single-value serialized to > binary: > https://iceberg.apache.org/spec/#appendix-d-single-value-serialization -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-10879) Add parquet stats to iceberg manifest
[ https://issues.apache.org/jira/browse/IMPALA-10879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-10879 started by Attila Jeges. - > Add parquet stats to iceberg manifest > - > > Key: IMPALA-10879 > URL: https://issues.apache.org/jira/browse/IMPALA-10879 > Project: IMPALA > Issue Type: Improvement > Components: Backend, Frontend >Affects Versions: Impala 4.0.0 >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > Labels: impala-iceberg > > Parquet stats should be written to iceberg manifest as per-datafile metrics. > This task is specifically about the following metrics: > - column_sizes : Map from column id to the total size on disk of all regions > that store the column. Does not include bytes necessary to read other > columns, like footers. Leave null for row-oriented formats > - null_value_counts : Map from column id to number of null values in the > column. > - lower_bounds : Map from column id to lower bound in the column serialized > as binary. Each value must be less than or equal to all non-null, non-NaN > values in the column for the file. > - upper_bounds : Map from column id to upper bound in the column serialized > as binary. Each value must be greater than or equal to all non-null, non-Nan > values in the column for the file. > Iceberg manifest doc: > https://iceberg.apache.org/spec/#manifests > lower_bounds and upper_bounds values should be Single-value serialized to > binary: > https://iceberg.apache.org/spec/#appendix-d-single-value-serialization -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10879) Add parquet stats to iceberg manifest
Attila Jeges created IMPALA-10879: - Summary: Add parquet stats to iceberg manifest Key: IMPALA-10879 URL: https://issues.apache.org/jira/browse/IMPALA-10879 Project: IMPALA Issue Type: Improvement Components: Backend, Frontend Affects Versions: Impala 4.0.0 Reporter: Attila Jeges Assignee: Attila Jeges Parquet stats should be written to iceberg manifest as per-datafile metrics. This task is specifically about the following metrics: - column_sizes : Map from column id to the total size on disk of all regions that store the column. Does not include bytes necessary to read other columns, like footers. Leave null for row-oriented formats - null_value_counts : Map from column id to number of null values in the column. - lower_bounds : Map from column id to lower bound in the column serialized as binary. Each value must be less than or equal to all non-null, non-NaN values in the column for the file. - upper_bounds : Map from column id to upper bound in the column serialized as binary. Each value must be greater than or equal to all non-null, non-Nan values in the column for the file. Iceberg manifest doc: https://iceberg.apache.org/spec/#manifests lower_bounds and upper_bounds values should be Single-value serialized to binary: https://iceberg.apache.org/spec/#appendix-d-single-value-serialization -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-8211) Add DATE support to the random query generator
[ https://issues.apache.org/jira/browse/IMPALA-8211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges reassigned IMPALA-8211: Assignee: (was: Attila Jeges) > Add DATE support to the random query generator > -- > > Key: IMPALA-8211 > URL: https://issues.apache.org/jira/browse/IMPALA-8211 > Project: IMPALA > Issue Type: Task > Components: Infrastructure >Reporter: Attila Jeges >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-2269) Allow non-int types for interval expression.
[ https://issues.apache.org/jira/browse/IMPALA-2269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges reassigned IMPALA-2269: Assignee: (was: Attila Jeges) > Allow non-int types for interval expression. > > > Key: IMPALA-2269 > URL: https://issues.apache.org/jira/browse/IMPALA-2269 > Project: IMPALA > Issue Type: Improvement > Components: Frontend >Affects Versions: Impala 2.2 >Reporter: Bharath Vissapragada >Priority: Minor > Labels: usability > > Currently we limit INT types for intervals. We can expand its scope to add > fine grained timestamps. For example, mysql supports the following, [1] > [1] > https://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_date-add -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9498) Allow Collection types in SELECT list for Parquet tables
[
https://issues.apache.org/jira/browse/IMPALA-9498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403623#comment-17403623
]
Attila Jeges commented on IMPALA-9498:
--
[~csringhofer] I'm re-assigning this to you, as my understanding is that you're
taking over this task.
> Allow Collection types in SELECT list for Parquet tables
>
>
> Key: IMPALA-9498
> URL: https://issues.apache.org/jira/browse/IMPALA-9498
> Project: IMPALA
> Issue Type: New Feature
> Components: Backend, Frontend
>Reporter: Gabor Kaszab
>Assignee: Csaba Ringhofer
>Priority: Major
> Labels: complextype
>
> This covers collections: Array, Map
> Expected printout format:
> Array: [null,1,2,null,3,null]
> Map: {"k1":2,"k2":null}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-9498) Allow Collection types in SELECT list for Parquet tables
[
https://issues.apache.org/jira/browse/IMPALA-9498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges reassigned IMPALA-9498:
Assignee: Csaba Ringhofer (was: Attila Jeges)
> Allow Collection types in SELECT list for Parquet tables
>
>
> Key: IMPALA-9498
> URL: https://issues.apache.org/jira/browse/IMPALA-9498
> Project: IMPALA
> Issue Type: New Feature
> Components: Backend, Frontend
>Reporter: Gabor Kaszab
>Assignee: Csaba Ringhofer
>Priority: Major
> Labels: complextype
>
> This covers collections: Array, Map
> Expected printout format:
> Array: [null,1,2,null,3,null]
> Map: {"k1":2,"k2":null}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-8943) When USE_CDP_HIVE=true, Impala should use the CDP version of Kudu
[ https://issues.apache.org/jira/browse/IMPALA-8943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403620#comment-17403620 ] Attila Jeges edited comment on IMPALA-8943 at 8/24/21, 7:58 AM: Resolving the JIRA as "Not A Problem". Feel free to reopen it if necessary. was (Author: attilaj): Resolving the JIRA as "Not A Problem". > When USE_CDP_HIVE=true, Impala should use the CDP version of Kudu > - > > Key: IMPALA-8943 > URL: https://issues.apache.org/jira/browse/IMPALA-8943 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Affects Versions: Impala 3.4.0 >Reporter: Joe McDonnell >Assignee: Attila Jeges >Priority: Minor > > Currently, Impala uses the version of Kudu that comes with the > CDH_BUILD_NUMBER, even when USE_CDP_HIVE=true. This is incorrect. The > USE_CDP_HIVE=true build of Impala should use the Kudu version from the > CDP_BUILD_NUMBER. > To avoid any cross-version issues, this Kudu will need to be built using the > native toolchain. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8943) When USE_CDP_HIVE=true, Impala should use the CDP version of Kudu
[ https://issues.apache.org/jira/browse/IMPALA-8943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403620#comment-17403620 ] Attila Jeges commented on IMPALA-8943: -- Resolving the JIRA as "Not A Problem". > When USE_CDP_HIVE=true, Impala should use the CDP version of Kudu > - > > Key: IMPALA-8943 > URL: https://issues.apache.org/jira/browse/IMPALA-8943 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Affects Versions: Impala 3.4.0 >Reporter: Joe McDonnell >Assignee: Attila Jeges >Priority: Minor > > Currently, Impala uses the version of Kudu that comes with the > CDH_BUILD_NUMBER, even when USE_CDP_HIVE=true. This is incorrect. The > USE_CDP_HIVE=true build of Impala should use the Kudu version from the > CDP_BUILD_NUMBER. > To avoid any cross-version issues, this Kudu will need to be built using the > native toolchain. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8943) When USE_CDP_HIVE=true, Impala should use the CDP version of Kudu
[ https://issues.apache.org/jira/browse/IMPALA-8943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges resolved IMPALA-8943. -- Resolution: Not A Problem > When USE_CDP_HIVE=true, Impala should use the CDP version of Kudu > - > > Key: IMPALA-8943 > URL: https://issues.apache.org/jira/browse/IMPALA-8943 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure >Affects Versions: Impala 3.4.0 >Reporter: Joe McDonnell >Assignee: Attila Jeges >Priority: Minor > > Currently, Impala uses the version of Kudu that comes with the > CDH_BUILD_NUMBER, even when USE_CDP_HIVE=true. This is incorrect. The > USE_CDP_HIVE=true build of Impala should use the Kudu version from the > CDP_BUILD_NUMBER. > To avoid any cross-version issues, this Kudu will need to be built using the > native toolchain. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-10739) Add support for ALTER TABLE tbl SET PARTITION SPEC for Iceberg tables
[
https://issues.apache.org/jira/browse/IMPALA-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges resolved IMPALA-10739.
---
Resolution: Implemented
> Add support for ALTER TABLE tbl SET PARTITION SPEC for Iceberg tables
> -
>
> Key: IMPALA-10739
> URL: https://issues.apache.org/jira/browse/IMPALA-10739
> Project: IMPALA
> Issue Type: New Feature
>Reporter: Zoltán Borók-Nagy
>Assignee: Attila Jeges
>Priority: Major
> Labels: impala-iceberg
>
> Impala should support partition evolution for Iceberg tables, i.e. it should
> be able to set a new partition spec for an Iceberg table via DDL.
> The command should be
> {noformat}
> ALTER TABLE SET PARTITION SPEC()
> {noformat}
> to be aligned with Hive.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-10627) Use standard Iceberg table properties
[ https://issues.apache.org/jira/browse/IMPALA-10627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges resolved IMPALA-10627. --- Resolution: Implemented > Use standard Iceberg table properties > - > > Key: IMPALA-10627 > URL: https://issues.apache.org/jira/browse/IMPALA-10627 > Project: IMPALA > Issue Type: Bug >Reporter: Zoltán Borók-Nagy >Assignee: Attila Jeges >Priority: Major > Labels: impala-iceberg > > Iceberg lists the following properties: > [https://iceberg.apache.org/configuration/] > We should also use these properties if possible, e.g. write.format.default, > write..compression-codec > Currently Impala use the table property 'iceberg.file_format' to determine > the data file format for reads/writes. In the future, read operations should > automatically detect the file formats (IMPALA-10610), but for writes we > should use 'write.format.default'. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-10820) TestInsertWideTable.test_insert_wide_table failed due to file size too small
[
https://issues.apache.org/jira/browse/IMPALA-10820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges resolved IMPALA-10820.
---
Fix Version/s: Impala 4.1
Resolution: Fixed
> TestInsertWideTable.test_insert_wide_table failed due to file size too small
>
>
> Key: IMPALA-10820
> URL: https://issues.apache.org/jira/browse/IMPALA-10820
> Project: IMPALA
> Issue Type: Bug
> Components: Backend, Frontend
>Affects Versions: Impala 4.1
>Reporter: Wenzhe Zhou
>Assignee: Attila Jeges
>Priority: Major
> Labels: broken-build
> Fix For: Impala 4.1
>
> Attachments: error_message_impala_10820.txt
>
>
> TestInsertWideTable.test_insert_wide_table failed in exhaustive tests.
> Error Message
> query_test/test_insert.py:259: in test_insert_wide_table
> self.client.execute(insert_stmt) common/impala_connection.py:208: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:189: in execute handle =
> self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:367: in __execute_query
> self.wait_for_finished(handle) beeswax/impala_beeswax.py:388: in
> wait_for_finished raise ImpalaBeeswaxException("Query aborted:" + error_log,
> None) E ImpalaBeeswaxException: ImpalaBeeswaxException: E Query
> aborted:Parquet file size 268435456 bytes is too small for a table with 2000
> non-partitioning columns. Set query option PARQUET_FILE_SIZE to at least
> 393216000.
> Stacktrace
> query_test/test_insert.py:259: in test_insert_wide_table
> self.client.execute(insert_stmt)
> common/impala_connection.py:208: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:189: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:367: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:388: in wait_for_finished
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E Query aborted:Parquet file size 268435456 bytes is too small for a table
> with 2000 non-partitioning columns. Set query option PARQUET_FILE_SIZE to at
> least 393216000.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10820) TestInsertWideTable.test_insert_wide_table failed due to file size too small
[
https://issues.apache.org/jira/browse/IMPALA-10820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17386142#comment-17386142
]
Attila Jeges commented on IMPALA-10820:
---
[~wzhou] You're correct, looks like IMPALA-10627 broke the test. Here's the
fix: https://gerrit.cloudera.org/#/c/17719/
> TestInsertWideTable.test_insert_wide_table failed due to file size too small
>
>
> Key: IMPALA-10820
> URL: https://issues.apache.org/jira/browse/IMPALA-10820
> Project: IMPALA
> Issue Type: Bug
> Components: Backend, Frontend
>Affects Versions: Impala 4.1
>Reporter: Wenzhe Zhou
>Assignee: Attila Jeges
>Priority: Major
> Labels: broken-build
> Attachments: error_message_impala_10820.txt
>
>
> TestInsertWideTable.test_insert_wide_table failed in exhaustive tests.
> Error Message
> query_test/test_insert.py:259: in test_insert_wide_table
> self.client.execute(insert_stmt) common/impala_connection.py:208: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:189: in execute handle =
> self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:367: in __execute_query
> self.wait_for_finished(handle) beeswax/impala_beeswax.py:388: in
> wait_for_finished raise ImpalaBeeswaxException("Query aborted:" + error_log,
> None) E ImpalaBeeswaxException: ImpalaBeeswaxException: E Query
> aborted:Parquet file size 268435456 bytes is too small for a table with 2000
> non-partitioning columns. Set query option PARQUET_FILE_SIZE to at least
> 393216000.
> Stacktrace
> query_test/test_insert.py:259: in test_insert_wide_table
> self.client.execute(insert_stmt)
> common/impala_connection.py:208: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:189: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:367: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:388: in wait_for_finished
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E Query aborted:Parquet file size 268435456 bytes is too small for a table
> with 2000 non-partitioning columns. Set query option PARQUET_FILE_SIZE to at
> least 393216000.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-10739) Add support for ALTER TABLE tbl SET PARTITION SPEC for Iceberg tables
[
https://issues.apache.org/jira/browse/IMPALA-10739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges reassigned IMPALA-10739:
-
Assignee: Attila Jeges
> Add support for ALTER TABLE tbl SET PARTITION SPEC for Iceberg tables
> -
>
> Key: IMPALA-10739
> URL: https://issues.apache.org/jira/browse/IMPALA-10739
> Project: IMPALA
> Issue Type: New Feature
>Reporter: Zoltán Borók-Nagy
>Assignee: Attila Jeges
>Priority: Major
> Labels: impala-iceberg
>
> Impala should support partition evolution for Iceberg tables, i.e. it should
> be able to set a new partition spec for an Iceberg table via DDL.
> The command should be
> {noformat}
> ALTER TABLE SET PARTITION SPEC()
> {noformat}
> to be aligned with Hive.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10627) Use standard Iceberg table properties
[ https://issues.apache.org/jira/browse/IMPALA-10627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17375753#comment-17375753 ] Attila Jeges commented on IMPALA-10627: --- https://gerrit.cloudera.org/#/c/17654/ > Use standard Iceberg table properties > - > > Key: IMPALA-10627 > URL: https://issues.apache.org/jira/browse/IMPALA-10627 > Project: IMPALA > Issue Type: Bug >Reporter: Zoltán Borók-Nagy >Assignee: Attila Jeges >Priority: Major > Labels: impala-iceberg > > Iceberg lists the following properties: > [https://iceberg.apache.org/configuration/] > We should also use these properties if possible, e.g. write.format.default, > write..compression-codec > Currently Impala use the table property 'iceberg.file_format' to determine > the data file format for reads/writes. In the future, read operations should > automatically detect the file formats (IMPALA-10610), but for writes we > should use 'write.format.default'. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-10627) Use standard Iceberg table properties
[ https://issues.apache.org/jira/browse/IMPALA-10627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-10627 started by Attila Jeges. - > Use standard Iceberg table properties > - > > Key: IMPALA-10627 > URL: https://issues.apache.org/jira/browse/IMPALA-10627 > Project: IMPALA > Issue Type: Bug >Reporter: Zoltán Borók-Nagy >Assignee: Attila Jeges >Priority: Major > Labels: impala-iceberg > > Iceberg lists the following properties: > [https://iceberg.apache.org/configuration/] > We should also use these properties if possible, e.g. write.format.default, > write..compression-codec > Currently Impala use the table property 'iceberg.file_format' to determine > the data file format for reads/writes. In the future, read operations should > automatically detect the file formats (IMPALA-10610), but for writes we > should use 'write.format.default'. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-10627) Use standard Iceberg table properties
[ https://issues.apache.org/jira/browse/IMPALA-10627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges reassigned IMPALA-10627: - Assignee: Attila Jeges > Use standard Iceberg table properties > - > > Key: IMPALA-10627 > URL: https://issues.apache.org/jira/browse/IMPALA-10627 > Project: IMPALA > Issue Type: Bug >Reporter: Zoltán Borók-Nagy >Assignee: Attila Jeges >Priority: Major > Labels: impala-iceberg > > Iceberg lists the following properties: > [https://iceberg.apache.org/configuration/] > We should also use these properties if possible, e.g. write.format.default, > write..compression-codec > Currently Impala use the table property 'iceberg.file_format' to determine > the data file format for reads/writes. In the future, read operations should > automatically detect the file formats (IMPALA-10610), but for writes we > should use 'write.format.default'. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-10383) Data race in AdmissionController::WaitOnQueued
[
https://issues.apache.org/jira/browse/IMPALA-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10383:
--
Labels: broken-build flaky (was: broken-build data flaky)
> Data race in AdmissionController::WaitOnQueued
> --
>
> Key: IMPALA-10383
> URL: https://issues.apache.org/jira/browse/IMPALA-10383
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.0
>Reporter: Attila Jeges
>Assignee: Thomas Tauber-Marshall
>Priority: Major
> Labels: broken-build, flaky
>
> TSAN is reporting a data race in {{AdmissionController::WaitOnQueued:}}
> {code:java}
> WARNING: ThreadSanitizer: data race (pid=4257)
> Write of size 8 at 0x7b58000901b0 by thread T416:
> #0 std::_Hashtable const, impala::AdmissionController::QueueNode>,
> std::allocator impala::AdmissionController::QueueNode> >, std::__detail::_Select1st,
> std::equal_to, std::hash,
> std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
> std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits false, true> >::_M_erase(unsigned long, std::__detail::_Hash_node_base*,
> std::__detail::_Hash_node impala::AdmissionController::QueueNode>, true>*)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/hashtable.h:1891:7
> (impalad+0x22bbe78)
> #1 std::_Hashtable const, impala::AdmissionController::QueueNode>,
> std::allocator impala::AdmissionController::QueueNode> >, std::__detail::_Select1st,
> std::equal_to, std::hash,
> std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
> std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits false, true> >::_M_erase(std::integral_constant,
> impala::UniqueIdPB const&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/hashtable.h:1916:7
> (impalad+0x22bbd3a)
> #2 std::_Hashtable const, impala::AdmissionController::QueueNode>,
> std::allocator impala::AdmissionController::QueueNode> >, std::__detail::_Select1st,
> std::equal_to, std::hash,
> std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
> std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits false, true> >::erase(impala::UniqueIdPB const&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/hashtable.h:759:16
> (impalad+0x22bbca0)
> #3 std::unordered_map impala::AdmissionController::QueueNode, std::hash,
> std::equal_to,
> std::allocator impala::AdmissionController::QueueNode> > >::erase(impala::UniqueIdPB const&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/unordered_map.h:814:21
> (impalad+0x22bbc50)
> #4 impala::AdmissionController::WaitOnQueued(impala::UniqueIdPB const&,
> std::unique_ptr std::default_delete >*, long,
> bool*)::$_6::operator()() const
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/admission-controller.cc:1278:49
> (impalad+0x229d199)
> #5
> impala::ScopeExitTrigger const&, std::unique_ptr std::default_delete >*, long,
> bool*)::$_6>::~ScopeExitTrigger()
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/scope-exit-trigger.h:40:25
> (impalad+0x2294ca9)
> #6 impala::AdmissionController::WaitOnQueued(impala::UniqueIdPB const&,
> std::unique_ptr std::default_delete >*, long, bool*)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/admission-controller.cc:1333:1
> (impalad+0x2294912)
> #7
> impala::LocalAdmissionControlClient::SubmitForAdmission(impala::AdmissionController::AdmissionRequest
> const&, impala::RuntimeProfile::EventSequence*,
> std::unique_ptr std::default_delete >*)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/local-admission-control-client.cc:45:62
> (impalad+0x2c1e40e)
> #8 impala::ClientRequestState::FinishExecQueryOrDmlRequest()
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/service/client-request-state.cc:578:52
> (impalad+0x245c651)
> #9 boost::_mfi::mf0 impala::ClientRequestState>::operator()(impala::ClientRequestState*) const
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/mem_fn_template.hpp:49:29
> (impalad+0x2468ef6)
> #10 void
[jira] [Updated] (IMPALA-10383) Data race in AdmissionController::WaitOnQueued
[
https://issues.apache.org/jira/browse/IMPALA-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10383:
--
Labels: broken-build data flaky (was: broken-build flaky)
> Data race in AdmissionController::WaitOnQueued
> --
>
> Key: IMPALA-10383
> URL: https://issues.apache.org/jira/browse/IMPALA-10383
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.0
>Reporter: Attila Jeges
>Assignee: Thomas Tauber-Marshall
>Priority: Major
> Labels: broken-build, data, flaky
>
> TSAN is reporting a data race in {{AdmissionController::WaitOnQueued:}}
> {code:java}
> WARNING: ThreadSanitizer: data race (pid=4257)
> Write of size 8 at 0x7b58000901b0 by thread T416:
> #0 std::_Hashtable const, impala::AdmissionController::QueueNode>,
> std::allocator impala::AdmissionController::QueueNode> >, std::__detail::_Select1st,
> std::equal_to, std::hash,
> std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
> std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits false, true> >::_M_erase(unsigned long, std::__detail::_Hash_node_base*,
> std::__detail::_Hash_node impala::AdmissionController::QueueNode>, true>*)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/hashtable.h:1891:7
> (impalad+0x22bbe78)
> #1 std::_Hashtable const, impala::AdmissionController::QueueNode>,
> std::allocator impala::AdmissionController::QueueNode> >, std::__detail::_Select1st,
> std::equal_to, std::hash,
> std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
> std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits false, true> >::_M_erase(std::integral_constant,
> impala::UniqueIdPB const&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/hashtable.h:1916:7
> (impalad+0x22bbd3a)
> #2 std::_Hashtable const, impala::AdmissionController::QueueNode>,
> std::allocator impala::AdmissionController::QueueNode> >, std::__detail::_Select1st,
> std::equal_to, std::hash,
> std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
> std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits false, true> >::erase(impala::UniqueIdPB const&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/hashtable.h:759:16
> (impalad+0x22bbca0)
> #3 std::unordered_map impala::AdmissionController::QueueNode, std::hash,
> std::equal_to,
> std::allocator impala::AdmissionController::QueueNode> > >::erase(impala::UniqueIdPB const&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/unordered_map.h:814:21
> (impalad+0x22bbc50)
> #4 impala::AdmissionController::WaitOnQueued(impala::UniqueIdPB const&,
> std::unique_ptr std::default_delete >*, long,
> bool*)::$_6::operator()() const
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/admission-controller.cc:1278:49
> (impalad+0x229d199)
> #5
> impala::ScopeExitTrigger const&, std::unique_ptr std::default_delete >*, long,
> bool*)::$_6>::~ScopeExitTrigger()
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/scope-exit-trigger.h:40:25
> (impalad+0x2294ca9)
> #6 impala::AdmissionController::WaitOnQueued(impala::UniqueIdPB const&,
> std::unique_ptr std::default_delete >*, long, bool*)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/admission-controller.cc:1333:1
> (impalad+0x2294912)
> #7
> impala::LocalAdmissionControlClient::SubmitForAdmission(impala::AdmissionController::AdmissionRequest
> const&, impala::RuntimeProfile::EventSequence*,
> std::unique_ptr std::default_delete >*)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/local-admission-control-client.cc:45:62
> (impalad+0x2c1e40e)
> #8 impala::ClientRequestState::FinishExecQueryOrDmlRequest()
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/service/client-request-state.cc:578:52
> (impalad+0x245c651)
> #9 boost::_mfi::mf0 impala::ClientRequestState>::operator()(impala::ClientRequestState*) const
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/mem_fn_template.hpp:49:29
> (impalad+0x2468ef6)
> #10 void
[jira] [Updated] (IMPALA-10383) Data race in AdmissionController::WaitOnQueued
[
https://issues.apache.org/jira/browse/IMPALA-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10383:
--
Labels: broken-build (was: )
> Data race in AdmissionController::WaitOnQueued
> --
>
> Key: IMPALA-10383
> URL: https://issues.apache.org/jira/browse/IMPALA-10383
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.0
>Reporter: Attila Jeges
>Assignee: Thomas Tauber-Marshall
>Priority: Major
> Labels: broken-build
>
> TSAN is reporting a data race in {{AdmissionController::WaitOnQueued:}}
> {code:java}
> WARNING: ThreadSanitizer: data race (pid=4257)
> Write of size 8 at 0x7b58000901b0 by thread T416:
> #0 std::_Hashtable const, impala::AdmissionController::QueueNode>,
> std::allocator impala::AdmissionController::QueueNode> >, std::__detail::_Select1st,
> std::equal_to, std::hash,
> std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
> std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits false, true> >::_M_erase(unsigned long, std::__detail::_Hash_node_base*,
> std::__detail::_Hash_node impala::AdmissionController::QueueNode>, true>*)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/hashtable.h:1891:7
> (impalad+0x22bbe78)
> #1 std::_Hashtable const, impala::AdmissionController::QueueNode>,
> std::allocator impala::AdmissionController::QueueNode> >, std::__detail::_Select1st,
> std::equal_to, std::hash,
> std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
> std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits false, true> >::_M_erase(std::integral_constant,
> impala::UniqueIdPB const&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/hashtable.h:1916:7
> (impalad+0x22bbd3a)
> #2 std::_Hashtable const, impala::AdmissionController::QueueNode>,
> std::allocator impala::AdmissionController::QueueNode> >, std::__detail::_Select1st,
> std::equal_to, std::hash,
> std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
> std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits false, true> >::erase(impala::UniqueIdPB const&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/hashtable.h:759:16
> (impalad+0x22bbca0)
> #3 std::unordered_map impala::AdmissionController::QueueNode, std::hash,
> std::equal_to,
> std::allocator impala::AdmissionController::QueueNode> > >::erase(impala::UniqueIdPB const&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/unordered_map.h:814:21
> (impalad+0x22bbc50)
> #4 impala::AdmissionController::WaitOnQueued(impala::UniqueIdPB const&,
> std::unique_ptr std::default_delete >*, long,
> bool*)::$_6::operator()() const
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/admission-controller.cc:1278:49
> (impalad+0x229d199)
> #5
> impala::ScopeExitTrigger const&, std::unique_ptr std::default_delete >*, long,
> bool*)::$_6>::~ScopeExitTrigger()
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/scope-exit-trigger.h:40:25
> (impalad+0x2294ca9)
> #6 impala::AdmissionController::WaitOnQueued(impala::UniqueIdPB const&,
> std::unique_ptr std::default_delete >*, long, bool*)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/admission-controller.cc:1333:1
> (impalad+0x2294912)
> #7
> impala::LocalAdmissionControlClient::SubmitForAdmission(impala::AdmissionController::AdmissionRequest
> const&, impala::RuntimeProfile::EventSequence*,
> std::unique_ptr std::default_delete >*)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/local-admission-control-client.cc:45:62
> (impalad+0x2c1e40e)
> #8 impala::ClientRequestState::FinishExecQueryOrDmlRequest()
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/service/client-request-state.cc:578:52
> (impalad+0x245c651)
> #9 boost::_mfi::mf0 impala::ClientRequestState>::operator()(impala::ClientRequestState*) const
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/mem_fn_template.hpp:49:29
> (impalad+0x2468ef6)
> #10 void boost::_bi::list1
> >::operator(),
>
[jira] [Updated] (IMPALA-10383) Data race in AdmissionController::WaitOnQueued
[
https://issues.apache.org/jira/browse/IMPALA-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10383:
--
Labels: broken-build flaky (was: broken-build)
> Data race in AdmissionController::WaitOnQueued
> --
>
> Key: IMPALA-10383
> URL: https://issues.apache.org/jira/browse/IMPALA-10383
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.0
>Reporter: Attila Jeges
>Assignee: Thomas Tauber-Marshall
>Priority: Major
> Labels: broken-build, flaky
>
> TSAN is reporting a data race in {{AdmissionController::WaitOnQueued:}}
> {code:java}
> WARNING: ThreadSanitizer: data race (pid=4257)
> Write of size 8 at 0x7b58000901b0 by thread T416:
> #0 std::_Hashtable const, impala::AdmissionController::QueueNode>,
> std::allocator impala::AdmissionController::QueueNode> >, std::__detail::_Select1st,
> std::equal_to, std::hash,
> std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
> std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits false, true> >::_M_erase(unsigned long, std::__detail::_Hash_node_base*,
> std::__detail::_Hash_node impala::AdmissionController::QueueNode>, true>*)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/hashtable.h:1891:7
> (impalad+0x22bbe78)
> #1 std::_Hashtable const, impala::AdmissionController::QueueNode>,
> std::allocator impala::AdmissionController::QueueNode> >, std::__detail::_Select1st,
> std::equal_to, std::hash,
> std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
> std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits false, true> >::_M_erase(std::integral_constant,
> impala::UniqueIdPB const&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/hashtable.h:1916:7
> (impalad+0x22bbd3a)
> #2 std::_Hashtable const, impala::AdmissionController::QueueNode>,
> std::allocator impala::AdmissionController::QueueNode> >, std::__detail::_Select1st,
> std::equal_to, std::hash,
> std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
> std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits false, true> >::erase(impala::UniqueIdPB const&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/hashtable.h:759:16
> (impalad+0x22bbca0)
> #3 std::unordered_map impala::AdmissionController::QueueNode, std::hash,
> std::equal_to,
> std::allocator impala::AdmissionController::QueueNode> > >::erase(impala::UniqueIdPB const&)
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/unordered_map.h:814:21
> (impalad+0x22bbc50)
> #4 impala::AdmissionController::WaitOnQueued(impala::UniqueIdPB const&,
> std::unique_ptr std::default_delete >*, long,
> bool*)::$_6::operator()() const
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/admission-controller.cc:1278:49
> (impalad+0x229d199)
> #5
> impala::ScopeExitTrigger const&, std::unique_ptr std::default_delete >*, long,
> bool*)::$_6>::~ScopeExitTrigger()
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/scope-exit-trigger.h:40:25
> (impalad+0x2294ca9)
> #6 impala::AdmissionController::WaitOnQueued(impala::UniqueIdPB const&,
> std::unique_ptr std::default_delete >*, long, bool*)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/admission-controller.cc:1333:1
> (impalad+0x2294912)
> #7
> impala::LocalAdmissionControlClient::SubmitForAdmission(impala::AdmissionController::AdmissionRequest
> const&, impala::RuntimeProfile::EventSequence*,
> std::unique_ptr std::default_delete >*)
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/local-admission-control-client.cc:45:62
> (impalad+0x2c1e40e)
> #8 impala::ClientRequestState::FinishExecQueryOrDmlRequest()
> /data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/service/client-request-state.cc:578:52
> (impalad+0x245c651)
> #9 boost::_mfi::mf0 impala::ClientRequestState>::operator()(impala::ClientRequestState*) const
> /data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/mem_fn_template.hpp:49:29
> (impalad+0x2468ef6)
> #10 void boost::_bi::list1
>
[jira] [Created] (IMPALA-10383) Data race in AdmissionController::WaitOnQueued
Attila Jeges created IMPALA-10383:
-
Summary: Data race in AdmissionController::WaitOnQueued
Key: IMPALA-10383
URL: https://issues.apache.org/jira/browse/IMPALA-10383
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 4.0
Reporter: Attila Jeges
Assignee: Thomas Tauber-Marshall
TSAN is reporting a data race in {{AdmissionController::WaitOnQueued:}}
{code:java}
WARNING: ThreadSanitizer: data race (pid=4257)
Write of size 8 at 0x7b58000901b0 by thread T416:
#0 std::_Hashtable,
std::allocator >, std::__detail::_Select1st,
std::equal_to, std::hash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(unsigned long, std::__detail::_Hash_node_base*,
std::__detail::_Hash_node, true>*)
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/hashtable.h:1891:7
(impalad+0x22bbe78)
#1 std::_Hashtable,
std::allocator >, std::__detail::_Select1st,
std::equal_to, std::hash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::_M_erase(std::integral_constant, impala::UniqueIdPB
const&)
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/hashtable.h:1916:7
(impalad+0x22bbd3a)
#2 std::_Hashtable,
std::allocator >, std::__detail::_Select1st,
std::equal_to, std::hash,
std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash,
std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits >::erase(impala::UniqueIdPB const&)
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/hashtable.h:759:16
(impalad+0x22bbca0)
#3 std::unordered_map,
std::equal_to, std::allocator > >::erase(impala::UniqueIdPB
const&)
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/unordered_map.h:814:21
(impalad+0x22bbc50)
#4 impala::AdmissionController::WaitOnQueued(impala::UniqueIdPB const&,
std::unique_ptr >*, long,
bool*)::$_6::operator()() const
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/admission-controller.cc:1278:49
(impalad+0x229d199)
#5
impala::ScopeExitTrigger >*, long,
bool*)::$_6>::~ScopeExitTrigger()
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/util/scope-exit-trigger.h:40:25
(impalad+0x2294ca9)
#6 impala::AdmissionController::WaitOnQueued(impala::UniqueIdPB const&,
std::unique_ptr >*, long, bool*)
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/admission-controller.cc:1333:1
(impalad+0x2294912)
#7
impala::LocalAdmissionControlClient::SubmitForAdmission(impala::AdmissionController::AdmissionRequest
const&, impala::RuntimeProfile::EventSequence*,
std::unique_ptr >*)
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/scheduling/local-admission-control-client.cc:45:62
(impalad+0x2c1e40e)
#8 impala::ClientRequestState::FinishExecQueryOrDmlRequest()
/data/jenkins/workspace/impala-asf-master-core-tsan/repos/Impala/be/src/service/client-request-state.cc:578:52
(impalad+0x245c651)
#9 boost::_mfi::mf0::operator()(impala::ClientRequestState*) const
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/mem_fn_template.hpp:49:29
(impalad+0x2468ef6)
#10 void boost::_bi::list1
>::operator(),
boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf0&, boost::_bi::list0&, int)
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:259:9
(impalad+0x2468e4a)
#11 boost::_bi::bind_t,
boost::_bi::list1 >
>::operator()()
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/bind/bind.hpp:1222:16
(impalad+0x2468dd3)
#12
boost::detail::function::void_function_obj_invoker0,
boost::_bi::list1 > >,
void>::invoke(boost::detail::function::function_buffer&)
/data/jenkins/workspace/impala-asf-master-core-tsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/boost-1.61.0-p2/include/boost/function/function_template.hpp:159:11
(impalad+0x2468bc9)
#13 boost::function0::operator()() const
[jira] [Reopened] (IMPALA-10125) webserver.test_web_pages.TestWebPage.test_catalog failed
[
https://issues.apache.org/jira/browse/IMPALA-10125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges reopened IMPALA-10125:
---
Seen this downstream again. Reopening.
> webserver.test_web_pages.TestWebPage.test_catalog failed
>
>
> Key: IMPALA-10125
> URL: https://issues.apache.org/jira/browse/IMPALA-10125
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Reporter: Yongzhi Chen
>Priority: Major
> Labels: broken-build, flaky
>
> In master-core-data-load, webserver.test_web_pages.TestWebPage.test_catalog
> failed with
> {noformat}
> Stacktrace
> webserver/test_web_pages.py:303: in test_catalog
> self.__test_table_metrics(unique_database, "foo_part", "alter-duration")
> webserver/test_web_pages.py:352: in __test_table_metrics
> "?name=%s.%s" % (db_name, tbl_name), metric,
> ports_to_test=self.CATALOG_TEST_PORT)
> webserver/test_web_pages.py:170: in get_and_check_status
> assert string_to_search in response.text, "URL: {0}
> Str:'{1}'\nResp:{2}".format(
> E AssertionError: URL:
> http://localhost:25020/table_metrics?name=test_catalog_caf8ffd1.foo_part
> Str:'alter-duration'
> E Resp:
> E
> E
> E
> E
> E Apache Impala
> E
> E
> E
> E href="/www/datatables-1.10.18.min.css"/>
> E src="/www/datatables-1.10.18.min.js">
> E rel='stylesheet' media='screen'>
> E
> E
> E @media (min-width: 1300px) {
> E #nav-options {
> E width: 1280px;
> E }
> E }
> E
> E body {
> E font-size: 14px;
> E }
> E
> E pre {
> E padding: 10px;
> E font-size: 12px;
> E border: 1px solid #ccc;
> E }
> E
> E /* Avoid unsightly padding around code element */
> E pre.code {
> E padding: 0;
> E }
> E
> E
> E
> E
> E
> E
> E catalogd
> E
> E role="navigation">
> E
> E
> E href="/">/
> E
> E href="/catalog">/catalog
> E
> E href="/jmx">/jmx
> E
> E href="/log_level">/log_level
> E
> E href="/logs">/logs
> E
> E href="/memz">/memz
> E
> E href="/metrics">/metrics
> E
> E href="/operations">/operations
> E
> E href="/profile_docs">/profile_docs
> E
> E href="/rpcz">/rpcz
> E
> E href="/threadz">/threadz
> E
> E href="/varz">/varz
> E
> E
> E
> E
> E
> E
> E
> E
> E // For Apache Knox compatibility, all urls that are accessed by
> javascript should have
> E // their path wrapped with this.
> E function make_url(path) {
> E var root_link = document.getElementById('root-link');
> E var s = root_link.href.split("?");
> E url = s[0] + path;
> E if (s.length > 1) {
> E if (path.includes("?")) {
> E url += "&"
> E } else {
> E url += "?";
> E }
> E url += s[1];
> E }
> E return url;
> E }
> E
> E
> E
> E Metrics for table test_catalog_caf8ffd1.foo_partare not available
> because the table is currently modified by another operation.
> E
> E
> E
> E
> E
> E
> E assert 'alter-duration' in
[jira] [Assigned] (IMPALA-8883) Update statistics of ACID tables during writes
[ https://issues.apache.org/jira/browse/IMPALA-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges reassigned IMPALA-8883: Assignee: (was: Attila Jeges) > Update statistics of ACID tables during writes > -- > > Key: IMPALA-8883 > URL: https://issues.apache.org/jira/browse/IMPALA-8883 > Project: IMPALA > Issue Type: Improvement >Reporter: Zoltán Borók-Nagy >Priority: Major > Labels: impala-acid > > When Impala INSERTs or TRUNCATEs an ACID table it simply removes the > COLUMN_STATS_ACCURATE property to invalidate the statistics in order to > prevent Hive using it. > Instead of it Impala should properly update the statistics. It should be > relatively simple for TRUNCATE since it erases all the data, but a bit more > complicated for INSERT, e.g.: > * Properly update _number of distinct values_ > * INSERT OVERWRITE partition should properly update table level _number of > rows_. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8883) Update statistics of ACID tables during writes
[ https://issues.apache.org/jira/browse/IMPALA-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243971#comment-17243971 ] Attila Jeges commented on IMPALA-8883: -- I'm not sure if this is still relevant, but I won't have time to work on this anytime soon. Unassigning. > Update statistics of ACID tables during writes > -- > > Key: IMPALA-8883 > URL: https://issues.apache.org/jira/browse/IMPALA-8883 > Project: IMPALA > Issue Type: Improvement >Reporter: Zoltán Borók-Nagy >Assignee: Attila Jeges >Priority: Major > Labels: impala-acid > > When Impala INSERTs or TRUNCATEs an ACID table it simply removes the > COLUMN_STATS_ACCURATE property to invalidate the statistics in order to > prevent Hive using it. > Instead of it Impala should properly update the statistics. It should be > relatively simple for TRUNCATE since it erases all the data, but a bit more > complicated for INSERT, e.g.: > * Properly update _number of distinct values_ > * INSERT OVERWRITE partition should properly update table level _number of > rows_. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-10224) Add startup flag not to expose debug web url via PingImpalaService/PingImpalaHS2Service RPC
[ https://issues.apache.org/jira/browse/IMPALA-10224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges resolved IMPALA-10224. --- Resolution: Fixed > Add startup flag not to expose debug web url via > PingImpalaService/PingImpalaHS2Service RPC > --- > > Key: IMPALA-10224 > URL: https://issues.apache.org/jira/browse/IMPALA-10224 > Project: IMPALA > Issue Type: New Feature > Components: Backend >Affects Versions: Impala 3.4.0 >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > > PingImpalaService/PingImpalaHS2Service RPC calls expose the coordinator's > debug web url to clients like impala shell. Since the debug web UI is not > something that end-users will necessarily have access to, we should have a > server option to send an empty string instead of the real url to the impala > client signalling that the debug web ui is not available. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-8827) Create tests for Hive stats interop in ACID tables
[ https://issues.apache.org/jira/browse/IMPALA-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges reassigned IMPALA-8827: Assignee: (was: Attila Jeges) > Create tests for Hive stats interop in ACID tables > -- > > Key: IMPALA-8827 > URL: https://issues.apache.org/jira/browse/IMPALA-8827 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Affects Versions: Impala 3.3.0 >Reporter: Csaba Ringhofer >Priority: Critical > Labels: impala-acid > > Hive started to use stats to answer some queries in ACID tables. Some tests > should check what happens if Hive uses stats computed by Impala. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-8827) Create tests for Hive stats interop in ACID tables
[ https://issues.apache.org/jira/browse/IMPALA-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243961#comment-17243961 ] Attila Jeges edited comment on IMPALA-8827 at 12/4/20, 12:01 PM: - Unassigned it as I won't be able to work on this anytime soon. was (Author: attilaj): Unassigned it as I won't be able to work on this > Create tests for Hive stats interop in ACID tables > -- > > Key: IMPALA-8827 > URL: https://issues.apache.org/jira/browse/IMPALA-8827 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Affects Versions: Impala 3.3.0 >Reporter: Csaba Ringhofer >Priority: Critical > Labels: impala-acid > > Hive started to use stats to answer some queries in ACID tables. Some tests > should check what happens if Hive uses stats computed by Impala. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8827) Create tests for Hive stats interop in ACID tables
[ https://issues.apache.org/jira/browse/IMPALA-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243961#comment-17243961 ] Attila Jeges commented on IMPALA-8827: -- Unassigned it as I won't be able to work on this > Create tests for Hive stats interop in ACID tables > -- > > Key: IMPALA-8827 > URL: https://issues.apache.org/jira/browse/IMPALA-8827 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Affects Versions: Impala 3.3.0 >Reporter: Csaba Ringhofer >Priority: Critical > Labels: impala-acid > > Hive started to use stats to answer some queries in ACID tables. Some tests > should check what happens if Hive uses stats computed by Impala. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-10362) FE test testShellLdapAuth() seems to be flaky
[
https://issues.apache.org/jira/browse/IMPALA-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10362:
--
Fix Version/s: Impala 4.0
> FE test testShellLdapAuth() seems to be flaky
> -
>
> Key: IMPALA-10362
> URL: https://issues.apache.org/jira/browse/IMPALA-10362
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.0
>Reporter: Fang-Yu Rao
>Assignee: Attila Jeges
>Priority: Major
> Labels: broken-build, flaky-test
> Fix For: Impala 4.0
>
> Attachments: org.apache.impala.customcluster.LdapImpalaShellTest.txt
>
>
> We found that sometimes {{LdapImpalaShellTest#testShellLdapAuth()}} failed at
> [https://github.com/apache/impala/blob/1c72c5a8f93ae3aa7e30b8bb59e0a18a654d0d34/fe/src/test/java/org/apache/impala/customcluster/LdapImpalaShellTest.java#L146]
> due to hitting an {{assertTrue()}} at
> [https://github.com/apache/impala/blob/master/fe/src/test/java/org/apache/impala/customcluster/RunShellCommand.java#L50].
> This test was slightly revised in IMPALA-10234. Maybe [~attilaj] could offer
> some insight into it.
> The corresponding log file is also provided. Let me know if other log files
> are needed as well. Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-10362) FE test testShellLdapAuth() seems to be flaky
[
https://issues.apache.org/jira/browse/IMPALA-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges resolved IMPALA-10362.
---
Resolution: Fixed
> FE test testShellLdapAuth() seems to be flaky
> -
>
> Key: IMPALA-10362
> URL: https://issues.apache.org/jira/browse/IMPALA-10362
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.0
>Reporter: Fang-Yu Rao
>Assignee: Attila Jeges
>Priority: Major
> Labels: broken-build, flaky-test
> Attachments: org.apache.impala.customcluster.LdapImpalaShellTest.txt
>
>
> We found that sometimes {{LdapImpalaShellTest#testShellLdapAuth()}} failed at
> [https://github.com/apache/impala/blob/1c72c5a8f93ae3aa7e30b8bb59e0a18a654d0d34/fe/src/test/java/org/apache/impala/customcluster/LdapImpalaShellTest.java#L146]
> due to hitting an {{assertTrue()}} at
> [https://github.com/apache/impala/blob/master/fe/src/test/java/org/apache/impala/customcluster/RunShellCommand.java#L50].
> This test was slightly revised in IMPALA-10234. Maybe [~attilaj] could offer
> some insight into it.
> The corresponding log file is also provided. Let me know if other log files
> are needed as well. Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Reopened] (IMPALA-10362) FE test testShellLdapAuth() seems to be flaky
[
https://issues.apache.org/jira/browse/IMPALA-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges reopened IMPALA-10362:
---
> FE test testShellLdapAuth() seems to be flaky
> -
>
> Key: IMPALA-10362
> URL: https://issues.apache.org/jira/browse/IMPALA-10362
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.0
>Reporter: Fang-Yu Rao
>Assignee: Attila Jeges
>Priority: Major
> Labels: broken-build, flaky-test
> Attachments: org.apache.impala.customcluster.LdapImpalaShellTest.txt
>
>
> We found that sometimes {{LdapImpalaShellTest#testShellLdapAuth()}} failed at
> [https://github.com/apache/impala/blob/1c72c5a8f93ae3aa7e30b8bb59e0a18a654d0d34/fe/src/test/java/org/apache/impala/customcluster/LdapImpalaShellTest.java#L146]
> due to hitting an {{assertTrue()}} at
> [https://github.com/apache/impala/blob/master/fe/src/test/java/org/apache/impala/customcluster/RunShellCommand.java#L50].
> This test was slightly revised in IMPALA-10234. Maybe [~attilaj] could offer
> some insight into it.
> The corresponding log file is also provided. Let me know if other log files
> are needed as well. Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-10362) FE test testShellLdapAuth() seems to be flaky
[
https://issues.apache.org/jira/browse/IMPALA-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges resolved IMPALA-10362.
---
Resolution: Not A Problem
> FE test testShellLdapAuth() seems to be flaky
> -
>
> Key: IMPALA-10362
> URL: https://issues.apache.org/jira/browse/IMPALA-10362
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.0
>Reporter: Fang-Yu Rao
>Assignee: Attila Jeges
>Priority: Major
> Labels: broken-build, flaky-test
> Attachments: org.apache.impala.customcluster.LdapImpalaShellTest.txt
>
>
> We found that sometimes {{LdapImpalaShellTest#testShellLdapAuth()}} failed at
> [https://github.com/apache/impala/blob/1c72c5a8f93ae3aa7e30b8bb59e0a18a654d0d34/fe/src/test/java/org/apache/impala/customcluster/LdapImpalaShellTest.java#L146]
> due to hitting an {{assertTrue()}} at
> [https://github.com/apache/impala/blob/master/fe/src/test/java/org/apache/impala/customcluster/RunShellCommand.java#L50].
> This test was slightly revised in IMPALA-10234. Maybe [~attilaj] could offer
> some insight into it.
> The corresponding log file is also provided. Let me know if other log files
> are needed as well. Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10362) FE test testShellLdapAuth() seems to be flaky
[
https://issues.apache.org/jira/browse/IMPALA-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17240642#comment-17240642
]
Attila Jeges commented on IMPALA-10362:
---
This is a downstream only issue.
> FE test testShellLdapAuth() seems to be flaky
> -
>
> Key: IMPALA-10362
> URL: https://issues.apache.org/jira/browse/IMPALA-10362
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 4.0
>Reporter: Fang-Yu Rao
>Assignee: Attila Jeges
>Priority: Major
> Labels: broken-build, flaky-test
> Attachments: org.apache.impala.customcluster.LdapImpalaShellTest.txt
>
>
> We found that sometimes {{LdapImpalaShellTest#testShellLdapAuth()}} failed at
> [https://github.com/apache/impala/blob/1c72c5a8f93ae3aa7e30b8bb59e0a18a654d0d34/fe/src/test/java/org/apache/impala/customcluster/LdapImpalaShellTest.java#L146]
> due to hitting an {{assertTrue()}} at
> [https://github.com/apache/impala/blob/master/fe/src/test/java/org/apache/impala/customcluster/RunShellCommand.java#L50].
> This test was slightly revised in IMPALA-10234. Maybe [~attilaj] could offer
> some insight into it.
> The corresponding log file is also provided. Let me know if other log files
> are needed as well. Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9121) heap-use-after-free in HdfsUtilTest CheckFilesystemsMatch
[
https://issues.apache.org/jira/browse/IMPALA-9121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17234941#comment-17234941
]
Attila Jeges commented on IMPALA-9121:
--
Unassigning the JIRA as I won't be able to work on this for a while.
> heap-use-after-free in HdfsUtilTest CheckFilesystemsMatch
> -
>
> Key: IMPALA-9121
> URL: https://issues.apache.org/jira/browse/IMPALA-9121
> Project: IMPALA
> Issue Type: Bug
>Reporter: Bikramjeet Vig
>Priority: Major
> Labels: asan, broken-build, flaky-test
>
> {noformat}
> --
> 85/113 Testing: hdfs-util-test
> 85/113 Test: hdfs-util-test
> Command:
> "/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/util/hdfs-util-test"
> "-log_dir=/data/jenkins/workspace
> /impala-asf-master-core-asan/repos/Impala/logs/be_tests"
> Directory:
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/util
> "hdfs-util-test" start time: Nov 01 22:21 PDT
> Output:
> --
> seed = 1572672103
> Note: Google Test filter = HdfsUtilTest.*
> [==] Running 2 tests from 1 test case.
> [--] Global test environment set-up.
> [--] 2 tests from HdfsUtilTest
> [ RUN ] HdfsUtilTest.CheckFilesystemsMatch
> 19/11/01 22:21:43 INFO util.JvmPauseMonitor: Starting JVM pause monitor
> [ OK ] HdfsUtilTest.CheckFilesystemsMatch (1012 ms)
> [ RUN ] HdfsUtilTest.CheckGetBaseName
> [ OK ] HdfsUtilTest.CheckGetBaseName (0 ms)
> [--] 2 tests from HdfsUtilTest (1012 ms total)
> [--] Global test environment tear-down
> [==] 2 tests from 1 test case ran. (1013 ms total)
> [ PASSED ] 2 tests.
> YOU HAVE 1 DISABLED TEST
> =
> ==87116==ERROR: AddressSanitizer: heap-use-after-free on address
> 0x60362878 at pc 0x01b67bac bp 0x7fc96cf57ae0 sp 0x7fc96cf57290
> READ of size 4 at 0x60362878 thread T4
>
> Test time = 2.64 sec
> --
> Test Passed.
> "hdfs-util-test" end time: Nov 01 22:21 PDT
> "hdfs-util-test" time elapsed: 00:00:02
> --
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-9121) heap-use-after-free in HdfsUtilTest CheckFilesystemsMatch
[
https://issues.apache.org/jira/browse/IMPALA-9121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges reassigned IMPALA-9121:
Assignee: (was: Attila Jeges)
> heap-use-after-free in HdfsUtilTest CheckFilesystemsMatch
> -
>
> Key: IMPALA-9121
> URL: https://issues.apache.org/jira/browse/IMPALA-9121
> Project: IMPALA
> Issue Type: Bug
>Reporter: Bikramjeet Vig
>Priority: Major
> Labels: asan, broken-build, flaky-test
>
> {noformat}
> --
> 85/113 Testing: hdfs-util-test
> 85/113 Test: hdfs-util-test
> Command:
> "/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/util/hdfs-util-test"
> "-log_dir=/data/jenkins/workspace
> /impala-asf-master-core-asan/repos/Impala/logs/be_tests"
> Directory:
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/util
> "hdfs-util-test" start time: Nov 01 22:21 PDT
> Output:
> --
> seed = 1572672103
> Note: Google Test filter = HdfsUtilTest.*
> [==] Running 2 tests from 1 test case.
> [--] Global test environment set-up.
> [--] 2 tests from HdfsUtilTest
> [ RUN ] HdfsUtilTest.CheckFilesystemsMatch
> 19/11/01 22:21:43 INFO util.JvmPauseMonitor: Starting JVM pause monitor
> [ OK ] HdfsUtilTest.CheckFilesystemsMatch (1012 ms)
> [ RUN ] HdfsUtilTest.CheckGetBaseName
> [ OK ] HdfsUtilTest.CheckGetBaseName (0 ms)
> [--] 2 tests from HdfsUtilTest (1012 ms total)
> [--] Global test environment tear-down
> [==] 2 tests from 1 test case ran. (1013 ms total)
> [ PASSED ] 2 tests.
> YOU HAVE 1 DISABLED TEST
> =
> ==87116==ERROR: AddressSanitizer: heap-use-after-free on address
> 0x60362878 at pc 0x01b67bac bp 0x7fc96cf57ae0 sp 0x7fc96cf57290
> READ of size 4 at 0x60362878 thread T4
>
> Test time = 2.64 sec
> --
> Test Passed.
> "hdfs-util-test" end time: Nov 01 22:21 PDT
> "hdfs-util-test" time elapsed: 00:00:02
> --
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8452) Avro scanner seems broken
[
https://issues.apache.org/jira/browse/IMPALA-8452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17234687#comment-17234687
]
Attila Jeges commented on IMPALA-8452:
--
[~tarmstrong] I'll not have time to work on this in the near future. Setting
to "unassigned".
> Avro scanner seems broken
> -
>
> Key: IMPALA-8452
> URL: https://issues.apache.org/jira/browse/IMPALA-8452
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.3.0
>Reporter: Michael Ho
>Priority: Major
> Labels: broken-build, wrongresults
>
> A few scanner tests started failing recently on Centos6. Coincidentally, both
> of them only started happening after [this
> commit|https://github.com/apache/impala/commit/b5805de3e65fd1c7154e4169b323bb38ddc54f4f].
> [~attilaj], can you please take a look and reassign if you think that commit
> is unrelated ?
> Oddly enough, this has shown up on Centos6. Other exhaustive runs with
> Centos7 seem to work fine. May be it's related to some platform's library ?
> In the first case, a select count star from an avro table hangs for 2 hours:
> {noformat}
> query_test/test_scanners_fuzz.py:83: in test_fuzz_alltypes
> self.run_fuzz_test(vector, src_db, table_name, unique_database,
> table_name)
> query_test/test_scanners_fuzz.py:201: in run_fuzz_test
> result = self.execute_query(query, query_options = query_options)
> common/impala_test_suite.py:619: in wrapper
> return function(*args, **kwargs)
> common/impala_test_suite.py:650: in execute_query
> return self.__execute_query(self.client, query, query_options)
> common/impala_test_suite.py:721: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:180: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:183: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:360: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:384: in wait_for_finished
> time.sleep(0.05)
> E Failed: Timeout >7200s
> SET
> client_identifier=query_test/test_scanners_fuzz.py::TestScannersFuzzing::()::test_fuzz_alltypes[protocol:beeswax|exec_option:{'debug_action':None;'abort_on_error':False;'mem_limit':'512m';'num_nodes':0}|table_format:avro/none];
> SET batch_size=1;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=True;
> SET abort_on_error=False;
> SET mem_limit=512m;
> -- executing against localhost:21000
> select count(*) from test_fuzz_alltypes_2cdcb963.alltypes q;
> -- 2019-04-24 04:14:31,857 INFO MainThread: Started query
> 2049069f9f5e3aa8:f2fd47ff
> {noformat}
> The second case has to do with incorrect number of rows in a select count
> star from tpch_avro.lineitem:
> {noformat}
> query_test/test_scanners.py:947: in test_tpch_scan_ranges
> self.run_test_case('tpch-scan-range-lengths', vector)
> common/impala_test_suite.py:517: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:370: in __verify_results_and_errors
> replace_filenames_with_placeholder)
> common/test_result_verifier.py:449: in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:271: in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E 6001215 != 6000679
> -- 2019-04-24 03:43:42,805 INFO MainThread: max_scan_range_length=8412307
> SET
> client_identifier=query_test/test_scanners.py::TestTpchScanRangeLengths::()::test_tpch_scan_ranges[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|;
> -- executing against localhost:21000
> use tpch_avro;
> -- 2019-04-24 03:43:42,814 INFO MainThread: Started query
> c04e1968443b52fc:5b99b1b3
> SET
> client_identifier=query_test/test_scanners.py::TestTpchScanRangeLengths::()::test_tpch_scan_ranges[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET max_scan_range_length=8412307;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select count(*)
> from lineitem;
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail:
[jira] [Assigned] (IMPALA-8452) Avro scanner seems broken
[
https://issues.apache.org/jira/browse/IMPALA-8452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges reassigned IMPALA-8452:
Assignee: (was: Attila Jeges)
> Avro scanner seems broken
> -
>
> Key: IMPALA-8452
> URL: https://issues.apache.org/jira/browse/IMPALA-8452
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.3.0
>Reporter: Michael Ho
>Priority: Major
> Labels: broken-build, wrongresults
>
> A few scanner tests started failing recently on Centos6. Coincidentally, both
> of them only started happening after [this
> commit|https://github.com/apache/impala/commit/b5805de3e65fd1c7154e4169b323bb38ddc54f4f].
> [~attilaj], can you please take a look and reassign if you think that commit
> is unrelated ?
> Oddly enough, this has shown up on Centos6. Other exhaustive runs with
> Centos7 seem to work fine. May be it's related to some platform's library ?
> In the first case, a select count star from an avro table hangs for 2 hours:
> {noformat}
> query_test/test_scanners_fuzz.py:83: in test_fuzz_alltypes
> self.run_fuzz_test(vector, src_db, table_name, unique_database,
> table_name)
> query_test/test_scanners_fuzz.py:201: in run_fuzz_test
> result = self.execute_query(query, query_options = query_options)
> common/impala_test_suite.py:619: in wrapper
> return function(*args, **kwargs)
> common/impala_test_suite.py:650: in execute_query
> return self.__execute_query(self.client, query, query_options)
> common/impala_test_suite.py:721: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:180: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:183: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:360: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:384: in wait_for_finished
> time.sleep(0.05)
> E Failed: Timeout >7200s
> SET
> client_identifier=query_test/test_scanners_fuzz.py::TestScannersFuzzing::()::test_fuzz_alltypes[protocol:beeswax|exec_option:{'debug_action':None;'abort_on_error':False;'mem_limit':'512m';'num_nodes':0}|table_format:avro/none];
> SET batch_size=1;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=True;
> SET abort_on_error=False;
> SET mem_limit=512m;
> -- executing against localhost:21000
> select count(*) from test_fuzz_alltypes_2cdcb963.alltypes q;
> -- 2019-04-24 04:14:31,857 INFO MainThread: Started query
> 2049069f9f5e3aa8:f2fd47ff
> {noformat}
> The second case has to do with incorrect number of rows in a select count
> star from tpch_avro.lineitem:
> {noformat}
> query_test/test_scanners.py:947: in test_tpch_scan_ranges
> self.run_test_case('tpch-scan-range-lengths', vector)
> common/impala_test_suite.py:517: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:370: in __verify_results_and_errors
> replace_filenames_with_placeholder)
> common/test_result_verifier.py:449: in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:271: in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E 6001215 != 6000679
> -- 2019-04-24 03:43:42,805 INFO MainThread: max_scan_range_length=8412307
> SET
> client_identifier=query_test/test_scanners.py::TestTpchScanRangeLengths::()::test_tpch_scan_ranges[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|;
> -- executing against localhost:21000
> use tpch_avro;
> -- 2019-04-24 03:43:42,814 INFO MainThread: Started query
> c04e1968443b52fc:5b99b1b3
> SET
> client_identifier=query_test/test_scanners.py::TestTpchScanRangeLengths::()::test_tpch_scan_ranges[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|;
> SET batch_size=0;
> SET num_nodes=0;
> SET disable_codegen_rows_threshold=0;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET max_scan_range_length=8412307;
> SET exec_single_node_rows_threshold=0;
> -- executing against localhost:21000
> select count(*)
> from lineitem;
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-10234) impala-shell: add support for cookie-based authentication
[ https://issues.apache.org/jira/browse/IMPALA-10234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges resolved IMPALA-10234. --- Resolution: Fixed > impala-shell: add support for cookie-based authentication > - > > Key: IMPALA-10234 > URL: https://issues.apache.org/jira/browse/IMPALA-10234 > Project: IMPALA > Issue Type: New Feature > Components: Clients >Affects Versions: Impala 3.4.0 >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > Fix For: Impala 4.0 > > > IMPALA-8584 added support for cookie authentication to Impala. Need to add > cookie authentication support to impala-shell as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10234) impala-shell: add support for cookie-based authentication
[ https://issues.apache.org/jira/browse/IMPALA-10234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221528#comment-17221528 ] Attila Jeges commented on IMPALA-10234: --- CR: https://gerrit.cloudera.org/#/c/16660/ > impala-shell: add support for cookie-based authentication > - > > Key: IMPALA-10234 > URL: https://issues.apache.org/jira/browse/IMPALA-10234 > Project: IMPALA > Issue Type: New Feature > Components: Clients >Affects Versions: Impala 3.4.0 >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > Fix For: Impala 4.0 > > > IMPALA-8584 added support for cookie authentication to Impala. Need to add > cookie authentication support to impala-shell as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-10234) impala-shell: add support for cookie-based authentication
[ https://issues.apache.org/jira/browse/IMPALA-10234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-10234 started by Attila Jeges. - > impala-shell: add support for cookie-based authentication > - > > Key: IMPALA-10234 > URL: https://issues.apache.org/jira/browse/IMPALA-10234 > Project: IMPALA > Issue Type: New Feature > Components: Clients >Affects Versions: Impala 3.4.0 >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > Fix For: Impala 4.0 > > > IMPALA-8584 added support for cookie authentication to Impala. Need to add > cookie authentication support to impala-shell as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10234) impala-shell: add support for cookie-based authentication
Attila Jeges created IMPALA-10234: - Summary: impala-shell: add support for cookie-based authentication Key: IMPALA-10234 URL: https://issues.apache.org/jira/browse/IMPALA-10234 Project: IMPALA Issue Type: New Feature Components: Clients Affects Versions: Impala 3.4.0 Reporter: Attila Jeges Assignee: Attila Jeges Fix For: Impala 4.0 IMPALA-8584 added support for cookie authentication to Impala. Need to add cookie authentication support to impala-shell as well. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10224) Add startup flag not to expose debug web url via PingImpalaService/PingImpalaHS2Service RPC
[ https://issues.apache.org/jira/browse/IMPALA-10224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17212250#comment-17212250 ] Attila Jeges commented on IMPALA-10224: --- CR: https://gerrit.cloudera.org/#/c/16573/ > Add startup flag not to expose debug web url via > PingImpalaService/PingImpalaHS2Service RPC > --- > > Key: IMPALA-10224 > URL: https://issues.apache.org/jira/browse/IMPALA-10224 > Project: IMPALA > Issue Type: New Feature > Components: Backend >Affects Versions: Impala 3.4.0 >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > > PingImpalaService/PingImpalaHS2Service RPC calls expose the coordinator's > debug web url to clients like impala shell. Since the debug web UI is not > something that end-users will necessarily have access to, we should have a > server option to send an empty string instead of the real url to the impala > client signalling that the debug web ui is not available. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-10224) Add startup flag not to expose debug web url via PingImpalaService/PingImpalaHS2Service RPC
[ https://issues.apache.org/jira/browse/IMPALA-10224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-10224 started by Attila Jeges. - > Add startup flag not to expose debug web url via > PingImpalaService/PingImpalaHS2Service RPC > --- > > Key: IMPALA-10224 > URL: https://issues.apache.org/jira/browse/IMPALA-10224 > Project: IMPALA > Issue Type: New Feature > Components: Backend >Affects Versions: Impala 3.4.0 >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > > PingImpalaService/PingImpalaHS2Service RPC calls expose the coordinator's > debug web url to clients like impala shell. Since the debug web UI is not > something that end-users will necessarily have access to, we should have a > server option to send an empty string instead of the real url to the impala > client signalling that the debug web ui is not available. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-10224) Add startup flag not to expose debug web url via PingImpalaService/PingImpalaHS2Service RPC
[ https://issues.apache.org/jira/browse/IMPALA-10224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Jeges updated IMPALA-10224: -- Affects Version/s: Impala 3.4.0 > Add startup flag not to expose debug web url via > PingImpalaService/PingImpalaHS2Service RPC > --- > > Key: IMPALA-10224 > URL: https://issues.apache.org/jira/browse/IMPALA-10224 > Project: IMPALA > Issue Type: New Feature > Components: Backend >Affects Versions: Impala 3.4.0 >Reporter: Attila Jeges >Assignee: Attila Jeges >Priority: Major > > PingImpalaService/PingImpalaHS2Service RPC calls expose the coordinator's > debug web url to clients like impala shell. Since the debug web UI is not > something that end-users will necessarily have access to, we should have a > server option to send an empty string instead of the real url to the impala > client signalling that the debug web ui is not available. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10224) Add startup flag not to expose debug web url via PingImpalaService/PingImpalaHS2Service RPC
Attila Jeges created IMPALA-10224: - Summary: Add startup flag not to expose debug web url via PingImpalaService/PingImpalaHS2Service RPC Key: IMPALA-10224 URL: https://issues.apache.org/jira/browse/IMPALA-10224 Project: IMPALA Issue Type: New Feature Components: Backend Reporter: Attila Jeges Assignee: Attila Jeges PingImpalaService/PingImpalaHS2Service RPC calls expose the coordinator's debug web url to clients like impala shell. Since the debug web UI is not something that end-users will necessarily have access to, we should have a server option to send an empty string instead of the real url to the impala client signalling that the debug web ui is not available. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10054) test_multiple_sort_run_bytes_limits fails in parallel-all-tests-nightly
[
https://issues.apache.org/jira/browse/IMPALA-10054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172429#comment-17172429
]
Attila Jeges commented on IMPALA-10054:
---
[~rizaon] I'm assigning this to you as the failing test was introduced in
IMPALA-6692 that you worked on.
> test_multiple_sort_run_bytes_limits fails in parallel-all-tests-nightly
> ---
>
> Key: IMPALA-10054
> URL: https://issues.apache.org/jira/browse/IMPALA-10054
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.0
>Reporter: Attila Jeges
>Assignee: Riza Suminto
>Priority: Blocker
> Labels: broken-build, flaky
> Fix For: Impala 4.0
>
>
> test_multiple_sort_run_bytes_limits introduced in IMPALA-6692 seems to be
> flaky.
> Jenkins job that triggered the error:
> https://jenkins.impala.io/job/parallel-all-tests-nightly/1173
> Failing job:
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/2899/testReport/
> {code}
> Stacktrace
> query_test/test_sort.py:89: in test_multiple_sort_run_bytes_limits
> assert "SpilledRuns: " + spilled_runs in query_result.runtime_profile
> E assert ('SpilledRuns: ' + '3') in 'Query
> (id=404da0b1e56e7248:120789cd):\n DEBUG MODE WARNING: Query profile
> created while running a DEBUG buil... 27.999ms\n - WriteIoBytes:
> 0\n - WriteIoOps: 0 (0)\n - WriteIoWaitTime:
> 0.000ns\n'
> E+ where 'Query (id=404da0b1e56e7248:120789cd):\n DEBUG MODE
> WARNING: Query profile created while running a DEBUG buil... 27.999ms\n
> - WriteIoBytes: 0\n - WriteIoOps: 0 (0)\n -
> WriteIoWaitTime: 0.000ns\n' =
> 0x7f51da77fb50>.runtime_profile
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10055) DCHECK was hit while executing e2e test TestQueries::test_subquery
[
https://issues.apache.org/jira/browse/IMPALA-10055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172422#comment-17172422
]
Attila Jeges commented on IMPALA-10055:
---
[~boroknagyz] I'm assigning this to you as it looks like the bug was introduced
in IMPALA-9515.
> DCHECK was hit while executing e2e test TestQueries::test_subquery
> --
>
> Key: IMPALA-10055
> URL: https://issues.apache.org/jira/browse/IMPALA-10055
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.0
>Reporter: Attila Jeges
>Assignee: Zoltán Borók-Nagy
>Priority: Blocker
> Labels: broken-build, crash, flaky
> Fix For: Impala 4.0
>
>
> A DCHECK was hit while executing e2e test. Time frame suggests that it
> possibly happened while executing TestQueries::test_subquery:
> {code}
> query_test/test_queries.py:149: in test_subquery
> self.run_test_case('QueryTest/subquery', vector)
> common/impala_test_suite.py:662: in run_test_case
> result = exec_fn(query, user=test_section.get('USER', '').strip() or None)
> common/impala_test_suite.py:600: in __exec_in_impala
> result = self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:909: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:334: in execute
> r = self.__fetch_results(handle, profile_format=profile_format)
> common/impala_connection.py:436: in __fetch_results
> result_tuples = cursor.fetchall()
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py:532:
> in fetchall
> self._wait_to_finish()
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py:405:
> in _wait_to_finish
> resp = self._last_operation._rpc('GetOperationStatus', req)
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py:992:
> in _rpc
> response = self._execute(func_name, request)
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py:1023:
> in _execute
> .format(self.retries))
> E HiveServer2Error: Failed after retrying 3 times
> {code}
> impalad log:
> {code}
> Log file created at: 2020/08/05 17:34:30
> Running on machine:
> impala-ec2-centos74-m5-4xlarge-ondemand-18a5.vpc.cloudera.com
> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
> F0805 17:34:30.003247 10887 orc-column-readers.cc:423]
> c34e87376f496a53:7ba6a2e40002] Check failed: (scanner_->row_batches_nee
> d_validation_ && scanner_->scan_node_->IsZeroSlotTableScan()) ||
> scanner_->acid_original_file
> {code}
> Stack trace:
> {code}
> CORE: ./fe/core.1596674070.14179.impalad
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/be/build/lat'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7efd6ec6e1f7 in raise () from /lib64/libc.so.6
> To enable execution of this file add
> add-auto-load-safe-path
> /data0/jenkins/workspace/impala-cdpd-master-core-ubsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib64/libstdc++.so.6.0.24-gdb.py
> line to your configuration file "/var/lib/jenkins/.gdbinit".
> To completely disable this security protection add
> set auto-load safe-path /
> line to your configuration file "/var/lib/jenkins/.gdbinit".
> For more information about this security protection see the
> "Auto-loading safe path" section in the GDB manual. E.g., run from the shell:
> info "(gdb)Auto-loading safe path"
> #0 0x7efd6ec6e1f7 in raise () from /lib64/libc.so.6
> #1 0x7efd6ec6f8e8 in abort () from /lib64/libc.so.6
> #2 0x086b8ea4 in google::DumpStackTraceAndExit() ()
> #3 0x086ae25d in google::LogMessage::Fail() ()
> #4 0x086afb4d in google::LogMessage::SendToLog() ()
> #5 0x086adbbb in google::LogMessage::Flush() ()
> #6 0x086b17b9 in google::LogMessageFatal::~LogMessageFatal() ()
> #7 0x0388e10a in impala::OrcStructReader::TopLevelReadValueBatch
> (this=0x61162630, scratch_batch=0x824831e0, pool=0x82483258) at
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/be/src/exec/orc-column-readers.cc:421
> #8 0x03810c92 in impala::HdfsOrcScanner::TransferTuples
> (this=0x27143c00, dst_batch=0x2e5ca820) at
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/be/src/exec/hdfs-orc-scanner.cc:808
> #9 0x03814e2a in
[jira] [Updated] (IMPALA-10054) test_multiple_sort_run_bytes_limits fails in parallel-all-tests-nightly
[
https://issues.apache.org/jira/browse/IMPALA-10054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10054:
--
Issue Type: Bug (was: Improvement)
> test_multiple_sort_run_bytes_limits fails in parallel-all-tests-nightly
> ---
>
> Key: IMPALA-10054
> URL: https://issues.apache.org/jira/browse/IMPALA-10054
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 4.0
>Reporter: Attila Jeges
>Assignee: Riza Suminto
>Priority: Blocker
> Labels: broken-build, flaky
> Fix For: Impala 4.0
>
>
> test_multiple_sort_run_bytes_limits introduced in IMPALA-6692 seems to be
> flaky.
> Jenkins job that triggered the error:
> https://jenkins.impala.io/job/parallel-all-tests-nightly/1173
> Failing job:
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/2899/testReport/
> {code}
> Stacktrace
> query_test/test_sort.py:89: in test_multiple_sort_run_bytes_limits
> assert "SpilledRuns: " + spilled_runs in query_result.runtime_profile
> E assert ('SpilledRuns: ' + '3') in 'Query
> (id=404da0b1e56e7248:120789cd):\n DEBUG MODE WARNING: Query profile
> created while running a DEBUG buil... 27.999ms\n - WriteIoBytes:
> 0\n - WriteIoOps: 0 (0)\n - WriteIoWaitTime:
> 0.000ns\n'
> E+ where 'Query (id=404da0b1e56e7248:120789cd):\n DEBUG MODE
> WARNING: Query profile created while running a DEBUG buil... 27.999ms\n
> - WriteIoBytes: 0\n - WriteIoOps: 0 (0)\n -
> WriteIoWaitTime: 0.000ns\n' =
> 0x7f51da77fb50>.runtime_profile
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-10050) DCHECK was hit possibly while executing TestFailpoints::test_failpoints
[
https://issues.apache.org/jira/browse/IMPALA-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10050:
--
Labels: broken-build crash flaky (was: broken-build crash)
> DCHECK was hit possibly while executing TestFailpoints::test_failpoints
> ---
>
> Key: IMPALA-10050
> URL: https://issues.apache.org/jira/browse/IMPALA-10050
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.0
>Reporter: Attila Jeges
>Assignee: Riza Suminto
>Priority: Blocker
> Labels: broken-build, crash, flaky
> Fix For: Impala 4.0
>
>
> A DCHECK was hit during ASAN core e2e tests. Time-frame suggests that it
> happened while executing TestFailpoints::test_failpoints e2e test.
> {code}
> 10:56:38 TestFailpoints.test_failpoints[protocol: beeswax | table_format:
> avro/snap/block | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | mt_dop: 4 |
> location: PREPARE | action: MEM_LIMIT_EXCEEDED | query: select 1 from
> alltypessmall a join alltypessmall b on a.id = b.id]
> 10:56:38 failure/test_failpoints.py:128: in test_failpoints
> 10:56:38 self.execute_query(query, vector.get_value('exec_option'))
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:811:
> in wrapper
> 10:56:38 return function(*args, **kwargs)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:843:
> in execute_query
> 10:56:38 return self.__execute_query(self.client, query, query_options)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:909:
> in __execute_query
> 10:56:38 return impalad_client.execute(query, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_connection.py:205:
> in execute
> 10:56:38 return self.__beeswax_client.execute(sql_stmt, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:187:
> in execute
> 10:56:38 handle = self.__execute_query(query_string.strip(), user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:365:
> in __execute_query
> 10:56:38 self.wait_for_finished(handle)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:386:
> in wait_for_finished
> 10:56:38 raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> 10:56:38 E ImpalaBeeswaxException: ImpalaBeeswaxException:
> 10:56:38 EQuery aborted:RPC from 127.0.0.1:27000 to 127.0.0.1:27002 failed
> 10:56:38 E TransmitData() to 127.0.0.1:27002 failed: Network error: Client
> connection negotiation failed: client connection to 127.0.0.1:27002: connect:
> Connection refused (error 111)
> {code}
> Impalad log:
> {code}
> Log file created at: 2020/08/05 01:52:56
> Running on machine:
> impala-ec2-centos74-r5-4xlarge-ondemand-017c.vpc.cloudera.com
> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
> F0805 01:52:56.979769 17313 query-state.cc:803]
> 3941a3d92a71e242:15c963f3] Check failed: is_cancelled_.Load() == 1 (0
> vs. 1)
> {code}
> Stack trace
> {code}
> Thread 368 (crashed)
> 0 libc-2.17.so + 0x351f7
> rax = 0x rdx = 0x0006
> rcx = 0x rbx = 0x0004
> rsi = 0x43a1 rdi = 0x37e4
> rbp = 0x7efcd4c53080 rsp = 0x7efcd4c52d08
> r8 = 0xr9 = 0x7efcd4c52b80
> r10 = 0x0008 r11 = 0x0206
> r12 = 0x093de7c0 r13 = 0x0086
> r14 = 0x093de7c4 r15 = 0x093d6de0
> rip = 0x7f05c9d231f7
> Found by: given as instruction pointer in context
> 1 impalad!google::LogMessage::Flush() + 0x1eb
> rbp = 0x7efcd4c53250 rsp = 0x7efcd4c53090
> rip = 0x05727e3b
> Found by: previous frame's frame pointer
> 2 impalad!google::LogMessageFatal::~LogMessageFatal() + 0x9
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53130 r12 = 0x0fe01a982628
> r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
> r15 = 0x7efcd4c53270 rip = 0x0572ba39
> Found by: call frame info
> 3 impalad!impala::QueryState::MonitorFInstances() [query-state.cc : 803 +
> 0x45]
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp =
[jira] [Updated] (IMPALA-10054) test_multiple_sort_run_bytes_limits fails in parallel-all-tests-nightly
[
https://issues.apache.org/jira/browse/IMPALA-10054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10054:
--
Labels: broken-build flaky (was: broken-build)
> test_multiple_sort_run_bytes_limits fails in parallel-all-tests-nightly
> ---
>
> Key: IMPALA-10054
> URL: https://issues.apache.org/jira/browse/IMPALA-10054
> Project: IMPALA
> Issue Type: Improvement
>Affects Versions: Impala 4.0
>Reporter: Attila Jeges
>Assignee: Riza Suminto
>Priority: Blocker
> Labels: broken-build, flaky
> Fix For: Impala 4.0
>
>
> test_multiple_sort_run_bytes_limits introduced in IMPALA-6692 seems to be
> flaky.
> Jenkins job that triggered the error:
> https://jenkins.impala.io/job/parallel-all-tests-nightly/1173
> Failing job:
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/2899/testReport/
> {code}
> Stacktrace
> query_test/test_sort.py:89: in test_multiple_sort_run_bytes_limits
> assert "SpilledRuns: " + spilled_runs in query_result.runtime_profile
> E assert ('SpilledRuns: ' + '3') in 'Query
> (id=404da0b1e56e7248:120789cd):\n DEBUG MODE WARNING: Query profile
> created while running a DEBUG buil... 27.999ms\n - WriteIoBytes:
> 0\n - WriteIoOps: 0 (0)\n - WriteIoWaitTime:
> 0.000ns\n'
> E+ where 'Query (id=404da0b1e56e7248:120789cd):\n DEBUG MODE
> WARNING: Query profile created while running a DEBUG buil... 27.999ms\n
> - WriteIoBytes: 0\n - WriteIoOps: 0 (0)\n -
> WriteIoWaitTime: 0.000ns\n' =
> 0x7f51da77fb50>.runtime_profile
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-10050) DCHECK was hit possibly while executing TestFailpoints::test_failpoints
[
https://issues.apache.org/jira/browse/IMPALA-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17172412#comment-17172412
]
Attila Jeges commented on IMPALA-10050:
---
[~rizaon] randomly assigning to you. Please feel free to reassign.
> DCHECK was hit possibly while executing TestFailpoints::test_failpoints
> ---
>
> Key: IMPALA-10050
> URL: https://issues.apache.org/jira/browse/IMPALA-10050
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.0
>Reporter: Attila Jeges
>Assignee: Riza Suminto
>Priority: Blocker
> Labels: broken-build, crash
> Fix For: Impala 4.0
>
>
> A DCHECK was hit during ASAN core e2e tests. Time-frame suggests that it
> happened while executing TestFailpoints::test_failpoints e2e test.
> {code}
> 10:56:38 TestFailpoints.test_failpoints[protocol: beeswax | table_format:
> avro/snap/block | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | mt_dop: 4 |
> location: PREPARE | action: MEM_LIMIT_EXCEEDED | query: select 1 from
> alltypessmall a join alltypessmall b on a.id = b.id]
> 10:56:38 failure/test_failpoints.py:128: in test_failpoints
> 10:56:38 self.execute_query(query, vector.get_value('exec_option'))
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:811:
> in wrapper
> 10:56:38 return function(*args, **kwargs)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:843:
> in execute_query
> 10:56:38 return self.__execute_query(self.client, query, query_options)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:909:
> in __execute_query
> 10:56:38 return impalad_client.execute(query, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_connection.py:205:
> in execute
> 10:56:38 return self.__beeswax_client.execute(sql_stmt, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:187:
> in execute
> 10:56:38 handle = self.__execute_query(query_string.strip(), user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:365:
> in __execute_query
> 10:56:38 self.wait_for_finished(handle)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:386:
> in wait_for_finished
> 10:56:38 raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> 10:56:38 E ImpalaBeeswaxException: ImpalaBeeswaxException:
> 10:56:38 EQuery aborted:RPC from 127.0.0.1:27000 to 127.0.0.1:27002 failed
> 10:56:38 E TransmitData() to 127.0.0.1:27002 failed: Network error: Client
> connection negotiation failed: client connection to 127.0.0.1:27002: connect:
> Connection refused (error 111)
> {code}
> Impalad log:
> {code}
> Log file created at: 2020/08/05 01:52:56
> Running on machine:
> impala-ec2-centos74-r5-4xlarge-ondemand-017c.vpc.cloudera.com
> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
> F0805 01:52:56.979769 17313 query-state.cc:803]
> 3941a3d92a71e242:15c963f3] Check failed: is_cancelled_.Load() == 1 (0
> vs. 1)
> {code}
> Stack trace
> {code}
> Thread 368 (crashed)
> 0 libc-2.17.so + 0x351f7
> rax = 0x rdx = 0x0006
> rcx = 0x rbx = 0x0004
> rsi = 0x43a1 rdi = 0x37e4
> rbp = 0x7efcd4c53080 rsp = 0x7efcd4c52d08
> r8 = 0xr9 = 0x7efcd4c52b80
> r10 = 0x0008 r11 = 0x0206
> r12 = 0x093de7c0 r13 = 0x0086
> r14 = 0x093de7c4 r15 = 0x093d6de0
> rip = 0x7f05c9d231f7
> Found by: given as instruction pointer in context
> 1 impalad!google::LogMessage::Flush() + 0x1eb
> rbp = 0x7efcd4c53250 rsp = 0x7efcd4c53090
> rip = 0x05727e3b
> Found by: previous frame's frame pointer
> 2 impalad!google::LogMessageFatal::~LogMessageFatal() + 0x9
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53130 r12 = 0x0fe01a982628
> r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
> r15 = 0x7efcd4c53270 rip = 0x0572ba39
> Found by: call frame info
> 3 impalad!impala::QueryState::MonitorFInstances() [query-state.cc : 803 +
> 0x45]
> rbx = 0x7efcd4c532a0 rbp =
[jira] [Assigned] (IMPALA-10050) DCHECK was hit possibly while executing TestFailpoints::test_failpoints
[
https://issues.apache.org/jira/browse/IMPALA-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges reassigned IMPALA-10050:
-
Assignee: Riza Suminto
> DCHECK was hit possibly while executing TestFailpoints::test_failpoints
> ---
>
> Key: IMPALA-10050
> URL: https://issues.apache.org/jira/browse/IMPALA-10050
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.0
>Reporter: Attila Jeges
>Assignee: Riza Suminto
>Priority: Blocker
> Labels: broken-build, crash
> Fix For: Impala 4.0
>
>
> A DCHECK was hit during ASAN core e2e tests. Time-frame suggests that it
> happened while executing TestFailpoints::test_failpoints e2e test.
> {code}
> 10:56:38 TestFailpoints.test_failpoints[protocol: beeswax | table_format:
> avro/snap/block | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | mt_dop: 4 |
> location: PREPARE | action: MEM_LIMIT_EXCEEDED | query: select 1 from
> alltypessmall a join alltypessmall b on a.id = b.id]
> 10:56:38 failure/test_failpoints.py:128: in test_failpoints
> 10:56:38 self.execute_query(query, vector.get_value('exec_option'))
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:811:
> in wrapper
> 10:56:38 return function(*args, **kwargs)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:843:
> in execute_query
> 10:56:38 return self.__execute_query(self.client, query, query_options)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:909:
> in __execute_query
> 10:56:38 return impalad_client.execute(query, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_connection.py:205:
> in execute
> 10:56:38 return self.__beeswax_client.execute(sql_stmt, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:187:
> in execute
> 10:56:38 handle = self.__execute_query(query_string.strip(), user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:365:
> in __execute_query
> 10:56:38 self.wait_for_finished(handle)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:386:
> in wait_for_finished
> 10:56:38 raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> 10:56:38 E ImpalaBeeswaxException: ImpalaBeeswaxException:
> 10:56:38 EQuery aborted:RPC from 127.0.0.1:27000 to 127.0.0.1:27002 failed
> 10:56:38 E TransmitData() to 127.0.0.1:27002 failed: Network error: Client
> connection negotiation failed: client connection to 127.0.0.1:27002: connect:
> Connection refused (error 111)
> {code}
> Impalad log:
> {code}
> Log file created at: 2020/08/05 01:52:56
> Running on machine:
> impala-ec2-centos74-r5-4xlarge-ondemand-017c.vpc.cloudera.com
> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
> F0805 01:52:56.979769 17313 query-state.cc:803]
> 3941a3d92a71e242:15c963f3] Check failed: is_cancelled_.Load() == 1 (0
> vs. 1)
> {code}
> Stack trace
> {code}
> Thread 368 (crashed)
> 0 libc-2.17.so + 0x351f7
> rax = 0x rdx = 0x0006
> rcx = 0x rbx = 0x0004
> rsi = 0x43a1 rdi = 0x37e4
> rbp = 0x7efcd4c53080 rsp = 0x7efcd4c52d08
> r8 = 0xr9 = 0x7efcd4c52b80
> r10 = 0x0008 r11 = 0x0206
> r12 = 0x093de7c0 r13 = 0x0086
> r14 = 0x093de7c4 r15 = 0x093d6de0
> rip = 0x7f05c9d231f7
> Found by: given as instruction pointer in context
> 1 impalad!google::LogMessage::Flush() + 0x1eb
> rbp = 0x7efcd4c53250 rsp = 0x7efcd4c53090
> rip = 0x05727e3b
> Found by: previous frame's frame pointer
> 2 impalad!google::LogMessageFatal::~LogMessageFatal() + 0x9
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53130 r12 = 0x0fe01a982628
> r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
> r15 = 0x7efcd4c53270 rip = 0x0572ba39
> Found by: call frame info
> 3 impalad!impala::QueryState::MonitorFInstances() [query-state.cc : 803 +
> 0x45]
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53140 r12 = 0x0fe01a982628
>
[jira] [Updated] (IMPALA-10055) DCHECK was hit while executing e2e test TestQueries::test_subquery
[
https://issues.apache.org/jira/browse/IMPALA-10055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10055:
--
Labels: broken-build crash (was: )
> DCHECK was hit while executing e2e test TestQueries::test_subquery
> --
>
> Key: IMPALA-10055
> URL: https://issues.apache.org/jira/browse/IMPALA-10055
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.0
>Reporter: Attila Jeges
>Assignee: Zoltán Borók-Nagy
>Priority: Blocker
> Labels: broken-build, crash
> Fix For: Impala 4.0
>
>
> A DCHECK was hit while executing e2e test. Time frame suggests that it
> possibly happened while executing TestQueries::test_subquery:
> {code}
> query_test/test_queries.py:149: in test_subquery
> self.run_test_case('QueryTest/subquery', vector)
> common/impala_test_suite.py:662: in run_test_case
> result = exec_fn(query, user=test_section.get('USER', '').strip() or None)
> common/impala_test_suite.py:600: in __exec_in_impala
> result = self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:909: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:334: in execute
> r = self.__fetch_results(handle, profile_format=profile_format)
> common/impala_connection.py:436: in __fetch_results
> result_tuples = cursor.fetchall()
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py:532:
> in fetchall
> self._wait_to_finish()
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py:405:
> in _wait_to_finish
> resp = self._last_operation._rpc('GetOperationStatus', req)
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py:992:
> in _rpc
> response = self._execute(func_name, request)
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py:1023:
> in _execute
> .format(self.retries))
> E HiveServer2Error: Failed after retrying 3 times
> {code}
> impalad log:
> {code}
> Log file created at: 2020/08/05 17:34:30
> Running on machine:
> impala-ec2-centos74-m5-4xlarge-ondemand-18a5.vpc.cloudera.com
> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
> F0805 17:34:30.003247 10887 orc-column-readers.cc:423]
> c34e87376f496a53:7ba6a2e40002] Check failed: (scanner_->row_batches_nee
> d_validation_ && scanner_->scan_node_->IsZeroSlotTableScan()) ||
> scanner_->acid_original_file
> {code}
> Stack trace:
> {code}
> CORE: ./fe/core.1596674070.14179.impalad
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/be/build/lat'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7efd6ec6e1f7 in raise () from /lib64/libc.so.6
> To enable execution of this file add
> add-auto-load-safe-path
> /data0/jenkins/workspace/impala-cdpd-master-core-ubsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib64/libstdc++.so.6.0.24-gdb.py
> line to your configuration file "/var/lib/jenkins/.gdbinit".
> To completely disable this security protection add
> set auto-load safe-path /
> line to your configuration file "/var/lib/jenkins/.gdbinit".
> For more information about this security protection see the
> "Auto-loading safe path" section in the GDB manual. E.g., run from the shell:
> info "(gdb)Auto-loading safe path"
> #0 0x7efd6ec6e1f7 in raise () from /lib64/libc.so.6
> #1 0x7efd6ec6f8e8 in abort () from /lib64/libc.so.6
> #2 0x086b8ea4 in google::DumpStackTraceAndExit() ()
> #3 0x086ae25d in google::LogMessage::Fail() ()
> #4 0x086afb4d in google::LogMessage::SendToLog() ()
> #5 0x086adbbb in google::LogMessage::Flush() ()
> #6 0x086b17b9 in google::LogMessageFatal::~LogMessageFatal() ()
> #7 0x0388e10a in impala::OrcStructReader::TopLevelReadValueBatch
> (this=0x61162630, scratch_batch=0x824831e0, pool=0x82483258) at
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/be/src/exec/orc-column-readers.cc:421
> #8 0x03810c92 in impala::HdfsOrcScanner::TransferTuples
> (this=0x27143c00, dst_batch=0x2e5ca820) at
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/be/src/exec/hdfs-orc-scanner.cc:808
> #9 0x03814e2a in impala::HdfsOrcScanner::AssembleRows
> (this=0x27143c00, row_batch=0x2e5ca820) at
>
[jira] [Updated] (IMPALA-10055) DCHECK was hit while executing e2e test TestQueries::test_subquery
[
https://issues.apache.org/jira/browse/IMPALA-10055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10055:
--
Labels: broken-build crash flaky (was: broken-build crash)
> DCHECK was hit while executing e2e test TestQueries::test_subquery
> --
>
> Key: IMPALA-10055
> URL: https://issues.apache.org/jira/browse/IMPALA-10055
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.0
>Reporter: Attila Jeges
>Assignee: Zoltán Borók-Nagy
>Priority: Blocker
> Labels: broken-build, crash, flaky
> Fix For: Impala 4.0
>
>
> A DCHECK was hit while executing e2e test. Time frame suggests that it
> possibly happened while executing TestQueries::test_subquery:
> {code}
> query_test/test_queries.py:149: in test_subquery
> self.run_test_case('QueryTest/subquery', vector)
> common/impala_test_suite.py:662: in run_test_case
> result = exec_fn(query, user=test_section.get('USER', '').strip() or None)
> common/impala_test_suite.py:600: in __exec_in_impala
> result = self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:909: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:334: in execute
> r = self.__fetch_results(handle, profile_format=profile_format)
> common/impala_connection.py:436: in __fetch_results
> result_tuples = cursor.fetchall()
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py:532:
> in fetchall
> self._wait_to_finish()
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py:405:
> in _wait_to_finish
> resp = self._last_operation._rpc('GetOperationStatus', req)
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py:992:
> in _rpc
> response = self._execute(func_name, request)
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py:1023:
> in _execute
> .format(self.retries))
> E HiveServer2Error: Failed after retrying 3 times
> {code}
> impalad log:
> {code}
> Log file created at: 2020/08/05 17:34:30
> Running on machine:
> impala-ec2-centos74-m5-4xlarge-ondemand-18a5.vpc.cloudera.com
> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
> F0805 17:34:30.003247 10887 orc-column-readers.cc:423]
> c34e87376f496a53:7ba6a2e40002] Check failed: (scanner_->row_batches_nee
> d_validation_ && scanner_->scan_node_->IsZeroSlotTableScan()) ||
> scanner_->acid_original_file
> {code}
> Stack trace:
> {code}
> CORE: ./fe/core.1596674070.14179.impalad
> BINARY: ./be/build/latest/service/impalad
> Core was generated by
> `/data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/be/build/lat'.
> Program terminated with signal SIGABRT, Aborted.
> #0 0x7efd6ec6e1f7 in raise () from /lib64/libc.so.6
> To enable execution of this file add
> add-auto-load-safe-path
> /data0/jenkins/workspace/impala-cdpd-master-core-ubsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib64/libstdc++.so.6.0.24-gdb.py
> line to your configuration file "/var/lib/jenkins/.gdbinit".
> To completely disable this security protection add
> set auto-load safe-path /
> line to your configuration file "/var/lib/jenkins/.gdbinit".
> For more information about this security protection see the
> "Auto-loading safe path" section in the GDB manual. E.g., run from the shell:
> info "(gdb)Auto-loading safe path"
> #0 0x7efd6ec6e1f7 in raise () from /lib64/libc.so.6
> #1 0x7efd6ec6f8e8 in abort () from /lib64/libc.so.6
> #2 0x086b8ea4 in google::DumpStackTraceAndExit() ()
> #3 0x086ae25d in google::LogMessage::Fail() ()
> #4 0x086afb4d in google::LogMessage::SendToLog() ()
> #5 0x086adbbb in google::LogMessage::Flush() ()
> #6 0x086b17b9 in google::LogMessageFatal::~LogMessageFatal() ()
> #7 0x0388e10a in impala::OrcStructReader::TopLevelReadValueBatch
> (this=0x61162630, scratch_batch=0x824831e0, pool=0x82483258) at
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/be/src/exec/orc-column-readers.cc:421
> #8 0x03810c92 in impala::HdfsOrcScanner::TransferTuples
> (this=0x27143c00, dst_batch=0x2e5ca820) at
> /data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/be/src/exec/hdfs-orc-scanner.cc:808
> #9 0x03814e2a in impala::HdfsOrcScanner::AssembleRows
> (this=0x27143c00,
[jira] [Created] (IMPALA-10055) DCHECK was hit while executing e2e test TestQueries::test_subquery
Attila Jeges created IMPALA-10055:
-
Summary: DCHECK was hit while executing e2e test
TestQueries::test_subquery
Key: IMPALA-10055
URL: https://issues.apache.org/jira/browse/IMPALA-10055
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 4.0
Reporter: Attila Jeges
Assignee: Zoltán Borók-Nagy
Fix For: Impala 4.0
A DCHECK was hit while executing e2e test. Time frame suggests that it possibly
happened while executing TestQueries::test_subquery:
{code}
query_test/test_queries.py:149: in test_subquery
self.run_test_case('QueryTest/subquery', vector)
common/impala_test_suite.py:662: in run_test_case
result = exec_fn(query, user=test_section.get('USER', '').strip() or None)
common/impala_test_suite.py:600: in __exec_in_impala
result = self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:909: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:334: in execute
r = self.__fetch_results(handle, profile_format=profile_format)
common/impala_connection.py:436: in __fetch_results
result_tuples = cursor.fetchall()
/data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py:532:
in fetchall
self._wait_to_finish()
/data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py:405:
in _wait_to_finish
resp = self._last_operation._rpc('GetOperationStatus', req)
/data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py:992:
in _rpc
response = self._execute(func_name, request)
/data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/infra/python/env-gcc7.5.0/lib/python2.7/site-packages/impala/hiveserver2.py:1023:
in _execute
.format(self.retries))
E HiveServer2Error: Failed after retrying 3 times
{code}
impalad log:
{code}
Log file created at: 2020/08/05 17:34:30
Running on machine:
impala-ec2-centos74-m5-4xlarge-ondemand-18a5.vpc.cloudera.com
Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
F0805 17:34:30.003247 10887 orc-column-readers.cc:423]
c34e87376f496a53:7ba6a2e40002] Check failed: (scanner_->row_batches_nee
d_validation_ && scanner_->scan_node_->IsZeroSlotTableScan()) ||
scanner_->acid_original_file
{code}
Stack trace:
{code}
CORE: ./fe/core.1596674070.14179.impalad
BINARY: ./be/build/latest/service/impalad
Core was generated by
`/data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/be/build/lat'.
Program terminated with signal SIGABRT, Aborted.
#0 0x7efd6ec6e1f7 in raise () from /lib64/libc.so.6
To enable execution of this file add
add-auto-load-safe-path
/data0/jenkins/workspace/impala-cdpd-master-core-ubsan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib64/libstdc++.so.6.0.24-gdb.py
line to your configuration file "/var/lib/jenkins/.gdbinit".
To completely disable this security protection add
set auto-load safe-path /
line to your configuration file "/var/lib/jenkins/.gdbinit".
For more information about this security protection see the
"Auto-loading safe path" section in the GDB manual. E.g., run from the shell:
info "(gdb)Auto-loading safe path"
#0 0x7efd6ec6e1f7 in raise () from /lib64/libc.so.6
#1 0x7efd6ec6f8e8 in abort () from /lib64/libc.so.6
#2 0x086b8ea4 in google::DumpStackTraceAndExit() ()
#3 0x086ae25d in google::LogMessage::Fail() ()
#4 0x086afb4d in google::LogMessage::SendToLog() ()
#5 0x086adbbb in google::LogMessage::Flush() ()
#6 0x086b17b9 in google::LogMessageFatal::~LogMessageFatal() ()
#7 0x0388e10a in impala::OrcStructReader::TopLevelReadValueBatch
(this=0x61162630, scratch_batch=0x824831e0, pool=0x82483258) at
/data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/be/src/exec/orc-column-readers.cc:421
#8 0x03810c92 in impala::HdfsOrcScanner::TransferTuples
(this=0x27143c00, dst_batch=0x2e5ca820) at
/data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/be/src/exec/hdfs-orc-scanner.cc:808
#9 0x03814e2a in impala::HdfsOrcScanner::AssembleRows
(this=0x27143c00, row_batch=0x2e5ca820) at
/data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/be/src/exec/hdfs-orc-scanner.cc:785
#10 0x0380f5fe in impala::HdfsOrcScanner::GetNextInternal
(this=0x27143c00, row_batch=0x2e5ca820) at
/data/jenkins/workspace/impala-cdpd-master-core-ubsan/repos/Impala/be/src/exec/hdfs-orc-scanner.cc:654
#11 0x0380c2bb in impala::HdfsOrcScanner::ProcessSplit
(this=0x27143c00) at
[jira] [Updated] (IMPALA-10054) test_multiple_sort_run_bytes_limits fails in parallel-all-tests-nightly
[
https://issues.apache.org/jira/browse/IMPALA-10054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10054:
--
Labels: broken-build (was: )
> test_multiple_sort_run_bytes_limits fails in parallel-all-tests-nightly
> ---
>
> Key: IMPALA-10054
> URL: https://issues.apache.org/jira/browse/IMPALA-10054
> Project: IMPALA
> Issue Type: Improvement
>Affects Versions: Impala 4.0
>Reporter: Attila Jeges
>Assignee: Riza Suminto
>Priority: Blocker
> Labels: broken-build
> Fix For: Impala 4.0
>
>
> test_multiple_sort_run_bytes_limits introduced in IMPALA-6692 seems to be
> flaky.
> Jenkins job that triggered the error:
> https://jenkins.impala.io/job/parallel-all-tests-nightly/1173
> Failing job:
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/2899/testReport/
> {code}
> Stacktrace
> query_test/test_sort.py:89: in test_multiple_sort_run_bytes_limits
> assert "SpilledRuns: " + spilled_runs in query_result.runtime_profile
> E assert ('SpilledRuns: ' + '3') in 'Query
> (id=404da0b1e56e7248:120789cd):\n DEBUG MODE WARNING: Query profile
> created while running a DEBUG buil... 27.999ms\n - WriteIoBytes:
> 0\n - WriteIoOps: 0 (0)\n - WriteIoWaitTime:
> 0.000ns\n'
> E+ where 'Query (id=404da0b1e56e7248:120789cd):\n DEBUG MODE
> WARNING: Query profile created while running a DEBUG buil... 27.999ms\n
> - WriteIoBytes: 0\n - WriteIoOps: 0 (0)\n -
> WriteIoWaitTime: 0.000ns\n' =
> 0x7f51da77fb50>.runtime_profile
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-10054) test_multiple_sort_run_bytes_limits fails in parallel-all-tests-nightly
Attila Jeges created IMPALA-10054:
-
Summary: test_multiple_sort_run_bytes_limits fails in
parallel-all-tests-nightly
Key: IMPALA-10054
URL: https://issues.apache.org/jira/browse/IMPALA-10054
Project: IMPALA
Issue Type: Improvement
Affects Versions: Impala 4.0
Reporter: Attila Jeges
Assignee: Riza Suminto
Fix For: Impala 4.0
test_multiple_sort_run_bytes_limits introduced in IMPALA-6692 seems to be
flaky.
Jenkins job that triggered the error:
https://jenkins.impala.io/job/parallel-all-tests-nightly/1173
Failing job:
https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/2899/testReport/
{code}
Stacktrace
query_test/test_sort.py:89: in test_multiple_sort_run_bytes_limits
assert "SpilledRuns: " + spilled_runs in query_result.runtime_profile
E assert ('SpilledRuns: ' + '3') in 'Query
(id=404da0b1e56e7248:120789cd):\n DEBUG MODE WARNING: Query profile
created while running a DEBUG buil... 27.999ms\n - WriteIoBytes:
0\n - WriteIoOps: 0 (0)\n - WriteIoWaitTime: 0.000ns\n'
E+ where 'Query (id=404da0b1e56e7248:120789cd):\n DEBUG MODE
WARNING: Query profile created while running a DEBUG buil... 27.999ms\n
- WriteIoBytes: 0\n - WriteIoOps: 0 (0)\n -
WriteIoWaitTime: 0.000ns\n' = .runtime_profile
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-10050) DCHECK was hit possibly while executing TestFailpoints::test_failpoints
[
https://issues.apache.org/jira/browse/IMPALA-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10050:
--
Affects Version/s: Impala 4.0
> DCHECK was hit possibly while executing TestFailpoints::test_failpoints
> ---
>
> Key: IMPALA-10050
> URL: https://issues.apache.org/jira/browse/IMPALA-10050
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.0
>Reporter: Attila Jeges
>Priority: Blocker
> Labels: broken-build, crash
>
> A DCHECK was hit during ASAN core e2e tests. Time-frame suggests that it
> happened while executing TestFailpoints::test_failpoints e2e test.
> {code}
> 10:56:38 TestFailpoints.test_failpoints[protocol: beeswax | table_format:
> avro/snap/block | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | mt_dop: 4 |
> location: PREPARE | action: MEM_LIMIT_EXCEEDED | query: select 1 from
> alltypessmall a join alltypessmall b on a.id = b.id]
> 10:56:38 failure/test_failpoints.py:128: in test_failpoints
> 10:56:38 self.execute_query(query, vector.get_value('exec_option'))
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:811:
> in wrapper
> 10:56:38 return function(*args, **kwargs)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:843:
> in execute_query
> 10:56:38 return self.__execute_query(self.client, query, query_options)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:909:
> in __execute_query
> 10:56:38 return impalad_client.execute(query, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_connection.py:205:
> in execute
> 10:56:38 return self.__beeswax_client.execute(sql_stmt, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:187:
> in execute
> 10:56:38 handle = self.__execute_query(query_string.strip(), user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:365:
> in __execute_query
> 10:56:38 self.wait_for_finished(handle)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:386:
> in wait_for_finished
> 10:56:38 raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> 10:56:38 E ImpalaBeeswaxException: ImpalaBeeswaxException:
> 10:56:38 EQuery aborted:RPC from 127.0.0.1:27000 to 127.0.0.1:27002 failed
> 10:56:38 E TransmitData() to 127.0.0.1:27002 failed: Network error: Client
> connection negotiation failed: client connection to 127.0.0.1:27002: connect:
> Connection refused (error 111)
> {code}
> Impalad log:
> {code}
> Log file created at: 2020/08/05 01:52:56
> Running on machine:
> impala-ec2-centos74-r5-4xlarge-ondemand-017c.vpc.cloudera.com
> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
> F0805 01:52:56.979769 17313 query-state.cc:803]
> 3941a3d92a71e242:15c963f3] Check failed: is_cancelled_.Load() == 1 (0
> vs. 1)
> {code}
> Stack trace
> {code}
> Thread 368 (crashed)
> 0 libc-2.17.so + 0x351f7
> rax = 0x rdx = 0x0006
> rcx = 0x rbx = 0x0004
> rsi = 0x43a1 rdi = 0x37e4
> rbp = 0x7efcd4c53080 rsp = 0x7efcd4c52d08
> r8 = 0xr9 = 0x7efcd4c52b80
> r10 = 0x0008 r11 = 0x0206
> r12 = 0x093de7c0 r13 = 0x0086
> r14 = 0x093de7c4 r15 = 0x093d6de0
> rip = 0x7f05c9d231f7
> Found by: given as instruction pointer in context
> 1 impalad!google::LogMessage::Flush() + 0x1eb
> rbp = 0x7efcd4c53250 rsp = 0x7efcd4c53090
> rip = 0x05727e3b
> Found by: previous frame's frame pointer
> 2 impalad!google::LogMessageFatal::~LogMessageFatal() + 0x9
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53130 r12 = 0x0fe01a982628
> r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
> r15 = 0x7efcd4c53270 rip = 0x0572ba39
> Found by: call frame info
> 3 impalad!impala::QueryState::MonitorFInstances() [query-state.cc : 803 +
> 0x45]
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53140 r12 = 0x0fe01a982628
> r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
> r15 =
[jira] [Updated] (IMPALA-10050) DCHECK was hit possibly while executing TestFailpoints::test_failpoints
[
https://issues.apache.org/jira/browse/IMPALA-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10050:
--
Fix Version/s: Impala 4.0
> DCHECK was hit possibly while executing TestFailpoints::test_failpoints
> ---
>
> Key: IMPALA-10050
> URL: https://issues.apache.org/jira/browse/IMPALA-10050
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.0
>Reporter: Attila Jeges
>Priority: Blocker
> Labels: broken-build, crash
> Fix For: Impala 4.0
>
>
> A DCHECK was hit during ASAN core e2e tests. Time-frame suggests that it
> happened while executing TestFailpoints::test_failpoints e2e test.
> {code}
> 10:56:38 TestFailpoints.test_failpoints[protocol: beeswax | table_format:
> avro/snap/block | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | mt_dop: 4 |
> location: PREPARE | action: MEM_LIMIT_EXCEEDED | query: select 1 from
> alltypessmall a join alltypessmall b on a.id = b.id]
> 10:56:38 failure/test_failpoints.py:128: in test_failpoints
> 10:56:38 self.execute_query(query, vector.get_value('exec_option'))
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:811:
> in wrapper
> 10:56:38 return function(*args, **kwargs)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:843:
> in execute_query
> 10:56:38 return self.__execute_query(self.client, query, query_options)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:909:
> in __execute_query
> 10:56:38 return impalad_client.execute(query, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_connection.py:205:
> in execute
> 10:56:38 return self.__beeswax_client.execute(sql_stmt, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:187:
> in execute
> 10:56:38 handle = self.__execute_query(query_string.strip(), user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:365:
> in __execute_query
> 10:56:38 self.wait_for_finished(handle)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:386:
> in wait_for_finished
> 10:56:38 raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> 10:56:38 E ImpalaBeeswaxException: ImpalaBeeswaxException:
> 10:56:38 EQuery aborted:RPC from 127.0.0.1:27000 to 127.0.0.1:27002 failed
> 10:56:38 E TransmitData() to 127.0.0.1:27002 failed: Network error: Client
> connection negotiation failed: client connection to 127.0.0.1:27002: connect:
> Connection refused (error 111)
> {code}
> Impalad log:
> {code}
> Log file created at: 2020/08/05 01:52:56
> Running on machine:
> impala-ec2-centos74-r5-4xlarge-ondemand-017c.vpc.cloudera.com
> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
> F0805 01:52:56.979769 17313 query-state.cc:803]
> 3941a3d92a71e242:15c963f3] Check failed: is_cancelled_.Load() == 1 (0
> vs. 1)
> {code}
> Stack trace
> {code}
> Thread 368 (crashed)
> 0 libc-2.17.so + 0x351f7
> rax = 0x rdx = 0x0006
> rcx = 0x rbx = 0x0004
> rsi = 0x43a1 rdi = 0x37e4
> rbp = 0x7efcd4c53080 rsp = 0x7efcd4c52d08
> r8 = 0xr9 = 0x7efcd4c52b80
> r10 = 0x0008 r11 = 0x0206
> r12 = 0x093de7c0 r13 = 0x0086
> r14 = 0x093de7c4 r15 = 0x093d6de0
> rip = 0x7f05c9d231f7
> Found by: given as instruction pointer in context
> 1 impalad!google::LogMessage::Flush() + 0x1eb
> rbp = 0x7efcd4c53250 rsp = 0x7efcd4c53090
> rip = 0x05727e3b
> Found by: previous frame's frame pointer
> 2 impalad!google::LogMessageFatal::~LogMessageFatal() + 0x9
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53130 r12 = 0x0fe01a982628
> r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
> r15 = 0x7efcd4c53270 rip = 0x0572ba39
> Found by: call frame info
> 3 impalad!impala::QueryState::MonitorFInstances() [query-state.cc : 803 +
> 0x45]
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53140 r12 = 0x0fe01a982628
> r13 = 0x61d000da0a6c r14 =
[jira] [Updated] (IMPALA-10050) DCHECK was hit possibly while executing TestFailpoints::test_failpoints
[
https://issues.apache.org/jira/browse/IMPALA-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10050:
--
Priority: Blocker (was: Major)
> DCHECK was hit possibly while executing TestFailpoints::test_failpoints
> ---
>
> Key: IMPALA-10050
> URL: https://issues.apache.org/jira/browse/IMPALA-10050
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: Attila Jeges
>Priority: Blocker
> Labels: broken-build, crash
>
> A DCHECK was hit during ASAN core e2e tests. Time-frame suggests that it
> happened while executing TestFailpoints::test_failpoints e2e test.
> {code}
> 10:56:38 TestFailpoints.test_failpoints[protocol: beeswax | table_format:
> avro/snap/block | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | mt_dop: 4 |
> location: PREPARE | action: MEM_LIMIT_EXCEEDED | query: select 1 from
> alltypessmall a join alltypessmall b on a.id = b.id]
> 10:56:38 failure/test_failpoints.py:128: in test_failpoints
> 10:56:38 self.execute_query(query, vector.get_value('exec_option'))
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:811:
> in wrapper
> 10:56:38 return function(*args, **kwargs)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:843:
> in execute_query
> 10:56:38 return self.__execute_query(self.client, query, query_options)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:909:
> in __execute_query
> 10:56:38 return impalad_client.execute(query, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_connection.py:205:
> in execute
> 10:56:38 return self.__beeswax_client.execute(sql_stmt, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:187:
> in execute
> 10:56:38 handle = self.__execute_query(query_string.strip(), user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:365:
> in __execute_query
> 10:56:38 self.wait_for_finished(handle)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:386:
> in wait_for_finished
> 10:56:38 raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> 10:56:38 E ImpalaBeeswaxException: ImpalaBeeswaxException:
> 10:56:38 EQuery aborted:RPC from 127.0.0.1:27000 to 127.0.0.1:27002 failed
> 10:56:38 E TransmitData() to 127.0.0.1:27002 failed: Network error: Client
> connection negotiation failed: client connection to 127.0.0.1:27002: connect:
> Connection refused (error 111)
> {code}
> Impalad log:
> {code}
> Log file created at: 2020/08/05 01:52:56
> Running on machine:
> impala-ec2-centos74-r5-4xlarge-ondemand-017c.vpc.cloudera.com
> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
> F0805 01:52:56.979769 17313 query-state.cc:803]
> 3941a3d92a71e242:15c963f3] Check failed: is_cancelled_.Load() == 1 (0
> vs. 1)
> {code}
> Stack trace
> {code}
> Thread 368 (crashed)
> 0 libc-2.17.so + 0x351f7
> rax = 0x rdx = 0x0006
> rcx = 0x rbx = 0x0004
> rsi = 0x43a1 rdi = 0x37e4
> rbp = 0x7efcd4c53080 rsp = 0x7efcd4c52d08
> r8 = 0xr9 = 0x7efcd4c52b80
> r10 = 0x0008 r11 = 0x0206
> r12 = 0x093de7c0 r13 = 0x0086
> r14 = 0x093de7c4 r15 = 0x093d6de0
> rip = 0x7f05c9d231f7
> Found by: given as instruction pointer in context
> 1 impalad!google::LogMessage::Flush() + 0x1eb
> rbp = 0x7efcd4c53250 rsp = 0x7efcd4c53090
> rip = 0x05727e3b
> Found by: previous frame's frame pointer
> 2 impalad!google::LogMessageFatal::~LogMessageFatal() + 0x9
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53130 r12 = 0x0fe01a982628
> r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
> r15 = 0x7efcd4c53270 rip = 0x0572ba39
> Found by: call frame info
> 3 impalad!impala::QueryState::MonitorFInstances() [query-state.cc : 803 +
> 0x45]
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53140 r12 = 0x0fe01a982628
> r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
> r15 = 0x7efcd4c53270 rip =
[jira] [Updated] (IMPALA-10050) DCHECK is hit possibly while executing TestFailpoints::test_failpoints
[
https://issues.apache.org/jira/browse/IMPALA-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10050:
--
Issue Type: Bug (was: Improvement)
> DCHECK is hit possibly while executing TestFailpoints::test_failpoints
> --
>
> Key: IMPALA-10050
> URL: https://issues.apache.org/jira/browse/IMPALA-10050
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: Attila Jeges
>Priority: Major
> Labels: broken-build, crash
>
> A DCHECK was hit during ASAN core e2e tests. Time-frame suggests that it
> happened while executing TestFailpoints::test_failpoints e2e test.
> {code}
> 10:56:38 TestFailpoints.test_failpoints[protocol: beeswax | table_format:
> avro/snap/block | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | mt_dop: 4 |
> location: PREPARE | action: MEM_LIMIT_EXCEEDED | query: select 1 from
> alltypessmall a join alltypessmall b on a.id = b.id]
> 10:56:38 failure/test_failpoints.py:128: in test_failpoints
> 10:56:38 self.execute_query(query, vector.get_value('exec_option'))
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:811:
> in wrapper
> 10:56:38 return function(*args, **kwargs)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:843:
> in execute_query
> 10:56:38 return self.__execute_query(self.client, query, query_options)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:909:
> in __execute_query
> 10:56:38 return impalad_client.execute(query, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_connection.py:205:
> in execute
> 10:56:38 return self.__beeswax_client.execute(sql_stmt, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:187:
> in execute
> 10:56:38 handle = self.__execute_query(query_string.strip(), user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:365:
> in __execute_query
> 10:56:38 self.wait_for_finished(handle)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:386:
> in wait_for_finished
> 10:56:38 raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> 10:56:38 E ImpalaBeeswaxException: ImpalaBeeswaxException:
> 10:56:38 EQuery aborted:RPC from 127.0.0.1:27000 to 127.0.0.1:27002 failed
> 10:56:38 E TransmitData() to 127.0.0.1:27002 failed: Network error: Client
> connection negotiation failed: client connection to 127.0.0.1:27002: connect:
> Connection refused (error 111)
> {code}
> Impalad log:
> {code}
> Log file created at: 2020/08/05 01:52:56
> Running on machine:
> impala-ec2-centos74-r5-4xlarge-ondemand-017c.vpc.cloudera.com
> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
> F0805 01:52:56.979769 17313 query-state.cc:803]
> 3941a3d92a71e242:15c963f3] Check failed: is_cancelled_.Load() == 1 (0
> vs. 1)
> {code}
> Stack trace
> {code}
> Thread 368 (crashed)
> 0 libc-2.17.so + 0x351f7
> rax = 0x rdx = 0x0006
> rcx = 0x rbx = 0x0004
> rsi = 0x43a1 rdi = 0x37e4
> rbp = 0x7efcd4c53080 rsp = 0x7efcd4c52d08
> r8 = 0xr9 = 0x7efcd4c52b80
> r10 = 0x0008 r11 = 0x0206
> r12 = 0x093de7c0 r13 = 0x0086
> r14 = 0x093de7c4 r15 = 0x093d6de0
> rip = 0x7f05c9d231f7
> Found by: given as instruction pointer in context
> 1 impalad!google::LogMessage::Flush() + 0x1eb
> rbp = 0x7efcd4c53250 rsp = 0x7efcd4c53090
> rip = 0x05727e3b
> Found by: previous frame's frame pointer
> 2 impalad!google::LogMessageFatal::~LogMessageFatal() + 0x9
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53130 r12 = 0x0fe01a982628
> r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
> r15 = 0x7efcd4c53270 rip = 0x0572ba39
> Found by: call frame info
> 3 impalad!impala::QueryState::MonitorFInstances() [query-state.cc : 803 +
> 0x45]
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53140 r12 = 0x0fe01a982628
> r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
> r15 = 0x7efcd4c53270 rip =
[jira] [Updated] (IMPALA-10050) DCHECK was hit possibly while executing TestFailpoints::test_failpoints
[
https://issues.apache.org/jira/browse/IMPALA-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10050:
--
Summary: DCHECK was hit possibly while executing
TestFailpoints::test_failpoints (was: DCHECK is hit possibly while executing
TestFailpoints::test_failpoints)
> DCHECK was hit possibly while executing TestFailpoints::test_failpoints
> ---
>
> Key: IMPALA-10050
> URL: https://issues.apache.org/jira/browse/IMPALA-10050
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Reporter: Attila Jeges
>Priority: Major
> Labels: broken-build, crash
>
> A DCHECK was hit during ASAN core e2e tests. Time-frame suggests that it
> happened while executing TestFailpoints::test_failpoints e2e test.
> {code}
> 10:56:38 TestFailpoints.test_failpoints[protocol: beeswax | table_format:
> avro/snap/block | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | mt_dop: 4 |
> location: PREPARE | action: MEM_LIMIT_EXCEEDED | query: select 1 from
> alltypessmall a join alltypessmall b on a.id = b.id]
> 10:56:38 failure/test_failpoints.py:128: in test_failpoints
> 10:56:38 self.execute_query(query, vector.get_value('exec_option'))
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:811:
> in wrapper
> 10:56:38 return function(*args, **kwargs)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:843:
> in execute_query
> 10:56:38 return self.__execute_query(self.client, query, query_options)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:909:
> in __execute_query
> 10:56:38 return impalad_client.execute(query, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_connection.py:205:
> in execute
> 10:56:38 return self.__beeswax_client.execute(sql_stmt, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:187:
> in execute
> 10:56:38 handle = self.__execute_query(query_string.strip(), user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:365:
> in __execute_query
> 10:56:38 self.wait_for_finished(handle)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:386:
> in wait_for_finished
> 10:56:38 raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> 10:56:38 E ImpalaBeeswaxException: ImpalaBeeswaxException:
> 10:56:38 EQuery aborted:RPC from 127.0.0.1:27000 to 127.0.0.1:27002 failed
> 10:56:38 E TransmitData() to 127.0.0.1:27002 failed: Network error: Client
> connection negotiation failed: client connection to 127.0.0.1:27002: connect:
> Connection refused (error 111)
> {code}
> Impalad log:
> {code}
> Log file created at: 2020/08/05 01:52:56
> Running on machine:
> impala-ec2-centos74-r5-4xlarge-ondemand-017c.vpc.cloudera.com
> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
> F0805 01:52:56.979769 17313 query-state.cc:803]
> 3941a3d92a71e242:15c963f3] Check failed: is_cancelled_.Load() == 1 (0
> vs. 1)
> {code}
> Stack trace
> {code}
> Thread 368 (crashed)
> 0 libc-2.17.so + 0x351f7
> rax = 0x rdx = 0x0006
> rcx = 0x rbx = 0x0004
> rsi = 0x43a1 rdi = 0x37e4
> rbp = 0x7efcd4c53080 rsp = 0x7efcd4c52d08
> r8 = 0xr9 = 0x7efcd4c52b80
> r10 = 0x0008 r11 = 0x0206
> r12 = 0x093de7c0 r13 = 0x0086
> r14 = 0x093de7c4 r15 = 0x093d6de0
> rip = 0x7f05c9d231f7
> Found by: given as instruction pointer in context
> 1 impalad!google::LogMessage::Flush() + 0x1eb
> rbp = 0x7efcd4c53250 rsp = 0x7efcd4c53090
> rip = 0x05727e3b
> Found by: previous frame's frame pointer
> 2 impalad!google::LogMessageFatal::~LogMessageFatal() + 0x9
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53130 r12 = 0x0fe01a982628
> r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
> r15 = 0x7efcd4c53270 rip = 0x0572ba39
> Found by: call frame info
> 3 impalad!impala::QueryState::MonitorFInstances() [query-state.cc : 803 +
> 0x45]
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53140 r12 =
[jira] [Updated] (IMPALA-10050) DCHECK is hit possibly while executing TestFailpoints::test_failpoints
[
https://issues.apache.org/jira/browse/IMPALA-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10050:
--
Description:
A DCHECK was hit during ASAN core e2e tests. Time-frame suggests that it
happened while executing TestFailpoints::test_failpoints e2e test.
{code}
10:56:38 TestFailpoints.test_failpoints[protocol: beeswax | table_format:
avro/snap/block | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | mt_dop: 4 |
location: PREPARE | action: MEM_LIMIT_EXCEEDED | query: select 1 from
alltypessmall a join alltypessmall b on a.id = b.id]
10:56:38 failure/test_failpoints.py:128: in test_failpoints
10:56:38 self.execute_query(query, vector.get_value('exec_option'))
10:56:38
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:811:
in wrapper
10:56:38 return function(*args, **kwargs)
10:56:38
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:843:
in execute_query
10:56:38 return self.__execute_query(self.client, query, query_options)
10:56:38
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:909:
in __execute_query
10:56:38 return impalad_client.execute(query, user=user)
10:56:38
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_connection.py:205:
in execute
10:56:38 return self.__beeswax_client.execute(sql_stmt, user=user)
10:56:38
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:187:
in execute
10:56:38 handle = self.__execute_query(query_string.strip(), user=user)
10:56:38
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:365:
in __execute_query
10:56:38 self.wait_for_finished(handle)
10:56:38
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:386:
in wait_for_finished
10:56:38 raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
10:56:38 E ImpalaBeeswaxException: ImpalaBeeswaxException:
10:56:38 EQuery aborted:RPC from 127.0.0.1:27000 to 127.0.0.1:27002 failed
10:56:38 E TransmitData() to 127.0.0.1:27002 failed: Network error: Client
connection negotiation failed: client connection to 127.0.0.1:27002: connect:
Connection refused (error 111)
{code}
Impalad log:
{code}
Log file created at: 2020/08/05 01:52:56
Running on machine:
impala-ec2-centos74-r5-4xlarge-ondemand-017c.vpc.cloudera.com
Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
F0805 01:52:56.979769 17313 query-state.cc:803]
3941a3d92a71e242:15c963f3] Check failed: is_cancelled_.Load() == 1 (0
vs. 1)
{code}
Stack trace
{code}
Thread 368 (crashed)
0 libc-2.17.so + 0x351f7
rax = 0x rdx = 0x0006
rcx = 0x rbx = 0x0004
rsi = 0x43a1 rdi = 0x37e4
rbp = 0x7efcd4c53080 rsp = 0x7efcd4c52d08
r8 = 0xr9 = 0x7efcd4c52b80
r10 = 0x0008 r11 = 0x0206
r12 = 0x093de7c0 r13 = 0x0086
r14 = 0x093de7c4 r15 = 0x093d6de0
rip = 0x7f05c9d231f7
Found by: given as instruction pointer in context
1 impalad!google::LogMessage::Flush() + 0x1eb
rbp = 0x7efcd4c53250 rsp = 0x7efcd4c53090
rip = 0x05727e3b
Found by: previous frame's frame pointer
2 impalad!google::LogMessageFatal::~LogMessageFatal() + 0x9
rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
rsp = 0x7efcd4c53130 r12 = 0x0fe01a982628
r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
r15 = 0x7efcd4c53270 rip = 0x0572ba39
Found by: call frame info
3 impalad!impala::QueryState::MonitorFInstances() [query-state.cc : 803 +
0x45]
rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
rsp = 0x7efcd4c53140 r12 = 0x0fe01a982628
r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
r15 = 0x7efcd4c53270 rip = 0x024c1713
Found by: call frame info
4 impalad!impala::QueryExecMgr::ExecuteQueryHelper(impala::QueryState*)
[query-exec-mgr.cc : 161 + 0x8]
rbx = 0x61d000da0280 rbp = 0x7efcd4c53330
rsp = 0x7efcd4c53320 r12 = 0x7efcd4c53360
r13 = 0x7efcd4c53340 r14 = 0x0fdf9a98a668
r15 = 0x rip = 0x024a767d
Found by: call frame info
5 impalad!boost::_bi::bind_tvoid, boost::_mfi::mf1void,
impala::QueryExecMgr, impala::QueryState*,
boost::_bi::list2boost::_bi::valueimpala::QueryExecMgr*,
boost::_bi::valueimpala::QueryState* ::operator()() [bind.hpp
: 1222 + 0xe]
rbx =
[jira] [Updated] (IMPALA-10050) DCHECK is hit possibly while executing TestFailpoints::test_failpoints
[
https://issues.apache.org/jira/browse/IMPALA-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10050:
--
Summary: DCHECK is hit possibly while executing
TestFailpoints::test_failpoints (was: DCHECK is hit while executing
TestFailpoints::test_failpoints)
> DCHECK is hit possibly while executing TestFailpoints::test_failpoints
> --
>
> Key: IMPALA-10050
> URL: https://issues.apache.org/jira/browse/IMPALA-10050
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Reporter: Attila Jeges
>Priority: Major
> Labels: broken-build, crash
>
> A DCHECK was hit during ASAN core e2e tests. Time-frame suggests that it
> happened while executing TestFailpoints::test_failpoints e2e test.
> {code}
> 0:56:38 === FAILURES
> ===
> 10:56:38 TestFailpoints.test_failpoints[protocol: beeswax | table_format:
> avro/snap/block | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | mt_dop: 4 |
> location: PREPARE | action: MEM_LIMIT_EXCEEDED | query: select 1 from
> alltypessmall a join alltypessmall b on a.id = b.id]
> 10:56:38 failure/test_failpoints.py:128: in test_failpoints
> 10:56:38 self.execute_query(query, vector.get_value('exec_option'))
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:811:
> in wrapper
> 10:56:38 return function(*args, **kwargs)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:843:
> in execute_query
> 10:56:38 return self.__execute_query(self.client, query, query_options)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:909:
> in __execute_query
> 10:56:38 return impalad_client.execute(query, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_connection.py:205:
> in execute
> 10:56:38 return self.__beeswax_client.execute(sql_stmt, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:187:
> in execute
> 10:56:38 handle = self.__execute_query(query_string.strip(), user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:365:
> in __execute_query
> 10:56:38 self.wait_for_finished(handle)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:386:
> in wait_for_finished
> 10:56:38 raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> 10:56:38 E ImpalaBeeswaxException: ImpalaBeeswaxException:
> 10:56:38 EQuery aborted:RPC from 127.0.0.1:27000 to 127.0.0.1:27002 failed
> 10:56:38 E TransmitData() to 127.0.0.1:27002 failed: Network error: Client
> connection negotiation failed: client connection to 127.0.0.1:27002: connect:
> Connection refused (error 111)
> {code}
> Impalad log:
> {code}
> Log file created at: 2020/08/05 01:52:56
> Running on machine:
> impala-ec2-centos74-r5-4xlarge-ondemand-017c.vpc.cloudera.com
> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
> F0805 01:52:56.979769 17313 query-state.cc:803]
> 3941a3d92a71e242:15c963f3] Check failed: is_cancelled_.Load() == 1 (0
> vs. 1)
> {code}
> Stack trace
> {code}
> Thread 368 (crashed)
> 0 libc-2.17.so + 0x351f7
> rax = 0x rdx = 0x0006
> rcx = 0x rbx = 0x0004
> rsi = 0x43a1 rdi = 0x37e4
> rbp = 0x7efcd4c53080 rsp = 0x7efcd4c52d08
> r8 = 0xr9 = 0x7efcd4c52b80
> r10 = 0x0008 r11 = 0x0206
> r12 = 0x093de7c0 r13 = 0x0086
> r14 = 0x093de7c4 r15 = 0x093d6de0
> rip = 0x7f05c9d231f7
> Found by: given as instruction pointer in context
> 1 impalad!google::LogMessage::Flush() + 0x1eb
> rbp = 0x7efcd4c53250 rsp = 0x7efcd4c53090
> rip = 0x05727e3b
> Found by: previous frame's frame pointer
> 2 impalad!google::LogMessageFatal::~LogMessageFatal() + 0x9
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53130 r12 = 0x0fe01a982628
> r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
> r15 = 0x7efcd4c53270 rip = 0x0572ba39
> Found by: call frame info
> 3 impalad!impala::QueryState::MonitorFInstances() [query-state.cc : 803 +
> 0x45]
>
[jira] [Updated] (IMPALA-10050) DCHECK is hit while executing TestFailpoints::test_failpoints
[
https://issues.apache.org/jira/browse/IMPALA-10050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-10050:
--
Labels: broken-build crash (was: )
> DCHECK is hit while executing TestFailpoints::test_failpoints
> -
>
> Key: IMPALA-10050
> URL: https://issues.apache.org/jira/browse/IMPALA-10050
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Reporter: Attila Jeges
>Priority: Major
> Labels: broken-build, crash
>
> A DCHECK was hit during ASAN core e2e tests. Time-frame suggests that it
> happened while executing TestFailpoints::test_failpoints e2e test.
> {code}
> 0:56:38 === FAILURES
> ===
> 10:56:38 TestFailpoints.test_failpoints[protocol: beeswax | table_format:
> avro/snap/block | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | mt_dop: 4 |
> location: PREPARE | action: MEM_LIMIT_EXCEEDED | query: select 1 from
> alltypessmall a join alltypessmall b on a.id = b.id]
> 10:56:38 failure/test_failpoints.py:128: in test_failpoints
> 10:56:38 self.execute_query(query, vector.get_value('exec_option'))
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:811:
> in wrapper
> 10:56:38 return function(*args, **kwargs)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:843:
> in execute_query
> 10:56:38 return self.__execute_query(self.client, query, query_options)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:909:
> in __execute_query
> 10:56:38 return impalad_client.execute(query, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_connection.py:205:
> in execute
> 10:56:38 return self.__beeswax_client.execute(sql_stmt, user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:187:
> in execute
> 10:56:38 handle = self.__execute_query(query_string.strip(), user=user)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:365:
> in __execute_query
> 10:56:38 self.wait_for_finished(handle)
> 10:56:38
> /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:386:
> in wait_for_finished
> 10:56:38 raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> 10:56:38 E ImpalaBeeswaxException: ImpalaBeeswaxException:
> 10:56:38 EQuery aborted:RPC from 127.0.0.1:27000 to 127.0.0.1:27002 failed
> 10:56:38 E TransmitData() to 127.0.0.1:27002 failed: Network error: Client
> connection negotiation failed: client connection to 127.0.0.1:27002: connect:
> Connection refused (error 111)
> {code}
> Impalad log:
> {code}
> Log file created at: 2020/08/05 01:52:56
> Running on machine:
> impala-ec2-centos74-r5-4xlarge-ondemand-017c.vpc.cloudera.com
> Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
> F0805 01:52:56.979769 17313 query-state.cc:803]
> 3941a3d92a71e242:15c963f3] Check failed: is_cancelled_.Load() == 1 (0
> vs. 1)
> {code}
> Stack trace
> {code}
> Thread 368 (crashed)
> 0 libc-2.17.so + 0x351f7
> rax = 0x rdx = 0x0006
> rcx = 0x rbx = 0x0004
> rsi = 0x43a1 rdi = 0x37e4
> rbp = 0x7efcd4c53080 rsp = 0x7efcd4c52d08
> r8 = 0xr9 = 0x7efcd4c52b80
> r10 = 0x0008 r11 = 0x0206
> r12 = 0x093de7c0 r13 = 0x0086
> r14 = 0x093de7c4 r15 = 0x093d6de0
> rip = 0x7f05c9d231f7
> Found by: given as instruction pointer in context
> 1 impalad!google::LogMessage::Flush() + 0x1eb
> rbp = 0x7efcd4c53250 rsp = 0x7efcd4c53090
> rip = 0x05727e3b
> Found by: previous frame's frame pointer
> 2 impalad!google::LogMessageFatal::~LogMessageFatal() + 0x9
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53130 r12 = 0x0fe01a982628
> r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
> r15 = 0x7efcd4c53270 rip = 0x0572ba39
> Found by: call frame info
> 3 impalad!impala::QueryState::MonitorFInstances() [query-state.cc : 803 +
> 0x45]
> rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
> rsp = 0x7efcd4c53140 r12 = 0x0fe01a982628
> r13 =
[jira] [Created] (IMPALA-10050) DCHECK is hit while executing TestFailpoints::test_failpoints
Attila Jeges created IMPALA-10050:
-
Summary: DCHECK is hit while executing
TestFailpoints::test_failpoints
Key: IMPALA-10050
URL: https://issues.apache.org/jira/browse/IMPALA-10050
Project: IMPALA
Issue Type: Improvement
Components: Backend
Reporter: Attila Jeges
A DCHECK was hit during ASAN core e2e tests. Time-frame suggests that it
happened while executing TestFailpoints::test_failpoints e2e test.
{code}
0:56:38 === FAILURES
===
10:56:38 TestFailpoints.test_failpoints[protocol: beeswax | table_format:
avro/snap/block | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | mt_dop: 4 |
location: PREPARE | action: MEM_LIMIT_EXCEEDED | query: select 1 from
alltypessmall a join alltypessmall b on a.id = b.id]
10:56:38 failure/test_failpoints.py:128: in test_failpoints
10:56:38 self.execute_query(query, vector.get_value('exec_option'))
10:56:38
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:811:
in wrapper
10:56:38 return function(*args, **kwargs)
10:56:38
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:843:
in execute_query
10:56:38 return self.__execute_query(self.client, query, query_options)
10:56:38
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_test_suite.py:909:
in __execute_query
10:56:38 return impalad_client.execute(query, user=user)
10:56:38
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_connection.py:205:
in execute
10:56:38 return self.__beeswax_client.execute(sql_stmt, user=user)
10:56:38
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:187:
in execute
10:56:38 handle = self.__execute_query(query_string.strip(), user=user)
10:56:38
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:365:
in __execute_query
10:56:38 self.wait_for_finished(handle)
10:56:38
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/beeswax/impala_beeswax.py:386:
in wait_for_finished
10:56:38 raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
10:56:38 E ImpalaBeeswaxException: ImpalaBeeswaxException:
10:56:38 EQuery aborted:RPC from 127.0.0.1:27000 to 127.0.0.1:27002 failed
10:56:38 E TransmitData() to 127.0.0.1:27002 failed: Network error: Client
connection negotiation failed: client connection to 127.0.0.1:27002: connect:
Connection refused (error 111)
{code}
Impalad log:
{code}
Log file created at: 2020/08/05 01:52:56
Running on machine:
impala-ec2-centos74-r5-4xlarge-ondemand-017c.vpc.cloudera.com
Log line format: [IWEF]mmdd hh:mm:ss.uu threadid file:line] msg
F0805 01:52:56.979769 17313 query-state.cc:803]
3941a3d92a71e242:15c963f3] Check failed: is_cancelled_.Load() == 1 (0
vs. 1)
{code}
Stack trace
{code}
Thread 368 (crashed)
0 libc-2.17.so + 0x351f7
rax = 0x rdx = 0x0006
rcx = 0x rbx = 0x0004
rsi = 0x43a1 rdi = 0x37e4
rbp = 0x7efcd4c53080 rsp = 0x7efcd4c52d08
r8 = 0xr9 = 0x7efcd4c52b80
r10 = 0x0008 r11 = 0x0206
r12 = 0x093de7c0 r13 = 0x0086
r14 = 0x093de7c4 r15 = 0x093d6de0
rip = 0x7f05c9d231f7
Found by: given as instruction pointer in context
1 impalad!google::LogMessage::Flush() + 0x1eb
rbp = 0x7efcd4c53250 rsp = 0x7efcd4c53090
rip = 0x05727e3b
Found by: previous frame's frame pointer
2 impalad!google::LogMessageFatal::~LogMessageFatal() + 0x9
rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
rsp = 0x7efcd4c53130 r12 = 0x0fe01a982628
r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
r15 = 0x7efcd4c53270 rip = 0x0572ba39
Found by: call frame info
3 impalad!impala::QueryState::MonitorFInstances() [query-state.cc : 803 +
0x45]
rbx = 0x7efcd4c532a0 rbp = 0x7efcd4c53310
rsp = 0x7efcd4c53140 r12 = 0x0fe01a982628
r13 = 0x61d000da0a6c r14 = 0x7efcd4c53250
r15 = 0x7efcd4c53270 rip = 0x024c1713
Found by: call frame info
4 impalad!impala::QueryExecMgr::ExecuteQueryHelper(impala::QueryState*)
[query-exec-mgr.cc : 161 + 0x8]
rbx = 0x61d000da0280 rbp = 0x7efcd4c53330
rsp = 0x7efcd4c53320 r12 = 0x7efcd4c53360
r13 = 0x7efcd4c53340 r14 = 0x0fdf9a98a668
r15 = 0x rip = 0x024a767d
Found
[jira] [Commented] (IMPALA-9351) AnalyzeDDLTest.TestCreateTableLikeFileOrc failed due to non-existing path
[
https://issues.apache.org/jira/browse/IMPALA-9351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17170717#comment-17170717
]
Attila Jeges commented on IMPALA-9351:
--
Another occurence here:
https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/11501/
{code}
Error during analysis:
org.apache.impala.common.AnalysisException: Cannot infer schema, path does not
exist:
hdfs://localhost:20500/test-warehouse/managed/complextypestbl_orc_def/base_001/bucket_0_0
sql:
create table if not exists newtbl_DNE like orc
'/test-warehouse/managed/complextypestbl_orc_def/base_001/bucket_0_0'
{code}
> AnalyzeDDLTest.TestCreateTableLikeFileOrc failed due to non-existing path
> -
>
> Key: IMPALA-9351
> URL: https://issues.apache.org/jira/browse/IMPALA-9351
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Reporter: Fang-Yu Rao
>Assignee: Norbert Luksa
>Priority: Blocker
> Labels: broken-build, flaky-test
> Fix For: Impala 3.4.0
>
>
> AnalyzeDDLTest.TestCreateTableLikeFileOrc failed due to a non-existing path.
> Specifically, we see the following error message.
> {code:java}
> Error Message
> Error during analysis:
> org.apache.impala.common.AnalysisException: Cannot infer schema, path does
> not exist:
> hdfs://localhost:20500/test-warehouse/functional_orc_def.db/complextypes_fileformat/00_0
> sql:
> create table if not exists newtbl_DNE like orc
> '/test-warehouse/functional_orc_def.db/complextypes_fileformat/00_0'
> {code}
> The stack trace is provided in the following.
> {code:java}
> Stacktrace
> java.lang.AssertionError:
> Error during analysis:
> org.apache.impala.common.AnalysisException: Cannot infer schema, path does
> not exist:
> hdfs://localhost:20500/test-warehouse/functional_orc_def.db/complextypes_fileformat/00_0
> sql:
> create table if not exists newtbl_DNE like orc
> '/test-warehouse/functional_orc_def.db/complextypes_fileformat/00_0'
> at org.junit.Assert.fail(Assert.java:88)
> at
> org.apache.impala.common.FrontendFixture.analyzeStmt(FrontendFixture.java:397)
> at
> org.apache.impala.common.FrontendTestBase.AnalyzesOk(FrontendTestBase.java:244)
> at
> org.apache.impala.common.FrontendTestBase.AnalyzesOk(FrontendTestBase.java:185)
> at
> org.apache.impala.analysis.AnalyzeDDLTest.TestCreateTableLikeFileOrc(AnalyzeDDLTest.java:2045)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
> at
>
[jira] [Created] (IMPALA-9615) Make re2's max_mem option configurable via an Impala startup flag.
Attila Jeges created IMPALA-9615: Summary: Make re2's max_mem option configurable via an Impala startup flag. Key: IMPALA-9615 URL: https://issues.apache.org/jira/browse/IMPALA-9615 Project: IMPALA Issue Type: Improvement Affects Versions: Impala 3.4.0 Reporter: Attila Jeges Right now Impala always uses the default max_mem value for re2 regexp pattern matching. For more memory consuming patterns this can cause the following error: "re2/re2.cc:667: DFA out of memory: size x, bytemap range xx, list count x". It would be nice if re2's max_mem option would be configurable via an Impala startup flag. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-9614) Disable error logging for all re2 library usage.
Attila Jeges created IMPALA-9614: Summary: Disable error logging for all re2 library usage. Key: IMPALA-9614 URL: https://issues.apache.org/jira/browse/IMPALA-9614 Project: IMPALA Issue Type: Improvement Components: Backend Affects Versions: Impala 3.4.0 Reporter: Attila Jeges Impala uses the re2 library for a number of purposes: - Built-in functions : REGEXP_REPLACE, REGEXP_EXTRACT, REGEXP_LIKE - SQL operators: REGEXP, RLIKE, LIKE - Log and query redaction. Logging errors is disabled for REGEXP_REPLACE and REGEXP_EXTRACT built-in functions. In all other cases logging errors is enabled. Right now this behavior is hard-coded and not configurable. In case every row causes an error, this can be problematic as re2 errors will flood the impala error logs. We should disable error logging for all re2 library usage or make it configurable through an Impala startup flag. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-9498) Allow Collection types in SELECT list for Parquet tables
[
https://issues.apache.org/jira/browse/IMPALA-9498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-9498 started by Attila Jeges.
> Allow Collection types in SELECT list for Parquet tables
>
>
> Key: IMPALA-9498
> URL: https://issues.apache.org/jira/browse/IMPALA-9498
> Project: IMPALA
> Issue Type: New Feature
>Reporter: Gabor Kaszab
>Assignee: Attila Jeges
>Priority: Major
> Labels: complextype
>
> This covers collections: Array, Map
> Expected printout format:
> Array: [null,1,2,null,3,null]
> Map: {"k1":2,"k2":null}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-9555) TestDateQueries.test_queries failing because Hive3 switched back to the hybrid Julian Gregorian calendar
[
https://issues.apache.org/jira/browse/IMPALA-9555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges resolved IMPALA-9555.
--
Resolution: Fixed
> TestDateQueries.test_queries failing because Hive3 switched back to the
> hybrid Julian Gregorian calendar
>
>
> Key: IMPALA-9555
> URL: https://issues.apache.org/jira/browse/IMPALA-9555
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.4.0
>Reporter: Attila Jeges
>Assignee: Attila Jeges
>Priority: Critical
> Fix For: Impala 3.4.0
>
>
> TestDateQueries.test_queries is failing after upgrading the CDP GBN with the
> following error:
> {code}
> query_test.test_date_queries.TestDateQueries.test_queries[protocol: beeswax |
> exec_option: {'disable_codegen_rows_threshold': 0, 'disable_codegen': 'true',
> 'batch_size': 1} | table_format: avro/snap/block] (from pytest)
> Error Message
> query_test/test_date_queries.py:60: in test_queries
> self.run_test_case('QueryTest/avro_date', vector)
> common/impala_test_suite.py:690: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:523: in __verify_results_and_errors
> replace_filenames_with_placeholder) common/test_result_verifier.py:456: in
> verify_raw_results VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results E assert Comparing
> QueryTestResults (expected vs actual): E 0,0001-01-01,0001-01-01 !=
> 10,1399-06-27,2017-11-28 E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
> E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31 E
> 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19 E 12,1399-06-27,2018-12-31
> != 21,2017-11-27,0001-06-20 E 2,0001-01-01,0002-01-01 !=
> 22,2017-11-27,0001-06-21 E 20,2017-11-27,0001-06-21 !=
> 23,2017-11-27,0001-06-22 E 21,2017-11-27,0001-06-22 !=
> 24,2017-11-27,0001-06-23 E 22,2017-11-27,0001-06-23 !=
> 25,2017-11-27,0001-06-24 E 23,2017-11-27,0001-06-24 !=
> 26,2017-11-27,0001-06-25 E 24,2017-11-27,0001-06-25 !=
> 27,2017-11-27,0001-06-26 E 25,2017-11-27,0001-06-26 !=
> 28,2017-11-27,0001-06-27 E 26,2017-11-27,0001-06-27 !=
> 29,2017-11-27,2017-11-28 E 27,2017-11-27,0001-06-28 !=
> 30,-12-31,-12-01 E 28,2017-11-27,0001-06-29 !=
> 31,-12-31,-12-31 E 29,2017-11-27,2017-11-28 != None E
> 3,0001-01-01,1399-12-31 != None E 30,-12-31,-12-01 != None E
> 31,-12-31,-12-31 != None E 4,0001-01-01,2017-11-28 != None E
> 5,0001-01-01,-12-31 != None E 6,0001-01-01,NULL != None E Number
> of rows returned (expected vs actual): 22 != 15
> Stacktrace
> query_test/test_date_queries.py:60: in test_queries
> self.run_test_case('QueryTest/avro_date', vector)
> common/impala_test_suite.py:690: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:523: in __verify_results_and_errors
> replace_filenames_with_placeholder)
> common/test_result_verifier.py:456: in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E 0,0001-01-01,0001-01-01 != 10,1399-06-27,2017-11-28
> E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
> E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31
> E 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19
> E 12,1399-06-27,2018-12-31 != 21,2017-11-27,0001-06-20
> E 2,0001-01-01,0002-01-01 != 22,2017-11-27,0001-06-21
> E 20,2017-11-27,0001-06-21 != 23,2017-11-27,0001-06-22
> E 21,2017-11-27,0001-06-22 != 24,2017-11-27,0001-06-23
> E 22,2017-11-27,0001-06-23 != 25,2017-11-27,0001-06-24
> E 23,2017-11-27,0001-06-24 != 26,2017-11-27,0001-06-25
> E 24,2017-11-27,0001-06-25 != 27,2017-11-27,0001-06-26
> E 25,2017-11-27,0001-06-26 != 28,2017-11-27,0001-06-27
> E 26,2017-11-27,0001-06-27 != 29,2017-11-27,2017-11-28
> E 27,2017-11-27,0001-06-28 != 30,-12-31,-12-01
> E 28,2017-11-27,0001-06-29 != 31,-12-31,-12-31
> E 29,2017-11-27,2017-11-28 != None
> E 3,0001-01-01,1399-12-31 != None
> E 30,-12-31,-12-01 != None
> E 31,-12-31,-12-31 != None
> E 4,0001-01-01,2017-11-28 != None
> E 5,0001-01-01,-12-31 != None
> E 6,0001-01-01,NULL != None
> E Number of rows returned (expected vs actual): 22 != 15
> Standard Error
> ERROR:test_configuration:Comparing QueryTestResults (expected vs
[jira] [Reopened] (IMPALA-9555) TestDateQueries.test_queries failing because Hive3 switched back to the hybrid Julian Gregorian calendar
[
https://issues.apache.org/jira/browse/IMPALA-9555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges reopened IMPALA-9555:
--
> TestDateQueries.test_queries failing because Hive3 switched back to the
> hybrid Julian Gregorian calendar
>
>
> Key: IMPALA-9555
> URL: https://issues.apache.org/jira/browse/IMPALA-9555
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.4.0
>Reporter: Attila Jeges
>Assignee: Attila Jeges
>Priority: Critical
> Fix For: Impala 3.4.0
>
>
> TestDateQueries.test_queries is failing after upgrading the CDP GBN with the
> following error:
> {code}
> query_test.test_date_queries.TestDateQueries.test_queries[protocol: beeswax |
> exec_option: {'disable_codegen_rows_threshold': 0, 'disable_codegen': 'true',
> 'batch_size': 1} | table_format: avro/snap/block] (from pytest)
> Error Message
> query_test/test_date_queries.py:60: in test_queries
> self.run_test_case('QueryTest/avro_date', vector)
> common/impala_test_suite.py:690: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:523: in __verify_results_and_errors
> replace_filenames_with_placeholder) common/test_result_verifier.py:456: in
> verify_raw_results VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results E assert Comparing
> QueryTestResults (expected vs actual): E 0,0001-01-01,0001-01-01 !=
> 10,1399-06-27,2017-11-28 E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
> E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31 E
> 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19 E 12,1399-06-27,2018-12-31
> != 21,2017-11-27,0001-06-20 E 2,0001-01-01,0002-01-01 !=
> 22,2017-11-27,0001-06-21 E 20,2017-11-27,0001-06-21 !=
> 23,2017-11-27,0001-06-22 E 21,2017-11-27,0001-06-22 !=
> 24,2017-11-27,0001-06-23 E 22,2017-11-27,0001-06-23 !=
> 25,2017-11-27,0001-06-24 E 23,2017-11-27,0001-06-24 !=
> 26,2017-11-27,0001-06-25 E 24,2017-11-27,0001-06-25 !=
> 27,2017-11-27,0001-06-26 E 25,2017-11-27,0001-06-26 !=
> 28,2017-11-27,0001-06-27 E 26,2017-11-27,0001-06-27 !=
> 29,2017-11-27,2017-11-28 E 27,2017-11-27,0001-06-28 !=
> 30,-12-31,-12-01 E 28,2017-11-27,0001-06-29 !=
> 31,-12-31,-12-31 E 29,2017-11-27,2017-11-28 != None E
> 3,0001-01-01,1399-12-31 != None E 30,-12-31,-12-01 != None E
> 31,-12-31,-12-31 != None E 4,0001-01-01,2017-11-28 != None E
> 5,0001-01-01,-12-31 != None E 6,0001-01-01,NULL != None E Number
> of rows returned (expected vs actual): 22 != 15
> Stacktrace
> query_test/test_date_queries.py:60: in test_queries
> self.run_test_case('QueryTest/avro_date', vector)
> common/impala_test_suite.py:690: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:523: in __verify_results_and_errors
> replace_filenames_with_placeholder)
> common/test_result_verifier.py:456: in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E 0,0001-01-01,0001-01-01 != 10,1399-06-27,2017-11-28
> E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
> E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31
> E 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19
> E 12,1399-06-27,2018-12-31 != 21,2017-11-27,0001-06-20
> E 2,0001-01-01,0002-01-01 != 22,2017-11-27,0001-06-21
> E 20,2017-11-27,0001-06-21 != 23,2017-11-27,0001-06-22
> E 21,2017-11-27,0001-06-22 != 24,2017-11-27,0001-06-23
> E 22,2017-11-27,0001-06-23 != 25,2017-11-27,0001-06-24
> E 23,2017-11-27,0001-06-24 != 26,2017-11-27,0001-06-25
> E 24,2017-11-27,0001-06-25 != 27,2017-11-27,0001-06-26
> E 25,2017-11-27,0001-06-26 != 28,2017-11-27,0001-06-27
> E 26,2017-11-27,0001-06-27 != 29,2017-11-27,2017-11-28
> E 27,2017-11-27,0001-06-28 != 30,-12-31,-12-01
> E 28,2017-11-27,0001-06-29 != 31,-12-31,-12-31
> E 29,2017-11-27,2017-11-28 != None
> E 3,0001-01-01,1399-12-31 != None
> E 30,-12-31,-12-01 != None
> E 31,-12-31,-12-31 != None
> E 4,0001-01-01,2017-11-28 != None
> E 5,0001-01-01,-12-31 != None
> E 6,0001-01-01,NULL != None
> E Number of rows returned (expected vs actual): 22 != 15
> Standard Error
> ERROR:test_configuration:Comparing QueryTestResults (expected vs actual):
>
[jira] [Work started] (IMPALA-9555) TestDateQueries.test_queries failing because Hive3 switched back to the hybrid Julian Gregorian calendar
[
https://issues.apache.org/jira/browse/IMPALA-9555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-9555 started by Attila Jeges.
> TestDateQueries.test_queries failing because Hive3 switched back to the
> hybrid Julian Gregorian calendar
>
>
> Key: IMPALA-9555
> URL: https://issues.apache.org/jira/browse/IMPALA-9555
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.4.0
>Reporter: Attila Jeges
>Assignee: Attila Jeges
>Priority: Critical
> Fix For: Impala 3.4.0
>
>
> TestDateQueries.test_queries is failing after upgrading the CDP GBN with the
> following error:
> {code}
> query_test.test_date_queries.TestDateQueries.test_queries[protocol: beeswax |
> exec_option: {'disable_codegen_rows_threshold': 0, 'disable_codegen': 'true',
> 'batch_size': 1} | table_format: avro/snap/block] (from pytest)
> Error Message
> query_test/test_date_queries.py:60: in test_queries
> self.run_test_case('QueryTest/avro_date', vector)
> common/impala_test_suite.py:690: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:523: in __verify_results_and_errors
> replace_filenames_with_placeholder) common/test_result_verifier.py:456: in
> verify_raw_results VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results E assert Comparing
> QueryTestResults (expected vs actual): E 0,0001-01-01,0001-01-01 !=
> 10,1399-06-27,2017-11-28 E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
> E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31 E
> 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19 E 12,1399-06-27,2018-12-31
> != 21,2017-11-27,0001-06-20 E 2,0001-01-01,0002-01-01 !=
> 22,2017-11-27,0001-06-21 E 20,2017-11-27,0001-06-21 !=
> 23,2017-11-27,0001-06-22 E 21,2017-11-27,0001-06-22 !=
> 24,2017-11-27,0001-06-23 E 22,2017-11-27,0001-06-23 !=
> 25,2017-11-27,0001-06-24 E 23,2017-11-27,0001-06-24 !=
> 26,2017-11-27,0001-06-25 E 24,2017-11-27,0001-06-25 !=
> 27,2017-11-27,0001-06-26 E 25,2017-11-27,0001-06-26 !=
> 28,2017-11-27,0001-06-27 E 26,2017-11-27,0001-06-27 !=
> 29,2017-11-27,2017-11-28 E 27,2017-11-27,0001-06-28 !=
> 30,-12-31,-12-01 E 28,2017-11-27,0001-06-29 !=
> 31,-12-31,-12-31 E 29,2017-11-27,2017-11-28 != None E
> 3,0001-01-01,1399-12-31 != None E 30,-12-31,-12-01 != None E
> 31,-12-31,-12-31 != None E 4,0001-01-01,2017-11-28 != None E
> 5,0001-01-01,-12-31 != None E 6,0001-01-01,NULL != None E Number
> of rows returned (expected vs actual): 22 != 15
> Stacktrace
> query_test/test_date_queries.py:60: in test_queries
> self.run_test_case('QueryTest/avro_date', vector)
> common/impala_test_suite.py:690: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:523: in __verify_results_and_errors
> replace_filenames_with_placeholder)
> common/test_result_verifier.py:456: in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E 0,0001-01-01,0001-01-01 != 10,1399-06-27,2017-11-28
> E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
> E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31
> E 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19
> E 12,1399-06-27,2018-12-31 != 21,2017-11-27,0001-06-20
> E 2,0001-01-01,0002-01-01 != 22,2017-11-27,0001-06-21
> E 20,2017-11-27,0001-06-21 != 23,2017-11-27,0001-06-22
> E 21,2017-11-27,0001-06-22 != 24,2017-11-27,0001-06-23
> E 22,2017-11-27,0001-06-23 != 25,2017-11-27,0001-06-24
> E 23,2017-11-27,0001-06-24 != 26,2017-11-27,0001-06-25
> E 24,2017-11-27,0001-06-25 != 27,2017-11-27,0001-06-26
> E 25,2017-11-27,0001-06-26 != 28,2017-11-27,0001-06-27
> E 26,2017-11-27,0001-06-27 != 29,2017-11-27,2017-11-28
> E 27,2017-11-27,0001-06-28 != 30,-12-31,-12-01
> E 28,2017-11-27,0001-06-29 != 31,-12-31,-12-31
> E 29,2017-11-27,2017-11-28 != None
> E 3,0001-01-01,1399-12-31 != None
> E 30,-12-31,-12-01 != None
> E 31,-12-31,-12-31 != None
> E 4,0001-01-01,2017-11-28 != None
> E 5,0001-01-01,-12-31 != None
> E 6,0001-01-01,NULL != None
> E Number of rows returned (expected vs actual): 22 != 15
> Standard Error
> ERROR:test_configuration:Comparing QueryTestResults (expected vs
[jira] [Issue Comment Deleted] (IMPALA-9555) TestDateQueries.test_queries failing because Hive3 switched back to the hybrid Julian Gregorian calendar
[
https://issues.apache.org/jira/browse/IMPALA-9555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-9555:
-
Comment: was deleted
(was: [~stakiar] You need to do a fresh data load with Hive3, otherwise the
test won't work.)
> TestDateQueries.test_queries failing because Hive3 switched back to the
> hybrid Julian Gregorian calendar
>
>
> Key: IMPALA-9555
> URL: https://issues.apache.org/jira/browse/IMPALA-9555
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.4.0
>Reporter: Attila Jeges
>Assignee: Attila Jeges
>Priority: Critical
> Fix For: Impala 3.4.0
>
>
> TestDateQueries.test_queries is failing after upgrading the CDP GBN with the
> following error:
> {code}
> query_test.test_date_queries.TestDateQueries.test_queries[protocol: beeswax |
> exec_option: {'disable_codegen_rows_threshold': 0, 'disable_codegen': 'true',
> 'batch_size': 1} | table_format: avro/snap/block] (from pytest)
> Error Message
> query_test/test_date_queries.py:60: in test_queries
> self.run_test_case('QueryTest/avro_date', vector)
> common/impala_test_suite.py:690: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:523: in __verify_results_and_errors
> replace_filenames_with_placeholder) common/test_result_verifier.py:456: in
> verify_raw_results VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results E assert Comparing
> QueryTestResults (expected vs actual): E 0,0001-01-01,0001-01-01 !=
> 10,1399-06-27,2017-11-28 E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
> E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31 E
> 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19 E 12,1399-06-27,2018-12-31
> != 21,2017-11-27,0001-06-20 E 2,0001-01-01,0002-01-01 !=
> 22,2017-11-27,0001-06-21 E 20,2017-11-27,0001-06-21 !=
> 23,2017-11-27,0001-06-22 E 21,2017-11-27,0001-06-22 !=
> 24,2017-11-27,0001-06-23 E 22,2017-11-27,0001-06-23 !=
> 25,2017-11-27,0001-06-24 E 23,2017-11-27,0001-06-24 !=
> 26,2017-11-27,0001-06-25 E 24,2017-11-27,0001-06-25 !=
> 27,2017-11-27,0001-06-26 E 25,2017-11-27,0001-06-26 !=
> 28,2017-11-27,0001-06-27 E 26,2017-11-27,0001-06-27 !=
> 29,2017-11-27,2017-11-28 E 27,2017-11-27,0001-06-28 !=
> 30,-12-31,-12-01 E 28,2017-11-27,0001-06-29 !=
> 31,-12-31,-12-31 E 29,2017-11-27,2017-11-28 != None E
> 3,0001-01-01,1399-12-31 != None E 30,-12-31,-12-01 != None E
> 31,-12-31,-12-31 != None E 4,0001-01-01,2017-11-28 != None E
> 5,0001-01-01,-12-31 != None E 6,0001-01-01,NULL != None E Number
> of rows returned (expected vs actual): 22 != 15
> Stacktrace
> query_test/test_date_queries.py:60: in test_queries
> self.run_test_case('QueryTest/avro_date', vector)
> common/impala_test_suite.py:690: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:523: in __verify_results_and_errors
> replace_filenames_with_placeholder)
> common/test_result_verifier.py:456: in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E 0,0001-01-01,0001-01-01 != 10,1399-06-27,2017-11-28
> E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
> E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31
> E 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19
> E 12,1399-06-27,2018-12-31 != 21,2017-11-27,0001-06-20
> E 2,0001-01-01,0002-01-01 != 22,2017-11-27,0001-06-21
> E 20,2017-11-27,0001-06-21 != 23,2017-11-27,0001-06-22
> E 21,2017-11-27,0001-06-22 != 24,2017-11-27,0001-06-23
> E 22,2017-11-27,0001-06-23 != 25,2017-11-27,0001-06-24
> E 23,2017-11-27,0001-06-24 != 26,2017-11-27,0001-06-25
> E 24,2017-11-27,0001-06-25 != 27,2017-11-27,0001-06-26
> E 25,2017-11-27,0001-06-26 != 28,2017-11-27,0001-06-27
> E 26,2017-11-27,0001-06-27 != 29,2017-11-27,2017-11-28
> E 27,2017-11-27,0001-06-28 != 30,-12-31,-12-01
> E 28,2017-11-27,0001-06-29 != 31,-12-31,-12-31
> E 29,2017-11-27,2017-11-28 != None
> E 3,0001-01-01,1399-12-31 != None
> E 30,-12-31,-12-01 != None
> E 31,-12-31,-12-31 != None
> E 4,0001-01-01,2017-11-28 != None
> E 5,0001-01-01,-12-31 != None
> E 6,0001-01-01,NULL != None
> E Number of rows returned (expected vs
[jira] [Commented] (IMPALA-9555) TestDateQueries.test_queries failing because Hive3 switched back to the hybrid Julian Gregorian calendar
[
https://issues.apache.org/jira/browse/IMPALA-9555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17071602#comment-17071602
]
Attila Jeges commented on IMPALA-9555:
--
[~stakiar] You need to do a fresh data load with Hive3, otherwise the test
won't work.
> TestDateQueries.test_queries failing because Hive3 switched back to the
> hybrid Julian Gregorian calendar
>
>
> Key: IMPALA-9555
> URL: https://issues.apache.org/jira/browse/IMPALA-9555
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.4.0
>Reporter: Attila Jeges
>Assignee: Attila Jeges
>Priority: Critical
> Fix For: Impala 3.4.0
>
>
> TestDateQueries.test_queries is failing after upgrading the CDP GBN with the
> following error:
> {code}
> query_test.test_date_queries.TestDateQueries.test_queries[protocol: beeswax |
> exec_option: {'disable_codegen_rows_threshold': 0, 'disable_codegen': 'true',
> 'batch_size': 1} | table_format: avro/snap/block] (from pytest)
> Error Message
> query_test/test_date_queries.py:60: in test_queries
> self.run_test_case('QueryTest/avro_date', vector)
> common/impala_test_suite.py:690: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:523: in __verify_results_and_errors
> replace_filenames_with_placeholder) common/test_result_verifier.py:456: in
> verify_raw_results VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results E assert Comparing
> QueryTestResults (expected vs actual): E 0,0001-01-01,0001-01-01 !=
> 10,1399-06-27,2017-11-28 E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
> E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31 E
> 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19 E 12,1399-06-27,2018-12-31
> != 21,2017-11-27,0001-06-20 E 2,0001-01-01,0002-01-01 !=
> 22,2017-11-27,0001-06-21 E 20,2017-11-27,0001-06-21 !=
> 23,2017-11-27,0001-06-22 E 21,2017-11-27,0001-06-22 !=
> 24,2017-11-27,0001-06-23 E 22,2017-11-27,0001-06-23 !=
> 25,2017-11-27,0001-06-24 E 23,2017-11-27,0001-06-24 !=
> 26,2017-11-27,0001-06-25 E 24,2017-11-27,0001-06-25 !=
> 27,2017-11-27,0001-06-26 E 25,2017-11-27,0001-06-26 !=
> 28,2017-11-27,0001-06-27 E 26,2017-11-27,0001-06-27 !=
> 29,2017-11-27,2017-11-28 E 27,2017-11-27,0001-06-28 !=
> 30,-12-31,-12-01 E 28,2017-11-27,0001-06-29 !=
> 31,-12-31,-12-31 E 29,2017-11-27,2017-11-28 != None E
> 3,0001-01-01,1399-12-31 != None E 30,-12-31,-12-01 != None E
> 31,-12-31,-12-31 != None E 4,0001-01-01,2017-11-28 != None E
> 5,0001-01-01,-12-31 != None E 6,0001-01-01,NULL != None E Number
> of rows returned (expected vs actual): 22 != 15
> Stacktrace
> query_test/test_date_queries.py:60: in test_queries
> self.run_test_case('QueryTest/avro_date', vector)
> common/impala_test_suite.py:690: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:523: in __verify_results_and_errors
> replace_filenames_with_placeholder)
> common/test_result_verifier.py:456: in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E 0,0001-01-01,0001-01-01 != 10,1399-06-27,2017-11-28
> E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
> E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31
> E 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19
> E 12,1399-06-27,2018-12-31 != 21,2017-11-27,0001-06-20
> E 2,0001-01-01,0002-01-01 != 22,2017-11-27,0001-06-21
> E 20,2017-11-27,0001-06-21 != 23,2017-11-27,0001-06-22
> E 21,2017-11-27,0001-06-22 != 24,2017-11-27,0001-06-23
> E 22,2017-11-27,0001-06-23 != 25,2017-11-27,0001-06-24
> E 23,2017-11-27,0001-06-24 != 26,2017-11-27,0001-06-25
> E 24,2017-11-27,0001-06-25 != 27,2017-11-27,0001-06-26
> E 25,2017-11-27,0001-06-26 != 28,2017-11-27,0001-06-27
> E 26,2017-11-27,0001-06-27 != 29,2017-11-27,2017-11-28
> E 27,2017-11-27,0001-06-28 != 30,-12-31,-12-01
> E 28,2017-11-27,0001-06-29 != 31,-12-31,-12-31
> E 29,2017-11-27,2017-11-28 != None
> E 3,0001-01-01,1399-12-31 != None
> E 30,-12-31,-12-01 != None
> E 31,-12-31,-12-31 != None
> E 4,0001-01-01,2017-11-28 != None
> E 5,0001-01-01,-12-31 != None
> E 6,0001-01-01,NULL != None
> E Number of rows returned (expected
[jira] [Resolved] (IMPALA-9555) TestDateQueries.test_queries failing because Hive3 switched back to the hybrid Julian Gregorian calendar
[
https://issues.apache.org/jira/browse/IMPALA-9555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges resolved IMPALA-9555.
--
Fix Version/s: Impala 3.4.0
Resolution: Fixed
> TestDateQueries.test_queries failing because Hive3 switched back to the
> hybrid Julian Gregorian calendar
>
>
> Key: IMPALA-9555
> URL: https://issues.apache.org/jira/browse/IMPALA-9555
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.4.0
>Reporter: Attila Jeges
>Assignee: Attila Jeges
>Priority: Critical
> Fix For: Impala 3.4.0
>
>
> TestDateQueries.test_queries is failing after upgrading the CDP GBN with the
> following error:
> {code}
> query_test.test_date_queries.TestDateQueries.test_queries[protocol: beeswax |
> exec_option: {'disable_codegen_rows_threshold': 0, 'disable_codegen': 'true',
> 'batch_size': 1} | table_format: avro/snap/block] (from pytest)
> Error Message
> query_test/test_date_queries.py:60: in test_queries
> self.run_test_case('QueryTest/avro_date', vector)
> common/impala_test_suite.py:690: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:523: in __verify_results_and_errors
> replace_filenames_with_placeholder) common/test_result_verifier.py:456: in
> verify_raw_results VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results E assert Comparing
> QueryTestResults (expected vs actual): E 0,0001-01-01,0001-01-01 !=
> 10,1399-06-27,2017-11-28 E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
> E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31 E
> 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19 E 12,1399-06-27,2018-12-31
> != 21,2017-11-27,0001-06-20 E 2,0001-01-01,0002-01-01 !=
> 22,2017-11-27,0001-06-21 E 20,2017-11-27,0001-06-21 !=
> 23,2017-11-27,0001-06-22 E 21,2017-11-27,0001-06-22 !=
> 24,2017-11-27,0001-06-23 E 22,2017-11-27,0001-06-23 !=
> 25,2017-11-27,0001-06-24 E 23,2017-11-27,0001-06-24 !=
> 26,2017-11-27,0001-06-25 E 24,2017-11-27,0001-06-25 !=
> 27,2017-11-27,0001-06-26 E 25,2017-11-27,0001-06-26 !=
> 28,2017-11-27,0001-06-27 E 26,2017-11-27,0001-06-27 !=
> 29,2017-11-27,2017-11-28 E 27,2017-11-27,0001-06-28 !=
> 30,-12-31,-12-01 E 28,2017-11-27,0001-06-29 !=
> 31,-12-31,-12-31 E 29,2017-11-27,2017-11-28 != None E
> 3,0001-01-01,1399-12-31 != None E 30,-12-31,-12-01 != None E
> 31,-12-31,-12-31 != None E 4,0001-01-01,2017-11-28 != None E
> 5,0001-01-01,-12-31 != None E 6,0001-01-01,NULL != None E Number
> of rows returned (expected vs actual): 22 != 15
> Stacktrace
> query_test/test_date_queries.py:60: in test_queries
> self.run_test_case('QueryTest/avro_date', vector)
> common/impala_test_suite.py:690: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:523: in __verify_results_and_errors
> replace_filenames_with_placeholder)
> common/test_result_verifier.py:456: in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E 0,0001-01-01,0001-01-01 != 10,1399-06-27,2017-11-28
> E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
> E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31
> E 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19
> E 12,1399-06-27,2018-12-31 != 21,2017-11-27,0001-06-20
> E 2,0001-01-01,0002-01-01 != 22,2017-11-27,0001-06-21
> E 20,2017-11-27,0001-06-21 != 23,2017-11-27,0001-06-22
> E 21,2017-11-27,0001-06-22 != 24,2017-11-27,0001-06-23
> E 22,2017-11-27,0001-06-23 != 25,2017-11-27,0001-06-24
> E 23,2017-11-27,0001-06-24 != 26,2017-11-27,0001-06-25
> E 24,2017-11-27,0001-06-25 != 27,2017-11-27,0001-06-26
> E 25,2017-11-27,0001-06-26 != 28,2017-11-27,0001-06-27
> E 26,2017-11-27,0001-06-27 != 29,2017-11-27,2017-11-28
> E 27,2017-11-27,0001-06-28 != 30,-12-31,-12-01
> E 28,2017-11-27,0001-06-29 != 31,-12-31,-12-31
> E 29,2017-11-27,2017-11-28 != None
> E 3,0001-01-01,1399-12-31 != None
> E 30,-12-31,-12-01 != None
> E 31,-12-31,-12-31 != None
> E 4,0001-01-01,2017-11-28 != None
> E 5,0001-01-01,-12-31 != None
> E 6,0001-01-01,NULL != None
> E Number of rows returned (expected vs actual): 22 != 15
> Standard Error
>
[jira] [Updated] (IMPALA-9555) TestDateQueries.test_queries failing because Hive3 switched back to the hybrid Julian Gregorian calendar
[
https://issues.apache.org/jira/browse/IMPALA-9555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-9555:
-
Affects Version/s: Impala 3.4.0
> TestDateQueries.test_queries failing because Hive3 switched back to the
> hybrid Julian Gregorian calendar
>
>
> Key: IMPALA-9555
> URL: https://issues.apache.org/jira/browse/IMPALA-9555
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.4.0
>Reporter: Attila Jeges
>Assignee: Attila Jeges
>Priority: Critical
>
> TestDateQueries.test_queries is failing after upgrading the CDP GBN with the
> following error:
> {code}
> query_test.test_date_queries.TestDateQueries.test_queries[protocol: beeswax |
> exec_option: {'disable_codegen_rows_threshold': 0, 'disable_codegen': 'true',
> 'batch_size': 1} | table_format: avro/snap/block] (from pytest)
> Error Message
> query_test/test_date_queries.py:60: in test_queries
> self.run_test_case('QueryTest/avro_date', vector)
> common/impala_test_suite.py:690: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:523: in __verify_results_and_errors
> replace_filenames_with_placeholder) common/test_result_verifier.py:456: in
> verify_raw_results VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results E assert Comparing
> QueryTestResults (expected vs actual): E 0,0001-01-01,0001-01-01 !=
> 10,1399-06-27,2017-11-28 E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
> E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31 E
> 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19 E 12,1399-06-27,2018-12-31
> != 21,2017-11-27,0001-06-20 E 2,0001-01-01,0002-01-01 !=
> 22,2017-11-27,0001-06-21 E 20,2017-11-27,0001-06-21 !=
> 23,2017-11-27,0001-06-22 E 21,2017-11-27,0001-06-22 !=
> 24,2017-11-27,0001-06-23 E 22,2017-11-27,0001-06-23 !=
> 25,2017-11-27,0001-06-24 E 23,2017-11-27,0001-06-24 !=
> 26,2017-11-27,0001-06-25 E 24,2017-11-27,0001-06-25 !=
> 27,2017-11-27,0001-06-26 E 25,2017-11-27,0001-06-26 !=
> 28,2017-11-27,0001-06-27 E 26,2017-11-27,0001-06-27 !=
> 29,2017-11-27,2017-11-28 E 27,2017-11-27,0001-06-28 !=
> 30,-12-31,-12-01 E 28,2017-11-27,0001-06-29 !=
> 31,-12-31,-12-31 E 29,2017-11-27,2017-11-28 != None E
> 3,0001-01-01,1399-12-31 != None E 30,-12-31,-12-01 != None E
> 31,-12-31,-12-31 != None E 4,0001-01-01,2017-11-28 != None E
> 5,0001-01-01,-12-31 != None E 6,0001-01-01,NULL != None E Number
> of rows returned (expected vs actual): 22 != 15
> Stacktrace
> query_test/test_date_queries.py:60: in test_queries
> self.run_test_case('QueryTest/avro_date', vector)
> common/impala_test_suite.py:690: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:523: in __verify_results_and_errors
> replace_filenames_with_placeholder)
> common/test_result_verifier.py:456: in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E 0,0001-01-01,0001-01-01 != 10,1399-06-27,2017-11-28
> E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
> E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31
> E 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19
> E 12,1399-06-27,2018-12-31 != 21,2017-11-27,0001-06-20
> E 2,0001-01-01,0002-01-01 != 22,2017-11-27,0001-06-21
> E 20,2017-11-27,0001-06-21 != 23,2017-11-27,0001-06-22
> E 21,2017-11-27,0001-06-22 != 24,2017-11-27,0001-06-23
> E 22,2017-11-27,0001-06-23 != 25,2017-11-27,0001-06-24
> E 23,2017-11-27,0001-06-24 != 26,2017-11-27,0001-06-25
> E 24,2017-11-27,0001-06-25 != 27,2017-11-27,0001-06-26
> E 25,2017-11-27,0001-06-26 != 28,2017-11-27,0001-06-27
> E 26,2017-11-27,0001-06-27 != 29,2017-11-27,2017-11-28
> E 27,2017-11-27,0001-06-28 != 30,-12-31,-12-01
> E 28,2017-11-27,0001-06-29 != 31,-12-31,-12-31
> E 29,2017-11-27,2017-11-28 != None
> E 3,0001-01-01,1399-12-31 != None
> E 30,-12-31,-12-01 != None
> E 31,-12-31,-12-31 != None
> E 4,0001-01-01,2017-11-28 != None
> E 5,0001-01-01,-12-31 != None
> E 6,0001-01-01,NULL != None
> E Number of rows returned (expected vs actual): 22 != 15
> Standard Error
> ERROR:test_configuration:Comparing QueryTestResults (expected vs actual):
>
[jira] [Work started] (IMPALA-9555) TestDateQueries.test_queries failing because Hive3 switched back to the hybrid Julian Gregorian calendar
[
https://issues.apache.org/jira/browse/IMPALA-9555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-9555 started by Attila Jeges.
> TestDateQueries.test_queries failing because Hive3 switched back to the
> hybrid Julian Gregorian calendar
>
>
> Key: IMPALA-9555
> URL: https://issues.apache.org/jira/browse/IMPALA-9555
> Project: IMPALA
> Issue Type: Bug
>Reporter: Attila Jeges
>Assignee: Attila Jeges
>Priority: Critical
>
> TestDateQueries.test_queries is failing after upgrading the CDP GBN with the
> following error:
> {code}
> query_test.test_date_queries.TestDateQueries.test_queries[protocol: beeswax |
> exec_option: {'disable_codegen_rows_threshold': 0, 'disable_codegen': 'true',
> 'batch_size': 1} | table_format: avro/snap/block] (from pytest)
> Error Message
> query_test/test_date_queries.py:60: in test_queries
> self.run_test_case('QueryTest/avro_date', vector)
> common/impala_test_suite.py:690: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:523: in __verify_results_and_errors
> replace_filenames_with_placeholder) common/test_result_verifier.py:456: in
> verify_raw_results VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results E assert Comparing
> QueryTestResults (expected vs actual): E 0,0001-01-01,0001-01-01 !=
> 10,1399-06-27,2017-11-28 E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
> E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31 E
> 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19 E 12,1399-06-27,2018-12-31
> != 21,2017-11-27,0001-06-20 E 2,0001-01-01,0002-01-01 !=
> 22,2017-11-27,0001-06-21 E 20,2017-11-27,0001-06-21 !=
> 23,2017-11-27,0001-06-22 E 21,2017-11-27,0001-06-22 !=
> 24,2017-11-27,0001-06-23 E 22,2017-11-27,0001-06-23 !=
> 25,2017-11-27,0001-06-24 E 23,2017-11-27,0001-06-24 !=
> 26,2017-11-27,0001-06-25 E 24,2017-11-27,0001-06-25 !=
> 27,2017-11-27,0001-06-26 E 25,2017-11-27,0001-06-26 !=
> 28,2017-11-27,0001-06-27 E 26,2017-11-27,0001-06-27 !=
> 29,2017-11-27,2017-11-28 E 27,2017-11-27,0001-06-28 !=
> 30,-12-31,-12-01 E 28,2017-11-27,0001-06-29 !=
> 31,-12-31,-12-31 E 29,2017-11-27,2017-11-28 != None E
> 3,0001-01-01,1399-12-31 != None E 30,-12-31,-12-01 != None E
> 31,-12-31,-12-31 != None E 4,0001-01-01,2017-11-28 != None E
> 5,0001-01-01,-12-31 != None E 6,0001-01-01,NULL != None E Number
> of rows returned (expected vs actual): 22 != 15
> Stacktrace
> query_test/test_date_queries.py:60: in test_queries
> self.run_test_case('QueryTest/avro_date', vector)
> common/impala_test_suite.py:690: in run_test_case
> self.__verify_results_and_errors(vector, test_section, result, use_db)
> common/impala_test_suite.py:523: in __verify_results_and_errors
> replace_filenames_with_placeholder)
> common/test_result_verifier.py:456: in verify_raw_results
> VERIFIER_MAP[verifier](expected, actual)
> common/test_result_verifier.py:278: in verify_query_result_is_equal
> assert expected_results == actual_results
> E assert Comparing QueryTestResults (expected vs actual):
> E 0,0001-01-01,0001-01-01 != 10,1399-06-27,2017-11-28
> E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
> E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31
> E 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19
> E 12,1399-06-27,2018-12-31 != 21,2017-11-27,0001-06-20
> E 2,0001-01-01,0002-01-01 != 22,2017-11-27,0001-06-21
> E 20,2017-11-27,0001-06-21 != 23,2017-11-27,0001-06-22
> E 21,2017-11-27,0001-06-22 != 24,2017-11-27,0001-06-23
> E 22,2017-11-27,0001-06-23 != 25,2017-11-27,0001-06-24
> E 23,2017-11-27,0001-06-24 != 26,2017-11-27,0001-06-25
> E 24,2017-11-27,0001-06-25 != 27,2017-11-27,0001-06-26
> E 25,2017-11-27,0001-06-26 != 28,2017-11-27,0001-06-27
> E 26,2017-11-27,0001-06-27 != 29,2017-11-27,2017-11-28
> E 27,2017-11-27,0001-06-28 != 30,-12-31,-12-01
> E 28,2017-11-27,0001-06-29 != 31,-12-31,-12-31
> E 29,2017-11-27,2017-11-28 != None
> E 3,0001-01-01,1399-12-31 != None
> E 30,-12-31,-12-01 != None
> E 31,-12-31,-12-31 != None
> E 4,0001-01-01,2017-11-28 != None
> E 5,0001-01-01,-12-31 != None
> E 6,0001-01-01,NULL != None
> E Number of rows returned (expected vs actual): 22 != 15
> Standard Error
> ERROR:test_configuration:Comparing QueryTestResults (expected vs actual):
> 0,0001-01-01,0001-01-01 != 10,1399-06-27,2017-11-28
>
[jira] [Updated] (IMPALA-9555) TestDateQueries.test_queries failing because Hive3 switched back to the hybrid Julian Gregorian calendar
[
https://issues.apache.org/jira/browse/IMPALA-9555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-9555:
-
Description:
TestDateQueries.test_queries is failing after upgrading the CDP GBN with the
following error:
{code}
query_test.test_date_queries.TestDateQueries.test_queries[protocol: beeswax |
exec_option: {'disable_codegen_rows_threshold': 0, 'disable_codegen': 'true',
'batch_size': 1} | table_format: avro/snap/block] (from pytest)
Error Message
query_test/test_date_queries.py:60: in test_queries
self.run_test_case('QueryTest/avro_date', vector)
common/impala_test_suite.py:690: in run_test_case
self.__verify_results_and_errors(vector, test_section, result, use_db)
common/impala_test_suite.py:523: in __verify_results_and_errors
replace_filenames_with_placeholder) common/test_result_verifier.py:456: in
verify_raw_results VERIFIER_MAP[verifier](expected, actual)
common/test_result_verifier.py:278: in verify_query_result_is_equal assert
expected_results == actual_results E assert Comparing QueryTestResults
(expected vs actual): E 0,0001-01-01,0001-01-01 != 10,1399-06-27,2017-11-28
E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL E
10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31 E 11,1399-06-27,NULL
!= 20,2017-11-27,0001-06-19 E 12,1399-06-27,2018-12-31 !=
21,2017-11-27,0001-06-20 E 2,0001-01-01,0002-01-01 !=
22,2017-11-27,0001-06-21 E 20,2017-11-27,0001-06-21 !=
23,2017-11-27,0001-06-22 E 21,2017-11-27,0001-06-22 !=
24,2017-11-27,0001-06-23 E 22,2017-11-27,0001-06-23 !=
25,2017-11-27,0001-06-24 E 23,2017-11-27,0001-06-24 !=
26,2017-11-27,0001-06-25 E 24,2017-11-27,0001-06-25 !=
27,2017-11-27,0001-06-26 E 25,2017-11-27,0001-06-26 !=
28,2017-11-27,0001-06-27 E 26,2017-11-27,0001-06-27 !=
29,2017-11-27,2017-11-28 E 27,2017-11-27,0001-06-28 !=
30,-12-31,-12-01 E 28,2017-11-27,0001-06-29 !=
31,-12-31,-12-31 E 29,2017-11-27,2017-11-28 != None E
3,0001-01-01,1399-12-31 != None E 30,-12-31,-12-01 != None E
31,-12-31,-12-31 != None E 4,0001-01-01,2017-11-28 != None E
5,0001-01-01,-12-31 != None E 6,0001-01-01,NULL != None E Number of
rows returned (expected vs actual): 22 != 15
Stacktrace
query_test/test_date_queries.py:60: in test_queries
self.run_test_case('QueryTest/avro_date', vector)
common/impala_test_suite.py:690: in run_test_case
self.__verify_results_and_errors(vector, test_section, result, use_db)
common/impala_test_suite.py:523: in __verify_results_and_errors
replace_filenames_with_placeholder)
common/test_result_verifier.py:456: in verify_raw_results
VERIFIER_MAP[verifier](expected, actual)
common/test_result_verifier.py:278: in verify_query_result_is_equal
assert expected_results == actual_results
E assert Comparing QueryTestResults (expected vs actual):
E 0,0001-01-01,0001-01-01 != 10,1399-06-27,2017-11-28
E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31
E 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19
E 12,1399-06-27,2018-12-31 != 21,2017-11-27,0001-06-20
E 2,0001-01-01,0002-01-01 != 22,2017-11-27,0001-06-21
E 20,2017-11-27,0001-06-21 != 23,2017-11-27,0001-06-22
E 21,2017-11-27,0001-06-22 != 24,2017-11-27,0001-06-23
E 22,2017-11-27,0001-06-23 != 25,2017-11-27,0001-06-24
E 23,2017-11-27,0001-06-24 != 26,2017-11-27,0001-06-25
E 24,2017-11-27,0001-06-25 != 27,2017-11-27,0001-06-26
E 25,2017-11-27,0001-06-26 != 28,2017-11-27,0001-06-27
E 26,2017-11-27,0001-06-27 != 29,2017-11-27,2017-11-28
E 27,2017-11-27,0001-06-28 != 30,-12-31,-12-01
E 28,2017-11-27,0001-06-29 != 31,-12-31,-12-31
E 29,2017-11-27,2017-11-28 != None
E 3,0001-01-01,1399-12-31 != None
E 30,-12-31,-12-01 != None
E 31,-12-31,-12-31 != None
E 4,0001-01-01,2017-11-28 != None
E 5,0001-01-01,-12-31 != None
E 6,0001-01-01,NULL != None
E Number of rows returned (expected vs actual): 22 != 15
Standard Error
ERROR:test_configuration:Comparing QueryTestResults (expected vs actual):
0,0001-01-01,0001-01-01 != 10,1399-06-27,2017-11-28
1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31
11,1399-06-27,NULL != 20,2017-11-27,0001-06-19
12,1399-06-27,2018-12-31 != 21,2017-11-27,0001-06-20
2,0001-01-01,0002-01-01 != 22,2017-11-27,0001-06-21
20,2017-11-27,0001-06-21 != 23,2017-11-27,0001-06-22
21,2017-11-27,0001-06-22 != 24,2017-11-27,0001-06-23
22,2017-11-27,0001-06-23 != 25,2017-11-27,0001-06-24
23,2017-11-27,0001-06-24 != 26,2017-11-27,0001-06-25
24,2017-11-27,0001-06-25 != 27,2017-11-27,0001-06-26
25,2017-11-27,0001-06-26 != 28,2017-11-27,0001-06-27
26,2017-11-27,0001-06-27 != 29,2017-11-27,2017-11-28
27,2017-11-27,0001-06-28
[jira] [Created] (IMPALA-9555) TestDateQueries.test_queries failing because Hive3 switched back to the hybrid Julian Gregorian calendar
Attila Jeges created IMPALA-9555:
Summary: TestDateQueries.test_queries failing because Hive3
switched back to the hybrid Julian Gregorian calendar
Key: IMPALA-9555
URL: https://issues.apache.org/jira/browse/IMPALA-9555
Project: IMPALA
Issue Type: Bug
Reporter: Attila Jeges
TestDateQueries.test_queries is failing after upgrading the CDP GBN with the
following error:
{code}
query_test.test_date_queries.TestDateQueries.test_queries[protocol: beeswax |
exec_option: {'disable_codegen_rows_threshold': 0, 'disable_codegen': 'true',
'batch_size': 1} | table_format: avro/snap/block] (from pytest)
Error Message
query_test/test_date_queries.py:60: in test_queries
self.run_test_case('QueryTest/avro_date', vector)
common/impala_test_suite.py:690: in run_test_case
self.__verify_results_and_errors(vector, test_section, result, use_db)
common/impala_test_suite.py:523: in __verify_results_and_errors
replace_filenames_with_placeholder) common/test_result_verifier.py:456: in
verify_raw_results VERIFIER_MAP[verifier](expected, actual)
common/test_result_verifier.py:278: in verify_query_result_is_equal assert
expected_results == actual_results E assert Comparing QueryTestResults
(expected vs actual): E 0,0001-01-01,0001-01-01 != 10,1399-06-27,2017-11-28
E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL E
10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31 E 11,1399-06-27,NULL
!= 20,2017-11-27,0001-06-19 E 12,1399-06-27,2018-12-31 !=
21,2017-11-27,0001-06-20 E 2,0001-01-01,0002-01-01 !=
22,2017-11-27,0001-06-21 E 20,2017-11-27,0001-06-21 !=
23,2017-11-27,0001-06-22 E 21,2017-11-27,0001-06-22 !=
24,2017-11-27,0001-06-23 E 22,2017-11-27,0001-06-23 !=
25,2017-11-27,0001-06-24 E 23,2017-11-27,0001-06-24 !=
26,2017-11-27,0001-06-25 E 24,2017-11-27,0001-06-25 !=
27,2017-11-27,0001-06-26 E 25,2017-11-27,0001-06-26 !=
28,2017-11-27,0001-06-27 E 26,2017-11-27,0001-06-27 !=
29,2017-11-27,2017-11-28 E 27,2017-11-27,0001-06-28 !=
30,-12-31,-12-01 E 28,2017-11-27,0001-06-29 !=
31,-12-31,-12-31 E 29,2017-11-27,2017-11-28 != None E
3,0001-01-01,1399-12-31 != None E 30,-12-31,-12-01 != None E
31,-12-31,-12-31 != None E 4,0001-01-01,2017-11-28 != None E
5,0001-01-01,-12-31 != None E 6,0001-01-01,NULL != None E Number of
rows returned (expected vs actual): 22 != 15
Stacktrace
query_test/test_date_queries.py:60: in test_queries
self.run_test_case('QueryTest/avro_date', vector)
common/impala_test_suite.py:690: in run_test_case
self.__verify_results_and_errors(vector, test_section, result, use_db)
common/impala_test_suite.py:523: in __verify_results_and_errors
replace_filenames_with_placeholder)
common/test_result_verifier.py:456: in verify_raw_results
VERIFIER_MAP[verifier](expected, actual)
common/test_result_verifier.py:278: in verify_query_result_is_equal
assert expected_results == actual_results
E assert Comparing QueryTestResults (expected vs actual):
E 0,0001-01-01,0001-01-01 != 10,1399-06-27,2017-11-28
E 1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
E 10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31
E 11,1399-06-27,NULL != 20,2017-11-27,0001-06-19
E 12,1399-06-27,2018-12-31 != 21,2017-11-27,0001-06-20
E 2,0001-01-01,0002-01-01 != 22,2017-11-27,0001-06-21
E 20,2017-11-27,0001-06-21 != 23,2017-11-27,0001-06-22
E 21,2017-11-27,0001-06-22 != 24,2017-11-27,0001-06-23
E 22,2017-11-27,0001-06-23 != 25,2017-11-27,0001-06-24
E 23,2017-11-27,0001-06-24 != 26,2017-11-27,0001-06-25
E 24,2017-11-27,0001-06-25 != 27,2017-11-27,0001-06-26
E 25,2017-11-27,0001-06-26 != 28,2017-11-27,0001-06-27
E 26,2017-11-27,0001-06-27 != 29,2017-11-27,2017-11-28
E 27,2017-11-27,0001-06-28 != 30,-12-31,-12-01
E 28,2017-11-27,0001-06-29 != 31,-12-31,-12-31
E 29,2017-11-27,2017-11-28 != None
E 3,0001-01-01,1399-12-31 != None
E 30,-12-31,-12-01 != None
E 31,-12-31,-12-31 != None
E 4,0001-01-01,2017-11-28 != None
E 5,0001-01-01,-12-31 != None
E 6,0001-01-01,NULL != None
E Number of rows returned (expected vs actual): 22 != 15
Standard Error
ERROR:test_configuration:Comparing QueryTestResults (expected vs actual):
0,0001-01-01,0001-01-01 != 10,1399-06-27,2017-11-28
1,0001-01-01,0001-12-31 != 11,1399-06-27,NULL
10,1399-06-27,2017-11-28 != 12,1399-06-27,2018-12-31
11,1399-06-27,NULL != 20,2017-11-27,0001-06-19
12,1399-06-27,2018-12-31 != 21,2017-11-27,0001-06-20
2,0001-01-01,0002-01-01 != 22,2017-11-27,0001-06-21
20,2017-11-27,0001-06-21 != 23,2017-11-27,0001-06-22
21,2017-11-27,0001-06-22 != 24,2017-11-27,0001-06-23
22,2017-11-27,0001-06-23 != 25,2017-11-27,0001-06-24
23,2017-11-27,0001-06-24 != 26,2017-11-27,0001-06-25
[jira] [Assigned] (IMPALA-9498) Allow Collection types in SELECT list for Parquet tables
[
https://issues.apache.org/jira/browse/IMPALA-9498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges reassigned IMPALA-9498:
Assignee: Attila Jeges
> Allow Collection types in SELECT list for Parquet tables
>
>
> Key: IMPALA-9498
> URL: https://issues.apache.org/jira/browse/IMPALA-9498
> Project: IMPALA
> Issue Type: New Feature
>Reporter: Gabor Kaszab
>Assignee: Attila Jeges
>Priority: Major
> Labels: complextype
>
> This covers collections: Array, Map
> Expected printout format:
> Array: [null,1,2,null,3,null]
> Map: {"k1":2,"k2":null}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-9421) Metadata operations are slow in impala-shell when using hs2-http with LDAP auth.
[
https://issues.apache.org/jira/browse/IMPALA-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-9421:
-
Description:
Show database operation takes ~ 3 - 4 seconds, sometimes ~ 8 - 9 seconds in
impala-shell when connecting to a coordinator using hs2-http with LDAP
authentication:
{code:java}
$ impala-shell.sh --protocol='hs2-http' --ssl -i "impala-coordinator:443" -u
username -l
impala-shell> show database;
++--+
| name | comment |
++--+
| _impala_builtins | System database for Impala builtin functions |
| airline_ontime_orc | |
| airline_ontime_parquet | |
| default | Default Hive database |
++--+
Fetched 4 row(s) in 8.87s
{code}
impala-coordinator logs show that there are multiple new connections set up and
authenticated:
{code:java}
I0225 16:07:58.143942 317 TAcceptQueueServer.cpp:340] New connection to
server hiveserver2-http-frontend from client
I0225 16:07:58.144042 321 TAcceptQueueServer.cpp:227] TAcceptQueueServer:
hiveserver2-http-frontend started connection setup for client
I0225 16:07:58.144101 321 TAcceptQueueServer.cpp:245] TAcceptQueueServer:
hiveserver2-http-frontend finished connection setup for client
I0225 16:07:58.144338 128883 authentication.cc:261] Trying simple LDAP bind
for:
uid=csso_attilaj,cn=users,cn=accounts,dc=attilaj,dc=xcu2-8y8x,dc=dev,dc=cldr,dc=work
I0225 16:07:58.155827 128883 authentication.cc:273] LDAP bind successful
I0225 16:07:58.155901 128883 impala-hs2-server.cc:1085] PingImpalaHS2Service():
request=TPingImpalaHS2ServiceReq {
01: sessionHandle (struct) = TSessionHandle {
01: sessionId (struct) = THandleIdentifier {
01: guid (string) = "\xab\x9bS/\r\xd1@\xab\x862z\xee(#\x14h",
02: secret (string) =
"\x81\x84\xf0\x7f\v\xac@\x9a\x9b\x9e\xdf#\xa1\xc3\xc4\x04",
},
},
}
I0225 16:07:58.876168 317 TAcceptQueueServer.cpp:340] New connection to
server hiveserver2-http-frontend from client
I0225 16:07:58.876317 320 TAcceptQueueServer.cpp:227] TAcceptQueueServer:
hiveserver2-http-frontend started connection setup for client
I0225 16:07:58.876364 320 TAcceptQueueServer.cpp:245] TAcceptQueueServer:
hiveserver2-http-frontend finished connection setup for client
I0225 16:07:58.876847 128884 authentication.cc:261] Trying simple LDAP bind
for:
uid=csso_attilaj,cn=users,cn=accounts,dc=attilaj,dc=xcu2-8y8x,dc=dev,dc=cldr,dc=work
I0225 16:07:58.887931 128884 authentication.cc:273] LDAP bind successful
I0225 16:07:58.888008 128884 impala-hs2-server.cc:442] ExecuteStatement():
request=TExecuteStatementReq {
01: sessionHandle (struct) = TSessionHandle {
01: sessionId (struct) = THandleIdentifier {
01: guid (string) = "\xab\x9bS/\r\xd1@\xab\x862z\xee(#\x14h",
02: secret (string) =
"\x81\x84\xf0\x7f\v\xac@\x9a\x9b\x9e\xdf#\xa1\xc3\xc4\x04",
},
},
02: statement (string) = "show databases",
03: confOverlay (map) = map[1] {
"CLIENT_IDENTIFIER" -> "Impala Shell v3.4.0-SNAPSHOT (cad1561) built on Fri
Feb 14 14:15:26 CET 2020",
},
04: runAsync (bool) = true,
}
I0225 16:07:58.888049 128884 impala-hs2-server.cc:230] TExecuteStatementReq:
TExecuteStatementReq {
01: sessionHandle (struct) = TSessionHandle {
01: sessionId (struct) = THandleIdentifier {
01: guid (string) = "\xab\x9bS/\r\xd1@\xab\x862z\xee(#\x14h",
02: secret (string) =
"\x81\x84\xf0\x7f\v\xac@\x9a\x9b\x9e\xdf#\xa1\xc3\xc4\x04",
},
},
02: statement (string) = "show databases",
03: confOverlay (map) = map[1] {
"CLIENT_IDENTIFIER" -> "Impala Shell v3.4.0-SNAPSHOT (cad1561) built on Fri
Feb 14 14:15:26 CET 2020",
},
04: runAsync (bool) = true,
}
I0225 16:07:58.898981 128884 impala-hs2-server.cc:268]
TClientRequest.queryOptions: TQueryOptions {
01: abort_on_error (bool) = false,
02: max_errors (i32) = 100,
03: disable_codegen (bool) = false,
04: batch_size (i32) = 0,
05: num_nodes (i32) = 0,
06: max_scan_range_length (i64) = 0,
07: num_scanner_threads (i32) = 0,
11: debug_action (string) = "",
12: mem_limit (i64) = 0,
15: hbase_caching (i32) = 0,
16: hbase_cache_blocks (bool) = false,
17: parquet_file_size (i64) = 0,
18: explain_level (i32) = 1,
19: sync_ddl (bool) = false,
24: disable_outermost_topn (bool) = false,
26: query_timeout_s (i32) = 0,
28: appx_count_distinct (bool) = false,
29: disable_unsafe_spills (bool) = false,
31: exec_single_node_rows_threshold (i32) = 100,
32: optimize_partition_key_scans (bool) = false,
33: replica_preference (i32) = 0,
34: schedule_random_replica (bool) = false,
36: disable_streaming_preaggregations (bool) = false,
37: runtime_filter_mode (i32) = 2,
38: runtime_bloom_filter_size (i32) = 1048576,
39:
[jira] [Updated] (IMPALA-9421) Metadata operations are slow in impala-shell when using hs2-http with LDAP auth.
[
https://issues.apache.org/jira/browse/IMPALA-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-9421:
-
Description:
Show database operation takes over 3-4 seconds in impala-shell when connecting
to a coordinator using hs2-http with LDAP authentication:
{code:java}
$ impala-shell.sh --protocol='hs2-http' --ssl -i "impala-coordinator:443" -u
username -l
impala-shell> show database;
++--+
| name | comment |
++--+
| _impala_builtins | System database for Impala builtin functions |
| airline_ontime_orc | |
| airline_ontime_parquet | |
| default | Default Hive database |
++--+
Fetched 4 row(s) in 3.79s
{code}
impala-coordinator logs show that there are multiple new connections set up and
authenticated:
{code:java}
I0225 15:45:28.478219 317 TAcceptQueueServer.cpp:340] New connection to
server hiveserver2-http-frontend from client
I0225 15:45:28.478384 321 TAcceptQueueServer.cpp:227] TAcceptQueueServer:
hiveserver2-http-frontend started connection setup for client
I0225 15:45:28.478454 321 TAcceptQueueServer.cpp:245] TAcceptQueueServer:
hiveserver2-http-frontend finished connection setup for client
I0225 15:45:28.478729 126270 authentication.cc:261] Trying simple LDAP bind
for:
uid=csso_attilaj,cn=users,cn=accounts,dc=attilaj,dc=xcu2-8y8x,dc=dev,dc=cldr,dc=work
I0225 15:45:28.491451 126270 authentication.cc:273] LDAP bind successful
I0225 15:45:28.491571 126270 impala-hs2-server.cc:1085] PingImpalaHS2Service():
request=TPingImpalaHS2ServiceReq {
01: sessionHandle (struct) = TSessionHandle {
01: sessionId (struct) = THandleIdentifier {
01: guid (string) =
"\xd7U\x11\x89\xf0\xd4J\x12\xbc\x9c\x0e\x19\xff\xd5\xec?",
02: secret (string) =
"\xcd\xf9\x86\a\x90\xa6E\xaf\x92\x19\xee\x1e6S\xea\x85",
},
},
}
I0225 15:45:29.199357 317 TAcceptQueueServer.cpp:340] New connection to
server hiveserver2-http-frontend from client
I0225 15:45:29.199455 320 TAcceptQueueServer.cpp:227] TAcceptQueueServer:
hiveserver2-http-frontend started connection setup for client
I0225 15:45:29.199498 320 TAcceptQueueServer.cpp:245] TAcceptQueueServer:
hiveserver2-http-frontend finished connection setup for client
I0225 15:45:29.199753 126271 authentication.cc:261] Trying simple LDAP bind
for:
uid=csso_attilaj,cn=users,cn=accounts,dc=attilaj,dc=xcu2-8y8x,dc=dev,dc=cldr,dc=work
I0225 15:45:29.210222 126271 authentication.cc:273] LDAP bind successful
I0225 15:45:29.210384 126271 impala-hs2-server.cc:442] ExecuteStatement():
request=TExecuteStatementReq {
01: sessionHandle (struct) = TSessionHandle {
01: sessionId (struct) = THandleIdentifier {
01: guid (string) =
"\xd7U\x11\x89\xf0\xd4J\x12\xbc\x9c\x0e\x19\xff\xd5\xec?",
02: secret (string) =
"\xcd\xf9\x86\a\x90\xa6E\xaf\x92\x19\xee\x1e6S\xea\x85",
},
},
02: statement (string) = "show databases",
03: confOverlay (map) = map[1] {
"CLIENT_IDENTIFIER" -> "Impala Shell v3.4.0-SNAPSHOT (cad1561) built on Fri
Feb 14 14:15:26 CET 2020",
},
04: runAsync (bool) = true,
}
I0225 15:45:29.210427 126271 impala-hs2-server.cc:230] TExecuteStatementReq:
TExecuteStatementReq {
01: sessionHandle (struct) = TSessionHandle {
01: sessionId (struct) = THandleIdentifier {
01: guid (string) =
"\xd7U\x11\x89\xf0\xd4J\x12\xbc\x9c\x0e\x19\xff\xd5\xec?",
02: secret (string) =
"\xcd\xf9\x86\a\x90\xa6E\xaf\x92\x19\xee\x1e6S\xea\x85",
},
},
02: statement (string) = "show databases",
03: confOverlay (map) = map[1] {
"CLIENT_IDENTIFIER" -> "Impala Shell v3.4.0-SNAPSHOT (cad1561) built on Fri
Feb 14 14:15:26 CET 2020",
},
04: runAsync (bool) = true,
}
I0225 15:45:29.220592 126271 impala-hs2-server.cc:268]
TClientRequest.queryOptions: TQueryOptions {
01: abort_on_error (bool) = false,
02: max_errors (i32) = 100,
03: disable_codegen (bool) = false,
04: batch_size (i32) = 0,
05: num_nodes (i32) = 0,
06: max_scan_range_length (i64) = 0,
07: num_scanner_threads (i32) = 0,
11: debug_action (string) = "",
12: mem_limit (i64) = 0,
15: hbase_caching (i32) = 0,
16: hbase_cache_blocks (bool) = false,
17: parquet_file_size (i64) = 0,
18: explain_level (i32) = 1,
19: sync_ddl (bool) = false,
24: disable_outermost_topn (bool) = false,
26: query_timeout_s (i32) = 0,
28: appx_count_distinct (bool) = false,
29: disable_unsafe_spills (bool) = false,
31: exec_single_node_rows_threshold (i32) = 100,
32: optimize_partition_key_scans (bool) = false,
33: replica_preference (i32) = 0,
34: schedule_random_replica (bool) = false,
36: disable_streaming_preaggregations (bool) = false,
37: runtime_filter_mode (i32) = 2,
38: runtime_bloom_filter_size (i32) = 1048576,
39:
[jira] [Updated] (IMPALA-9421) Metadata operations are slow in impala-shell when using hs2-http with LDAP auth.
[
https://issues.apache.org/jira/browse/IMPALA-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-9421:
-
Description:
Show database operation takes over 3-4 seconds in impala-shell when connecting
to a coordinator using hs2-http with LDAP authentication:
{code:java}
$ impala-shell.sh --protocol='hs2-http' --ssl -i "impala-coordinator:443" -u
username -l
impala-shell> show database;
++--+
| name | comment |
++--+
| _impala_builtins | System database for Impala builtin functions |
| airline_ontime_orc | |
| airline_ontime_parquet | |
| default | Default Hive database |
++--+
Fetched 4 row(s) in 3.66s
{code}
impala-coordinator logs show that there are multiple new connections set up and
authenticated:
{code:java}
I0225 14:15:48.976776 317 TAcceptQueueServer.cpp:340] New connection to
server hiveserver2-http-frontend from client
I0225 14:15:48.976878 320 TAcceptQueueServer.cpp:227] TAcceptQueueServer:
hiveserver2-http-frontend started connection setup for client
I0225 14:15:48.976912 320 TAcceptQueueServer.cpp:245] TAcceptQueueServer:
hiveserver2-http-frontend finished connection setup for client
I0225 14:15:48.977216 115929 authentication.cc:261] Trying simple LDAP bind
for:
uid=csso_attilaj,cn=users,cn=accounts,dc=attilaj,dc=xcu2-8y8x,dc=dev,dc=cldr,dc=work
I0225 14:15:48.989554 115929 authentication.cc:273] LDAP bind successful
I0225 14:15:48.989639 115929 impala-hs2-server.cc:1085] PingImpalaHS2Service():
request=TPingImpalaHS2ServiceReq {
01: sessionHandle (struct) = TSessionHandle {
01: sessionId (struct) = THandleIdentifier {
01: guid (string) = "#\x8f\xdf\x01\xd7\xd6Bv\xa5\xec\xcd\x17Q\xb9q\x93",
02: secret (string) = "\xd6\xaaO\v\xedXE!\x89}x\xbds\x1f\xe1\xf0",
},
},
}
I0225 14:15:50.152348 317 TAcceptQueueServer.cpp:340] New connection to
server hiveserver2-http-frontend from client
I0225 14:15:50.152446 321 TAcceptQueueServer.cpp:227] TAcceptQueueServer:
hiveserver2-http-frontend started connection setup for client
I0225 14:15:50.152493 321 TAcceptQueueServer.cpp:245] TAcceptQueueServer:
hiveserver2-http-frontend finished connection setup for client
I0225 14:15:50.152722 115930 authentication.cc:261] Trying simple LDAP bind
for:
uid=csso_attilaj,cn=users,cn=accounts,dc=attilaj,dc=xcu2-8y8x,dc=dev,dc=cldr,dc=work
I0225 14:15:50.163576 115930 authentication.cc:273] LDAP bind successful
I0225 14:15:50.163733 115930 impala-hs2-server.cc:442] ExecuteStatement():
request=TExecuteStatementReq {
01: sessionHandle (struct) = TSessionHandle {
01: sessionId (struct) = THandleIdentifier {
01: guid (string) = "#\x8f\xdf\x01\xd7\xd6Bv\xa5\xec\xcd\x17Q\xb9q\x93",
02: secret (string) = "\xd6\xaaO\v\xedXE!\x89}x\xbds\x1f\xe1\xf0",
},
},
02: statement (string) = "show databases",
03: confOverlay (map) = map[1] {
"CLIENT_IDENTIFIER" -> "Impala Shell v3.4.0-SNAPSHOT (cad1561) built on Fri
Feb 14 14:15:26 CET 2020",
},
04: runAsync (bool) = true,
}
I0225 14:15:50.163775 115930 impala-hs2-server.cc:230] TExecuteStatementReq:
TExecuteStatementReq {
01: sessionHandle (struct) = TSessionHandle {
01: sessionId (struct) = THandleIdentifier {
01: guid (string) = "#\x8f\xdf\x01\xd7\xd6Bv\xa5\xec\xcd\x17Q\xb9q\x93",
02: secret (string) = "\xd6\xaaO\v\xedXE!\x89}x\xbds\x1f\xe1\xf0",
},
},
02: statement (string) = "show databases",
03: confOverlay (map) = map[1] {
"CLIENT_IDENTIFIER" -> "Impala Shell v3.4.0-SNAPSHOT (cad1561) built on Fri
Feb 14 14:15:26 CET 2020",
},
04: runAsync (bool) = true,
}
I0225 14:15:50.173715 115930 impala-hs2-server.cc:268]
TClientRequest.queryOptions: TQueryOptions {
01: abort_on_error (bool) = false,
02: max_errors (i32) = 100,
03: disable_codegen (bool) = false,
04: batch_size (i32) = 0,
05: num_nodes (i32) = 0,
06: max_scan_range_length (i64) = 0,
07: num_scanner_threads (i32) = 0,
11: debug_action (string) = "",
12: mem_limit (i64) = 0,
15: hbase_caching (i32) = 0,
16: hbase_cache_blocks (bool) = false,
17: parquet_file_size (i64) = 0,
18: explain_level (i32) = 1,
19: sync_ddl (bool) = false,
24: disable_outermost_topn (bool) = false,
26: query_timeout_s (i32) = 0,
28: appx_count_distinct (bool) = false,
29: disable_unsafe_spills (bool) = false,
31: exec_single_node_rows_threshold (i32) = 100,
32: optimize_partition_key_scans (bool) = false,
33: replica_preference (i32) = 0,
34: schedule_random_replica (bool) = false,
36: disable_streaming_preaggregations (bool) = false,
37: runtime_filter_mode (i32) = 2,
38: runtime_bloom_filter_size (i32) = 1048576,
39: runtime_filter_wait_time_ms (i32) = 0,
40:
[jira] [Created] (IMPALA-9421) Metadata operations are slow in impala-shell when using hs2-http with LDAP auth.
Attila Jeges created IMPALA-9421:
Summary: Metadata operations are slow in impala-shell when using
hs2-http with LDAP auth.
Key: IMPALA-9421
URL: https://issues.apache.org/jira/browse/IMPALA-9421
Project: IMPALA
Issue Type: Improvement
Components: Clients
Affects Versions: Impala 3.4.0
Reporter: Attila Jeges
Show database operation takes over 3-4 seconds in impala-shell when connecting
to an CDW Azure environment:
{code:java}
$ impala-shell.sh --protocol='hs2-http' --ssl -i
"coordinator-attilaj-test-impala-vw.env-q52cn6.dwx.workload-dev.cloudera.com:443"
-u csso_attilaj -l
impala-shell> show database;
++--+
| name | comment |
++--+
| _impala_builtins | System database for Impala builtin functions |
| airline_ontime_orc | |
| airline_ontime_parquet | |
| default | Default Hive database |
++--+
Fetched 4 row(s) in 3.66s
{code}
impala-coordinator logs show that there are multiple new connections set up and
authenticated:
{code:java}
I0225 14:15:48.976776 317 TAcceptQueueServer.cpp:340] New connection to
server hiveserver2-http-frontend from client
I0225 14:15:48.976878 320 TAcceptQueueServer.cpp:227] TAcceptQueueServer:
hiveserver2-http-frontend started connection setup for client
I0225 14:15:48.976912 320 TAcceptQueueServer.cpp:245] TAcceptQueueServer:
hiveserver2-http-frontend finished connection setup for client
I0225 14:15:48.977216 115929 authentication.cc:261] Trying simple LDAP bind
for:
uid=csso_attilaj,cn=users,cn=accounts,dc=attilaj,dc=xcu2-8y8x,dc=dev,dc=cldr,dc=work
I0225 14:15:48.989554 115929 authentication.cc:273] LDAP bind successful
I0225 14:15:48.989639 115929 impala-hs2-server.cc:1085] PingImpalaHS2Service():
request=TPingImpalaHS2ServiceReq {
01: sessionHandle (struct) = TSessionHandle {
01: sessionId (struct) = THandleIdentifier {
01: guid (string) = "#\x8f\xdf\x01\xd7\xd6Bv\xa5\xec\xcd\x17Q\xb9q\x93",
02: secret (string) = "\xd6\xaaO\v\xedXE!\x89}x\xbds\x1f\xe1\xf0",
},
},
}
I0225 14:15:50.152348 317 TAcceptQueueServer.cpp:340] New connection to
server hiveserver2-http-frontend from client
I0225 14:15:50.152446 321 TAcceptQueueServer.cpp:227] TAcceptQueueServer:
hiveserver2-http-frontend started connection setup for client
I0225 14:15:50.152493 321 TAcceptQueueServer.cpp:245] TAcceptQueueServer:
hiveserver2-http-frontend finished connection setup for client
I0225 14:15:50.152722 115930 authentication.cc:261] Trying simple LDAP bind
for:
uid=csso_attilaj,cn=users,cn=accounts,dc=attilaj,dc=xcu2-8y8x,dc=dev,dc=cldr,dc=work
I0225 14:15:50.163576 115930 authentication.cc:273] LDAP bind successful
I0225 14:15:50.163733 115930 impala-hs2-server.cc:442] ExecuteStatement():
request=TExecuteStatementReq {
01: sessionHandle (struct) = TSessionHandle {
01: sessionId (struct) = THandleIdentifier {
01: guid (string) = "#\x8f\xdf\x01\xd7\xd6Bv\xa5\xec\xcd\x17Q\xb9q\x93",
02: secret (string) = "\xd6\xaaO\v\xedXE!\x89}x\xbds\x1f\xe1\xf0",
},
},
02: statement (string) = "show databases",
03: confOverlay (map) = map[1] {
"CLIENT_IDENTIFIER" -> "Impala Shell v3.4.0-SNAPSHOT (cad1561) built on Fri
Feb 14 14:15:26 CET 2020",
},
04: runAsync (bool) = true,
}
I0225 14:15:50.163775 115930 impala-hs2-server.cc:230] TExecuteStatementReq:
TExecuteStatementReq {
01: sessionHandle (struct) = TSessionHandle {
01: sessionId (struct) = THandleIdentifier {
01: guid (string) = "#\x8f\xdf\x01\xd7\xd6Bv\xa5\xec\xcd\x17Q\xb9q\x93",
02: secret (string) = "\xd6\xaaO\v\xedXE!\x89}x\xbds\x1f\xe1\xf0",
},
},
02: statement (string) = "show databases",
03: confOverlay (map) = map[1] {
"CLIENT_IDENTIFIER" -> "Impala Shell v3.4.0-SNAPSHOT (cad1561) built on Fri
Feb 14 14:15:26 CET 2020",
},
04: runAsync (bool) = true,
}
I0225 14:15:50.173715 115930 impala-hs2-server.cc:268]
TClientRequest.queryOptions: TQueryOptions {
01: abort_on_error (bool) = false,
02: max_errors (i32) = 100,
03: disable_codegen (bool) = false,
04: batch_size (i32) = 0,
05: num_nodes (i32) = 0,
06: max_scan_range_length (i64) = 0,
07: num_scanner_threads (i32) = 0,
11: debug_action (string) = "",
12: mem_limit (i64) = 0,
15: hbase_caching (i32) = 0,
16: hbase_cache_blocks (bool) = false,
17: parquet_file_size (i64) = 0,
18: explain_level (i32) = 1,
19: sync_ddl (bool) = false,
24: disable_outermost_topn (bool) = false,
26: query_timeout_s (i32) = 0,
28: appx_count_distinct (bool) = false,
29: disable_unsafe_spills (bool) = false,
31: exec_single_node_rows_threshold (i32) = 100,
32: optimize_partition_key_scans (bool) = false,
33: replica_preference
[jira] [Updated] (IMPALA-9407) Impalad crash in RuntimeProfile::EventSequence
[
https://issues.apache.org/jira/browse/IMPALA-9407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-9407:
-
Labels: broken-build flaky (was: broken-build)
> Impalad crash in RuntimeProfile::EventSequence
> --
>
> Key: IMPALA-9407
> URL: https://issues.apache.org/jira/browse/IMPALA-9407
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.4.0
>Reporter: Attila Jeges
>Assignee: Tim Armstrong
>Priority: Blocker
> Labels: broken-build, flaky
>
> impalad instance crashed during exhaustive tests, probably while running
> TestExprs.test_exprs:
> {code:java}
> 18:00:32 TestExprs.test_exprs[protocol: beeswax | exec_option:
> {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
> 'disable_codegen': False, 'abort_on_error': 1,
> 'exec_single_node_rows_threshold': 0} | table_format: orc/def/block |
> enable_expr_rewrites: 0]
> 18:00:32 [gw1] linux2 -- Python 2.6.6
> /data/jenkins/workspace/impala-asf-master-exhaustive-centos6-shard2/repos/Impala/bin/../infra/python/env/bin/python
> 18:00:32 query_test/test_exprs.py:60: in test_exprs
> 18:00:32 self.run_test_case('QueryTest/exprs', vector)
> 18:00:32 common/impala_test_suite.py:659: in run_test_case
> 18:00:32 result = exec_fn(query, user=test_section.get('USER',
> '').strip() or None)
> 18:00:32 common/impala_test_suite.py:594: in __exec_in_impala
> 18:00:32 result = self.__execute_query(target_impalad_client, query,
> user=user)
> 18:00:32 common/impala_test_suite.py:933: in __execute_query
> 18:00:32 return impalad_client.execute(query, user=user)
> 18:00:32 common/impala_connection.py:205: in execute
> 18:00:32 return self.__beeswax_client.execute(sql_stmt, user=user)
> 18:00:32 beeswax/impala_beeswax.py:187: in execute
> 18:00:32 handle = self.__execute_query(query_string.strip(), user=user)
> 18:00:32 beeswax/impala_beeswax.py:365: in __execute_query
> 18:00:32 self.wait_for_finished(handle)
> 18:00:32 beeswax/impala_beeswax.py:386: in wait_for_finished
> 18:00:32 raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> 18:00:32 E ImpalaBeeswaxException: ImpalaBeeswaxException:
> 18:00:32 EQuery aborted:Failed due to unreachable impalad(s):
> shared-centos64-ec2-m2-4xlarge-ondemand-13c1.vpc.cloudera.com:22002
> {code}
> impalad minidump:
> {code:java}
> Operating system: Linux
> 0.0.0 Linux 2.6.32-358.14.1.el6.centos.plus.x86_64 #1 SMP
> Tue Jul 16 21:33:24 UTC 2013 x86_64
> CPU: amd64
> family 6 model 45 stepping 7
> 8 CPUs
> GPU: UNKNOWN
> Crash reason: SIGABRT
> Crash address: 0x4524ed0
> Process uptime: not available
> Thread 373 (crashed)
> 0 libc-2.12.so + 0x328e5
> rax = 0x rdx = 0x0006
> rcx = 0x rbx = 0x072153c0
> rsi = 0x3a25 rdi = 0x4ed0
> rbp = 0x7f2b804a9de0 rsp = 0x7f2b804a9a78
> r8 = 0x7f2b804a9b00r9 = 0x3a25
> r10 = 0x0008 r11 = 0x0202
> r12 = 0x07215440 r13 = 0x004c
> r14 = 0x0721cd84 r15 = 0x072153c0
> rip = 0x0034f4c328e5
> Found by: given as instruction pointer in context
> 1 libc-2.12.so + 0x340c5
> rbp = 0x7f2b804a9de0 rsp = 0x7f2b804a9a80
> rip = 0x0034f4c340c5
> Found by: stack scanning
> 2
> impalad!boost::detail::function::void_function_obj_invoker0,
> void>::invoke [function_template.hpp : 153 + 0xb]
> rbp = 0x7f2b804a9de0 rsp = 0x7f2b804a9a88
> rip = 0x021b94da
> Found by: stack scanning
> 3 impalad!google::LogMessage::Flush() + 0x157
> rbp = 0x7f2c83fb27a0 rsp = 0x7f2b804a9df0
> rip = 0x04e4e3b7
> Found by: call frame info
> 4 impalad!google::LogMessageFatal::~LogMessageFatal() + 0xe
> rbx = 0x7f2b804a9eb0 rbp = 0x7f2b804a9ed0
> rsp = 0x7f2b804a9e70 r12 = 0x0001
> r13 = 0x0001 r14 = 0x0001
> r15 = 0x0001 rip = 0x04e5197e
> Found by: call frame info
> 5 impalad!impala::RuntimeProfile::EventSequence::Start(long)
> [runtime-profile-counters.h : 572 + 0xc]
> rbx = 0x0caaee00 rbp = 0x7f2b804a9ed0
> rsp = 0x7f2b804a9e90 r12 = 0x0001
> r13 = 0x0001 r14 = 0x0001
> r15 = 0x0001 rip = 0x021aa49c
> Found by: call frame info
> 6 impalad!impala::FragmentInstanceState::Prepare()
> [fragment-instance-state.cc : 153 + 0x2e]
> rbx = 0x0caaee00 rbp = 0x7f2b804aa4f0
> rsp = 0x7f2b804a9ee0 r12 = 0x0001
> r13 = 0x0001 r14 =
[jira] [Created] (IMPALA-9407) Impalad crash in RuntimeProfile::EventSequence
Attila Jeges created IMPALA-9407:
Summary: Impalad crash in RuntimeProfile::EventSequence
Key: IMPALA-9407
URL: https://issues.apache.org/jira/browse/IMPALA-9407
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 3.4.0
Reporter: Attila Jeges
Assignee: Tim Armstrong
impalad instance crashed during exhaustive tests, probably while running
TestExprs.test_exprs:
{code:java}
18:00:32 TestExprs.test_exprs[protocol: beeswax | exec_option: {'batch_size':
0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen':
False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} |
table_format: orc/def/block | enable_expr_rewrites: 0]
18:00:32 [gw1] linux2 -- Python 2.6.6
/data/jenkins/workspace/impala-asf-master-exhaustive-centos6-shard2/repos/Impala/bin/../infra/python/env/bin/python
18:00:32 query_test/test_exprs.py:60: in test_exprs
18:00:32 self.run_test_case('QueryTest/exprs', vector)
18:00:32 common/impala_test_suite.py:659: in run_test_case
18:00:32 result = exec_fn(query, user=test_section.get('USER', '').strip()
or None)
18:00:32 common/impala_test_suite.py:594: in __exec_in_impala
18:00:32 result = self.__execute_query(target_impalad_client, query,
user=user)
18:00:32 common/impala_test_suite.py:933: in __execute_query
18:00:32 return impalad_client.execute(query, user=user)
18:00:32 common/impala_connection.py:205: in execute
18:00:32 return self.__beeswax_client.execute(sql_stmt, user=user)
18:00:32 beeswax/impala_beeswax.py:187: in execute
18:00:32 handle = self.__execute_query(query_string.strip(), user=user)
18:00:32 beeswax/impala_beeswax.py:365: in __execute_query
18:00:32 self.wait_for_finished(handle)
18:00:32 beeswax/impala_beeswax.py:386: in wait_for_finished
18:00:32 raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
18:00:32 E ImpalaBeeswaxException: ImpalaBeeswaxException:
18:00:32 EQuery aborted:Failed due to unreachable impalad(s):
shared-centos64-ec2-m2-4xlarge-ondemand-13c1.vpc.cloudera.com:22002
{code}
impalad minidump:
{code:java}
Operating system: Linux
0.0.0 Linux 2.6.32-358.14.1.el6.centos.plus.x86_64 #1 SMP Tue
Jul 16 21:33:24 UTC 2013 x86_64
CPU: amd64
family 6 model 45 stepping 7
8 CPUs
GPU: UNKNOWN
Crash reason: SIGABRT
Crash address: 0x4524ed0
Process uptime: not available
Thread 373 (crashed)
0 libc-2.12.so + 0x328e5
rax = 0x rdx = 0x0006
rcx = 0x rbx = 0x072153c0
rsi = 0x3a25 rdi = 0x4ed0
rbp = 0x7f2b804a9de0 rsp = 0x7f2b804a9a78
r8 = 0x7f2b804a9b00r9 = 0x3a25
r10 = 0x0008 r11 = 0x0202
r12 = 0x07215440 r13 = 0x004c
r14 = 0x0721cd84 r15 = 0x072153c0
rip = 0x0034f4c328e5
Found by: given as instruction pointer in context
1 libc-2.12.so + 0x340c5
rbp = 0x7f2b804a9de0 rsp = 0x7f2b804a9a80
rip = 0x0034f4c340c5
Found by: stack scanning
2
impalad!boost::detail::function::void_function_obj_invoker0,
void>::invoke [function_template.hpp : 153 + 0xb]
rbp = 0x7f2b804a9de0 rsp = 0x7f2b804a9a88
rip = 0x021b94da
Found by: stack scanning
3 impalad!google::LogMessage::Flush() + 0x157
rbp = 0x7f2c83fb27a0 rsp = 0x7f2b804a9df0
rip = 0x04e4e3b7
Found by: call frame info
4 impalad!google::LogMessageFatal::~LogMessageFatal() + 0xe
rbx = 0x7f2b804a9eb0 rbp = 0x7f2b804a9ed0
rsp = 0x7f2b804a9e70 r12 = 0x0001
r13 = 0x0001 r14 = 0x0001
r15 = 0x0001 rip = 0x04e5197e
Found by: call frame info
5 impalad!impala::RuntimeProfile::EventSequence::Start(long)
[runtime-profile-counters.h : 572 + 0xc]
rbx = 0x0caaee00 rbp = 0x7f2b804a9ed0
rsp = 0x7f2b804a9e90 r12 = 0x0001
r13 = 0x0001 r14 = 0x0001
r15 = 0x0001 rip = 0x021aa49c
Found by: call frame info
6 impalad!impala::FragmentInstanceState::Prepare()
[fragment-instance-state.cc : 153 + 0x2e]
rbx = 0x0caaee00 rbp = 0x7f2b804aa4f0
rsp = 0x7f2b804a9ee0 r12 = 0x0001
r13 = 0x0001 r14 = 0x0001
r15 = 0x0001 rip = 0x021a42f8
Found by: call frame info
7 impalad!impala::FragmentInstanceState::Exec() [fragment-instance-state.cc :
75 + 0x19]
rbx = 0x0001 rbp = 0x7f2b804aa650
rsp = 0x7f2b804aa500 r12 = 0x0001
r13 = 0x0001 r14 = 0x0001
r15 = 0x0001 rip = 0x021a366e
Found by: call frame info
8
[jira] [Reopened] (IMPALA-7983) Failed verification/decryption while reading spilled data causing test failure
[
https://issues.apache.org/jira/browse/IMPALA-7983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges reopened IMPALA-7983:
--
> Failed verification/decryption while reading spilled data causing test failure
> --
>
> Key: IMPALA-7983
> URL: https://issues.apache.org/jira/browse/IMPALA-7983
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.1.0
>Reporter: Bharath Vissapragada
>Assignee: Tim Armstrong
>Priority: Major
> Labels: broken-build
>
> Failed test:
> {noformat}
> query_test.test_spilling.TestSpillingDebugActionDimensions.test_spilling[protocol:
> beeswax | exec_option: {'debug_action':
> '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0',
> 'default_spillable_buffer_size': '256k'} | table_format: parquet/none]
> {noformat}
> Build type: {{exhaustive/centos6}}
> Build logs:
> {noformat}
> query_test/test_spilling.py:59: in test_spilling
> self.run_test_case('QueryTest/spilling', vector)
> common/impala_test_suite.py:467: in run_test_case result =
> self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:688: in __execute_query return
> impalad_client.execute(query, user=user) common/impala_connection.py:170: in
> execute return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:182: in execute handle =
> self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:359: in __execute_query
> self.wait_for_finished(handle) beeswax/impala_beeswax.py:380: in
> wait_for_finished raise ImpalaBeeswaxException("Query aborted:" +
> error_log, None) E ImpalaBeeswaxException: ImpalaBeeswaxException: E
> Query aborted:Error reading 524288 bytes from scratch file
> '/tmp/impala-scratch/4b417a5b42d94b12:2ba23210_d1ff959f-84da-4176-8b29-ff8e1560e941'
> on backend
> shared-centos64-ec2-m2-4xlarge-ondemand-1d0a.vpc.cloudera.com:22001 at offset
> 806092800: verification of read data failed. E OpenSSL error in
> EVP_DecryptFinal decrypting:
> {noformat}
> Stacktrace
> {noformat}
> query_test/test_spilling.py:59: in test_spilling
> self.run_test_case('QueryTest/spilling', vector)
> common/impala_test_suite.py:467: in run_test_case
> result = self.__execute_query(target_impalad_client, query, user=user)
> common/impala_test_suite.py:688: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:170: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:182: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:359: in __execute_query
> self.wait_for_finished(handle)
> beeswax/impala_beeswax.py:380: in wait_for_finished
> raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EQuery aborted:Error reading 524288 bytes from scratch file
> '/tmp/impala-scratch/4b417a5b42d94b12:2ba23210_d1ff959f-84da-4176-8b29-ff8e1560e941'
> on backend
> shared-centos64-ec2-m2-4xlarge-ondemand-1d0a.vpc.cloudera.com:22001 at offset
> 806092800: verification of read data failed.
> E OpenSSL error in EVP_DecryptFinal decrypting:
> {noformat}
> Build stderr:
> {noformat}
> -- executing against localhost:21000
> use tpch_parquet;
> -- 2018-12-14 00:17:21,945 INFO MainThread: Started query
> 244b8ebf1a985cca:df5b0e3a
> SET debug_action=-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@1.0;
> SET default_spillable_buffer_size=256k;
> -- executing against localhost:21000
> set buffer_pool_limit=215m;
> -- 2018-12-14 00:17:21,990 INFO MainThread: Started query
> 1a46eec9e244c88a:49a6b2df
> -- executing against localhost:21000
> select count(l1.l_tax)
> from
> lineitem l1,
> lineitem l2,
> lineitem l3
> where
> l1.l_tax < 0.01 and
> l2.l_tax < 0.04 and
> l1.l_orderkey = l2.l_orderkey and
> l1.l_orderkey = l3.l_orderkey and
> l1.l_comment = l3.l_comment and
> l1.l_shipdate = l3.l_shipdate;
> -- 2018-12-14 00:17:22,059 INFO MainThread: Started query
> ef4546e5750f98b3:d67f7716
> -- executing against localhost:21000
> SET BUFFER_POOL_LIMIT="";
> -- 2018-12-14 00:17:47,105 INFO MainThread: Started query
> b84dc37674cb5ab9:e080595b
> -- executing against localhost:21000
> set buffer_pool_limit=50m;
> -- 2018-12-14 00:17:47,132 INFO MainThread: Started query
> be4b201e2b52176a:2f88f483
> -- executing against localhost:21000
> select max(t1.total_count), max(t1.l_shipinstruct), max(t1.l_comment) from
> (select l_shipinstruct, l_comment, count(*) over () total_count from
> lineitem) t1;
> -- 2018-12-14 00:17:47,213 INFO
[jira] [Updated] (IMPALA-9397) AuthorizationStmtTest.testSelect failed against Sentry
[
https://issues.apache.org/jira/browse/IMPALA-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-9397:
-
Labels: broken-build flaky (was: broken-build)
> AuthorizationStmtTest.testSelect failed against Sentry
> --
>
> Key: IMPALA-9397
> URL: https://issues.apache.org/jira/browse/IMPALA-9397
> Project: IMPALA
> Issue Type: Bug
> Components: Security
>Affects Versions: Impala 3.4.0
>Reporter: Attila Jeges
>Assignee: Csaba Ringhofer
>Priority: Blocker
> Labels: broken-build, flaky
>
> Error Message
> {code}
> Principal: non_owner
> Statement: select id from functional.alltypes_view
> Error: User 'non_owner' does not have privileges to execute 'SELECT' on:
> functional.alltypes_view
> {code}
> Stacktrace
> {code}
> org.apache.impala.authorization.AuthorizationException:
> Principal: non_owner
> Statement: select id from functional.alltypes_view
> Error: User 'non_owner' does not have privileges to execute 'SELECT' on:
> functional.alltypes_view
> at
> org.apache.impala.authorization.AuthorizationTestBase.authzOk(AuthorizationTestBase.java:581)
> at
> org.apache.impala.authorization.AuthorizationTestBase.authzOk(AuthorizationTestBase.java:571)
> at
> org.apache.impala.authorization.AuthorizationTestBase.access$300(AuthorizationTestBase.java:75)
> at
> org.apache.impala.authorization.AuthorizationTestBase$AuthzTest.ok(AuthorizationTestBase.java:414)
> at
> org.apache.impala.authorization.AuthorizationStmtTest.testSelect(AuthorizationStmtTest.java:498)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail:
[jira] [Updated] (IMPALA-9397) AuthorizationStmtTest.testSelect failed against Sentry
[
https://issues.apache.org/jira/browse/IMPALA-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-9397:
-
Labels: broken-build (was: )
> AuthorizationStmtTest.testSelect failed against Sentry
> --
>
> Key: IMPALA-9397
> URL: https://issues.apache.org/jira/browse/IMPALA-9397
> Project: IMPALA
> Issue Type: Bug
> Components: Security
>Affects Versions: Impala 3.4.0
>Reporter: Attila Jeges
>Assignee: Csaba Ringhofer
>Priority: Blocker
> Labels: broken-build
>
> Error Message
> {code}
> Principal: non_owner
> Statement: select id from functional.alltypes_view
> Error: User 'non_owner' does not have privileges to execute 'SELECT' on:
> functional.alltypes_view
> {code}
> Stacktrace
> {code}
> org.apache.impala.authorization.AuthorizationException:
> Principal: non_owner
> Statement: select id from functional.alltypes_view
> Error: User 'non_owner' does not have privileges to execute 'SELECT' on:
> functional.alltypes_view
> at
> org.apache.impala.authorization.AuthorizationTestBase.authzOk(AuthorizationTestBase.java:581)
> at
> org.apache.impala.authorization.AuthorizationTestBase.authzOk(AuthorizationTestBase.java:571)
> at
> org.apache.impala.authorization.AuthorizationTestBase.access$300(AuthorizationTestBase.java:75)
> at
> org.apache.impala.authorization.AuthorizationTestBase$AuthzTest.ok(AuthorizationTestBase.java:414)
> at
> org.apache.impala.authorization.AuthorizationStmtTest.testSelect(AuthorizationStmtTest.java:498)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail:
[jira] [Assigned] (IMPALA-9397) AuthorizationStmtTest.testSelect failed
[
https://issues.apache.org/jira/browse/IMPALA-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges reassigned IMPALA-9397:
Assignee: Csaba Ringhofer
> AuthorizationStmtTest.testSelect failed
> ---
>
> Key: IMPALA-9397
> URL: https://issues.apache.org/jira/browse/IMPALA-9397
> Project: IMPALA
> Issue Type: Bug
> Components: Security
>Affects Versions: Impala 3.4.0
>Reporter: Attila Jeges
>Assignee: Csaba Ringhofer
>Priority: Blocker
>
> Error Message
> {code}
> Principal: non_owner
> Statement: select id from functional.alltypes_view
> Error: User 'non_owner' does not have privileges to execute 'SELECT' on:
> functional.alltypes_view
> {code}
> Stacktrace
> {code}
> org.apache.impala.authorization.AuthorizationException:
> Principal: non_owner
> Statement: select id from functional.alltypes_view
> Error: User 'non_owner' does not have privileges to execute 'SELECT' on:
> functional.alltypes_view
> at
> org.apache.impala.authorization.AuthorizationTestBase.authzOk(AuthorizationTestBase.java:581)
> at
> org.apache.impala.authorization.AuthorizationTestBase.authzOk(AuthorizationTestBase.java:571)
> at
> org.apache.impala.authorization.AuthorizationTestBase.access$300(AuthorizationTestBase.java:75)
> at
> org.apache.impala.authorization.AuthorizationTestBase$AuthzTest.ok(AuthorizationTestBase.java:414)
> at
> org.apache.impala.authorization.AuthorizationStmtTest.testSelect(AuthorizationStmtTest.java:498)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail:
[jira] [Updated] (IMPALA-9397) AuthorizationStmtTest.testSelect failed against Sentry
[
https://issues.apache.org/jira/browse/IMPALA-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-9397:
-
Summary: AuthorizationStmtTest.testSelect failed against Sentry (was:
AuthorizationStmtTest.testSelect failed)
> AuthorizationStmtTest.testSelect failed against Sentry
> --
>
> Key: IMPALA-9397
> URL: https://issues.apache.org/jira/browse/IMPALA-9397
> Project: IMPALA
> Issue Type: Bug
> Components: Security
>Affects Versions: Impala 3.4.0
>Reporter: Attila Jeges
>Assignee: Csaba Ringhofer
>Priority: Blocker
>
> Error Message
> {code}
> Principal: non_owner
> Statement: select id from functional.alltypes_view
> Error: User 'non_owner' does not have privileges to execute 'SELECT' on:
> functional.alltypes_view
> {code}
> Stacktrace
> {code}
> org.apache.impala.authorization.AuthorizationException:
> Principal: non_owner
> Statement: select id from functional.alltypes_view
> Error: User 'non_owner' does not have privileges to execute 'SELECT' on:
> functional.alltypes_view
> at
> org.apache.impala.authorization.AuthorizationTestBase.authzOk(AuthorizationTestBase.java:581)
> at
> org.apache.impala.authorization.AuthorizationTestBase.authzOk(AuthorizationTestBase.java:571)
> at
> org.apache.impala.authorization.AuthorizationTestBase.access$300(AuthorizationTestBase.java:75)
> at
> org.apache.impala.authorization.AuthorizationTestBase$AuthzTest.ok(AuthorizationTestBase.java:414)
> at
> org.apache.impala.authorization.AuthorizationStmtTest.testSelect(AuthorizationStmtTest.java:498)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To
[jira] [Updated] (IMPALA-9397) AuthorizationStmtTest.testSelect failed
[
https://issues.apache.org/jira/browse/IMPALA-9397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-9397:
-
Summary: AuthorizationStmtTest.testSelect failed (was:
AuthorizationStmtTest.testSelect failure)
> AuthorizationStmtTest.testSelect failed
> ---
>
> Key: IMPALA-9397
> URL: https://issues.apache.org/jira/browse/IMPALA-9397
> Project: IMPALA
> Issue Type: Bug
> Components: Security
>Affects Versions: Impala 3.4.0
>Reporter: Attila Jeges
>Priority: Blocker
>
> Error Message
> {code}
> Principal: non_owner
> Statement: select id from functional.alltypes_view
> Error: User 'non_owner' does not have privileges to execute 'SELECT' on:
> functional.alltypes_view
> {code}
> Stacktrace
> {code}
> org.apache.impala.authorization.AuthorizationException:
> Principal: non_owner
> Statement: select id from functional.alltypes_view
> Error: User 'non_owner' does not have privileges to execute 'SELECT' on:
> functional.alltypes_view
> at
> org.apache.impala.authorization.AuthorizationTestBase.authzOk(AuthorizationTestBase.java:581)
> at
> org.apache.impala.authorization.AuthorizationTestBase.authzOk(AuthorizationTestBase.java:571)
> at
> org.apache.impala.authorization.AuthorizationTestBase.access$300(AuthorizationTestBase.java:75)
> at
> org.apache.impala.authorization.AuthorizationTestBase$AuthzTest.ok(AuthorizationTestBase.java:414)
> at
> org.apache.impala.authorization.AuthorizationStmtTest.testSelect(AuthorizationStmtTest.java:498)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands,
[jira] [Created] (IMPALA-9397) AuthorizationStmtTest.testSelect failure
Attila Jeges created IMPALA-9397:
Summary: AuthorizationStmtTest.testSelect failure
Key: IMPALA-9397
URL: https://issues.apache.org/jira/browse/IMPALA-9397
Project: IMPALA
Issue Type: Bug
Components: Security
Affects Versions: Impala 3.4.0
Reporter: Attila Jeges
Error Message
{code}
Principal: non_owner
Statement: select id from functional.alltypes_view
Error: User 'non_owner' does not have privileges to execute 'SELECT' on:
functional.alltypes_view
{code}
Stacktrace
{code}
org.apache.impala.authorization.AuthorizationException:
Principal: non_owner
Statement: select id from functional.alltypes_view
Error: User 'non_owner' does not have privileges to execute 'SELECT' on:
functional.alltypes_view
at
org.apache.impala.authorization.AuthorizationTestBase.authzOk(AuthorizationTestBase.java:581)
at
org.apache.impala.authorization.AuthorizationTestBase.authzOk(AuthorizationTestBase.java:571)
at
org.apache.impala.authorization.AuthorizationTestBase.access$300(AuthorizationTestBase.java:75)
at
org.apache.impala.authorization.AuthorizationTestBase$AuthzTest.ok(AuthorizationTestBase.java:414)
at
org.apache.impala.authorization.AuthorizationStmtTest.testSelect(AuthorizationStmtTest.java:498)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:27)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:272)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:236)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:386)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:323)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:143)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-9395) RuntimeFilter causes impalad crash
[
https://issues.apache.org/jira/browse/IMPALA-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-9395:
-
Description:
impalad crashed while running TestRuntimeFilters::test_basic_filters:
{code}
16:55:53 TestRuntimeFilters.test_basic_filters[protocol: beeswax |
exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': True, 'abort_on_error':
1, 'exec_single_node_rows_threshold': 0} | table_format: seq/none]
16:55:53 query_test/test_runtime_filters.py:55: in test_basic_filters
16:55:53 test_file_vars={'$RUNTIME_FILTER_WAIT_TIME_MS' :
str(WAIT_TIME_MS)})
16:55:53
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/tests/common/impala_test_suite.py:659:
in run_test_case
16:55:53 result = exec_fn(query, user=test_section.get('USER', '').strip()
or None)
16:55:53
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/tests/common/impala_test_suite.py:594:
in __exec_in_impala
16:55:53 result = self.__execute_query(target_impalad_client, queryl
user=user)
16:55:53
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/tests/common/impala_test_suite.py:933:
in __execute_query
16:55:53 return impalad_client.execute(query, user=user)
16:55:53
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/tests/common/impala_connection.py:205:
in execute
16:55:53 return self.__beeswax_client.execute(sql_stmt, user=user)
16:55:53
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/tests/beeswax/impala_beeswax.py:187:
in execute
16:55:53 handle = self._ execute_query(query_string.strip(), user=user)
16:55:53
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/tests/beeswax/impala_beeswax.py:365:
in __execute_query
16:55:53 self.wait_for_finished(handle)
16:55:53
/data/jenkins/workspace/impala-asf-master-exhaustive/repos/Impala/tests/beeswax/impala_beeswax.py:386:
in wait_for_finished
16:55:53 raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
16:55:53 E ImpalaBeeswaxException: ImpalaBeeswaxException:
16:55:53 EQuery aborted:ExecQueryFInstances rpc
query_id=1b4b52ada7e0b713:a6ca09e2 failed: Exec() rpc failed: Network
error: Client connection negotiation failed: client connection to
127.0.0.1:27002: connect: Connection refused (error 111)
{code}
minidump:
{code}
Operating system: Linux
0.0.0 Linux 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20
20:32:50 UTC 2017 x86_64
CPU: amd64
family 6 model 85 stepping 4
1 CPU
GPU: UNKNOWN
Crash reason: SIGABRT
Crash address: 0x7d16057
Process uptime: not available
Thread 122 (crashed)
0 libc-2.17.so + 0x351f7
rax = 0x rdx = 0x0006
rcx = 0x rbx = 0x07208c80
rsi = 0x6366 rdi = 0x6057
rbp = 0x7fb39ecf7da0 rsp = 0x7fb39ecf7a38
r8 = 0xr9 = 0x7fb39ecf78b0
r10 = 0x0008 r11 = 0x0202
r12 = 0x07208d00 r13 = 0x004b
r14 = 0x07210644 r15 = 0x07208c80
rip = 0x7fb40b9501f7
Found by: given as instruction pointer in context
1 libc-2.17.so + 0x368e8
rbp = 0x7fb39ecf7da0 rsp = 0x7fb39ecf7a40
rip = 0x7fb40b9518e8
Found by: stack scanning
2 impalad!google_breakpad::ExceptionHandler::HandleSignal(int, siginfo_t*,
void*) + 0x1e0
rbp = 0x7fb39ecf7da0 rsp = 0x7fb39ecf7ac8
rip = 0x04e866f0
Found by: stack scanning
3 impalad!boost::detail::lexical_converter_impl::try_convert(impala::extdatasource::TColumnDesc
const&, std::string&) [converter_lexical.hpp : 508 + 0x1]
rbp = 0x7fb39ecf7da0 rsp = 0x7fb39ecf7ae0
rip = 0x01f90130
Found by: stack scanning
4 impalad!google::LogMessage::Flush() + 0x157
rbx = 0x rbp = 0x7fb40c5006a0
rsp = 0x7fb39ecf7db0 rip = 0x04e4e837
Found by: call frame info
5 impalad!google::LogMessageFatal::~LogMessageFatal() + 0xe
rbx = 0x7fb39ecf7e70 rbp = 0x7fb39ecf7f80
rsp = 0x7fb39ecf7e30 r12 = 0x0001
r13 = 0x07217b78 r14 = 0x145d27c8
r15 = 0x rip = 0x04e51dfe
Found by: call frame info
6 impalad!impala::RuntimeFilter::SetFilter(impala::BloomFilter*,
impala::MinMaxFilter*) [runtime-filter.cc : 29 + 0x10]
rbx = 0x0001 rbp = 0x7fb39ecf7f80
rsp = 0x7fb39ecf7e50 r12 = 0x0001
r13 = 0x07217b78 r14 = 0x145d27c8
r15 = 0x rip = 0x02207e37
Found by: call frame info
7
impalad!impala::RuntimeFilterBank::PublishGlobalFilter(impala::PublishFilterParamsPB
const&, kudu::rpc::RpcContext*) [runtime-filter-bank.cc : 359 +
[jira] [Updated] (IMPALA-9395) RuntimeFilter causes impalad crash
[
https://issues.apache.org/jira/browse/IMPALA-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-9395:
-
Description:
impalad crashed while running TestRuntimeFilters::test_basic_filters:
{code}
Operating system: Linux
0.0.0 Linux 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20
20:32:50 UTC 2017 x86_64
CPU: amd64
family 6 model 85 stepping 4
1 CPU
GPU: UNKNOWN
Crash reason: SIGABRT
Crash address: 0x7d16057
Process uptime: not available
Thread 122 (crashed)
0 libc-2.17.so + 0x351f7
rax = 0x rdx = 0x0006
rcx = 0x rbx = 0x07208c80
rsi = 0x6366 rdi = 0x6057
rbp = 0x7fb39ecf7da0 rsp = 0x7fb39ecf7a38
r8 = 0xr9 = 0x7fb39ecf78b0
r10 = 0x0008 r11 = 0x0202
r12 = 0x07208d00 r13 = 0x004b
r14 = 0x07210644 r15 = 0x07208c80
rip = 0x7fb40b9501f7
Found by: given as instruction pointer in context
1 libc-2.17.so + 0x368e8
rbp = 0x7fb39ecf7da0 rsp = 0x7fb39ecf7a40
rip = 0x7fb40b9518e8
Found by: stack scanning
2 impalad!google_breakpad::ExceptionHandler::HandleSignal(int, siginfo_t*,
void*) + 0x1e0
rbp = 0x7fb39ecf7da0 rsp = 0x7fb39ecf7ac8
rip = 0x04e866f0
Found by: stack scanning
3 impalad!boost::detail::lexical_converter_impl::try_convert(impala::extdatasource::TColumnDesc
const&, std::string&) [converter_lexical.hpp : 508 + 0x1]
rbp = 0x7fb39ecf7da0 rsp = 0x7fb39ecf7ae0
rip = 0x01f90130
Found by: stack scanning
4 impalad!google::LogMessage::Flush() + 0x157
rbx = 0x rbp = 0x7fb40c5006a0
rsp = 0x7fb39ecf7db0 rip = 0x04e4e837
Found by: call frame info
5 impalad!google::LogMessageFatal::~LogMessageFatal() + 0xe
rbx = 0x7fb39ecf7e70 rbp = 0x7fb39ecf7f80
rsp = 0x7fb39ecf7e30 r12 = 0x0001
r13 = 0x07217b78 r14 = 0x145d27c8
r15 = 0x rip = 0x04e51dfe
Found by: call frame info
6 impalad!impala::RuntimeFilter::SetFilter(impala::BloomFilter*,
impala::MinMaxFilter*) [runtime-filter.cc : 29 + 0x10]
rbx = 0x0001 rbp = 0x7fb39ecf7f80
rsp = 0x7fb39ecf7e50 r12 = 0x0001
r13 = 0x07217b78 r14 = 0x145d27c8
r15 = 0x rip = 0x02207e37
Found by: call frame info
7
impalad!impala::RuntimeFilterBank::PublishGlobalFilter(impala::PublishFilterParamsPB
const&, kudu::rpc::RpcContext*) [runtime-filter-bank.cc : 359 + 0x21]
rbx = 0x02532b2a rbp = 0x7fb39ecf82a0
rsp = 0x7fb39ecf7f90 r12 = 0x
r13 = 0x1737d5a0 r14 = 0x145d27c8
r15 = 0x rip = 0x021c953c
Found by: call frame info
8 impalad!impala::QueryState::PublishFilter(impala::PublishFilterParamsPB
const&, kudu::rpc::RpcContext*) [query-state.cc : 722 + 0x28]
rbx = 0x rbp = 0x7fb39ecf82f0
rsp = 0x7fb39ecf82b0 r12 = 0x
r13 = 0x1737d5a0 r14 = 0x145d27c8
r15 = 0x rip = 0x021b8127
Found by: call frame info
9
impalad!impala::DataStreamService::PublishFilter(impala::PublishFilterParamsPB
const*, impala::PublishFilterResultPB*, kudu::rpc::RpcContext*)
[data-stream-service.cc : 132 + 0x2b]
rbx = 0x rbp = 0x7fb39ecf8490
rsp = 0x7fb39ecf8300 r12 = 0x
r13 = 0x1737d5a0 r14 = 0x145d27c8
r15 = 0x rip = 0x022e4c26
Found by: call frame info
10
impalad!impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr
const&, scoped_refptr
const&)::{lambda(google::protobuf::Message const*, google::protobuf::Message*,
kudu::rpc::RpcContext*)#8}::operator()(google::protobuf::Message const*,
google::protobuf::Message*, kudu::rpc::RpcContext*) const + 0x3e
rbx = 0x129d96c0 rbp = 0x7fb39ecf84c0
rsp = 0x7fb39ecf84a0 r12 = 0x176cf1a0
r13 = 0x1737d5a0 r14 = 0x145d27c8
r15 = 0x rip = 0x023a59ec
Found by: call frame info
11 impalad!std::_Function_handler&, const
scoped_refptr&):: >::_M_invoke [functional : 2039 + 0x20]
rbx = 0x129d96c0 rbp = 0x7fb39ecf8510
rsp = 0x7fb39ecf84d0 r12 = 0x176cf1a0
r13 = 0x1737d5a0 r14 = 0x145d27c8
r15 = 0x rip = 0x023a7975
Found by: call frame info
12 impalad!std::function::operator()(google::protobuf::Message const*,
google::protobuf::Message*, kudu::rpc::RpcContext*) const [functional : 2439 +
0x44]
[jira] [Updated] (IMPALA-9395) RuntimeFilter causes impalad crash
[
https://issues.apache.org/jira/browse/IMPALA-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Jeges updated IMPALA-9395:
-
Labels: broken-build flaky (was: )
> RuntimeFilter causes impalad crash
> --
>
> Key: IMPALA-9395
> URL: https://issues.apache.org/jira/browse/IMPALA-9395
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.4.0
>Reporter: Attila Jeges
>Priority: Blocker
> Labels: broken-build, flaky
>
> impalad crashed while running exhaustive tests.
> {code}
> Operating system: Linux
> 0.0.0 Linux 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20
> 20:32:50 UTC 2017 x86_64
> CPU: amd64
> family 6 model 85 stepping 4
> 1 CPU
> GPU: UNKNOWN
> Crash reason: SIGABRT
> Crash address: 0x7d16057
> Process uptime: not available
> Thread 122 (crashed)
> 0 libc-2.17.so + 0x351f7
> rax = 0x rdx = 0x0006
> rcx = 0x rbx = 0x07208c80
> rsi = 0x6366 rdi = 0x6057
> rbp = 0x7fb39ecf7da0 rsp = 0x7fb39ecf7a38
> r8 = 0xr9 = 0x7fb39ecf78b0
> r10 = 0x0008 r11 = 0x0202
> r12 = 0x07208d00 r13 = 0x004b
> r14 = 0x07210644 r15 = 0x07208c80
> rip = 0x7fb40b9501f7
> Found by: given as instruction pointer in context
> 1 libc-2.17.so + 0x368e8
> rbp = 0x7fb39ecf7da0 rsp = 0x7fb39ecf7a40
> rip = 0x7fb40b9518e8
> Found by: stack scanning
> 2 impalad!google_breakpad::ExceptionHandler::HandleSignal(int, siginfo_t*,
> void*) + 0x1e0
> rbp = 0x7fb39ecf7da0 rsp = 0x7fb39ecf7ac8
> rip = 0x04e866f0
> Found by: stack scanning
> 3 impalad!boost::detail::lexical_converter_impl impala::extdatasource::TColumnDesc>::try_convert(impala::extdatasource::TColumnDesc
> const&, std::string&) [converter_lexical.hpp : 508 + 0x1]
> rbp = 0x7fb39ecf7da0 rsp = 0x7fb39ecf7ae0
> rip = 0x01f90130
> Found by: stack scanning
> 4 impalad!google::LogMessage::Flush() + 0x157
> rbx = 0x rbp = 0x7fb40c5006a0
> rsp = 0x7fb39ecf7db0 rip = 0x04e4e837
> Found by: call frame info
> 5 impalad!google::LogMessageFatal::~LogMessageFatal() + 0xe
> rbx = 0x7fb39ecf7e70 rbp = 0x7fb39ecf7f80
> rsp = 0x7fb39ecf7e30 r12 = 0x0001
> r13 = 0x07217b78 r14 = 0x145d27c8
> r15 = 0x rip = 0x04e51dfe
> Found by: call frame info
> 6 impalad!impala::RuntimeFilter::SetFilter(impala::BloomFilter*,
> impala::MinMaxFilter*) [runtime-filter.cc : 29 + 0x10]
> rbx = 0x0001 rbp = 0x7fb39ecf7f80
> rsp = 0x7fb39ecf7e50 r12 = 0x0001
> r13 = 0x07217b78 r14 = 0x145d27c8
> r15 = 0x rip = 0x02207e37
> Found by: call frame info
> 7
> impalad!impala::RuntimeFilterBank::PublishGlobalFilter(impala::PublishFilterParamsPB
> const&, kudu::rpc::RpcContext*) [runtime-filter-bank.cc : 359 + 0x21]
> rbx = 0x02532b2a rbp = 0x7fb39ecf82a0
> rsp = 0x7fb39ecf7f90 r12 = 0x
> r13 = 0x1737d5a0 r14 = 0x145d27c8
> r15 = 0x rip = 0x021c953c
> Found by: call frame info
> 8 impalad!impala::QueryState::PublishFilter(impala::PublishFilterParamsPB
> const&, kudu::rpc::RpcContext*) [query-state.cc : 722 + 0x28]
> rbx = 0x rbp = 0x7fb39ecf82f0
> rsp = 0x7fb39ecf82b0 r12 = 0x
> r13 = 0x1737d5a0 r14 = 0x145d27c8
> r15 = 0x rip = 0x021b8127
> Found by: call frame info
> 9
> impalad!impala::DataStreamService::PublishFilter(impala::PublishFilterParamsPB
> const*, impala::PublishFilterResultPB*, kudu::rpc::RpcContext*)
> [data-stream-service.cc : 132 + 0x2b]
> rbx = 0x rbp = 0x7fb39ecf8490
> rsp = 0x7fb39ecf8300 r12 = 0x
> r13 = 0x1737d5a0 r14 = 0x145d27c8
> r15 = 0x rip = 0x022e4c26
> Found by: call frame info
> 10
> impalad!impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr
> const&, scoped_refptr
> const&)::{lambda(google::protobuf::Message const*,
> google::protobuf::Message*,
> kudu::rpc::RpcContext*)#8}::operator()(google::protobuf::Message const*,
> google::protobuf::Message*, kudu::rpc::RpcContext*) const + 0x3e
> rbx = 0x129d96c0 rbp = 0x7fb39ecf84c0
> rsp = 0x7fb39ecf84a0 r12 = 0x176cf1a0
> r13 = 0x1737d5a0
