[jira] [Commented] (IMPALA-8738) Add a column representing the type（table or view） in the show tables output
[
https://issues.apache.org/jira/browse/IMPALA-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903509#comment-16903509
]
baotuquan commented on IMPALA-8738:
---
Ok. Use the same command as hive 'SHOW EXTENDED TABLES';
The result has two columns, name and table_type;
Name: is the name of the table or view.
Table_type: [VIRTUAL_VIEW | MANAGED_TABLE | EXTERNL_TABLE ]
> Add a column representing the type（table or view） in the show tables output
> ---
>
> Key: IMPALA-8738
> URL: https://issues.apache.org/jira/browse/IMPALA-8738
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog, Frontend
>Reporter: baotuquan
>Assignee: baotuquan
>Priority: Minor
> Labels: features
>
> Now the output of the +*show tables*+ command in the system is as follows：
> {code:java}
> default> show tables;
> Query: show tables
> +--+
> | name |
> +--+
> | table1 |
> | view1 |
> +--+
> {code}
> I think we should add a column for the representation type，The output should
> be like this
> {code:java}
> default> show tables;
> Query: show tables
> +---+
> | name | type |
> ++
> | table1 | table |
> | view1 | view |
> ++
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8851) Drop table if exists throws authorization exception when table does not exist
Vihang Karajgaonkar created IMPALA-8851:
---
Summary: Drop table if exists throws authorization exception when
table does not exist
Key: IMPALA-8851
URL: https://issues.apache.org/jira/browse/IMPALA-8851
Project: IMPALA
Issue Type: Bug
Reporter: Vihang Karajgaonkar
Assignee: Vihang Karajgaonkar
When authorization is enabled, a {{drop table if exists }} on a
non-existing table throws an authorization exception. In such a case if the
user has required permissions to the query the tables on the database, this is
unnecessary and the SQL should succeed saying the table does not exists instead
of erroring out.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8851) Drop table if exists throws authorization exception when table does not exist
[
https://issues.apache.org/jira/browse/IMPALA-8851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-8851 started by Vihang Karajgaonkar.
---
> Drop table if exists throws authorization exception when table does not exist
> -
>
> Key: IMPALA-8851
> URL: https://issues.apache.org/jira/browse/IMPALA-8851
> Project: IMPALA
> Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
>
> When authorization is enabled, a {{drop table if exists }} on a
> non-existing table throws an authorization exception. In such a case if the
> user has required permissions to the query the tables on the database, this
> is unnecessary and the SQL should succeed saying the table does not exists
> instead of erroring out.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8124) Build failure: webserver/test_web_pages.py
[
https://issues.apache.org/jira/browse/IMPALA-8124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-8124 started by Anurag Mantripragada.
> Build failure: webserver/test_web_pages.py
> --
>
> Key: IMPALA-8124
> URL: https://issues.apache.org/jira/browse/IMPALA-8124
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 3.1.0, Impala 3.2.0
>Reporter: Paul Rogers
>Assignee: Anurag Mantripragada
>Priority: Major
> Labels: build-failure, flaky
> Fix For: Impala 3.3.0
>
>
> Build log output:
> {noformat}
> 04:20:18 === FAILURES
> ===
> 04:20:18 ___ TestWebPage.test_catalog
> ___
> 04:20:18 [gw4] linux2 -- Python 2.7.5
> /data/jenkins/workspace/impala-cdh6.x-core/repos/Impala/bin/../infra/python/env/bin/python
> 04:20:18 webserver/test_web_pages.py:265: in test_catalog
> 04:20:18 self.__test_table_metrics("functional", "alltypes",
> "total-file-size-bytes")
> 04:20:18 webserver/test_web_pages.py:283: in __test_table_metrics
> 04:20:18 "?name=%s.%s" % (db_name, tbl_name), metric,
> ports_to_test=self.CATALOG_TEST_PORT)
> 04:20:18 webserver/test_web_pages.py:144: in get_and_check_status
> 04:20:18 assert response.status_code == requests.codes.ok\
> 04:20:18 E AssertionError: Offending url:
> http://localhost:25020/table_metrics?name=functional.alltypes
> 04:20:18 E assert (200 == 200 and 'total-file-size-bytes' in
>
[jira] [Commented] (IMPALA-8786) BufferedPlanRootSink should directly write to a QueryResultSet if one is available
[
https://issues.apache.org/jira/browse/IMPALA-8786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903421#comment-16903421
]
Sahil Takiar commented on IMPALA-8786:
--
I've been thinking about this some more, as well as a few other short-circuit
optimizations. Leaving some notes here for reference:
* For queries that return less than {{BATCH_SIZE}} rows (e.g. the query has a
limit so we know the max number of rows it returns) we could probably
automatically disable result spooling and just use the current implementation
(e.g. use {{BlockingPlanRootSink}})
** This avoids the overhead of reading / writing to a {{BufferedTupleStream}}
and avoids the need for a memory estimate / reservation
** In order to take advantage of the changes in IMPALA-8845, we might have to
modify {{BlockingPlanRootSink}} so that it blocks in {{FlushFinal}} rather than
{{Send}} (similar to how {{BufferedPlanRootSink}} works in IMPALA-8780)
> BufferedPlanRootSink should directly write to a QueryResultSet if one is
> available
> --
>
> Key: IMPALA-8786
> URL: https://issues.apache.org/jira/browse/IMPALA-8786
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
>
> {{BufferedPlanRootSink}} uses a {{RowBatchQueue}} to buffer {{RowBatch}}-es
> and then the consumer thread reads them and writes them to a given
> {{QueryResultSet}}. Implementations of {{RowBatchQueue}} might end up copying
> the buffered {{RowBatch}}-es (e.g. if the queue is backed by a
> {{BufferedTupleStream}}). An optimization would be for the producer thread to
> directly write to the consumer {{QueryResultSet}}. This optimization would
> only be triggered if (1) the queue is empty, and (2) the consumer thread has
> a {{QueryResultSet}} available for writing.
> This "fast path" is useful in a few different scenarios:
> * If the consumer is faster than at reading rows than the producer is at
> sending them; in this case, the overhead of buffering rows in a
> {{RowBatchQueue}} can be completely avoided
> * For queries that return under 1024 its likely that the consumer will
> produce a {{QueryResultSet}} before the first {{RowBatch}} is returned
> (except perhaps for very trivial queries)
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8124) Build failure: webserver/test_web_pages.py
[
https://issues.apache.org/jira/browse/IMPALA-8124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903416#comment-16903416
]
Anurag Mantripragada commented on IMPALA-8124:
--
Thanks [~Xiaomeng Zhang] for letting us know, I will take a look at it.
> Build failure: webserver/test_web_pages.py
> --
>
> Key: IMPALA-8124
> URL: https://issues.apache.org/jira/browse/IMPALA-8124
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 3.1.0, Impala 3.2.0
>Reporter: Paul Rogers
>Assignee: Anurag Mantripragada
>Priority: Major
> Labels: build-failure, flaky
> Fix For: Impala 3.3.0
>
>
> Build log output:
> {noformat}
> 04:20:18 === FAILURES
> ===
> 04:20:18 ___ TestWebPage.test_catalog
> ___
> 04:20:18 [gw4] linux2 -- Python 2.7.5
> /data/jenkins/workspace/impala-cdh6.x-core/repos/Impala/bin/../infra/python/env/bin/python
> 04:20:18 webserver/test_web_pages.py:265: in test_catalog
> 04:20:18 self.__test_table_metrics("functional", "alltypes",
> "total-file-size-bytes")
> 04:20:18 webserver/test_web_pages.py:283: in __test_table_metrics
> 04:20:18 "?name=%s.%s" % (db_name, tbl_name), metric,
> ports_to_test=self.CATALOG_TEST_PORT)
> 04:20:18 webserver/test_web_pages.py:144: in get_and_check_status
> 04:20:18 assert response.status_code == requests.codes.ok\
> 04:20:18 E AssertionError: Offending url:
> http://localhost:25020/table_metrics?name=functional.alltypes
> 04:20:18 E assert (200 == 200 and 'total-file-size-bytes' in
>
[jira] [Closed] (IMPALA-8334) Impala Doc: Document impala-shell config_file format
[ https://issues.apache.org/jira/browse/IMPALA-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Rodoni closed IMPALA-8334. --- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Impala Doc: Document impala-shell config_file format > > > Key: IMPALA-8334 > URL: https://issues.apache.org/jira/browse/IMPALA-8334 > Project: IMPALA > Issue Type: Sub-task > Components: Docs >Reporter: Alex Rodoni >Assignee: Alex Rodoni >Priority: Major > Fix For: Impala 3.3.0 > > > https://gerrit.cloudera.org/#/c/14034/ -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8637) Implement transaction handling and locking for ACID queries
[ https://issues.apache.org/jira/browse/IMPALA-8637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903365#comment-16903365 ] ASF subversion and git services commented on IMPALA-8637: - Commit 8ca8ca3aad504d30521546ed0ef49e3a8c1b4038 in impala's branch refs/heads/master from Zoltan Borok-Nagy [ https://gitbox.apache.org/repos/asf?p=impala.git;h=8ca8ca3 ] IMPALA-8637: Implement transaction handling and locking for ACID queries Adds background thread to the Frontend that periodically heartbeats the opened transactions and locks. Most of the logic is implemented in the newly added TransactionKeepalive class. TransactionKeepalive keeps track of the creation time of the transactions and locks, and only heartbeat them if they are older then the sleep interval. This way we don't heartbeat short-running queries unnecessarily. Testing: Added an exhaustive test that checks heartbeating. It creates a long running transaction with sleep(). While the transaction is executing the test periodically checks whether there is an OPEN transaction that has sent a heartbeat to HMS. If it found one then the test succeeds. TODOs: * add metrics Change-Id: Iaa37899b24aa114be642bf8772b4e0f882865cfa Reviewed-on: http://gerrit.cloudera.org:8080/13968 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Implement transaction handling and locking for ACID queries > --- > > Key: IMPALA-8637 > URL: https://issues.apache.org/jira/browse/IMPALA-8637 > Project: IMPALA > Issue Type: Improvement >Reporter: Zoltán Borók-Nagy >Assignee: Zoltán Borók-Nagy >Priority: Major > Labels: impala-acid > > * Start a transaction before planning > * lock tables > * heartbeat during execution > * mark committed after execution finishes -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8124) Build failure: webserver/test_web_pages.py
[
https://issues.apache.org/jira/browse/IMPALA-8124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903360#comment-16903360
]
Xiaomeng Zhang commented on IMPALA-8124:
Hit this issue again in
[webserver.test_web_pages.TestWebPage.test_catalog|https://master-02.jenkins.cloudera.com/job/impala-asf-master-core-data-load/712/testReport/junit/webserver.test_web_pages/TestWebPage/test_catalog/]
> Build failure: webserver/test_web_pages.py
> --
>
> Key: IMPALA-8124
> URL: https://issues.apache.org/jira/browse/IMPALA-8124
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Affects Versions: Impala 3.1.0, Impala 3.2.0
>Reporter: Paul Rogers
>Assignee: Anurag Mantripragada
>Priority: Major
> Labels: build-failure, flaky
> Fix For: Impala 3.3.0
>
>
> Build log output:
> {noformat}
> 04:20:18 === FAILURES
> ===
> 04:20:18 ___ TestWebPage.test_catalog
> ___
> 04:20:18 [gw4] linux2 -- Python 2.7.5
> /data/jenkins/workspace/impala-cdh6.x-core/repos/Impala/bin/../infra/python/env/bin/python
> 04:20:18 webserver/test_web_pages.py:265: in test_catalog
> 04:20:18 self.__test_table_metrics("functional", "alltypes",
> "total-file-size-bytes")
> 04:20:18 webserver/test_web_pages.py:283: in __test_table_metrics
> 04:20:18 "?name=%s.%s" % (db_name, tbl_name), metric,
> ports_to_test=self.CATALOG_TEST_PORT)
> 04:20:18 webserver/test_web_pages.py:144: in get_and_check_status
> 04:20:18 assert response.status_code == requests.codes.ok\
> 04:20:18 E AssertionError: Offending url:
> http://localhost:25020/table_metrics?name=functional.alltypes
> 04:20:18 E assert (200 == 200 and 'total-file-size-bytes' in
>
[jira] [Commented] (IMPALA-8849) IllegalStateException in HashJoinNode because of missing memory estimate
[
https://issues.apache.org/jira/browse/IMPALA-8849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903356#comment-16903356
]
Tim Armstrong commented on IMPALA-8849:
---
Reverting this commit fixed the issue for me
commit c2516d220da8e532b6ebdb6f3a12e7ad97c4f597
Author: Csaba Ringhofer
Date: Wed Apr 17 15:01:07 2019 +0200
IMPALA-8409: Fix row-size for STRING columns with unknown stats
Explain returned row-size=11B for STRING columns without statistics.
The issue was caused by adding -1 (meaning unknown) to the 12 byte
slot size (sizeof(StringValue)). The code in TupleDescriptor.java
tried to handle this by checking if the size is -1, but it was
already 11 at this point.
There is more potential for cleanup, but I wanted to keep this
change minimal.
Testing:
- revived some tests in CatalogTest.java that were removed
in 2013 due to flakiness
- added an EE test that checks row size with and without stats
- fixed a similar test, test_explain_validate_cardinality_estimates
(the format of the line it looks for has changed, which lead to
skipping the actual verification and accepting everything)
- ran core FE and EE tests
Change-Id: I866acf10b2c011a735dee019f4bc29358f2ec4e5
Reviewed-on: http://gerrit.cloudera.org:8080/13190
Reviewed-by: Impala Public Jenkins
Tested-by: Impala Public Jenkins
> IllegalStateException in HashJoinNode because of missing memory estimate
>
>
> Key: IMPALA-8849
> URL: https://issues.apache.org/jira/browse/IMPALA-8849
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Blocker
> Attachments: query4.sql
>
>
> [~grahn] reports the below error when running the attached TPC-DS query on
> uncompressed text with stats.
> {noformat}
> I0808 03:27:16.621085 150090 jni-util.cc:288]
> 8945e9f76f33dca0:8bf31c47] java.lang.IllegalStateException: Mem
> estimate must be set
> at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.impala.planner.ResourceProfileBuilder.build(ResourceProfileBuilder.java:73)
> at
> org.apache.impala.planner.HashJoinNode.computeNodeResourceProfile(HashJoinNode.java:252)
> at
> org.apache.impala.planner.PlanFragment.computeResourceProfile(PlanFragment.java:238)
> at org.apache.impala.planner.Planner.computeResourceReqs(Planner.java:416)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1162)
> at
> org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:1477)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1341)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1236)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1206)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:154)
> I0808 03:27:16.621861 150090 status.cc:124]
> 8945e9f76f33dca0:8bf31c47] IllegalStateException: Mem estimate must
> be set
> @ 0xb9af99
> @ 0x11f785e
> @ 0x10ab9a3
> @ 0x10d0f54
> @ 0x10e2a2c
> @ 0x1125f8d
> @ 0x1458c54
> @ 0x145810c
> @ 0xb68769
> @ 0xf91ca0
> @ 0xf8791e
> @ 0xf887b1
> @ 0x127cd2f
> @ 0x127d8d9
> @ 0x1ac0709
> @ 0x7f11ea4d7dd4
> @ 0x7f11e6f1e02c
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8849) IllegalStateException in HashJoinNode because of missing memory estimate
[
https://issues.apache.org/jira/browse/IMPALA-8849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903351#comment-16903351
]
Tim Armstrong commented on IMPALA-8849:
---
So it looks like it only happens when stats are computed and the column is a
VARCHAR.
{noformat}
[localhost:21000] tpcds> use tpcds_1_text;
Query: use tpcds_1_text
[localhost:21000] tpcds_1_text> explain select c_customer_id from
tpcds_1_text.customer;
Query: explain select c_customer_id from tpcds_1_text.customer
++
| Explain String |
++
| Max Per-Host Resource Reservation: Memory=8.00MB Threads=3 |
| Per-Host Resource Estimates: Memory=48MB |
||
| PLAN-ROOT SINK |
| | |
| 01:EXCHANGE [UNPARTITIONED]|
| | |
| 00:SCAN HDFS [tpcds_1_text.customer] |
|HDFS partitions=1/1 files=1 size=12.50MB|
|row-size=-1B cardinality=100.00K|
++
{noformat}
Here's a repro in my dev env:
{noformat}
[localhost:21000] tpcds> create external table repro_cust(c_customer_sk INT,
c_customer_id VARCHAR(16)) row format delimited fields terminated by '|' WITH
SERDEPROPERTIES ('field.delim'='|', 'serialization.format'='|') STORED AS
TEXTFILE LOCATION 'hdfs://localhost:20500/test-warehouse/tpcds.customer';
Query: create external table repro_cust(c_customer_sk INT, c_customer_id
VARCHAR(16)) row format delimited fields terminated by '|' WITH SERDEPROPERTIES
('field.delim'='|', 'serialization.format'='|') STORED AS TEXTFILE LOCATION
'hdfs://localhost:20500/test-warehouse/tpcds.customer'
+-+
| summary |
+-+
| Table has been created. |
+-+
Fetched 1 row(s) in 0.39s
[localhost:21000] tpcds> compute stats repro_cust;
Query: compute stats repro_cust
+-+
| summary |
+-+
| Updated 1 partition(s) and 2 column(s). |
+-+
Fetched 1 row(s) in 3.62s
[localhost:21000] tpcds> explain select c_customer_id from repro_cust;
Query: explain select c_customer_id from repro_cust
++
| Explain String |
++
| Max Per-Host Resource Reservation: Memory=8.00MB Threads=3 |
| Per-Host Resource Estimates: Memory=48MB |
||
| PLAN-ROOT SINK |
| | |
| 01:EXCHANGE [UNPARTITIONED]|
| | |
| 00:SCAN HDFS [tpcds.repro_cust]|
|HDFS partitions=1/1 files=1 size=12.60MB|
|row-size=-1B cardinality=100.00K|
++
Fetched 10 row(s) in 0.01s
{noformat}
> IllegalStateException in HashJoinNode because of missing memory estimate
>
>
> Key: IMPALA-8849
> URL: https://issues.apache.org/jira/browse/IMPALA-8849
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Blocker
> Attachments: query4.sql
>
>
> [~grahn] reports the below error when running the attached TPC-DS query on
> uncompressed text with stats.
> {noformat}
> I0808 03:27:16.621085 150090 jni-util.cc:288]
> 8945e9f76f33dca0:8bf31c47] java.lang.IllegalStateException: Mem
> estimate must be set
> at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.impala.planner.ResourceProfileBuilder.build(ResourceProfileBuilder.java:73)
> at
> org.apache.impala.planner.HashJoinNode.computeNodeResourceProfile(HashJoinNode.java:252)
> at
> org.apache.impala.planner.PlanFragment.computeResourceProfile(PlanFragment.java:238)
> at org.apache.impala.planner.Planner.computeResourceReqs(Planner.java:416)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1162)
> at
[jira] [Commented] (IMPALA-8849) IllegalStateException in HashJoinNode because of missing memory estimate
[
https://issues.apache.org/jira/browse/IMPALA-8849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903342#comment-16903342
]
Tim Armstrong commented on IMPALA-8849:
---
{noformat}
[nightly6x-unsecure-1.nightly6x-unsecure.root.hwx.site:21000] default> describe
tpcds.customer;
Query: describe tpcds.customer
++-+-+
| name | type| comment |
++-+-+
| c_customer_sk | int | |
| c_customer_id | varchar(16) | |
| c_current_cdemo_sk | int | |
| c_current_hdemo_sk | int | |
| c_current_addr_sk | int | |
| c_first_shipto_date_sk | int | |
| c_first_sales_date_sk | int | |
| c_salutation | varchar(10) | |
| c_first_name | varchar(20) | |
| c_last_name| varchar(30) | |
| c_preferred_cust_flag | varchar(1) | |
| c_birth_day| int | |
| c_birth_month | int | |
| c_birth_year | int | |
| c_birth_country| varchar(20) | |
| c_login| varchar(13) | |
| c_email_address| varchar(50) | |
| c_last_review_date_sk | int | |
++-+-+
Fetched 18 row(s) in 0.01s
[nightly6x-unsecure-1.nightly6x-unsecure.root.hwx.site:21000] default> explain
select c_customer_id from tpcds.customer;
Query: explain select c_customer_id from tpcds.customer
++
| Explain String |
++
| Max Per-Host Resource Reservation: Memory=8.00MB Threads=3 |
| Per-Host Resource Estimates: Memory=48MB |
||
| PLAN-ROOT SINK |
| | |
| 01:EXCHANGE [UNPARTITIONED]|
| | |
| 00:SCAN HDFS [tpcds.customer] |
|HDFS partitions=1/1 files=1 size=12.50MB|
|row-size=-1B cardinality=100.00K|
++
Fetched 10 row(s) in 0.02s
[nightly6x-unsecure-1.nightly6x-unsecure.root.hwx.site:21000] default> show
column stats tpcds.customer;
Query: show column stats tpcds.customer
++-+--++--+---+
| Column | Type| #Distinct Values | #Nulls | Max Size |
Avg Size |
++-+--++--+---+
| c_customer_sk | INT | 10 | 0 | 4|
4 |
| c_customer_id | VARCHAR(16) | 10 | 0 | 16 |
16|
| c_current_cdemo_sk | INT | 91558| 3438 | 4|
4 |
| c_current_hdemo_sk | INT | 7376 | 3431 | 4|
4 |
| c_current_addr_sk | INT | 42003| 0 | 4|
4 |
| c_first_shipto_date_sk | INT | 3754 | 3443 | 4|
4 |
| c_first_sales_date_sk | INT | 3734 | 3518 | 4|
4 |
| c_salutation | VARCHAR(10) | 6| 3410 | 4|
3.24143433228 |
| c_first_name | VARCHAR(20) | 4013 | 3492 | 11 |
5.839499950408936 |
| c_last_name| VARCHAR(30) | 4951 | 3497 | 13 |
6.124800205230713 |
| c_preferred_cust_flag | VARCHAR(1) | 2| 3426 | 1|
1 |
| c_birth_day| INT | 31 | 3461 | 4|
4 |
| c_birth_month | INT | 12 | 3449 | 4|
4 |
| c_birth_year | INT | 67 | 3453 | 4|
4 |
| c_birth_country| VARCHAR(20) | 207 | 3439 | 20 |
8.708800315856934 |
| c_login| VARCHAR(13) | 0| 10 | 0|
0 |
| c_email_address| VARCHAR(50) | 10 | 3521 | 46 |
27.45219993591309 |
| c_last_review_date_sk | INT | 350 | 3484 | 4|
4 |
++-+--++--+---+
{noformat}
[jira] [Work started] (IMPALA-8848) Cardinality of UnionNode does not handle missing input cardinality correctly
[
https://issues.apache.org/jira/browse/IMPALA-8848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-8848 started by Tim Armstrong.
-
> Cardinality of UnionNode does not handle missing input cardinality correctly
>
>
> Key: IMPALA-8848
> URL: https://issues.apache.org/jira/browse/IMPALA-8848
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Critical
> Labels: planner
> Attachments: profile_4d48b2a3bb0236f1_bfa8157d
>
>
> {noformat}
> | 35:UNION
> | | mem-estimate=0B mem-reservation=0B thread-reservation=0
> | | tuple-ids=82 row-size=28B cardinality=0
> | | in pipelines: 75(GETNEXT)
> | |
> | 75:AGGREGATE [FINALIZE]
> | | output: sum:mergews_ext_list_price - ws_ext_wholesale_cost -
> ws_ext_discount_amt) + ws_ext_sales_price) / 2))
> | | group by: c_customer_id, c_first_name, c_last_name,
> c_preferred_cust_flag, c_birth_country, c_login, c_email_address, d_year
> | | mem-estimate=128.00MB mem-reservation=34.00MB spill-buffer=2.00MB
> thread-reservation=0
> | | tuple-ids=81 row-size=104B cardinality=unavailable
> | | in pipelines: 75(GETNEXT), 36(OPEN)
> {noformat}
> I expect that the cardinality should be unavailable, not 0.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8847) Add partition events may contain empty partition object list
[
https://issues.apache.org/jira/browse/IMPALA-8847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-8847 started by Vihang Karajgaonkar.
---
> Add partition events may contain empty partition object list
>
>
> Key: IMPALA-8847
> URL: https://issues.apache.org/jira/browse/IMPALA-8847
> Project: IMPALA
> Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
>
> When event polling is ON and when an external application like Hive issues a
> {{alter table add if not exists partition ()}} it is
> possble that command did not add a partition since it is preexisting.
> However, metastore still generates a ADD_PARTITION event in such a case with
> empty list of added partitions. Such events cause a Precondition to fail
> while processing on the EventsProcessor side and event polling goes into
> error state.
> The fix would be simple. Ignore such events.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8849) IllegalStateException in HashJoinNode because of missing memory estimate
[
https://issues.apache.org/jira/browse/IMPALA-8849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903290#comment-16903290
]
Tim Armstrong commented on IMPALA-8849:
---
Looks like HdfsScanNode can produce a negative row size estimate, which would
explain the negative memory estimate.
{noformat}
I0808 20:25:37.342036 110162 DistributedPlanner.java:498]
114bef3456667eb2:458377fe] 36:SCAN HDFS [tpcds.customer
, RANDOM]
HDFS partitions=1/1 files=1 size=12.50MB
stored statistics:
table: rows=100.00K size=12.50MB
columns: all
extrapolated-rows=disabled max-scan-range-rows=100.00K
mem-estimate=invalid mem-reservation=invalid thread-reservation=invalid
tuple-ids=78 row-size=-3B cardinality=100.00K
in pipelines:
{noformat}
> IllegalStateException in HashJoinNode because of missing memory estimate
>
>
> Key: IMPALA-8849
> URL: https://issues.apache.org/jira/browse/IMPALA-8849
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Blocker
> Attachments: query4.sql
>
>
> [~grahn] reports the below error when running the attached TPC-DS query on
> uncompressed text with stats.
> {noformat}
> I0808 03:27:16.621085 150090 jni-util.cc:288]
> 8945e9f76f33dca0:8bf31c47] java.lang.IllegalStateException: Mem
> estimate must be set
> at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.impala.planner.ResourceProfileBuilder.build(ResourceProfileBuilder.java:73)
> at
> org.apache.impala.planner.HashJoinNode.computeNodeResourceProfile(HashJoinNode.java:252)
> at
> org.apache.impala.planner.PlanFragment.computeResourceProfile(PlanFragment.java:238)
> at org.apache.impala.planner.Planner.computeResourceReqs(Planner.java:416)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1162)
> at
> org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:1477)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1341)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1236)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1206)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:154)
> I0808 03:27:16.621861 150090 status.cc:124]
> 8945e9f76f33dca0:8bf31c47] IllegalStateException: Mem estimate must
> be set
> @ 0xb9af99
> @ 0x11f785e
> @ 0x10ab9a3
> @ 0x10d0f54
> @ 0x10e2a2c
> @ 0x1125f8d
> @ 0x1458c54
> @ 0x145810c
> @ 0xb68769
> @ 0xf91ca0
> @ 0xf8791e
> @ 0xf887b1
> @ 0x127cd2f
> @ 0x127d8d9
> @ 0x1ac0709
> @ 0x7f11ea4d7dd4
> @ 0x7f11e6f1e02c
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8849) IllegalStateException in HashJoinNode because of missing memory estimate
[
https://issues.apache.org/jira/browse/IMPALA-8849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-8849:
--
Summary: IllegalStateException in HashJoinNode because of missing memory
estimate (was: IllegalStateException in HashJoinNode because of missing memory
estimtae)
> IllegalStateException in HashJoinNode because of missing memory estimate
>
>
> Key: IMPALA-8849
> URL: https://issues.apache.org/jira/browse/IMPALA-8849
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Blocker
> Attachments: query4.sql
>
>
> [~grahn] reports the below error when running the attached TPC-DS query on
> uncompressed text with stats.
> {noformat}
> I0808 03:27:16.621085 150090 jni-util.cc:288]
> 8945e9f76f33dca0:8bf31c47] java.lang.IllegalStateException: Mem
> estimate must be set
> at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.impala.planner.ResourceProfileBuilder.build(ResourceProfileBuilder.java:73)
> at
> org.apache.impala.planner.HashJoinNode.computeNodeResourceProfile(HashJoinNode.java:252)
> at
> org.apache.impala.planner.PlanFragment.computeResourceProfile(PlanFragment.java:238)
> at org.apache.impala.planner.Planner.computeResourceReqs(Planner.java:416)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1162)
> at
> org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:1477)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1341)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1236)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1206)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:154)
> I0808 03:27:16.621861 150090 status.cc:124]
> 8945e9f76f33dca0:8bf31c47] IllegalStateException: Mem estimate must
> be set
> @ 0xb9af99
> @ 0x11f785e
> @ 0x10ab9a3
> @ 0x10d0f54
> @ 0x10e2a2c
> @ 0x1125f8d
> @ 0x1458c54
> @ 0x145810c
> @ 0xb68769
> @ 0xf91ca0
> @ 0xf8791e
> @ 0xf887b1
> @ 0x127cd2f
> @ 0x127d8d9
> @ 0x1ac0709
> @ 0x7f11ea4d7dd4
> @ 0x7f11e6f1e02c
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8849) IllegalStateException in HashJoinNode because of missing memory estimtae
[
https://issues.apache.org/jira/browse/IMPALA-8849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-8849:
--
Summary: IllegalStateException in HashJoinNode because of missing memory
estimtae (was: IllegalStateException in HashJoinNode because of missing memory
limit)
> IllegalStateException in HashJoinNode because of missing memory estimtae
>
>
> Key: IMPALA-8849
> URL: https://issues.apache.org/jira/browse/IMPALA-8849
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Blocker
> Attachments: query4.sql
>
>
> [~grahn] reports the below error when running the attached TPC-DS query on
> uncompressed text with stats.
> {noformat}
> I0808 03:27:16.621085 150090 jni-util.cc:288]
> 8945e9f76f33dca0:8bf31c47] java.lang.IllegalStateException: Mem
> estimate must be set
> at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.impala.planner.ResourceProfileBuilder.build(ResourceProfileBuilder.java:73)
> at
> org.apache.impala.planner.HashJoinNode.computeNodeResourceProfile(HashJoinNode.java:252)
> at
> org.apache.impala.planner.PlanFragment.computeResourceProfile(PlanFragment.java:238)
> at org.apache.impala.planner.Planner.computeResourceReqs(Planner.java:416)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1162)
> at
> org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:1477)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1341)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1236)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1206)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:154)
> I0808 03:27:16.621861 150090 status.cc:124]
> 8945e9f76f33dca0:8bf31c47] IllegalStateException: Mem estimate must
> be set
> @ 0xb9af99
> @ 0x11f785e
> @ 0x10ab9a3
> @ 0x10d0f54
> @ 0x10e2a2c
> @ 0x1125f8d
> @ 0x1458c54
> @ 0x145810c
> @ 0xb68769
> @ 0xf91ca0
> @ 0xf8791e
> @ 0xf887b1
> @ 0x127cd2f
> @ 0x127d8d9
> @ 0x1ac0709
> @ 0x7f11ea4d7dd4
> @ 0x7f11e6f1e02c
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-8849) IllegalStateException in HashJoinNode because of missing memory limit
[
https://issues.apache.org/jira/browse/IMPALA-8849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903232#comment-16903232
]
Greg Rahn edited comment on IMPALA-8849 at 8/8/19 7:05 PM:
---
IIRC it reproed with or without stats
First was discovered using:
{noformat}
cd /tmp
curl -ROL https://www.dropbox.com/s/bd6jm1nyis2onwv/sf1.zip
unzip sf1.zip
hdfs dfs -put sf1 /tmp
curl -ROL
https://gist.githubusercontent.com/gregrahn/9c731947643be2817fc5048135a1420f/raw/25d908e4d7452f4c92a51b7cf0ea78880cdc9a23/tpcds-impala-date.ddl
impala-shell -f tpcds-impala-date.ddl
curl -ROL
https://raw.githubusercontent.com/cloudera/impala-tpcds-kit/master/queries/query4.sql
impala-shell -d tpcds -f query4.sql
{noformat}
was (Author: grahn):
IIRC it reproed with or without stats
First was discovered using:
{{noformat}}
cd /tmp
curl -ROL https://www.dropbox.com/s/bd6jm1nyis2onwv/sf1.zip
unzip sf1.zip
hdfs dfs -put sf1 /tmp
curl -ROL
https://gist.githubusercontent.com/gregrahn/9c731947643be2817fc5048135a1420f/raw/25d908e4d7452f4c92a51b7cf0ea78880cdc9a23/tpcds-impala-date.ddl
impala-shell -f tpcds-impala-date.ddl
curl -ROL
https://raw.githubusercontent.com/cloudera/impala-tpcds-kit/master/queries/query4.sql
impala-shell -d tpcds -f query4.sql
{{noformat}}
> IllegalStateException in HashJoinNode because of missing memory limit
> -
>
> Key: IMPALA-8849
> URL: https://issues.apache.org/jira/browse/IMPALA-8849
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Blocker
> Attachments: query4.sql
>
>
> [~grahn] reports the below error when running the attached TPC-DS query on
> uncompressed text with stats.
> {noformat}
> I0808 03:27:16.621085 150090 jni-util.cc:288]
> 8945e9f76f33dca0:8bf31c47] java.lang.IllegalStateException: Mem
> estimate must be set
> at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.impala.planner.ResourceProfileBuilder.build(ResourceProfileBuilder.java:73)
> at
> org.apache.impala.planner.HashJoinNode.computeNodeResourceProfile(HashJoinNode.java:252)
> at
> org.apache.impala.planner.PlanFragment.computeResourceProfile(PlanFragment.java:238)
> at org.apache.impala.planner.Planner.computeResourceReqs(Planner.java:416)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1162)
> at
> org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:1477)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1341)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1236)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1206)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:154)
> I0808 03:27:16.621861 150090 status.cc:124]
> 8945e9f76f33dca0:8bf31c47] IllegalStateException: Mem estimate must
> be set
> @ 0xb9af99
> @ 0x11f785e
> @ 0x10ab9a3
> @ 0x10d0f54
> @ 0x10e2a2c
> @ 0x1125f8d
> @ 0x1458c54
> @ 0x145810c
> @ 0xb68769
> @ 0xf91ca0
> @ 0xf8791e
> @ 0xf887b1
> @ 0x127cd2f
> @ 0x127d8d9
> @ 0x1ac0709
> @ 0x7f11ea4d7dd4
> @ 0x7f11e6f1e02c
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-8849) IllegalStateException in HashJoinNode because of missing memory limit
[
https://issues.apache.org/jira/browse/IMPALA-8849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903232#comment-16903232
]
Greg Rahn edited comment on IMPALA-8849 at 8/8/19 7:04 PM:
---
IIRC it reproed with or without stats
First was discovered using:
{{noformat}}
cd /tmp
curl -ROL https://www.dropbox.com/s/bd6jm1nyis2onwv/sf1.zip
unzip sf1.zip
hdfs dfs -put sf1 /tmp
curl -ROL
https://gist.githubusercontent.com/gregrahn/9c731947643be2817fc5048135a1420f/raw/25d908e4d7452f4c92a51b7cf0ea78880cdc9a23/tpcds-impala-date.ddl
impala-shell -f tpcds-impala-date.ddl
curl -ROL
https://raw.githubusercontent.com/cloudera/impala-tpcds-kit/master/queries/query4.sql
impala-shell -d tpcds -f query4.sql
{{noformat}}
was (Author: grahn):
IIRC it reproed with or without stats
> IllegalStateException in HashJoinNode because of missing memory limit
> -
>
> Key: IMPALA-8849
> URL: https://issues.apache.org/jira/browse/IMPALA-8849
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Blocker
> Attachments: query4.sql
>
>
> [~grahn] reports the below error when running the attached TPC-DS query on
> uncompressed text with stats.
> {noformat}
> I0808 03:27:16.621085 150090 jni-util.cc:288]
> 8945e9f76f33dca0:8bf31c47] java.lang.IllegalStateException: Mem
> estimate must be set
> at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.impala.planner.ResourceProfileBuilder.build(ResourceProfileBuilder.java:73)
> at
> org.apache.impala.planner.HashJoinNode.computeNodeResourceProfile(HashJoinNode.java:252)
> at
> org.apache.impala.planner.PlanFragment.computeResourceProfile(PlanFragment.java:238)
> at org.apache.impala.planner.Planner.computeResourceReqs(Planner.java:416)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1162)
> at
> org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:1477)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1341)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1236)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1206)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:154)
> I0808 03:27:16.621861 150090 status.cc:124]
> 8945e9f76f33dca0:8bf31c47] IllegalStateException: Mem estimate must
> be set
> @ 0xb9af99
> @ 0x11f785e
> @ 0x10ab9a3
> @ 0x10d0f54
> @ 0x10e2a2c
> @ 0x1125f8d
> @ 0x1458c54
> @ 0x145810c
> @ 0xb68769
> @ 0xf91ca0
> @ 0xf8791e
> @ 0xf887b1
> @ 0x127cd2f
> @ 0x127d8d9
> @ 0x1ac0709
> @ 0x7f11ea4d7dd4
> @ 0x7f11e6f1e02c
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8849) IllegalStateException in HashJoinNode because of missing memory limit
[
https://issues.apache.org/jira/browse/IMPALA-8849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903232#comment-16903232
]
Greg Rahn commented on IMPALA-8849:
---
IIRC it reproed with or without stats
> IllegalStateException in HashJoinNode because of missing memory limit
> -
>
> Key: IMPALA-8849
> URL: https://issues.apache.org/jira/browse/IMPALA-8849
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Blocker
> Attachments: query4.sql
>
>
> [~grahn] reports the below error when running the attached TPC-DS query on
> uncompressed text with stats.
> {noformat}
> I0808 03:27:16.621085 150090 jni-util.cc:288]
> 8945e9f76f33dca0:8bf31c47] java.lang.IllegalStateException: Mem
> estimate must be set
> at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.impala.planner.ResourceProfileBuilder.build(ResourceProfileBuilder.java:73)
> at
> org.apache.impala.planner.HashJoinNode.computeNodeResourceProfile(HashJoinNode.java:252)
> at
> org.apache.impala.planner.PlanFragment.computeResourceProfile(PlanFragment.java:238)
> at org.apache.impala.planner.Planner.computeResourceReqs(Planner.java:416)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1162)
> at
> org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:1477)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1341)
> at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1236)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1206)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:154)
> I0808 03:27:16.621861 150090 status.cc:124]
> 8945e9f76f33dca0:8bf31c47] IllegalStateException: Mem estimate must
> be set
> @ 0xb9af99
> @ 0x11f785e
> @ 0x10ab9a3
> @ 0x10d0f54
> @ 0x10e2a2c
> @ 0x1125f8d
> @ 0x1458c54
> @ 0x145810c
> @ 0xb68769
> @ 0xf91ca0
> @ 0xf8791e
> @ 0xf887b1
> @ 0x127cd2f
> @ 0x127d8d9
> @ 0x1ac0709
> @ 0x7f11ea4d7dd4
> @ 0x7f11e6f1e02c
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8849) IllegalStateException in HashJoinNode because of missing memory limit
[
https://issues.apache.org/jira/browse/IMPALA-8849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Rahn updated IMPALA-8849:
--
Description:
[~grahn] reports the below error when running the attached TPC-DS query on
uncompressed text with stats.
{noformat}
I0808 03:27:16.621085 150090 jni-util.cc:288]
8945e9f76f33dca0:8bf31c47] java.lang.IllegalStateException: Mem
estimate must be set
at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
at
org.apache.impala.planner.ResourceProfileBuilder.build(ResourceProfileBuilder.java:73)
at
org.apache.impala.planner.HashJoinNode.computeNodeResourceProfile(HashJoinNode.java:252)
at
org.apache.impala.planner.PlanFragment.computeResourceProfile(PlanFragment.java:238)
at org.apache.impala.planner.Planner.computeResourceReqs(Planner.java:416)
at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1162)
at
org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:1477)
at
org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1341)
at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1236)
at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1206)
at
org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:154)
I0808 03:27:16.621861 150090 status.cc:124] 8945e9f76f33dca0:8bf31c47]
IllegalStateException: Mem estimate must be set
@ 0xb9af99
@ 0x11f785e
@ 0x10ab9a3
@ 0x10d0f54
@ 0x10e2a2c
@ 0x1125f8d
@ 0x1458c54
@ 0x145810c
@ 0xb68769
@ 0xf91ca0
@ 0xf8791e
@ 0xf887b1
@ 0x127cd2f
@ 0x127d8d9
@ 0x1ac0709
@ 0x7f11ea4d7dd4
@ 0x7f11e6f1e02c
{noformat}
was:
[~grahn] reports the below error when running the attached TPC-DS query on
uncompressed text with no stats.
{noformat}
I0808 03:27:16.621085 150090 jni-util.cc:288]
8945e9f76f33dca0:8bf31c47] java.lang.IllegalStateException: Mem
estimate must be set
at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
at
org.apache.impala.planner.ResourceProfileBuilder.build(ResourceProfileBuilder.java:73)
at
org.apache.impala.planner.HashJoinNode.computeNodeResourceProfile(HashJoinNode.java:252)
at
org.apache.impala.planner.PlanFragment.computeResourceProfile(PlanFragment.java:238)
at org.apache.impala.planner.Planner.computeResourceReqs(Planner.java:416)
at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1162)
at
org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:1477)
at
org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1341)
at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1236)
at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1206)
at
org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:154)
I0808 03:27:16.621861 150090 status.cc:124] 8945e9f76f33dca0:8bf31c47]
IllegalStateException: Mem estimate must be set
@ 0xb9af99
@ 0x11f785e
@ 0x10ab9a3
@ 0x10d0f54
@ 0x10e2a2c
@ 0x1125f8d
@ 0x1458c54
@ 0x145810c
@ 0xb68769
@ 0xf91ca0
@ 0xf8791e
@ 0xf887b1
@ 0x127cd2f
@ 0x127d8d9
@ 0x1ac0709
@ 0x7f11ea4d7dd4
@ 0x7f11e6f1e02c
{noformat}
> IllegalStateException in HashJoinNode because of missing memory limit
> -
>
> Key: IMPALA-8849
> URL: https://issues.apache.org/jira/browse/IMPALA-8849
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Tim Armstrong
>Priority: Blocker
> Attachments: query4.sql
>
>
> [~grahn] reports the below error when running the attached TPC-DS query on
> uncompressed text with stats.
> {noformat}
> I0808 03:27:16.621085 150090 jni-util.cc:288]
> 8945e9f76f33dca0:8bf31c47] java.lang.IllegalStateException: Mem
> estimate must be set
> at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
> at
> org.apache.impala.planner.ResourceProfileBuilder.build(ResourceProfileBuilder.java:73)
> at
> org.apache.impala.planner.HashJoinNode.computeNodeResourceProfile(HashJoinNode.java:252)
> at
> org.apache.impala.planner.PlanFragment.computeResourceProfile(PlanFragment.java:238)
> at org.apache.impala.planner.Planner.computeResourceReqs(Planner.java:416)
> at
[jira] [Created] (IMPALA-8849) IllegalStateException in HashJoinNode because of missing memory limit
Tim Armstrong created IMPALA-8849:
-
Summary: IllegalStateException in HashJoinNode because of missing
memory limit
Key: IMPALA-8849
URL: https://issues.apache.org/jira/browse/IMPALA-8849
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 3.3.0
Reporter: Tim Armstrong
Assignee: Tim Armstrong
Attachments: query4.sql
[~grahn] reports the below error when running the attached TPC-DS query on
uncompressed text with no stats.
{noformat}
I0808 03:27:16.621085 150090 jni-util.cc:288]
8945e9f76f33dca0:8bf31c47] java.lang.IllegalStateException: Mem
estimate must be set
at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
at
org.apache.impala.planner.ResourceProfileBuilder.build(ResourceProfileBuilder.java:73)
at
org.apache.impala.planner.HashJoinNode.computeNodeResourceProfile(HashJoinNode.java:252)
at
org.apache.impala.planner.PlanFragment.computeResourceProfile(PlanFragment.java:238)
at org.apache.impala.planner.Planner.computeResourceReqs(Planner.java:416)
at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1162)
at
org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:1477)
at
org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1341)
at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1236)
at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1206)
at
org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:154)
I0808 03:27:16.621861 150090 status.cc:124] 8945e9f76f33dca0:8bf31c47]
IllegalStateException: Mem estimate must be set
@ 0xb9af99
@ 0x11f785e
@ 0x10ab9a3
@ 0x10d0f54
@ 0x10e2a2c
@ 0x1125f8d
@ 0x1458c54
@ 0x145810c
@ 0xb68769
@ 0xf91ca0
@ 0xf8791e
@ 0xf887b1
@ 0x127cd2f
@ 0x127d8d9
@ 0x1ac0709
@ 0x7f11ea4d7dd4
@ 0x7f11e6f1e02c
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8848) Cardinality of UnionNode does not handle missing input cardinality correctly
Tim Armstrong created IMPALA-8848:
-
Summary: Cardinality of UnionNode does not handle missing input
cardinality correctly
Key: IMPALA-8848
URL: https://issues.apache.org/jira/browse/IMPALA-8848
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 3.3.0
Reporter: Tim Armstrong
Assignee: Tim Armstrong
Attachments: profile_4d48b2a3bb0236f1_bfa8157d
{noformat}
| 35:UNION
| | mem-estimate=0B mem-reservation=0B thread-reservation=0
| | tuple-ids=82 row-size=28B cardinality=0
| | in pipelines: 75(GETNEXT)
| |
| 75:AGGREGATE [FINALIZE]
| | output: sum:mergews_ext_list_price - ws_ext_wholesale_cost -
ws_ext_discount_amt) + ws_ext_sales_price) / 2))
| | group by: c_customer_id, c_first_name, c_last_name,
c_preferred_cust_flag, c_birth_country, c_login, c_email_address, d_year
| | mem-estimate=128.00MB mem-reservation=34.00MB spill-buffer=2.00MB
thread-reservation=0
| | tuple-ids=81 row-size=104B cardinality=unavailable
| | in pipelines: 75(GETNEXT), 36(OPEN)
{noformat}
I expect that the cardinality should be unavailable, not 0.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8847) Add partition events may contain empty partition object list
Vihang Karajgaonkar created IMPALA-8847:
---
Summary: Add partition events may contain empty partition object
list
Key: IMPALA-8847
URL: https://issues.apache.org/jira/browse/IMPALA-8847
Project: IMPALA
Issue Type: Bug
Reporter: Vihang Karajgaonkar
Assignee: Vihang Karajgaonkar
When event polling is ON and when an external application like Hive issues a
{{alter table add if not exists partition ()}} it is possble
that command did not add a partition since it is preexisting. However,
metastore still generates a ADD_PARTITION event in such a case with empty list
of added partitions. Such events cause a Precondition to fail while processing
on the EventsProcessor side and event polling goes into error state.
The fix would be simple. Ignore such events.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8627) Re-enable catalog v2 in containers
[ https://issues.apache.org/jira/browse/IMPALA-8627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar resolved IMPALA-8627. - Resolution: Fixed > Re-enable catalog v2 in containers > -- > > Key: IMPALA-8627 > URL: https://issues.apache.org/jira/browse/IMPALA-8627 > Project: IMPALA > Issue Type: Sub-task > Components: Infrastructure >Affects Versions: Impala 3.3.0 >Reporter: Tim Armstrong >Assignee: Vihang Karajgaonkar >Priority: Major > Labels: catalog-v2 > Fix For: Impala 3.3.0 > > > We also need to set --invalidate_tables_on_memory_pressure on the impalads > for that to be fully effective - the impalads send table usage info to the > catalogd -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Work started] (IMPALA-8795) Enable event polling by default in tests
[ https://issues.apache.org/jira/browse/IMPALA-8795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on IMPALA-8795 started by Vihang Karajgaonkar. --- > Enable event polling by default in tests > > > Key: IMPALA-8795 > URL: https://issues.apache.org/jira/browse/IMPALA-8795 > Project: IMPALA > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > > We should turn on event processing by default in all the tests to make sure > that there are no regressions when we turn ON the feature by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8795) Enable event polling by default in tests
[ https://issues.apache.org/jira/browse/IMPALA-8795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903214#comment-16903214 ] Vihang Karajgaonkar commented on IMPALA-8795: - Code is in review > Enable event polling by default in tests > > > Key: IMPALA-8795 > URL: https://issues.apache.org/jira/browse/IMPALA-8795 > Project: IMPALA > Issue Type: Sub-task >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > > We should turn on event processing by default in all the tests to make sure > that there are no regressions when we turn ON the feature by default. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8846) Undefined behaviour in RleEncoder::Put
[ https://issues.apache.org/jira/browse/IMPALA-8846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903201#comment-16903201 ] Daniel Becker commented on IMPALA-8846: --- Added some benchmarks. > Undefined behaviour in RleEncoder::Put > -- > > Key: IMPALA-8846 > URL: https://issues.apache.org/jira/browse/IMPALA-8846 > Project: IMPALA > Issue Type: Bug >Reporter: Daniel Becker >Assignee: Daniel Becker >Priority: Major > Attachments: original.txt, with_check.txt > > > On line > [https://github.com/apache/impala/blob/4000da35be69e469500f5f11e0e5fdec119cf5c7/be/src/util/rle-encoding.h#L346,] > we test repeat_count_ <= std::numeric_limits::max(), which is > always true (repeat_count_ is an int), then we increment repeat_count which > could be std::numeric_limits::max() and overflow, which is undefined > behaviour for signed integers. > > We should either change <= to < or if we think that this never happens, > remove the misleading check. > If we correct the check, it may lead to some (probably small) performance > regression because the compiler could have optimised this out. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8846) Undefined behaviour in RleEncoder::Put
[ https://issues.apache.org/jira/browse/IMPALA-8846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Becker updated IMPALA-8846: -- Attachment: with_check.txt original.txt > Undefined behaviour in RleEncoder::Put > -- > > Key: IMPALA-8846 > URL: https://issues.apache.org/jira/browse/IMPALA-8846 > Project: IMPALA > Issue Type: Bug >Reporter: Daniel Becker >Assignee: Daniel Becker >Priority: Major > Attachments: original.txt, with_check.txt > > > On line > [https://github.com/apache/impala/blob/4000da35be69e469500f5f11e0e5fdec119cf5c7/be/src/util/rle-encoding.h#L346,] > we test repeat_count_ <= std::numeric_limits::max(), which is > always true (repeat_count_ is an int), then we increment repeat_count which > could be std::numeric_limits::max() and overflow, which is undefined > behaviour for signed integers. > > We should either change <= to < or if we think that this never happens, > remove the misleading check. > If we correct the check, it may lead to some (probably small) performance > regression because the compiler could have optimised this out. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8846) Undefined behaviour in RleEncoder::Put
Daniel Becker created IMPALA-8846: - Summary: Undefined behaviour in RleEncoder::Put Key: IMPALA-8846 URL: https://issues.apache.org/jira/browse/IMPALA-8846 Project: IMPALA Issue Type: Bug Reporter: Daniel Becker Assignee: Daniel Becker On line [https://github.com/apache/impala/blob/4000da35be69e469500f5f11e0e5fdec119cf5c7/be/src/util/rle-encoding.h#L346,] we test repeat_count_ <= std::numeric_limits::max(), which is always true (repeat_count_ is an int), then we increment repeat_count which could be std::numeric_limits::max() and overflow, which is undefined behaviour for signed integers. We should either change <= to < or if we think that this never happens, remove the misleading check. If we correct the check, it may lead to some (probably small) performance regression because the compiler could have optimised this out. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8833) Check failed: bit_width <= sizeof(T) * 8 (40 vs. 32) in BatchedBitReader::UnpackBatch()
[
https://issues.apache.org/jira/browse/IMPALA-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903101#comment-16903101
]
ASF subversion and git services commented on IMPALA-8833:
-
Commit ef796639451af63bc9f4dbdc2a79bfc2f5048a93 in impala's branch
refs/heads/master from Daniel Becker
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=ef79663 ]
IMPALA-8840: Check failed: num_bytes <= sizeof(T) (5 vs. 4)
The actual DCHECK failure was fixed by the change for IMPALA-8833.
Added a DCHECK to RleBatchDecoder so that it does not accept bit widths
higher than the width of its type parameter.
Also preventing UnpackAndDecodeValues from using higher bit widths than
32, the width of the dictionary index type, with a static assert. This
also reduces compile time because the compiler does not have to generate
code for invalid bit widths.
Change-Id: I93461ba2cabb5ec7e0b65dcd62844fcbfa597d16
Reviewed-on: http://gerrit.cloudera.org:8080/14029
Reviewed-by: Csaba Ringhofer
Tested-by: Impala Public Jenkins
> Check failed: bit_width <= sizeof(T) * 8 (40 vs. 32) in
> BatchedBitReader::UnpackBatch()
>
>
> Key: IMPALA-8833
> URL: https://issues.apache.org/jira/browse/IMPALA-8833
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Daniel Becker
>Priority: Blocker
> Labels: broken-build, crash, flaky
> Fix For: Impala 3.3.0
>
>
> {noformat}
> F0801 21:24:10.571285 15993 bit-stream-utils.inline.h:126]
> d04ba69d5da8ffd1:a9045b820001] Check failed: bit_width <= sizeof(T) * 8
> (40 vs. 32)
> *** Check failure stack trace: ***
> @ 0x52f63ac google::LogMessage::Fail()
> @ 0x52f7c51 google::LogMessage::SendToLog()
> @ 0x52f5d86 google::LogMessage::Flush()
> @ 0x52f934d google::LogMessageFatal::~LogMessageFatal()
> @ 0x2b265b5 impala::BatchedBitReader::UnpackBatch<>()
> @ 0x2ae8623 impala::RleBatchDecoder<>::FillLiteralBuffer()
> @ 0x2b2cadb impala::RleBatchDecoder<>::DecodeLiteralValues<>()
> @ 0x2b27bfb impala::DictDecoder<>::DecodeNextValue()
> @ 0x2b16fed
> impala::ScalarColumnReader<>::ReadSlotsNoConversion()
> @ 0x2ac7252 impala::ScalarColumnReader<>::ReadSlots()
> @ 0x2a76cef
> impala::ScalarColumnReader<>::MaterializeValueBatchRepeatedDefLevel()
> @ 0x2a58faa impala::ScalarColumnReader<>::ReadValueBatch<>()
> @ 0x2a20e8e
> impala::ScalarColumnReader<>::ReadNonRepeatedValueBatch()
> @ 0x29b189c impala::HdfsParquetScanner::AssembleRows()
> @ 0x29ac6de impala::HdfsParquetScanner::GetNextInternal()
> @ 0x29aa656 impala::HdfsParquetScanner::ProcessSplit()
> @ 0x249172d impala::HdfsScanNode::ProcessSplit()
> @ 0x2490902 impala::HdfsScanNode::ScannerThread()
> @ 0x248fc8b
> _ZZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS_18ThreadResourcePoolEENKUlvE_clEv
> @ 0x2492253
> {noformat}
> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/6915
> Log lines around the failure:
> {noformat}
> [gw5] PASSED
> query_test/test_scanners.py::TestParquet::test_bad_compression_codec[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold':
> 0} | table_format: parquet/none]
> query_test/test_nested_types.py::TestMaxNestingDepth::test_load_hive_table[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> parquet/none]
> query_test/test_scanners.py::TestParquet::test_bad_compression_codec[protocol:
> beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': True,
> 'abort_on_error': 1, 'debug_action':
> '-1:OPEN:SET_DENY_RESERVATION_PROBABILITY@0.5',
> 'exec_single_node_rows_threshold': 0} | table_format: parquet/none]
> [gw1] PASSED
> query_test/test_tpcds_queries.py::TestTpcdsQuery::test_tpcds_q7[protocol:
> beeswax | exec_option: {'decimal_v2': 0, 'batch_size': 0, 'num_nodes': 0,
> 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> parquet/none]
> query_test/test_tpcds_queries.py::TestTpcdsQuery::test_tpcds_q8[protocol:
> beeswax | exec_option: {'decimal_v2': 0, 'batch_size': 0, 'num_nodes':
[jira] [Commented] (IMPALA-8840) Check failed: num_bytes <= sizeof(T) (5 vs. 4)
[
https://issues.apache.org/jira/browse/IMPALA-8840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903100#comment-16903100
]
ASF subversion and git services commented on IMPALA-8840:
-
Commit ef796639451af63bc9f4dbdc2a79bfc2f5048a93 in impala's branch
refs/heads/master from Daniel Becker
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=ef79663 ]
IMPALA-8840: Check failed: num_bytes <= sizeof(T) (5 vs. 4)
The actual DCHECK failure was fixed by the change for IMPALA-8833.
Added a DCHECK to RleBatchDecoder so that it does not accept bit widths
higher than the width of its type parameter.
Also preventing UnpackAndDecodeValues from using higher bit widths than
32, the width of the dictionary index type, with a static assert. This
also reduces compile time because the compiler does not have to generate
code for invalid bit widths.
Change-Id: I93461ba2cabb5ec7e0b65dcd62844fcbfa597d16
Reviewed-on: http://gerrit.cloudera.org:8080/14029
Reviewed-by: Csaba Ringhofer
Tested-by: Impala Public Jenkins
> Check failed: num_bytes <= sizeof(T) (5 vs. 4)
> ---
>
> Key: IMPALA-8840
> URL: https://issues.apache.org/jira/browse/IMPALA-8840
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.3.0
>Reporter: Xiaomeng Zhang
>Assignee: Daniel Becker
>Priority: Blocker
> Labels: broken-build, crash
>
> Not sure if this is due to same issue as
> https://issues.apache.org/jira/browse/IMPALA-8833#, the error message is a
> little different.
> {code:java}
> F0805 18:48:08.737411 5488 bit-stream-utils.inline.h:173]
> 284731e5d1aad693:05c883020001] Check failed: num_bytes <= sizeof(T) (8
> vs. 4)
> *** Check failure stack trace: ***
> @ 0x52fb9bc google::LogMessage::Fail()
> @ 0x52fd261 google::LogMessage::SendToLog()
> @ 0x52fb396 google::LogMessage::Flush()
> @ 0x52fe95d google::LogMessageFatal::~LogMessageFatal()
> @ 0x2b2b867 impala::BatchedBitReader::GetBytes<>()
> @ 0x2aeda65 impala::RleBatchDecoder<>::NextCounts()
> @ 0x2a82896 impala::RleBatchDecoder<>::NextNumRepeats()
> @ 0x2b1927f impala::ScalarColumnReader<>::ReadSlotsNoConversion()
> @ 0x2ac7c2c impala::ScalarColumnReader<>::ReadSlots()
> @ 0x2a7b861
> impala::ScalarColumnReader<>::MaterializeValueBatchRepeatedDefLevel()
> @ 0x2a5b3b0 impala::ScalarColumnReader<>::ReadValueBatch<>()
> @ 0x2a256a4 impala::ScalarColumnReader<>::ReadNonRepeatedValueBatch()
> @ 0x29b6eb6 impala::HdfsParquetScanner::AssembleRows()
> @ 0x29b1cf8 impala::HdfsParquetScanner::GetNextInternal()
> @ 0x29afc70 impala::HdfsParquetScanner::ProcessSplit()
> @ 0x2494bc3 impala::HdfsScanNode::ProcessSplit()
> @ 0x2493d98 impala::HdfsScanNode::ScannerThread()
> @ 0x2493121
> _ZZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS_18ThreadResourcePoolEENKUlvE_clEv
> @ 0x24956e9
> _ZN5boost6detail8function26void_function_obj_invoker0IZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS3_18ThreadResourcePoolEEUlvE_vE6invokeERNS1_15function_bufferE
> @ 0x1ea0241 boost::function0<>::operator()()
> @ 0x23de77a impala::Thread::SuperviseThread()
> @ 0x23e6afe boost::_bi::list5<>::operator()<>()
> @ 0x23e6a22 boost::_bi::bind_t<>::operator()()
> @ 0x23e69e5 boost::detail::thread_data<>::run()
> @ 0x4224819 thread_proxy
> @ 0x7fc1818c5e24 start_thread
> @ 0x7fc17e01f34c __clone
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8840) Check failed: num_bytes <= sizeof(T) (5 vs. 4)
[
https://issues.apache.org/jira/browse/IMPALA-8840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Becker resolved IMPALA-8840.
---
Resolution: Fixed
> Check failed: num_bytes <= sizeof(T) (5 vs. 4)
> ---
>
> Key: IMPALA-8840
> URL: https://issues.apache.org/jira/browse/IMPALA-8840
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.3.0
>Reporter: Xiaomeng Zhang
>Assignee: Daniel Becker
>Priority: Blocker
> Labels: broken-build, crash
>
> Not sure if this is due to same issue as
> https://issues.apache.org/jira/browse/IMPALA-8833#, the error message is a
> little different.
> {code:java}
> F0805 18:48:08.737411 5488 bit-stream-utils.inline.h:173]
> 284731e5d1aad693:05c883020001] Check failed: num_bytes <= sizeof(T) (8
> vs. 4)
> *** Check failure stack trace: ***
> @ 0x52fb9bc google::LogMessage::Fail()
> @ 0x52fd261 google::LogMessage::SendToLog()
> @ 0x52fb396 google::LogMessage::Flush()
> @ 0x52fe95d google::LogMessageFatal::~LogMessageFatal()
> @ 0x2b2b867 impala::BatchedBitReader::GetBytes<>()
> @ 0x2aeda65 impala::RleBatchDecoder<>::NextCounts()
> @ 0x2a82896 impala::RleBatchDecoder<>::NextNumRepeats()
> @ 0x2b1927f impala::ScalarColumnReader<>::ReadSlotsNoConversion()
> @ 0x2ac7c2c impala::ScalarColumnReader<>::ReadSlots()
> @ 0x2a7b861
> impala::ScalarColumnReader<>::MaterializeValueBatchRepeatedDefLevel()
> @ 0x2a5b3b0 impala::ScalarColumnReader<>::ReadValueBatch<>()
> @ 0x2a256a4 impala::ScalarColumnReader<>::ReadNonRepeatedValueBatch()
> @ 0x29b6eb6 impala::HdfsParquetScanner::AssembleRows()
> @ 0x29b1cf8 impala::HdfsParquetScanner::GetNextInternal()
> @ 0x29afc70 impala::HdfsParquetScanner::ProcessSplit()
> @ 0x2494bc3 impala::HdfsScanNode::ProcessSplit()
> @ 0x2493d98 impala::HdfsScanNode::ScannerThread()
> @ 0x2493121
> _ZZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS_18ThreadResourcePoolEENKUlvE_clEv
> @ 0x24956e9
> _ZN5boost6detail8function26void_function_obj_invoker0IZN6impala12HdfsScanNode22ThreadTokenAvailableCbEPNS3_18ThreadResourcePoolEEUlvE_vE6invokeERNS1_15function_bufferE
> @ 0x1ea0241 boost::function0<>::operator()()
> @ 0x23de77a impala::Thread::SuperviseThread()
> @ 0x23e6afe boost::_bi::list5<>::operator()<>()
> @ 0x23e6a22 boost::_bi::bind_t<>::operator()()
> @ 0x23e69e5 boost::detail::thread_data<>::run()
> @ 0x4224819 thread_proxy
> @ 0x7fc1818c5e24 start_thread
> @ 0x7fc17e01f34c __clone
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
