[jira] [Created] (IMPALA-8569) Periodically scrub deleted files from the file handle cache
Todd Lipcon created IMPALA-8569: --- Summary: Periodically scrub deleted files from the file handle cache Key: IMPALA-8569 URL: https://issues.apache.org/jira/browse/IMPALA-8569 Project: IMPALA Issue Type: Improvement Components: Backend Reporter: Todd Lipcon Currently, if you query a file, and then later delete that file (eg drop the partition or table), the file will still stay in the impalad's file handle cache. Because the file is open, the space can't be reclaimed on disk until the impalad restarts or churns through its cache enough to drop the handle. Typically this isn't a big deal in practice, since most files don't get deleted shortly after being read, and the FH cache should cycle through after 6 hours by default. Additionally, fixing it would be a bit of a pain since we'd need to add HDFS and libhdfs hooks to get HDFS to tell us if the underlying short circuit FD is unlinked, which probably also means adding JNI code to let Java call to fstat() in order to check st_nlink. Given that, I'm not sure it's worth fixing, or if we should just consider a shorter default expiry on the FH cache. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8569) Periodically scrub deleted files from the file handle cache
Todd Lipcon created IMPALA-8569: --- Summary: Periodically scrub deleted files from the file handle cache Key: IMPALA-8569 URL: https://issues.apache.org/jira/browse/IMPALA-8569 Project: IMPALA Issue Type: Improvement Components: Backend Reporter: Todd Lipcon Currently, if you query a file, and then later delete that file (eg drop the partition or table), the file will still stay in the impalad's file handle cache. Because the file is open, the space can't be reclaimed on disk until the impalad restarts or churns through its cache enough to drop the handle. Typically this isn't a big deal in practice, since most files don't get deleted shortly after being read, and the FH cache should cycle through after 6 hours by default. Additionally, fixing it would be a bit of a pain since we'd need to add HDFS and libhdfs hooks to get HDFS to tell us if the underlying short circuit FD is unlinked, which probably also means adding JNI code to let Java call to fstat() in order to check st_nlink. Given that, I'm not sure it's worth fixing, or if we should just consider a shorter default expiry on the FH cache. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IMPALA-8544) Expose additional S3A / S3Guard metrics
[
https://issues.apache.org/jira/browse/IMPALA-8544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845479#comment-16845479
]
Todd Lipcon commented on IMPALA-8544:
-
I think the problem with using the global metrics is that they are only pushed
up to the global counters when a stream is closed. Impala now keeps open file
handles to S3. Perhaps we can make unbuffer() also push the counters to the
global store?
bq. the fields are all non-atomic, non-volatile values so that the cost of
incrementing them is ~0. If things are being collected, that may change.
I don't think that's necessarily the case. Given they are counters, I'm sure
we're OK with slight raciness on read. We could always use Unsafe or VarHandle
to do barrier-less reads and writes knowing that we're OK without those
barriers.
> Expose additional S3A / S3Guard metrics
> ---
>
> Key: IMPALA-8544
> URL: https://issues.apache.org/jira/browse/IMPALA-8544
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Labels: s3
>
> S3A / S3Guard internally collects several useful metrics that we should
> consider exposing to Impala users. The full list of statistics can be found
> in {{o.a.h.fs.s3a.Statistic}}. The stats include: the number of S3 operations
> performed (put, get, etc.), invocation counts for various {{FileSystem}}
> methods, stream statistics (bytes read, written, etc.), etc.
> Some interesting stats that stand out:
> * "stream_aborted": "Count of times the TCP stream was aborted" - the number
> of TCP connection aborts, a high value would indicate performance issues
> * "stream_read_exceptions" : "Number of exceptions invoked on input streams"
> - incremented whenever an {{IOException}} is caught while reading (these
> exception don't always get propagated to Impala because they trigger a retry)
> * "store_io_throttled": "Requests throttled and retried" - looks like it
> tracks the number of times the fs retries an operation because the original
> request hit a throttling exception
> * "s3guard_metadatastore_retry": "S3Guard metadata store retry events" -
> looks like it tracks the number of times the fs retries S3Guard operations
> * "s3guard_metadatastore_throttled" : "S3Guard metadata store throttled
> events" - similar to "store_io_throttled" but looks like it is specific to
> S3Guard
> We should consider how to expose these metrics via Impala logs / runtime
> profiles.
> There are a few options:
> * {{S3AFileSystem}} exposes {{StorageStatistics}} specific to S3A / S3Guard
> via the {{FileSystem#getStorageStatistics}} method; the
> {{S3AStorageStatistics}} seems to include all the S3A / S3Guard metrics,
> however, I think the stats might be aggregated globally, which would make it
> hard to create per-query specific metrics
> * {{S3AInstrumentation}} exposes all the metrics as well, and looks like it
> is per-fs instance, so it is not aggregated globally; {{S3AInstrumentation}}
> extends {{o.a.h.metrics2.MetricsSource}} so perhaps it is exposed via some
> API (haven't looked into this yet)
> * {{S3AInputStream#toString}} dumps the statistics from
> {{o.a.h.fs.s3a.S3AInstrumentation.InputStreamStatistics}} and
> {{S3AFileSystem#toString}} dumps them all as well
> * {{S3AFileSystem}} updates the stats in
> {{o.a.h.fs.Statistics.StatisticsData}} as well (e.g. bytesRead, bytesWritten,
> etc.)
> Impala has a {{hdfs-fs-cache}} as well, so {{hdfsFs}} objects get shared
> across threads.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8544) Expose additional S3A / S3Guard metrics
[
https://issues.apache.org/jira/browse/IMPALA-8544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845313#comment-16845313
]
Sahil Takiar commented on IMPALA-8544:
--
Thanks for the input everyone.
It sounds like there is some additional effort to expose these metrics on a
per-query basis. As a first step, I think there are several S3A metrics that
would make sense to expose on an impalad process-level (rather than a per query
basis). My guess is that metrics like "store_io_throttled" and
"s3guard_metadatastore_throttled" are likely to be most relevant process wide.
For example, given an impalad running five concurrent queries against the same
bucket, if one of the queries has a high value for "store_io_throttled" its
likely that the rest of the queries do as well. I can't remember where I read
it, but I think S3 throttling is done on a per-bucket basis? If thats the case,
all queries against a bucket will likely be throttled at the same time. So the
ability to see the value of "store_io_throttled" on an impalad process-level is
probably sufficient to notice this. I think the story for S3Guard is similar,
since you specify a maximum request rate per second for a single table.
Impala already has support for exposing process wide metrics, so implementing
this using the exposed {{StorageStatistics}} should be straightforward. The
only issue I see is that {{StorageStatistics}} are aggregated globally, but I'm
curious if we have considered aggregating them on a per-bucket granularity
instead?
> Expose additional S3A / S3Guard metrics
> ---
>
> Key: IMPALA-8544
> URL: https://issues.apache.org/jira/browse/IMPALA-8544
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Labels: s3
>
> S3A / S3Guard internally collects several useful metrics that we should
> consider exposing to Impala users. The full list of statistics can be found
> in {{o.a.h.fs.s3a.Statistic}}. The stats include: the number of S3 operations
> performed (put, get, etc.), invocation counts for various {{FileSystem}}
> methods, stream statistics (bytes read, written, etc.), etc.
> Some interesting stats that stand out:
> * "stream_aborted": "Count of times the TCP stream was aborted" - the number
> of TCP connection aborts, a high value would indicate performance issues
> * "stream_read_exceptions" : "Number of exceptions invoked on input streams"
> - incremented whenever an {{IOException}} is caught while reading (these
> exception don't always get propagated to Impala because they trigger a retry)
> * "store_io_throttled": "Requests throttled and retried" - looks like it
> tracks the number of times the fs retries an operation because the original
> request hit a throttling exception
> * "s3guard_metadatastore_retry": "S3Guard metadata store retry events" -
> looks like it tracks the number of times the fs retries S3Guard operations
> * "s3guard_metadatastore_throttled" : "S3Guard metadata store throttled
> events" - similar to "store_io_throttled" but looks like it is specific to
> S3Guard
> We should consider how to expose these metrics via Impala logs / runtime
> profiles.
> There are a few options:
> * {{S3AFileSystem}} exposes {{StorageStatistics}} specific to S3A / S3Guard
> via the {{FileSystem#getStorageStatistics}} method; the
> {{S3AStorageStatistics}} seems to include all the S3A / S3Guard metrics,
> however, I think the stats might be aggregated globally, which would make it
> hard to create per-query specific metrics
> * {{S3AInstrumentation}} exposes all the metrics as well, and looks like it
> is per-fs instance, so it is not aggregated globally; {{S3AInstrumentation}}
> extends {{o.a.h.metrics2.MetricsSource}} so perhaps it is exposed via some
> API (haven't looked into this yet)
> * {{S3AInputStream#toString}} dumps the statistics from
> {{o.a.h.fs.s3a.S3AInstrumentation.InputStreamStatistics}} and
> {{S3AFileSystem#toString}} dumps them all as well
> * {{S3AFileSystem}} updates the stats in
> {{o.a.h.fs.Statistics.StatisticsData}} as well (e.g. bytesRead, bytesWritten,
> etc.)
> Impala has a {{hdfs-fs-cache}} as well, so {{hdfsFs}} objects get shared
> across threads.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8537) Negative values reported for tmp-file-mgr.scratch-space-bytes-used under heavy spilling load
[ https://issues.apache.org/jira/browse/IMPALA-8537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Rawat resolved IMPALA-8537. Resolution: Fixed Fix Version/s: Impala 3.3.0 > Negative values reported for tmp-file-mgr.scratch-space-bytes-used under > heavy spilling load > > > Key: IMPALA-8537 > URL: https://issues.apache.org/jira/browse/IMPALA-8537 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 3.3.0 >Reporter: David Rorke >Assignee: Abhishek Rawat >Priority: Major > Fix For: Impala 3.3.0 > > Attachments: bad_spill_metrics.json > > > I'm running a workload that does a lot of spilling and noticed the value > reported for tmp-file-mgr.scratch-space-bytes-used is negative on all nodes. > Some details of the workload and cluster configuration: > * Generating a 10 TB TPC-DS partitioned parquet data set (very large sort). > * 30 impalads, each with 48 GB RAM and 14 scratch directories (each on a > separate drive) > * Rough estimate (based on query metrics) of total cumulative aggregate > memory spilled across the cluster since restart is 6.5 TB. > Snapshot of the bad metrics attached. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8537) Negative values reported for tmp-file-mgr.scratch-space-bytes-used under heavy spilling load
[ https://issues.apache.org/jira/browse/IMPALA-8537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Rawat resolved IMPALA-8537. Resolution: Fixed Fix Version/s: Impala 3.3.0 > Negative values reported for tmp-file-mgr.scratch-space-bytes-used under > heavy spilling load > > > Key: IMPALA-8537 > URL: https://issues.apache.org/jira/browse/IMPALA-8537 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 3.3.0 >Reporter: David Rorke >Assignee: Abhishek Rawat >Priority: Major > Fix For: Impala 3.3.0 > > Attachments: bad_spill_metrics.json > > > I'm running a workload that does a lot of spilling and noticed the value > reported for tmp-file-mgr.scratch-space-bytes-used is negative on all nodes. > Some details of the workload and cluster configuration: > * Generating a 10 TB TPC-DS partitioned parquet data set (very large sort). > * 30 impalads, each with 48 GB RAM and 14 scratch directories (each on a > separate drive) > * Rough estimate (based on query metrics) of total cumulative aggregate > memory spilled across the cluster since restart is 6.5 TB. > Snapshot of the bad metrics attached. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IMPALA-7936) Enable better control over Parquet writing
[ https://issues.apache.org/jira/browse/IMPALA-7936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845262#comment-16845262 ] ASF subversion and git services commented on IMPALA-7936: - Commit 95be560e0c43a5f8ea7b00f3ba9f83ec3f734ca2 in impala's branch refs/heads/master from Zoltan Borok-Nagy [ https://gitbox.apache.org/repos/asf?p=impala.git;h=95be560 ] IMPALA-7936: Enable better control over Parquet writing This commit adds two new query options to Impala. One is to disable/ enable parquet page index writing (by default it is enabled), the other is to set a row count limit per Parquet page (by default there is no row count limit). It removes the old command-line flag that controlled the enablement of page index writing. Since page index writing is the default since IMPALA-5843, I moved the tests from the "custom cluster" test suite to the "query test" test suite. This way the tests run faster because we don't need to restart the Impala daemons. Testing: Added new test cases to test the effect of the query options. Change-Id: Ib9ec8b16036e1fd35886e887809be8eca52a6982 Reviewed-on: http://gerrit.cloudera.org:8080/13361 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Enable better control over Parquet writing > -- > > Key: IMPALA-7936 > URL: https://issues.apache.org/jira/browse/IMPALA-7936 > Project: IMPALA > Issue Type: Improvement >Reporter: Zoltán Borók-Nagy >Assignee: Zoltán Borók-Nagy >Priority: Major > > With the introduction of the Parquet page indexes it became desirable to have > more control over how Impala writes Parquet files. > These configuration options (probably implemented as query options) would be: > * enable/disable Parquet page index writing (currently we can do it with a > command-line argument) > * set page-size limits based on row count > * -Set truncation length for statistics about string values- (current > truncation length is 64, it is unlikely to have user data that needs longer > truncation than that) > They'd enable writing more complete tests for page filtering. They'd be also > useful for fine-tuning the page index for better performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-5843) Use page index in Parquet files to skip pages
[ https://issues.apache.org/jira/browse/IMPALA-5843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845263#comment-16845263 ] ASF subversion and git services commented on IMPALA-5843: - Commit 95be560e0c43a5f8ea7b00f3ba9f83ec3f734ca2 in impala's branch refs/heads/master from Zoltan Borok-Nagy [ https://gitbox.apache.org/repos/asf?p=impala.git;h=95be560 ] IMPALA-7936: Enable better control over Parquet writing This commit adds two new query options to Impala. One is to disable/ enable parquet page index writing (by default it is enabled), the other is to set a row count limit per Parquet page (by default there is no row count limit). It removes the old command-line flag that controlled the enablement of page index writing. Since page index writing is the default since IMPALA-5843, I moved the tests from the "custom cluster" test suite to the "query test" test suite. This way the tests run faster because we don't need to restart the Impala daemons. Testing: Added new test cases to test the effect of the query options. Change-Id: Ib9ec8b16036e1fd35886e887809be8eca52a6982 Reviewed-on: http://gerrit.cloudera.org:8080/13361 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > Use page index in Parquet files to skip pages > - > > Key: IMPALA-5843 > URL: https://issues.apache.org/jira/browse/IMPALA-5843 > Project: IMPALA > Issue Type: New Feature > Components: Backend >Affects Versions: Impala 2.10.0 >Reporter: Lars Volker >Assignee: Zoltán Borók-Nagy >Priority: Critical > Labels: parquet, performance > Fix For: Impala 3.3.0 > > > Once IMPALA-5842 has been resolved, we should skip pages based on the page > index in Parquet files. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7637) Include more hash table stats in profile
[ https://issues.apache.org/jira/browse/IMPALA-7637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845206#comment-16845206 ] Sahil Takiar commented on IMPALA-7637: -- Assigned this to myself by mistake. Re-assigning to Yongzhi. > Include more hash table stats in profile > > > Key: IMPALA-7637 > URL: https://issues.apache.org/jira/browse/IMPALA-7637 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Tim Armstrong >Assignee: Yongzhi Chen >Priority: Major > Labels: observability, ramp-up > > Our hash table collects some useful stats about collisions and travel length, > but then we don't do anything to expose them: > https://github.com/apache/impala/blob/540611e863fe99b3d3ae35f8b94a745a68b9eba2/be/src/exec/hash-table.h#L989 > We should add some of them to the profile, maybe: > * the number of probes > * the average travel length per probe > * the number of hash collisions > * (optional) the number of hash table resizes. We already have the hash table > size and the resize time, which I think is sufficient to debug most problems > with resizes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8568) Fix Impala JNI warnings when -Xcheck:jni is enabled
Sahil Takiar created IMPALA-8568: Summary: Fix Impala JNI warnings when -Xcheck:jni is enabled Key: IMPALA-8568 URL: https://issues.apache.org/jira/browse/IMPALA-8568 Project: IMPALA Issue Type: Improvement Reporter: Sahil Takiar Assignee: Sahil Takiar IMPALA-8250 made a lot of improvements to our usage of the JNI. Impala no longer crashes when running exhaustive tests with -Xcheck:jni enabled. We made some progress in cleaning up libhdfs JNI usage in HDFS-14321 and HDFS-14348 as well. However, re-running exhaustive tests with -Xcheck:jni still shows a lot of warnings. It's not clear if these warnings are from libhdfs or Impala, but either way we should drive a fix. The most concerning of the current list of JNI warnings produced by Impala, are the "JNI call made without checking exceptions when required to from ..." warnings. Essentially, this means that when making a JNI call, we are not properly checking for exceptions. This can be problematic because a JNI call make throw an exception, and we end up swallowing it. There are lots of warnings about "WARNING: JNI local refs: [x], exceeds capacity: [y]". Based on some digging (e.g. https://community.oracle.com/message/13290783) it looks like these warnings aren't fatal, but are just bad practice. I think we can fix the most egregious offenders (looks like the HBase code is one of them), and hopefully live with the rest (a lot of the warnings are thrown by internal Java code as well). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IMPALA-7637) Include more hash table stats in profile
[ https://issues.apache.org/jira/browse/IMPALA-7637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar reassigned IMPALA-7637: Assignee: Yongzhi Chen (was: Sahil Takiar) > Include more hash table stats in profile > > > Key: IMPALA-7637 > URL: https://issues.apache.org/jira/browse/IMPALA-7637 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Tim Armstrong >Assignee: Yongzhi Chen >Priority: Major > Labels: observability, ramp-up > > Our hash table collects some useful stats about collisions and travel length, > but then we don't do anything to expose them: > https://github.com/apache/impala/blob/540611e863fe99b3d3ae35f8b94a745a68b9eba2/be/src/exec/hash-table.h#L989 > We should add some of them to the profile, maybe: > * the number of probes > * the average travel length per probe > * the number of hash collisions > * (optional) the number of hash table resizes. We already have the hash table > size and the resize time, which I think is sufficient to debug most problems > with resizes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8568) Fix Impala JNI warnings when -Xcheck:jni is enabled
Sahil Takiar created IMPALA-8568: Summary: Fix Impala JNI warnings when -Xcheck:jni is enabled Key: IMPALA-8568 URL: https://issues.apache.org/jira/browse/IMPALA-8568 Project: IMPALA Issue Type: Improvement Reporter: Sahil Takiar Assignee: Sahil Takiar IMPALA-8250 made a lot of improvements to our usage of the JNI. Impala no longer crashes when running exhaustive tests with -Xcheck:jni enabled. We made some progress in cleaning up libhdfs JNI usage in HDFS-14321 and HDFS-14348 as well. However, re-running exhaustive tests with -Xcheck:jni still shows a lot of warnings. It's not clear if these warnings are from libhdfs or Impala, but either way we should drive a fix. The most concerning of the current list of JNI warnings produced by Impala, are the "JNI call made without checking exceptions when required to from ..." warnings. Essentially, this means that when making a JNI call, we are not properly checking for exceptions. This can be problematic because a JNI call make throw an exception, and we end up swallowing it. There are lots of warnings about "WARNING: JNI local refs: [x], exceeds capacity: [y]". Based on some digging (e.g. https://community.oracle.com/message/13290783) it looks like these warnings aren't fatal, but are just bad practice. I think we can fix the most egregious offenders (looks like the HBase code is one of them), and hopefully live with the rest (a lot of the warnings are thrown by internal Java code as well). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8250) Impala crashes with -Xcheck:jni
[ https://issues.apache.org/jira/browse/IMPALA-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar resolved IMPALA-8250. -- Resolution: Fixed I'm closing this. I re-ran exhaustive tests against Impala with -Xcheck:jni and Impala no longer crashes; all tests pass. There are still a ton of JNI warnings, so I'm going to file a follow up JIRA to fix them. > Impala crashes with -Xcheck:jni > --- > > Key: IMPALA-8250 > URL: https://issues.apache.org/jira/browse/IMPALA-8250 > Project: IMPALA > Issue Type: Task >Reporter: Philip Zeyliger >Priority: Major > Fix For: Impala 3.2.0 > > > The JVM has a checker for JNI usage, and Impala (and libhdfs) have some > violations. This ticket captures figuring that out. At least one of the > issues can crash Impala. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8250) Impala crashes with -Xcheck:jni
[ https://issues.apache.org/jira/browse/IMPALA-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar resolved IMPALA-8250. -- Resolution: Fixed I'm closing this. I re-ran exhaustive tests against Impala with -Xcheck:jni and Impala no longer crashes; all tests pass. There are still a ton of JNI warnings, so I'm going to file a follow up JIRA to fix them. > Impala crashes with -Xcheck:jni > --- > > Key: IMPALA-8250 > URL: https://issues.apache.org/jira/browse/IMPALA-8250 > Project: IMPALA > Issue Type: Task >Reporter: Philip Zeyliger >Priority: Major > Fix For: Impala 3.2.0 > > > The JVM has a checker for JNI usage, and Impala (and libhdfs) have some > violations. This ticket captures figuring that out. At least one of the > issues can crash Impala. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IMPALA-8567) Many random catalog consistency issues with catalog v2/event processor
[ https://issues.apache.org/jira/browse/IMPALA-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845122#comment-16845122 ] Vihang Karajgaonkar commented on IMPALA-8567: - Thanks [~tarmstrong] for doing the initial triage. Let me try to see if I can reproduce this locally > Many random catalog consistency issues with catalog v2/event processor > -- > > Key: IMPALA-8567 > URL: https://issues.apache.org/jira/browse/IMPALA-8567 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 3.3.0 >Reporter: Tim Armstrong >Assignee: Vihang Karajgaonkar >Priority: Blocker > Labels: broken-build, catalog, flaky > > [~tlipcon] [~vihangk1] FYI. I'm not sure whether the local catalog or the > event processor is likely to blame here so I'll let you look. The general > theme is tables and databases not existing when they should. > https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/289/testReport/junit/metadata.test_refresh_partition/TestRefreshPartition/test_drop_hive_partition_and_refresh_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ > https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/267/testReport/junit/query_test.test_kudu/TestKuduOperations/test_kudu_insert_protocol__beeswax___exec_optionkudu_read_modeREAD_AT_SNAPSHOTbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ > https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/286/testReport/junit/metadata.test_metadata_query_statements/TestMetadataQueryStatements/test_describe_db_protocol__beeswax___exec_optionsync_ddl___0___batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ > https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/286/testReport/junit/metadata.test_hms_integration/TestHmsIntegrationSanity/test_sanity_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ > https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/288/testReport/junit/query_test.test_insert_parquet/TestHdfsParquetTableStatsWriter/test_write_statistics_multiple_row_groups_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__parquet_none_/ > I'll include the output of each job in a follow-on comment. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8567) Many random catalog consistency issues with catalog v2/event processor
[ https://issues.apache.org/jira/browse/IMPALA-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845078#comment-16845078 ] Tim Armstrong commented on IMPALA-8567: --- We're missing the logs, which is a major problem for debugging. > Many random catalog consistency issues with catalog v2/event processor > -- > > Key: IMPALA-8567 > URL: https://issues.apache.org/jira/browse/IMPALA-8567 > Project: IMPALA > Issue Type: Bug > Components: Frontend >Affects Versions: Impala 3.3.0 >Reporter: Tim Armstrong >Assignee: Vihang Karajgaonkar >Priority: Blocker > Labels: broken-build, catalog, flaky > > [~tlipcon] [~vihangk1] FYI. I'm not sure whether the local catalog or the > event processor is likely to blame here so I'll let you look. The general > theme is tables and databases not existing when they should. > https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/289/testReport/junit/metadata.test_refresh_partition/TestRefreshPartition/test_drop_hive_partition_and_refresh_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ > https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/267/testReport/junit/query_test.test_kudu/TestKuduOperations/test_kudu_insert_protocol__beeswax___exec_optionkudu_read_modeREAD_AT_SNAPSHOTbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ > https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/286/testReport/junit/metadata.test_metadata_query_statements/TestMetadataQueryStatements/test_describe_db_protocol__beeswax___exec_optionsync_ddl___0___batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ > https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/286/testReport/junit/metadata.test_hms_integration/TestHmsIntegrationSanity/test_sanity_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ > https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/288/testReport/junit/query_test.test_insert_parquet/TestHdfsParquetTableStatsWriter/test_write_statistics_multiple_row_groups_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__parquet_none_/ > I'll include the output of each job in a follow-on comment. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8567) Many random catalog consistency issues with catalog v2/event processor
[
https://issues.apache.org/jira/browse/IMPALA-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845073#comment-16845073
]
Tim Armstrong commented on IMPALA-8567:
---
{noformat}
metadata.test_refresh_partition.TestRefreshPartition.test_drop_hive_partition_and_refresh[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 5000, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
text/none] (from pytest)
Failing for the past 1 build (Since Failed#289 )
Took 7.3 sec.
add description
Error Message
metadata/test_refresh_partition.py:106: in test_drop_hive_partition_and_refresh
table_name) common/impala_connection.py:180: in execute return
self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:187: in execute handle =
self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:362: in __execute_query handle =
self.execute_query_async(query_string, user=user)
beeswax/impala_beeswax.py:356: in execute_query_async handle =
self.__do_rpc(lambda: self.imp_service.query(query,))
beeswax/impala_beeswax.py:516: in __do_rpc raise
ImpalaBeeswaxException(self.__build_error_message(b), b) E
ImpalaBeeswaxException: ImpalaBeeswaxException: EINNER EXCEPTION: EMESSAGE: AnalysisException: Database
does not exist: test_drop_hive_partition_and_refresh_f9f45992
Stacktrace
metadata/test_refresh_partition.py:106: in test_drop_hive_partition_and_refresh
table_name)
common/impala_connection.py:180: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:187: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:362: in __execute_query
handle = self.execute_query_async(query_string, user=user)
beeswax/impala_beeswax.py:356: in execute_query_async
handle = self.__do_rpc(lambda: self.imp_service.query(query,))
beeswax/impala_beeswax.py:516: in __do_rpc
raise ImpalaBeeswaxException(self.__build_error_message(b), b)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
EINNER EXCEPTION:
EMESSAGE: AnalysisException: Database does not exist:
test_drop_hive_partition_and_refresh_f9f45992
Standard Error
SET
client_identifier=metadata/test_refresh_partition.py::TestRefreshPartition::()::test_drop_hive_partition_and_refresh[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':5000;'disable_codegen':False;'abort_on_error':1;'exec_single_nod;
-- connecting to: localhost:21000
-- connecting to localhost:21050 with impyla
-- 2019-05-21 06:42:13,293 INFO MainThread: Closing active operation
SET
client_identifier=metadata/test_refresh_partition.py::TestRefreshPartition::()::test_drop_hive_partition_and_refresh[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':5000;'disable_codegen':False;'abort_on_error':1;'exec_single_nod;
SET sync_ddl=False;
-- executing against localhost:21000
DROP DATABASE IF EXISTS `test_drop_hive_partition_and_refresh_f9f45992` CASCADE;
-- 2019-05-21 06:42:16,981 INFO MainThread: Started query
2b400cf62bec2536:5c6787d4
SET
client_identifier=metadata/test_refresh_partition.py::TestRefreshPartition::()::test_drop_hive_partition_and_refresh[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':5000;'disable_codegen':False;'abort_on_error':1;'exec_single_nod;
SET sync_ddl=False;
-- executing against localhost:21000
CREATE DATABASE `test_drop_hive_partition_and_refresh_f9f45992`;
-- 2019-05-21 06:42:18,813 INFO MainThread: Started query
5047962ce847957a:97629321
-- 2019-05-21 06:42:18,819 INFO MainThread: Created database
"test_drop_hive_partition_and_refresh_f9f45992" for test ID
"metadata/test_refresh_partition.py::TestRefreshPartition::()::test_drop_hive_partition_and_refresh[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 5000, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
text/none]"
-- executing against localhost:21000
create table test_drop_hive_partition_and_refresh_f9f45992.partition_test_table
(x int) partitioned by (y int, z int);
{noformat}
> Many random catalog consistency issues with catalog v2/event processor
> --
>
> Key: IMPALA-8567
> URL: https://issues.apache.org/jira/browse/IMPALA-8567
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Vihang Karajgaonkar
>Priority: Blocker
> Labels: broken-build, catalog, flaky
>
[jira] [Commented] (IMPALA-8567) Many random catalog consistency issues with catalog v2/event processor
[
https://issues.apache.org/jira/browse/IMPALA-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845071#comment-16845071
]
Tim Armstrong commented on IMPALA-8567:
---
{noformat}
metadata.test_metadata_query_statements.TestMetadataQueryStatements.test_describe_db[protocol:
beeswax | exec_option: {'sync_ddl': 0, 'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
text/none] (from pytest)
Failing for the past 1 build (Since Failed#286 )
Took 1 min 25 sec.
add description
Error Message
Exception: ("DB {0} didn't show up after {1}s", 'hive_test_desc_db', 30)
Stacktrace
metadata/test_metadata_query_statements.py:175: in test_describe_db
self.wait_for_db_to_appear("hive_test_desc_db", timeout_s=30)
common/impala_test_suite.py:924: in wait_for_db_to_appear
raise Exception("DB {0} didn't show up after {1}s", db_name, timeout_s)
E Exception: ("DB {0} didn't show up after {1}s", 'hive_test_desc_db', 30)
Standard Error
SET
client_identifier=metadata/test_metadata_query_statements.py::TestMetadataQueryStatements::()::test_describe_db[protocol:beeswax|exec_option:{'sync_ddl':0;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_singl;
-- connecting to: localhost:21000
-- connecting to localhost:21050 with impyla
-- 2019-05-21 00:29:47,861 INFO MainThread: Closing active operation
-- executing against localhost:21000
use default;
-- 2019-05-21 00:29:47,875 INFO MainThread: Started query
c7445fdd14b1955e:c7932836
SET
client_identifier=metadata/test_metadata_query_statements.py::TestMetadataQueryStatements::()::test_describe_db[protocol:beeswax|exec_option:{'sync_ddl':0;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_singl;
SET sync_ddl=1;
-- executing against localhost:21000
drop database if exists `hive_test_desc_db` cascade;
-- 2019-05-21 00:29:47,883 INFO MainThread: Started query
1f4cb8bc98227065:e93afed5
-- executing against localhost:21000
use default;
-- 2019-05-21 00:29:47,888 INFO MainThread: Started query
bf4fcac2e50cf412:1d6539da
SET
client_identifier=metadata/test_metadata_query_statements.py::TestMetadataQueryStatements::()::test_describe_db[protocol:beeswax|exec_option:{'sync_ddl':0;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_singl;
SET sync_ddl=1;
-- executing against localhost:21000
drop database if exists `impala_test_desc_db1` cascade;
-- 2019-05-21 00:29:47,895 INFO MainThread: Started query
0f4f4c812e4e17fa:adddaca9
-- executing against localhost:21000
use default;
-- 2019-05-21 00:29:47,901 INFO MainThread: Started query
e0426ed1f4848374:994f4ab6
SET
client_identifier=metadata/test_metadata_query_statements.py::TestMetadataQueryStatements::()::test_describe_db[protocol:beeswax|exec_option:{'sync_ddl':0;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_singl;
SET sync_ddl=1;
-- executing against localhost:21000
drop database if exists `impala_test_desc_db2` cascade;
-- 2019-05-21 00:29:47,907 INFO MainThread: Started query
a34f886f81c23a93:eb70ba41
-- executing against localhost:21000
use default;
-- 2019-05-21 00:29:47,913 INFO MainThread: Started query
16475e090fe3171c:06911d89
SET
client_identifier=metadata/test_metadata_query_statements.py::TestMetadataQueryStatements::()::test_describe_db[protocol:beeswax|exec_option:{'sync_ddl':0;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_singl;
SET sync_ddl=1;
-- executing against localhost:21000
drop database if exists `impala_test_desc_db3` cascade;
-- 2019-05-21 00:29:47,920 INFO MainThread: Started query
3347d669173b5de8:29cfbbba
-- executing against localhost:21000
use default;
-- 2019-05-21 00:29:47,925 INFO MainThread: Started query
7b418e344ea71e92:fefa7b70
SET
client_identifier=metadata/test_metadata_query_statements.py::TestMetadataQueryStatements::()::test_describe_db[protocol:beeswax|exec_option:{'sync_ddl':0;'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_singl;
SET sync_ddl=1;
-- executing against localhost:21000
drop database if exists `impala_test_desc_db4` cascade;
-- 2019-05-21 00:29:47,932 INFO MainThread: Started query
e942a0407bf227fc:3c9681d9
-- executing against localhost:21000
create database impala_test_desc_db1;
-- 2019-05-21 00:29:55,143 INFO MainThread: Started query
8841a5e12c6dcc9f:58a7de6c
-- executing against localhost:21000
create database impala_test_desc_db2 comment 'test
[jira] [Commented] (IMPALA-8567) Many random catalog consistency issues with catalog v2/event processor
[
https://issues.apache.org/jira/browse/IMPALA-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845070#comment-16845070
]
Tim Armstrong commented on IMPALA-8567:
---
{noformat}
metadata.test_hms_integration.TestHmsIntegrationSanity.test_sanity[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 5000, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
text/none] (from pytest)
Failing for the past 1 build (Since Failed#286 )
Took 42 sec.
add description
Error Message
Exception: ("DB {0} didn't show up after {1}s", 'hms_sanity_db', 30)
Stacktrace
metadata/test_hms_integration.py:68: in test_sanity
self.wait_for_db_to_appear("hms_sanity_db", timeout_s=30)
common/impala_test_suite.py:924: in wait_for_db_to_appear
raise Exception("DB {0} didn't show up after {1}s", db_name, timeout_s)
E Exception: ("DB {0} didn't show up after {1}s", 'hms_sanity_db', 30)
Standard Error
SET
client_identifier=metadata/test_hms_integration.py::TestHmsIntegrationSanity::()::test_sanity[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':5000;'disable_codegen':False;'abort_on_error':1;'exec_single_node_rows_threshold':0}|ta;
-- connecting to: localhost:21000
-- connecting to localhost:21050 with impyla
-- 2019-05-21 00:29:04,883 INFO MainThread: Closing active operation
-- executing against localhost:21000
describe database `hms_sanity_db`;
-- executing against localhost:21000
describe database `hms_sanity_db`;
-- executing against localhost:21000
describe database `hms_sanity_db`;
-- executing against localhost:21000
describe database `hms_sanity_db`;
... repeated many times
{noformat}
> Many random catalog consistency issues with catalog v2/event processor
> --
>
> Key: IMPALA-8567
> URL: https://issues.apache.org/jira/browse/IMPALA-8567
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Vihang Karajgaonkar
>Priority: Blocker
> Labels: broken-build, catalog, flaky
>
> [~tlipcon] [~vihangk1] FYI. I'm not sure whether the local catalog or the
> event processor is likely to blame here so I'll let you look. The general
> theme is tables and databases not existing when they should.
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/289/testReport/junit/metadata.test_refresh_partition/TestRefreshPartition/test_drop_hive_partition_and_refresh_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/267/testReport/junit/query_test.test_kudu/TestKuduOperations/test_kudu_insert_protocol__beeswax___exec_optionkudu_read_modeREAD_AT_SNAPSHOTbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/286/testReport/junit/metadata.test_metadata_query_statements/TestMetadataQueryStatements/test_describe_db_protocol__beeswax___exec_optionsync_ddl___0___batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/286/testReport/junit/metadata.test_hms_integration/TestHmsIntegrationSanity/test_sanity_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/288/testReport/junit/query_test.test_insert_parquet/TestHdfsParquetTableStatsWriter/test_write_statistics_multiple_row_groups_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__parquet_none_/
> I'll include the output of each job in a follow-on comment.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-8567) Many random catalog consistency issues with catalog v2/event processor
[
https://issues.apache.org/jira/browse/IMPALA-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845069#comment-16845069
]
Tim Armstrong edited comment on IMPALA-8567 at 5/21/19 5:37 PM:
{noformat}
query_test.test_insert_parquet.TestHdfsParquetTableStatsWriter.test_write_statistics_multiple_row_groups[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
parquet/none] (from pytest)
Failing for the past 1 build (Since Failed#288 )
Took 7.6 sec.
add description
Error Message
query_test/test_insert_parquet.py:787: in
test_write_statistics_multiple_row_groups self.execute_query(query,
vector.get_value('exec_option')) common/impala_test_suite.py:644: in wrapper
return function(*args, **kwargs) common/impala_test_suite.py:675: in
execute_query return self.__execute_query(self.client, query,
query_options) common/impala_test_suite.py:746: in __execute_query return
impalad_client.execute(query, user=user) common/impala_connection.py:180: in
execute return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:187: in execute handle =
self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:362: in __execute_query handle =
self.execute_query_async(query_string, user=user)
beeswax/impala_beeswax.py:356: in execute_query_async handle =
self.__do_rpc(lambda: self.imp_service.query(query,))
beeswax/impala_beeswax.py:516: in __do_rpc raise
ImpalaBeeswaxException(self.__build_error_message(b), b) E
ImpalaBeeswaxException: ImpalaBeeswaxException: EINNER EXCEPTION: EMESSAGE: AnalysisException: Table does
not exist:
test_write_statistics_multiple_row_groups_3bed993c.test_hdfs_parquet_table_writer
Stacktrace
query_test/test_insert_parquet.py:787: in
test_write_statistics_multiple_row_groups
self.execute_query(query, vector.get_value('exec_option'))
common/impala_test_suite.py:644: in wrapper
return function(*args, **kwargs)
common/impala_test_suite.py:675: in execute_query
return self.__execute_query(self.client, query, query_options)
common/impala_test_suite.py:746: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:180: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:187: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:362: in __execute_query
handle = self.execute_query_async(query_string, user=user)
beeswax/impala_beeswax.py:356: in execute_query_async
handle = self.__do_rpc(lambda: self.imp_service.query(query,))
beeswax/impala_beeswax.py:516: in __do_rpc
raise ImpalaBeeswaxException(self.__build_error_message(b), b)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
EINNER EXCEPTION:
EMESSAGE: AnalysisException: Table does not exist:
test_write_statistics_multiple_row_groups_3bed993c.test_hdfs_parquet_table_writer
Standard Error
SET
client_identifier=query_test/test_insert_parquet.py::TestHdfsParquetTableStatsWriter::()::test_write_statistics_multiple_row_groups[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exe;
SET sync_ddl=False;
-- executing against localhost:21000
DROP DATABASE IF EXISTS `test_write_statistics_multiple_row_groups_3bed993c`
CASCADE;
-- 2019-05-21 02:25:33,533 INFO MainThread: Started query
ae4e9c0107c68e9e:6c0d8b41
SET
client_identifier=query_test/test_insert_parquet.py::TestHdfsParquetTableStatsWriter::()::test_write_statistics_multiple_row_groups[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exe;
SET sync_ddl=False;
-- executing against localhost:21000
CREATE DATABASE `test_write_statistics_multiple_row_groups_3bed993c`;
-- 2019-05-21 02:25:34,034 INFO MainThread: Started query
f1487560c0eb8826:21fa1b96
-- 2019-05-21 02:25:34,038 INFO MainThread: Created database
"test_write_statistics_multiple_row_groups_3bed993c" for test ID
"query_test/test_insert_parquet.py::TestHdfsParquetTableStatsWriter::()::test_write_statistics_multiple_row_groups[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
parquet/none]"
SET
[jira] [Commented] (IMPALA-8567) Many random catalog consistency issues with catalog v2/event processor
[
https://issues.apache.org/jira/browse/IMPALA-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845072#comment-16845072
]
Tim Armstrong commented on IMPALA-8567:
---
{noformat}
query_test.test_kudu.TestKuduOperations.test_kudu_insert[protocol: beeswax |
exec_option: {'kudu_read_mode': 'READ_AT_SNAPSHOT', 'batch_size': 0,
'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
text/none] (from pytest)
Failing for the past 1 build (Since Failed#267 )
Took 29 sec.
add description
Error Message
query_test/test_kudu.py:97: in test_kudu_insert
self.run_test_case('QueryTest/kudu_insert', vector, use_db=unique_database)
common/impala_test_suite.py:512: in run_test_case result =
self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:746: in __execute_query return
impalad_client.execute(query, user=user) common/impala_connection.py:180: in
execute return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:187: in execute handle =
self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:362: in __execute_query handle =
self.execute_query_async(query_string, user=user)
beeswax/impala_beeswax.py:356: in execute_query_async handle =
self.__do_rpc(lambda: self.imp_service.query(query,))
beeswax/impala_beeswax.py:516: in __do_rpc raise
ImpalaBeeswaxException(self.__build_error_message(b), b) E
ImpalaBeeswaxException: ImpalaBeeswaxException: EINNER EXCEPTION: EMESSAGE: AnalysisException: Table does
not exist: test_kudu_insert_361abd7a.allkeytypes
Stacktrace
query_test/test_kudu.py:97: in test_kudu_insert
self.run_test_case('QueryTest/kudu_insert', vector, use_db=unique_database)
common/impala_test_suite.py:512: in run_test_case
result = self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:746: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:180: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:187: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:362: in __execute_query
handle = self.execute_query_async(query_string, user=user)
beeswax/impala_beeswax.py:356: in execute_query_async
handle = self.__do_rpc(lambda: self.imp_service.query(query,))
beeswax/impala_beeswax.py:516: in __do_rpc
raise ImpalaBeeswaxException(self.__build_error_message(b), b)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
EINNER EXCEPTION:
EMESSAGE: AnalysisException: Table does not exist:
test_kudu_insert_361abd7a.allkeytypes
Standard Error
SET
client_identifier=query_test/test_kudu.py::TestKuduOperations::()::test_kudu_insert[protocol:beeswax|exec_option:{'kudu_read_mode':'READ_AT_SNAPSHOT';'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_nod;
SET sync_ddl=False;
-- executing against localhost:21000
DROP DATABASE IF EXISTS `test_kudu_insert_361abd7a` CASCADE;
-- 2019-05-16 22:51:36,992 INFO MainThread: Started query
cb42abc8159a0ad5:450d1848
SET
client_identifier=query_test/test_kudu.py::TestKuduOperations::()::test_kudu_insert[protocol:beeswax|exec_option:{'kudu_read_mode':'READ_AT_SNAPSHOT';'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_nod;
SET sync_ddl=False;
-- executing against localhost:21000
CREATE DATABASE `test_kudu_insert_361abd7a`;
-- 2019-05-16 22:51:37,127 INFO MainThread: Started query
7645a05d781c8580:a89c083e
-- 2019-05-16 22:51:37,143 INFO MainThread: Created database
"test_kudu_insert_361abd7a" for test ID
"query_test/test_kudu.py::TestKuduOperations::()::test_kudu_insert[protocol:
beeswax | exec_option: {'kudu_read_mode': 'READ_AT_SNAPSHOT', 'batch_size': 0,
'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
text/none]"
SET
client_identifier=query_test/test_kudu.py::TestKuduOperations::()::test_kudu_insert[protocol:beeswax|exec_option:{'kudu_read_mode':'READ_AT_SNAPSHOT';'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exec_single_nod;
-- executing against localhost:21000
use test_kudu_insert_361abd7a;
-- 2019-05-16 22:51:37,152 INFO MainThread: Started query
6c47252392c65cd6:1ee784f6
SET
[jira] [Commented] (IMPALA-8567) Many random catalog consistency issues with catalog v2/event processor
[
https://issues.apache.org/jira/browse/IMPALA-8567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845069#comment-16845069
]
Tim Armstrong commented on IMPALA-8567:
---
query_test.test_insert_parquet.TestHdfsParquetTableStatsWriter.test_write_statistics_multiple_row_groups[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
parquet/none] (from pytest)
Failing for the past 1 build (Since Failed#288 )
Took 7.6 sec.
add description
Error Message
query_test/test_insert_parquet.py:787: in
test_write_statistics_multiple_row_groups self.execute_query(query,
vector.get_value('exec_option')) common/impala_test_suite.py:644: in wrapper
return function(*args, **kwargs) common/impala_test_suite.py:675: in
execute_query return self.__execute_query(self.client, query,
query_options) common/impala_test_suite.py:746: in __execute_query return
impalad_client.execute(query, user=user) common/impala_connection.py:180: in
execute return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:187: in execute handle =
self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:362: in __execute_query handle =
self.execute_query_async(query_string, user=user)
beeswax/impala_beeswax.py:356: in execute_query_async handle =
self.__do_rpc(lambda: self.imp_service.query(query,))
beeswax/impala_beeswax.py:516: in __do_rpc raise
ImpalaBeeswaxException(self.__build_error_message(b), b) E
ImpalaBeeswaxException: ImpalaBeeswaxException: EINNER EXCEPTION: EMESSAGE: AnalysisException: Table does
not exist:
test_write_statistics_multiple_row_groups_3bed993c.test_hdfs_parquet_table_writer
Stacktrace
query_test/test_insert_parquet.py:787: in
test_write_statistics_multiple_row_groups
self.execute_query(query, vector.get_value('exec_option'))
common/impala_test_suite.py:644: in wrapper
return function(*args, **kwargs)
common/impala_test_suite.py:675: in execute_query
return self.__execute_query(self.client, query, query_options)
common/impala_test_suite.py:746: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:180: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:187: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:362: in __execute_query
handle = self.execute_query_async(query_string, user=user)
beeswax/impala_beeswax.py:356: in execute_query_async
handle = self.__do_rpc(lambda: self.imp_service.query(query,))
beeswax/impala_beeswax.py:516: in __do_rpc
raise ImpalaBeeswaxException(self.__build_error_message(b), b)
E ImpalaBeeswaxException: ImpalaBeeswaxException:
EINNER EXCEPTION:
EMESSAGE: AnalysisException: Table does not exist:
test_write_statistics_multiple_row_groups_3bed993c.test_hdfs_parquet_table_writer
Standard Error
SET
client_identifier=query_test/test_insert_parquet.py::TestHdfsParquetTableStatsWriter::()::test_write_statistics_multiple_row_groups[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exe;
SET sync_ddl=False;
-- executing against localhost:21000
DROP DATABASE IF EXISTS `test_write_statistics_multiple_row_groups_3bed993c`
CASCADE;
-- 2019-05-21 02:25:33,533 INFO MainThread: Started query
ae4e9c0107c68e9e:6c0d8b41
SET
client_identifier=query_test/test_insert_parquet.py::TestHdfsParquetTableStatsWriter::()::test_write_statistics_multiple_row_groups[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exe;
SET sync_ddl=False;
-- executing against localhost:21000
CREATE DATABASE `test_write_statistics_multiple_row_groups_3bed993c`;
-- 2019-05-21 02:25:34,034 INFO MainThread: Started query
f1487560c0eb8826:21fa1b96
-- 2019-05-21 02:25:34,038 INFO MainThread: Created database
"test_write_statistics_multiple_row_groups_3bed993c" for test ID
"query_test/test_insert_parquet.py::TestHdfsParquetTableStatsWriter::()::test_write_statistics_multiple_row_groups[protocol:
beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0,
'disable_codegen_rows_threshold': 0, 'disable_codegen': False,
'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
parquet/none]"
SET
client_identifier=query_test/test_insert_parquet.py::TestHdfsParquetTableStatsWriter::()::test_write_statistics_multiple_row_groups[protocol:beeswax|exec_option:{'batch_size':0;'num_nodes':0;'disable_codegen_rows_threshold':0;'disable_codegen':False;'abort_on_error':1;'exe;
SET batch_size=0;
SET num_nodes=0;
SET
[jira] [Created] (IMPALA-8567) Many random catalog consistency issues with catalog v2/event processor
Tim Armstrong created IMPALA-8567: - Summary: Many random catalog consistency issues with catalog v2/event processor Key: IMPALA-8567 URL: https://issues.apache.org/jira/browse/IMPALA-8567 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 3.3.0 Reporter: Tim Armstrong Assignee: Vihang Karajgaonkar [~tlipcon] [~vihangk1] FYI. I'm not sure whether the local catalog or the event processor is likely to blame here so I'll let you look. The general theme is tables and databases not existing when they should. https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/289/testReport/junit/metadata.test_refresh_partition/TestRefreshPartition/test_drop_hive_partition_and_refresh_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/267/testReport/junit/query_test.test_kudu/TestKuduOperations/test_kudu_insert_protocol__beeswax___exec_optionkudu_read_modeREAD_AT_SNAPSHOTbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/286/testReport/junit/metadata.test_metadata_query_statements/TestMetadataQueryStatements/test_describe_db_protocol__beeswax___exec_optionsync_ddl___0___batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/286/testReport/junit/metadata.test_hms_integration/TestHmsIntegrationSanity/test_sanity_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/288/testReport/junit/query_test.test_insert_parquet/TestHdfsParquetTableStatsWriter/test_write_statistics_multiple_row_groups_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__parquet_none_/ I'll include the output of each job in a follow-on comment. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-8567) Many random catalog consistency issues with catalog v2/event processor
Tim Armstrong created IMPALA-8567: - Summary: Many random catalog consistency issues with catalog v2/event processor Key: IMPALA-8567 URL: https://issues.apache.org/jira/browse/IMPALA-8567 Project: IMPALA Issue Type: Bug Components: Frontend Affects Versions: Impala 3.3.0 Reporter: Tim Armstrong Assignee: Vihang Karajgaonkar [~tlipcon] [~vihangk1] FYI. I'm not sure whether the local catalog or the event processor is likely to blame here so I'll let you look. The general theme is tables and databases not existing when they should. https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/289/testReport/junit/metadata.test_refresh_partition/TestRefreshPartition/test_drop_hive_partition_and_refresh_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/267/testReport/junit/query_test.test_kudu/TestKuduOperations/test_kudu_insert_protocol__beeswax___exec_optionkudu_read_modeREAD_AT_SNAPSHOTbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/286/testReport/junit/metadata.test_metadata_query_statements/TestMetadataQueryStatements/test_describe_db_protocol__beeswax___exec_optionsync_ddl___0___batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/286/testReport/junit/metadata.test_hms_integration/TestHmsIntegrationSanity/test_sanity_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___5000___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__text_none_/ https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/288/testReport/junit/query_test.test_insert_parquet/TestHdfsParquetTableStatsWriter/test_write_statistics_multiple_row_groups_protocol__beeswax___exec_optionbatch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0table_format__parquet_none_/ I'll include the output of each job in a follow-on comment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8566) COMPUTE INCREMENTAL STATS sets num_nulls off-by-one
[ https://issues.apache.org/jira/browse/IMPALA-8566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved IMPALA-8566. - Resolution: Fixed Fix Version/s: Impala 3.3.0 > COMPUTE INCREMENTAL STATS sets num_nulls off-by-one > --- > > Key: IMPALA-8566 > URL: https://issues.apache.org/jira/browse/IMPALA-8566 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 3.2.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Major > Fix For: Impala 3.3.0 > > > IMPALA-7659 added the population of NULL counts while computing stats, but > this functionality isn't working properly for incremental stats. The query is > produced correctly, but the null count set in the table is one lower than it > should be. In the case that the table has no nulls, this ends up setting a > '-1' count, which is interpreted as 'unknown'. In the case that there are > nulls, we'll just be a little inaccurate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8566) COMPUTE INCREMENTAL STATS sets num_nulls off-by-one
[ https://issues.apache.org/jira/browse/IMPALA-8566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16845048#comment-16845048 ] ASF subversion and git services commented on IMPALA-8566: - Commit f9bf62eefab7fb807f4e5d6900064b612b455a5e in impala's branch refs/heads/master from Todd Lipcon [ https://gitbox.apache.org/repos/asf?p=impala.git;h=f9bf62e ] IMPALA-8566. Fix computation of num_nulls for incremental stats The calculation for num_nulls in the incremental stats code path initialized the counter to -1 instead of 0. This meant that, if there were no nulls (reasonably common), the num_nulls counter would be set to -1, indicating unknown, rather than 0. This simply fixes the initialization and updates the tests. Change-Id: Ie42103ad21d719cac45abc160c8d5422dd33fb28 Reviewed-on: http://gerrit.cloudera.org:8080/13378 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins > COMPUTE INCREMENTAL STATS sets num_nulls off-by-one > --- > > Key: IMPALA-8566 > URL: https://issues.apache.org/jira/browse/IMPALA-8566 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 3.2.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Major > > IMPALA-7659 added the population of NULL counts while computing stats, but > this functionality isn't working properly for incremental stats. The query is > produced correctly, but the null count set in the table is one lower than it > should be. In the case that the table has no nulls, this ends up setting a > '-1' count, which is interpreted as 'unknown'. In the case that there are > nulls, we'll just be a little inaccurate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8566) COMPUTE INCREMENTAL STATS sets num_nulls off-by-one
[ https://issues.apache.org/jira/browse/IMPALA-8566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved IMPALA-8566. - Resolution: Fixed Fix Version/s: Impala 3.3.0 > COMPUTE INCREMENTAL STATS sets num_nulls off-by-one > --- > > Key: IMPALA-8566 > URL: https://issues.apache.org/jira/browse/IMPALA-8566 > Project: IMPALA > Issue Type: Bug > Components: Backend >Affects Versions: Impala 3.2.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Major > Fix For: Impala 3.3.0 > > > IMPALA-7659 added the population of NULL counts while computing stats, but > this functionality isn't working properly for incremental stats. The query is > produced correctly, but the null count set in the table is one lower than it > should be. In the case that the table has no nulls, this ends up setting a > '-1' count, which is interpreted as 'unknown'. In the case that there are > nulls, we'll just be a little inaccurate. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IMPALA-7637) Include more hash table stats in profile
[ https://issues.apache.org/jira/browse/IMPALA-7637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar reassigned IMPALA-7637: Assignee: Sahil Takiar (was: Yongzhi Chen) > Include more hash table stats in profile > > > Key: IMPALA-7637 > URL: https://issues.apache.org/jira/browse/IMPALA-7637 > Project: IMPALA > Issue Type: Improvement > Components: Backend >Reporter: Tim Armstrong >Assignee: Sahil Takiar >Priority: Major > Labels: observability, ramp-up > > Our hash table collects some useful stats about collisions and travel length, > but then we don't do anything to expose them: > https://github.com/apache/impala/blob/540611e863fe99b3d3ae35f8b94a745a68b9eba2/be/src/exec/hash-table.h#L989 > We should add some of them to the profile, maybe: > * the number of probes > * the average travel length per probe > * the number of hash collisions > * (optional) the number of hash table resizes. We already have the hash table > size and the resize time, which I think is sufficient to debug most problems > with resizes. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-478) Create a Hash builtin function
[ https://issues.apache.org/jira/browse/IMPALA-478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844990#comment-16844990 ] Michael Olschimke commented on IMPALA-478: -- Hi Tim, please get in touch with me at molschi...@scalefree.com > Create a Hash builtin function > -- > > Key: IMPALA-478 > URL: https://issues.apache.org/jira/browse/IMPALA-478 > Project: IMPALA > Issue Type: Task >Affects Versions: Impala 1.0 >Reporter: Nong Li >Assignee: Matthew Jacobs >Priority: Minor > Fix For: Product Backlog > > > It would be good to add a Hash() builtin. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8544) Expose additional S3A / S3Guard metrics
[
https://issues.apache.org/jira/browse/IMPALA-8544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844719#comment-16844719
]
Steve Loughran commented on IMPALA-8544:
The trouble with thread-local tracking is that operations span threads, e.g.
writes up are done block-by-block in the thread pool, rename/copy will soon do
the same, etc. This is why the current statistics underreport, while the
aggregate value overreports on a per-query basis (see the aggregate stats in a
_SUCCESS file for a spark query as an example)
w.r.t exposing our stream statistics method, -1 as is
# it removes the option for us to change that data structure.
# the fields are all non-atomic, non-volatile values so that the cost of
incrementing them is ~0. If things are being collected, that may change.
# if people start wrapping/proxying streams, it ceases to be valid.
if you want to have some per-input stream statistics, it'd be better to have
something which all input streams can implement, so HDFS, ABFS, etc can also
implement. I'll take suggestions as to what is the best design here, given that
goal of keeping incrementing the statistics low cost.
As usual, anything which goes near the filesystem APIs will need spec updates,
tests for all the stores to implement etc. We need that to stop us breaking
your code later.
The other thing to consider is passing more of a stats context down to
read/write/copy/commit operations so that the work done across threads can be
tied back to the final operation. e.g. every async write would update some
counters which in outputstream.close() would be merged back into the classic
per-thread counters.
+[~DanielZhou] for his thoughts on ABFS stats gathering
> Expose additional S3A / S3Guard metrics
> ---
>
> Key: IMPALA-8544
> URL: https://issues.apache.org/jira/browse/IMPALA-8544
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Labels: s3
>
> S3A / S3Guard internally collects several useful metrics that we should
> consider exposing to Impala users. The full list of statistics can be found
> in {{o.a.h.fs.s3a.Statistic}}. The stats include: the number of S3 operations
> performed (put, get, etc.), invocation counts for various {{FileSystem}}
> methods, stream statistics (bytes read, written, etc.), etc.
> Some interesting stats that stand out:
> * "stream_aborted": "Count of times the TCP stream was aborted" - the number
> of TCP connection aborts, a high value would indicate performance issues
> * "stream_read_exceptions" : "Number of exceptions invoked on input streams"
> - incremented whenever an {{IOException}} is caught while reading (these
> exception don't always get propagated to Impala because they trigger a retry)
> * "store_io_throttled": "Requests throttled and retried" - looks like it
> tracks the number of times the fs retries an operation because the original
> request hit a throttling exception
> * "s3guard_metadatastore_retry": "S3Guard metadata store retry events" -
> looks like it tracks the number of times the fs retries S3Guard operations
> * "s3guard_metadatastore_throttled" : "S3Guard metadata store throttled
> events" - similar to "store_io_throttled" but looks like it is specific to
> S3Guard
> We should consider how to expose these metrics via Impala logs / runtime
> profiles.
> There are a few options:
> * {{S3AFileSystem}} exposes {{StorageStatistics}} specific to S3A / S3Guard
> via the {{FileSystem#getStorageStatistics}} method; the
> {{S3AStorageStatistics}} seems to include all the S3A / S3Guard metrics,
> however, I think the stats might be aggregated globally, which would make it
> hard to create per-query specific metrics
> * {{S3AInstrumentation}} exposes all the metrics as well, and looks like it
> is per-fs instance, so it is not aggregated globally; {{S3AInstrumentation}}
> extends {{o.a.h.metrics2.MetricsSource}} so perhaps it is exposed via some
> API (haven't looked into this yet)
> * {{S3AInputStream#toString}} dumps the statistics from
> {{o.a.h.fs.s3a.S3AInstrumentation.InputStreamStatistics}} and
> {{S3AFileSystem#toString}} dumps them all as well
> * {{S3AFileSystem}} updates the stats in
> {{o.a.h.fs.Statistics.StatisticsData}} as well (e.g. bytesRead, bytesWritten,
> etc.)
> Impala has a {{hdfs-fs-cache}} as well, so {{hdfsFs}} objects get shared
> across threads.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8553) Several tests failing with connection errors on deployed clusters
[
https://issues.apache.org/jira/browse/IMPALA-8553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-8553:
--
Target Version: Impala 3.3.0
> Several tests failing with connection errors on deployed clusters
> -
>
> Key: IMPALA-8553
> URL: https://issues.apache.org/jira/browse/IMPALA-8553
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Affects Versions: Impala 3.3.0
>Reporter: David Knupp
>Assignee: Tim Armstrong
>Priority: Critical
>
> The errors look fairly similar. I suspect this commit introduced a regression:
> https://github.com/apache/impala/commit/79c5f875
> *Stacktrace*
> {noformat}
> metadata/test_hms_integration.py:66: in test_sanity
> if IMPALA_TEST_CLUSTER_PROPERTIES.is_catalog_v2_cluster():
> common/environ.py:307: in is_catalog_v2_cluster
> flags = self._get_flags_from_web_ui(web_ui_url)
> common/environ.py:295: in _get_flags_from_web_ui
> response = requests.get(impala_url + "/varz?json")
> ../infra/python/env/lib/python2.7/site-packages/requests/api.py:69: in get
> return request('get', url, params=params, **kwargs)
> ../infra/python/env/lib/python2.7/site-packages/requests/api.py:50: in request
> response = session.request(method=method, url=url, **kwargs)
> ../infra/python/env/lib/python2.7/site-packages/requests/sessions.py:465: in
> request
> resp = self.send(prep, **send_kwargs)
> ../infra/python/env/lib/python2.7/site-packages/requests/sessions.py:573: in
> send
> r = adapter.send(request, **kwargs)
> ../infra/python/env/lib/python2.7/site-packages/requests/adapters.py:415: in
> send
> raise ConnectionError(err, request=request)
> E ConnectionError: ('Connection aborted.', error(111, 'Connection refused'))
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-8565) Test failure in TestAdmissionControllerStress.test_mem_limit
[
https://issues.apache.org/jira/browse/IMPALA-8565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong updated IMPALA-8565:
--
Target Version: Impala 3.3.0
> Test failure in TestAdmissionControllerStress.test_mem_limit
>
>
> Key: IMPALA-8565
> URL: https://issues.apache.org/jira/browse/IMPALA-8565
> Project: IMPALA
> Issue Type: Bug
>Reporter: David Rorke
>Assignee: Tim Armstrong
>Priority: Major
>
> We had a recent test failure in test_mem_limit:
> {noformat}
> custom_cluster/test_admission_controller.py:1504: in test_mem_limit
> {'request_pool': self.pool_name, 'mem_limit': query_mem_limit})
> custom_cluster/test_admission_controller.py:1365: in run_admission_test
> assert metric_deltas['dequeued'] == 0,\
> E AssertionError: Queued queries should not run until others are made to
> finish
> E assert 3 == 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-6876) Entries in CatalogUsageMonitor are not cleared after invalidation
[ https://issues.apache.org/jira/browse/IMPALA-6876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated IMPALA-6876: Labels: memory-leak ramp-up (was: memory-leak) > Entries in CatalogUsageMonitor are not cleared after invalidation > - > > Key: IMPALA-6876 > URL: https://issues.apache.org/jira/browse/IMPALA-6876 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 3.0, Impala 2.12.0 >Reporter: Dimitris Tsirogiannis >Priority: Major > Labels: memory-leak, ramp-up > > The CatalogUsageMonitor in the catalog maintains a small cache of references > to tables that: a) are accessed frequently in the catalog and b) have the > highest memory requirements. These entries are not cleared upon server or > table invalidation, thus preventing the GC from collecting the memory of > these tables. We should make sure that the CatalogUsageMonitor does not > maintain entries of tables that have been invalidated or deleted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-6876) Entries in CatalogUsageMonitor are not cleared after invalidation
[ https://issues.apache.org/jira/browse/IMPALA-6876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844538#comment-16844538 ] Todd Lipcon commented on IMPALA-6876: - Seems like the simplest fix here would be to use weak references from this data structure > Entries in CatalogUsageMonitor are not cleared after invalidation > - > > Key: IMPALA-6876 > URL: https://issues.apache.org/jira/browse/IMPALA-6876 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Affects Versions: Impala 3.0, Impala 2.12.0 >Reporter: Dimitris Tsirogiannis >Priority: Major > Labels: memory-leak, ramp-up > > The CatalogUsageMonitor in the catalog maintains a small cache of references > to tables that: a) are accessed frequently in the catalog and b) have the > highest memory requirements. These entries are not cleared upon server or > table invalidation, thus preventing the GC from collecting the memory of > these tables. We should make sure that the CatalogUsageMonitor does not > maintain entries of tables that have been invalidated or deleted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-8383) Bump toolchain version
[ https://issues.apache.org/jira/browse/IMPALA-8383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16844536#comment-16844536 ] Todd Lipcon commented on IMPALA-8383: - [~hacosta] is this still an issue? Seems it's been bumped several times since this was filed. > Bump toolchain version > -- > > Key: IMPALA-8383 > URL: https://issues.apache.org/jira/browse/IMPALA-8383 > Project: IMPALA > Issue Type: Bug >Reporter: Hector Acosta >Priority: Major > > The current $IMPALA_TOOLCHAIN_BUILD_ID has a bug where the fastbinary shared > object is missing for some distributions. We should bump the version to an id > that includes fastbinary.so. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-8529) ccache is ignored when using ninja generator
[ https://issues.apache.org/jira/browse/IMPALA-8529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved IMPALA-8529. - Resolution: Fixed Fix Version/s: Impala 3.3.0 Seems this works now after we upgraded to CMake 3.14, even without changes to CMakeFiles (at least when I run ninja -v I see ccache in use) > ccache is ignored when using ninja generator > > > Key: IMPALA-8529 > URL: https://issues.apache.org/jira/browse/IMPALA-8529 > Project: IMPALA > Issue Type: Task >Reporter: Todd Lipcon >Priority: Major > Fix For: Impala 3.3.0 > > > The CMakeLists.txt sets up ccache by using RULE_LAUNCH_PREFIX, which is only > respected by the Makefile generator. So, if we use ninja (which is generally > better at parallelism) then ccache won't kick in. Newer versions of cmake > have more explicit support for ccache that ought to also work with the ninja > generator. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (IMPALA-8529) ccache is ignored when using ninja generator
[ https://issues.apache.org/jira/browse/IMPALA-8529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved IMPALA-8529. - Resolution: Fixed Fix Version/s: Impala 3.3.0 Seems this works now after we upgraded to CMake 3.14, even without changes to CMakeFiles (at least when I run ninja -v I see ccache in use) > ccache is ignored when using ninja generator > > > Key: IMPALA-8529 > URL: https://issues.apache.org/jira/browse/IMPALA-8529 > Project: IMPALA > Issue Type: Task >Reporter: Todd Lipcon >Priority: Major > Fix For: Impala 3.3.0 > > > The CMakeLists.txt sets up ccache by using RULE_LAUNCH_PREFIX, which is only > respected by the Makefile generator. So, if we use ninja (which is generally > better at parallelism) then ccache won't kick in. Newer versions of cmake > have more explicit support for ccache that ought to also work with the ninja > generator. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
