[jira] [Updated] (IMPALA-10585) retry_failed_queries=true should not apply to DMLs
[
https://issues.apache.org/jira/browse/IMPALA-10585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-10585:
Priority: Major (was: Critical)
> retry_failed_queries=true should not apply to DMLs
> --
>
> Key: IMPALA-10585
> URL: https://issues.apache.org/jira/browse/IMPALA-10585
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.0.0
>Reporter: Joe McDonnell
>Priority: Major
>
> I noticed that retry_failed_queries=true will retry insert statements:
> {noformat}
> [localhost:21050] joetest> insert into retrytest select count(*) from
> functional.alltypes where bool_col = sleep(50);
> Query: insert into retrytest select count(*) from functional.alltypes where
> bool_col = sleep(50)
> Query submitted at: 2021-03-15 10:23:32 (Coordinator:
> http://joemcdonnell:25000)
> Query progress can be monitored at:
> http://joemcdonnell:25000/query_plan?query_id=5f4b8c0224faa31a:4a585cf7
> ...
> Failed due to unreachable impalad(s): joemcdonnell:27002
>
> Retried query link:
> http://joemcdonnell:25000/query_plan?query_id=824b6b103ea68ea3:bc804b4Failed
> due to unreachable impalad(s): joemcdonnell:27002
> ...
> Query has been retried using query id: 824b6b103ea68ea3:bc804b45
> Retried query link:
> http://joemcdonnell:25000/query_plan?query_id=824b6b103ea68ea3:bc804b45
> Modified 1 row(s) in 47.71s{noformat}
> I don't think this was intended to work, because
> https://issues.apache.org/jira/browse/IMPALA-9734 was closed saying that we
> don't do retries for write statements. There also aren't any tests for these
> cases.
> I think we intended to exempt DML statements from retry_failed_queries=true.
> We should implement that and add tests to make sure DMLs don't get retried.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-10585) retry_failed_queries=true should not apply to DMLs
[
https://issues.apache.org/jira/browse/IMPALA-10585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-10585:
Priority: Critical (was: Major)
> retry_failed_queries=true should not apply to DMLs
> --
>
> Key: IMPALA-10585
> URL: https://issues.apache.org/jira/browse/IMPALA-10585
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 4.0.0
>Reporter: Joe McDonnell
>Priority: Critical
>
> I noticed that retry_failed_queries=true will retry insert statements:
> {noformat}
> [localhost:21050] joetest> insert into retrytest select count(*) from
> functional.alltypes where bool_col = sleep(50);
> Query: insert into retrytest select count(*) from functional.alltypes where
> bool_col = sleep(50)
> Query submitted at: 2021-03-15 10:23:32 (Coordinator:
> http://joemcdonnell:25000)
> Query progress can be monitored at:
> http://joemcdonnell:25000/query_plan?query_id=5f4b8c0224faa31a:4a585cf7
> ...
> Failed due to unreachable impalad(s): joemcdonnell:27002
>
> Retried query link:
> http://joemcdonnell:25000/query_plan?query_id=824b6b103ea68ea3:bc804b4Failed
> due to unreachable impalad(s): joemcdonnell:27002
> ...
> Query has been retried using query id: 824b6b103ea68ea3:bc804b45
> Retried query link:
> http://joemcdonnell:25000/query_plan?query_id=824b6b103ea68ea3:bc804b45
> Modified 1 row(s) in 47.71s{noformat}
> I don't think this was intended to work, because
> https://issues.apache.org/jira/browse/IMPALA-9734 was closed saying that we
> don't do retries for write statements. There also aren't any tests for these
> cases.
> I think we intended to exempt DML statements from retry_failed_queries=true.
> We should implement that and add tests to make sure DMLs don't get retried.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Created] (IMPALA-12189) updateCatalog not releasing the catalog lock if createTblTransaction() throws exceptions
Quanlong Huang created IMPALA-12189:
---
Summary: updateCatalog not releasing the catalog lock if
createTblTransaction() throws exceptions
Key: IMPALA-12189
URL: https://issues.apache.org/jira/browse/IMPALA-12189
Project: IMPALA
Issue Type: Bug
Components: Catalog
Reporter: Quanlong Huang
Assignee: Quanlong Huang
We saw an issue that catalogd can't finish RPC requests after this error:
{code:java}
I0605 21:04:49.356642 6145 jni-util.cc:288]
org.apache.impala.common.TransactionException: Internal error processing
allocate_table_write_ids
at
org.apache.impala.catalog.Hive3MetastoreShimBase.allocateTableWriteId(Hive3MetastoreShimBase.java:763)
at
org.apache.impala.catalog.Hive3MetastoreShimBase.createTblTransaction(Hive3MetastoreShimBase.java:129)
at
org.apache.impala.service.CatalogOpExecutor.updateCatalog(CatalogOpExecutor.java:6394)
at
org.apache.impala.service.JniCatalog.updateCatalog(JniCatalog.java:507)
I0605 21:04:49.356665 6145 status.cc:129] TransactionException: Internal error
processing allocate_table_write_ids
{code}
Code snipper of the downstream branch:
{code:java}
6370 public TUpdateCatalogResponse updateCatalog(TUpdateCatalogRequest update)
6371 throws ImpalaException {
6372 TUpdateCatalogResponse response = new TUpdateCatalogResponse();
6373 // Only update metastore for Hdfs tables.
6374 Table table = getExistingTable(update.getDb_name(),
update.getTarget_table(),
6375 "Load for INSERT");
6376 if (!(table instanceof FeFsTable)) {
6377 throw new InternalException("Unexpected table type: " +
6378 update.getTarget_table());
6379 }
6380
6381 tryWriteLock(table, "updating the catalog");
6382 final Timer.Context context
6383 =
table.getMetrics().getTimer(HdfsTable.CATALOG_UPDATE_DURATION_METRIC).time();
6384
6385 long transactionId = -1;
6386 TblTransaction tblTxn = null;
6387 if (update.isSetTransaction_id()) {
6388 transactionId = update.getTransaction_id();
6389 Preconditions.checkState(transactionId > 0);
6390 try (MetaStoreClient msClient = catalog_.getMetaStoreClient()) {
6391 // Setup transactional parameters needed to do alter
table/partitions later.
6392 // TODO: Could be optimized to possibly save some RPCs, as these
parameters are
6393 // not always needed + the writeId of the INSERT could be
probably reused.
6394 tblTxn = MetastoreShim.createTblTransaction(
6395 msClient.getHiveClient(), table.getMetaStoreTable(),
transactionId);
6396 }
6397 }
6398
6399 try {
6400 // Get new catalog version for table in insert.
6401 long newCatalogVersion = catalog_.incrementAndGetCatalogVersion();
6402 catalog_.getLock().writeLock().unlock();
...
6617 } finally {
6618 context.stop();
6619 UnlockWriteLockIfErronouslyLocked();
6620 table.releaseWriteLock();
6621 }
{code}
The catalog lock (versionLock) is acquired at line 6381 if the current thread
get the table lock. In normal workload, it will be released at line 6402.
However, if MetastoreShim.createTblTransaction() throws exceptions, there are
no place to release the lock. Note that there is a finally-clause at line 6619
that can release the lock. But it's not guarding the code that calls
createTblTransaction().
If the write lock of versionLock is not released, other threads can't proceed
in their catalog operations, including table loading and the event-processor.
I'm able to reproduce the issue by modifying the code to explicitly throws an
exception at
[https://github.com/apache/impala/blob/4cf0bfa83f9641eb95d83c76af7962e6a3f1e064/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L6636]
CC [~csringhofer] [~gfurnstahl]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-532) Impala should tolerate bad locale settings.
[
https://issues.apache.org/jira/browse/IMPALA-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729871#comment-17729871
]
Quanlong Huang commented on IMPALA-532:
---
We now print the locale at startup:
{code:cpp}
LOG(INFO) << "Using locale: " << std::locale("").name(); {code}
[https://github.com/apache/impala/blob/4cf0bfa83f9641eb95d83c76af7962e6a3f1e064/be/src/common/init.cc#L497]
We can capture the error there and print the effective locale name with a
meaningful exit message. Admins then can check whether the specific locale is
missing on the node and install it if missing.
> Impala should tolerate bad locale settings.
> ---
>
> Key: IMPALA-532
> URL: https://issues.apache.org/jira/browse/IMPALA-532
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 1.1
>Reporter: Ishaan Joshi
>Assignee: Pranav Yogi Lodha
>Priority: Major
> Labels: newbie, ramp-up, supportability
>
> Currently, the Statestore does not tolerate a bad locale setting and crashes
> while starting up.
> {code}
> USE_DEBUG_BUILD=false
> + perl -pi -e
> 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/2469-impala-STATESTORE#g'
>
> /var/run/cloudera-scm-agent/process/2469-impala-STATESTORE/impala-conf/state_store_flags
> perl: warning: Setting locale failed.
> perl: warning: Please check that your locale settings:
> LANGUAGE = (unset),
> LC_ALL = (unset),
> LANG = "fr_FR.UTF-8"
> are supported and installed on your system.
> perl: warning: Falling back to the standard locale ("C").
> + '[' -f
> /var/run/cloudera-scm-agent/process/2469-impala-STATESTORE/impala-conf/.htpasswd
> ']'
> + chmod 600
> /var/run/cloudera-scm-agent/process/2469-impala-STATESTORE/impala-conf/.htpasswd
> + false
> + export
> IMPALA_BIN=/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/sbin-retail
> + IMPALA_BIN=/opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/sbin-retail
> + '[' impalad = statestore ']'
> + '[' statestore = statestore ']'
> + exec
> /opt/cloudera/parcels/IMPALA-1.1-1.p0.8/lib/impala/../../bin/statestored
> --flagfile=/var/run/cloudera-scm-agent/process/2469-impala-STATESTORE/impala-conf/state_store_flags
> terminate called after throwing an instance of 'std::runtime_error'
> what(): locale::facet::_S_create_c_locale name not valid
> {code}
> It should fall back to the standard locale ("C"), if the user's locale is
> messed up.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Resolved] (IMPALA-12059) Make Thrift compilation stop spewing warnings
[ https://issues.apache.org/jira/browse/IMPALA-12059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe McDonnell resolved IMPALA-12059. Fix Version/s: Impala 4.3.0 Assignee: Joe McDonnell Resolution: Fixed > Make Thrift compilation stop spewing warnings > - > > Key: IMPALA-12059 > URL: https://issues.apache.org/jira/browse/IMPALA-12059 > Project: IMPALA > Issue Type: Improvement > Components: Infrastructure >Affects Versions: Impala 4.3.0 >Reporter: Joe McDonnell >Assignee: Joe McDonnell >Priority: Major > Fix For: Impala 4.3.0 > > > Thrift compilation produces enormous output due to warnings that are repeated > countless times. Most of the warnings are worthless, so we should just filter > them out rather than printing them to the console. There are a mixture of > warnings that we have no intention of addressing and warnings that come from > non-Impala thrift files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-12185) bootstrap_system.sh fails to install snappy-devel on Redhat 8.4
[
https://issues.apache.org/jira/browse/IMPALA-12185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Gaal reassigned IMPALA-12185:
Assignee: Laszlo Gaal
> bootstrap_system.sh fails to install snappy-devel on Redhat 8.4
> ---
>
> Key: IMPALA-12185
> URL: https://issues.apache.org/jira/browse/IMPALA-12185
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Reporter: Quanlong Huang
>Assignee: Laszlo Gaal
>Priority: Major
>
> Saw bin/bootstrap_system.sh fail in installing snappy-devel:
> {code:java}
> + sudo yum install -y '--enablerepo=[Pp]ower[Tt]ools*' snappy-devel
> Error: Unknown repo: '[Pp]ower[Tt]ools*'{code}
> I'm able to proceed after removing the "enablerepo" option.
> OS info:
> {noformat}
> # cat /etc/os-release
> NAME="Red Hat Enterprise Linux"
> VERSION="8.4 (Ootpa)"
> ID="rhel"
> ID_LIKE="fedora"
> VERSION_ID="8.4"
> PLATFORM_ID="platform:el8"
> PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
> ANSI_COLOR="0;31"
> CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
> HOME_URL="https://www.redhat.com/";
> DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/";
> BUG_REPORT_URL="https://bugzilla.redhat.com/";
> REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
> REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
> REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
> REDHAT_SUPPORT_PRODUCT_VERSION="8.4" {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Comment Edited] (IMPALA-12185) bootstrap_system.sh fails to install snappy-devel on Redhat 8.4
[
https://issues.apache.org/jira/browse/IMPALA-12185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729669#comment-17729669
]
Laszlo Gaal edited comment on IMPALA-12185 at 6/6/23 9:18 AM:
--
Funnily enough, I bumped into the very same thing last night.
Seems like the PowerTools repo is specific to Centos, and may be specific to
older (read: pre-Stream) versions too.
In Red Hat releases {{snappy-devel}} comes from the CodeReady Linux Builder
repo, which may have to be enabled separately, as some installations come with
that repo disabled.
The magic stanza from https://stackoverflow.com/a/61584360/143681 is:{code}
$ sudo dnf config-manager --set-enabled codeready-builder-for-rhel-8-rhui-rpms
{code}
was (Author: laszlog):
Funnily enough, I bimped into the very same thing last night.
Seems like the PowerTools repo is specific to Centos, and may be specific to
older (read: pre-Stream) versions too.
In Red Hat releases {{snappy-devel}} comes from the CodeReady Linux Builder
repo, which may have to be enabled separately, as some installations come with
that repo disabled.
The magic stanza from https://stackoverflow.com/a/61584360/143681 is:{code}
$ sudo dnf config-manager --set-enabled codeready-builder-for-rhel-8-rhui-rpms
{code}
> bootstrap_system.sh fails to install snappy-devel on Redhat 8.4
> ---
>
> Key: IMPALA-12185
> URL: https://issues.apache.org/jira/browse/IMPALA-12185
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Reporter: Quanlong Huang
>Priority: Major
>
> Saw bin/bootstrap_system.sh fail in installing snappy-devel:
> {code:java}
> + sudo yum install -y '--enablerepo=[Pp]ower[Tt]ools*' snappy-devel
> Error: Unknown repo: '[Pp]ower[Tt]ools*'{code}
> I'm able to proceed after removing the "enablerepo" option.
> OS info:
> {noformat}
> # cat /etc/os-release
> NAME="Red Hat Enterprise Linux"
> VERSION="8.4 (Ootpa)"
> ID="rhel"
> ID_LIKE="fedora"
> VERSION_ID="8.4"
> PLATFORM_ID="platform:el8"
> PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
> ANSI_COLOR="0;31"
> CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
> HOME_URL="https://www.redhat.com/";
> DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/";
> BUG_REPORT_URL="https://bugzilla.redhat.com/";
> REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
> REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
> REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
> REDHAT_SUPPORT_PRODUCT_VERSION="8.4" {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-12185) bootstrap_system.sh fails to install snappy-devel on Redhat 8.4
[
https://issues.apache.org/jira/browse/IMPALA-12185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729669#comment-17729669
]
Laszlo Gaal commented on IMPALA-12185:
--
Funnily enough, I bimped into the very same thing last night.
Seems like the PowerTools repo is specific to Centos, and may be specific to
older (read: pre-Stream) versions too.
In Red Hat releases {{snappy-devel}} comes from the CodeReady Linux Builder
repo, which may have to be enabled separately, as some installations come with
that repo disabled.
The magic stanza from https://stackoverflow.com/a/61584360/143681 is:{code}
$ sudo dnf config-manager --set-enabled codeready-builder-for-rhel-8-rhui-rpms
{code}
> bootstrap_system.sh fails to install snappy-devel on Redhat 8.4
> ---
>
> Key: IMPALA-12185
> URL: https://issues.apache.org/jira/browse/IMPALA-12185
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
>Reporter: Quanlong Huang
>Priority: Major
>
> Saw bin/bootstrap_system.sh fail in installing snappy-devel:
> {code:java}
> + sudo yum install -y '--enablerepo=[Pp]ower[Tt]ools*' snappy-devel
> Error: Unknown repo: '[Pp]ower[Tt]ools*'{code}
> I'm able to proceed after removing the "enablerepo" option.
> OS info:
> {noformat}
> # cat /etc/os-release
> NAME="Red Hat Enterprise Linux"
> VERSION="8.4 (Ootpa)"
> ID="rhel"
> ID_LIKE="fedora"
> VERSION_ID="8.4"
> PLATFORM_ID="platform:el8"
> PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)"
> ANSI_COLOR="0;31"
> CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA"
> HOME_URL="https://www.redhat.com/";
> DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/";
> BUG_REPORT_URL="https://bugzilla.redhat.com/";
> REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8"
> REDHAT_BUGZILLA_PRODUCT_VERSION=8.4
> REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
> REDHAT_SUPPORT_PRODUCT_VERSION="8.4" {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
