[jira] [Updated] (IMPALA-12827) Precondition was hit in MutableValidReaderWriteIdList
[
https://issues.apache.org/jira/browse/IMPALA-12827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naveen Gangam updated IMPALA-12827:
---
Component/s: Catalog
> Precondition was hit in MutableValidReaderWriteIdList
> -
>
> Key: IMPALA-12827
> URL: https://issues.apache.org/jira/browse/IMPALA-12827
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
>Reporter: Csaba Ringhofer
>Assignee: Quanlong Huang
>Priority: Critical
> Labels: ACID, catalog
> Fix For: Impala 4.4.0
>
>
> The callstack below led to stopping metastore event processor during an abort
> transaction event:
> {code}
> MetastoreEventsProcessor.java:899] Unexpected exception received while
> processing event
> Java exception follows:
> java.lang.IllegalStateException
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:486)
> at
> org.apache.impala.hive.common.MutableValidReaderWriteIdList.addAbortedWriteIds(MutableValidReaderWriteIdList.java:274)
> at org.apache.impala.catalog.HdfsTable.addWriteIds(HdfsTable.java:3101)
> at
> org.apache.impala.catalog.CatalogServiceCatalog.addWriteIdsToTable(CatalogServiceCatalog.java:3885)
> at
> org.apache.impala.catalog.events.MetastoreEvents$AbortTxnEvent.addAbortedWriteIdsToTables(MetastoreEvents.java:2775)
> at
> org.apache.impala.catalog.events.MetastoreEvents$AbortTxnEvent.process(MetastoreEvents.java:2761)
> at
> org.apache.impala.catalog.events.MetastoreEvents$MetastoreEvent.processIfEnabled(MetastoreEvents.java:522)
> at
> org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:1052)
> at
> org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:881)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
> {code}
> Precondition:
> https://github.com/apache/impala/blob/2f14fd29c0b47fc2c170a7f0eb1cecaf6b9704f4/fe/src/main/java/org/apache/impala/hive/common/MutableValidReaderWriteIdList.java#L274
> I was not able to reproduce this so far.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-9093) Fix ACID upgrade tests to account for HIVE-22158 (table translation)
[
https://issues.apache.org/jira/browse/IMPALA-9093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961131#comment-16961131
]
Naveen Gangam commented on IMPALA-9093:
---
[~csringhofer]
#1 is expected as designed. Going forward, MANAGED tables are by definition
ACID-only. So the current/old non-ACID managed tables will be external tables
with purging=on (aka table data gets deleted when the table is dropped unlike a
true "EXTERNAL" table.
#2 You can upgrade a non-ACID table (becomes EXTERNAL with these changes) to
ACID table via an alter table command that sets location, transactional
properties and EXTERNAL=false on the table.
#3 I am not sure I understand the scenario here. Can you give me an example? An
older HMS client (Hive or Impala) will not have the capabilities set on the
client. So the behavior could be different depending on the action.
Hope this helps. Thanks
> Fix ACID upgrade tests to account for HIVE-22158 (table translation)
>
>
> Key: IMPALA-9093
> URL: https://issues.apache.org/jira/browse/IMPALA-9093
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.4.0
>Reporter: Joe McDonnell
>Assignee: Gabor Kaszab
>Priority: Blocker
>
> There are tests that create a normal managed table and upgrade that table to
> a transactional table. For example, in test_acid.py, we run acid.test, which
> has:
> {noformat}
> create table upgraded_table (x int);
> insert into upgraded_table values (1);
> # Upgrade to the table to insert only acid when there are already values in
> it.
> alter table upgraded_table set tblproperties
> ('transactional' = 'true', 'transactional_properties' = 'insert_only');
> insert into upgraded_table values (2);
> insert into upgraded_table values (3);{noformat}
> With HIVE-22158, the create table is now translated to an external table, and
> this now fails with:
> {noformat}
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> EINNER EXCEPTION:
> EMESSAGE: ImpalaRuntimeException: Error making 'alter_table' RPC to Hive
> Metastore:
> E CAUSED BY: MetaException: test_acid_basic_5d04240b.upgraded_table cannot
> be declared transactional because it's an external table{noformat}
> If external tables can't be upgraded and all managed tables are now external,
> then this test case is invalid and can be removed. We should make sure that
> this is how it is supposed to work.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7734) Catalog memz page shows useless memory breakdown
[
https://issues.apache.org/jira/browse/IMPALA-7734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16839686#comment-16839686
]
Naveen Gangam commented on IMPALA-7734:
---
[~bikramjeet.vig] Sorry, this was meant to be a rampup jira for me in Impala.
But I have been re-assigned back to hive. If there are any takers, could you
please re-assign it? Thanks
> Catalog memz page shows useless memory breakdown
>
>
> Key: IMPALA-7734
> URL: https://issues.apache.org/jira/browse/IMPALA-7734
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.0, Impala 2.12.0, Impala 3.1.0
>Reporter: Tim Armstrong
>Priority: Minor
> Labels: newbie, observability, ramp-up, supportability
>
> If you look at catalogd memz, e.g. at localhost:25020/memz, it has a bogus
> breakdown. The catalogd does not use the MemTracker infrastructure since the
> vast majority of it's memory consumption is in the JVM.
> {noformat}
> Breakdown
> : Total=0 Peak=0
> Untracked Memory: Total=0
> {noformat}
> Reported by [~alanj_impala_5a78]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Commented] (IMPALA-7734) Catalog memz page shows useless memory breakdown
[
https://issues.apache.org/jira/browse/IMPALA-7734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16830386#comment-16830386
]
Naveen Gangam commented on IMPALA-7734:
---
[~ychena] Do you have to take a look at this? Thanks
> Catalog memz page shows useless memory breakdown
>
>
> Key: IMPALA-7734
> URL: https://issues.apache.org/jira/browse/IMPALA-7734
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.0, Impala 2.12.0, Impala 3.1.0
>Reporter: Tim Armstrong
>Priority: Minor
> Labels: newbie, observability, ramp-up, supportability
>
> If you look at catalogd memz, e.g. at localhost:25020/memz, it has a bogus
> breakdown. The catalogd does not use the MemTracker infrastructure since the
> vast majority of it's memory consumption is in the JVM.
> {noformat}
> Breakdown
> : Total=0 Peak=0
> Untracked Memory: Total=0
> {noformat}
> Reported by [~alanj_impala_5a78]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-7734) Catalog memz page shows useless memory breakdown
[
https://issues.apache.org/jira/browse/IMPALA-7734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naveen Gangam reassigned IMPALA-7734:
-
Assignee: (was: Naveen Gangam)
> Catalog memz page shows useless memory breakdown
>
>
> Key: IMPALA-7734
> URL: https://issues.apache.org/jira/browse/IMPALA-7734
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.0, Impala 2.12.0, Impala 3.1.0
>Reporter: Tim Armstrong
>Priority: Minor
> Labels: newbie, observability, ramp-up, supportability
>
> If you look at catalogd memz, e.g. at localhost:25020/memz, it has a bogus
> breakdown. The catalogd does not use the MemTracker infrastructure since the
> vast majority of it's memory consumption is in the JVM.
> {noformat}
> Breakdown
> : Total=0 Peak=0
> Untracked Memory: Total=0
> {noformat}
> Reported by [~alanj_impala_5a78]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Assigned] (IMPALA-7734) Catalog memz page shows useless memory breakdown
[
https://issues.apache.org/jira/browse/IMPALA-7734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naveen Gangam reassigned IMPALA-7734:
-
Assignee: Naveen Gangam
> Catalog memz page shows useless memory breakdown
>
>
> Key: IMPALA-7734
> URL: https://issues.apache.org/jira/browse/IMPALA-7734
> Project: IMPALA
> Issue Type: Bug
>Affects Versions: Impala 3.0, Impala 2.12.0, Impala 3.1.0
>Reporter: Tim Armstrong
>Assignee: Naveen Gangam
>Priority: Minor
> Labels: newbie, observability, ramp-up, supportability
>
> If you look at catalogd memz, e.g. at localhost:25020/memz, it has a bogus
> breakdown. The catalogd does not use the MemTracker infrastructure since the
> vast majority of it's memory consumption is in the JVM.
> {noformat}
> Breakdown
> : Total=0 Peak=0
> Untracked Memory: Total=0
> {noformat}
> Reported by [~alanj_impala_5a78]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
