[jira] [Resolved] (IMPALA-8686) container entrypoint should exec binary
[ https://issues.apache.org/jira/browse/IMPALA-8686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tim Armstrong resolved IMPALA-8686. --- Resolution: Fixed Fix Version/s: Impala 3.3.0 > container entrypoint should exec binary > --- > > Key: IMPALA-8686 > URL: https://issues.apache.org/jira/browse/IMPALA-8686 > Project: IMPALA > Issue Type: Sub-task > Components: Infrastructure >Affects Versions: Impala 3.3.0 >Reporter: Tim Armstrong >Assignee: Tim Armstrong >Priority: Major > Labels: docker > Fix For: Impala 3.3.0 > > > Currently the daemon_entrypoint forks the daemon process. It should really > exec it instead. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (IMPALA-8766) Change hadoop cloud dependencies to use hadoop-cloud-storage
Joe McDonnell created IMPALA-8766: - Summary: Change hadoop cloud dependencies to use hadoop-cloud-storage Key: IMPALA-8766 URL: https://issues.apache.org/jira/browse/IMPALA-8766 Project: IMPALA Issue Type: Improvement Components: Infrastructure Affects Versions: Impala 3.3.0 Reporter: Joe McDonnell Assignee: Joe McDonnell Currently, fe/pom.xml specifically includes hadoop-aws, hadoop-azure, and hadoop-azure-datalake directly. There is a meta-package in hadoop called hadoop-cloud-storage that includes these dependencies and others as customized by the hadoop provider, with appropriate exclusions applied to each package. Migrating Impala to use this meta-package would make it easier for different providers of hadoop to customize hadoop-cloud-storage and the resulting CLASSPATH without needing to change Impala. For example, a hadoop provider may want to include Apache Knox for cloud identity management. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Resolved] (IMPALA-8439) Add Hive ACID tables during dataload if Hive 3.1 is enabled
[ https://issues.apache.org/jira/browse/IMPALA-8439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongzhi Chen resolved IMPALA-8439. -- Resolution: Fixed Fix Version/s: Impala 3.3.0 > Add Hive ACID tables during dataload if Hive 3.1 is enabled > --- > > Key: IMPALA-8439 > URL: https://issues.apache.org/jira/browse/IMPALA-8439 > Project: IMPALA > Issue Type: Story > Components: Infrastructure >Affects Versions: Impala 3.2.0 >Reporter: Csaba Ringhofer >Assignee: Yongzhi Chen >Priority: Critical > Labels: impala-acid > Fix For: Impala 3.3.0 > > > Test warehouse should include a few transactional tables (insert-only, not > insert-only, partitioned, not partitioned, bucketed, not-bucketed, compacted > and uncompacted) to enable the testing of ACID features. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Resolved] (IMPALA-8486) test_udf_update_via_drop and test_udf_update_via_create fail on local catalog
[
https://issues.apache.org/jira/browse/IMPALA-8486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang resolved IMPALA-8486.
Resolution: Fixed
Fix Version/s: Impala 3.3.0
> test_udf_update_via_drop and test_udf_update_via_create fail on local catalog
> -
>
> Key: IMPALA-8486
> URL: https://issues.apache.org/jira/browse/IMPALA-8486
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
>Affects Versions: Impala 3.3.0
>Reporter: Tim Armstrong
>Assignee: Quanlong Huang
>Priority: Critical
> Labels: catalog-v2
> Fix For: Impala 3.3.0
>
>
> {noformat}
> TestUdfTargeted.test_udf_update_via_drop[protocol: beeswax | exec_option:
> {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0,
> 'disable_codegen': False, 'abort_on_error': 1,
> 'exec_single_node_rows_threshold': 0} | table_format: text/none]
> tests/query_test/test_udfs.py:541: in test_udf_update_via_drop
> self._run_query_all_impalads(exec_options, query_stmt, ["New UDF"])
> tests/query_test/test_udfs.py:52: in _run_query_all_impalads
> assert result.data == expected
> E assert ['Old UDF'] == ['New UDF']
> E At index 0 diff: 'Old UDF' != 'New UDF'
> E Full diff:
> E - ['Old UDF']
> E + ['New UDF']
>
> {noformat}
> The tests are checking that the local UDF caches on each impalad get
> invalidated by a drop/create of a function referencing the HDFS file
> containing the UDF. The test fails because the local catalog, unlike the
> regular catalog, doesn't invalidate LibCache entries upon receiving a catalog
> update.
> I looked at this for long enough to realise that the invalidation mechanism
> is fundamentally broken - it doesn't work with dedicated executors. It also
> creates a race between the statestore updates and queries referencing the
> UDFs - if the queries win the race, then they can incorrectly use the old
> version that should have been invalidated.
> I think this is a potentially problematic issue because old JAR/SO versions
> could persist in the cache indefinitely if old versions are overwritten in
> place.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Resolved] (IMPALA-8758) Misleading error message "Unknown executor group" during cluster startup with dedicated coordinator
[
https://issues.apache.org/jira/browse/IMPALA-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Volker resolved IMPALA-8758.
-
Resolution: Fixed
Assignee: Lars Volker
Fix Version/s: Impala 3.3.0
> Misleading error message "Unknown executor group" during cluster startup with
> dedicated coordinator
> ---
>
> Key: IMPALA-8758
> URL: https://issues.apache.org/jira/browse/IMPALA-8758
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
>Affects Versions: Impala 3.3.0
>Reporter: Lars Volker
>Assignee: Lars Volker
>Priority: Major
> Labels: cluster-membership, dedicated-coordinator, ramp-up,
> scheduler
> Fix For: Impala 3.3.0
>
>
> During cluster startup the Scheduler will return an error until the local
> backend has started up ("Local backend has not been registered in the cluster
> membership"). Afterwards, it will assume that the default executor group
> exists. However, if the coordinator is not also an executor (i.e. it is a
> dedicated coordinator), then it will not actually create the default executor
> group in cluster-membership-mgr.cc:256.
> Queries are expected to fail in this scenario, but the error message should
> certainly be improved to indicate that no executors could be found. For this
> purpose, we should make sure that the default executor group gets created as
> soon as the local backend has started, but keep it empty if it is not an
> executor. Then we can warn in the scheduler that no executors have been
> registered so far.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (IMPALA-8767) Jenkins failures due to "Could not get lock /var/lib/dpkg/lock"
Sahil Takiar created IMPALA-8767: Summary: Jenkins failures due to "Could not get lock /var/lib/dpkg/lock" Key: IMPALA-8767 URL: https://issues.apache.org/jira/browse/IMPALA-8767 Project: IMPALA Issue Type: Bug Reporter: Sahil Takiar -- This message was sent by Atlassian JIRA (v7.6.14#76016)
