[ https://issues.apache.org/jira/browse/IMPALA-8121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16832084#comment-16832084 ]
ASF subversion and git services commented on IMPALA-8121: --------------------------------------------------------- Commit 79c5f87565467074697a7d98e01c9742f7228991 in impala's branch refs/heads/master from Tim Armstrong [ https://gitbox.apache.org/repos/asf?p=impala.git;h=79c5f87 ] IMPALA-8121: part 1: some test fixes for catalog v2 This fixes some test issues encountered when running the tests against a cluster with catalog V2 enabled, meaning the local catalog with HMS notifications enabled. More fixes are to come but I preferred to do them in smaller batches as they're ready. Test fixes: * Detect whether catalog v2 features are enabled from web UI. * test_describe_db waits for metadata event processor to pick up new database and doesn't need to change database owner * TestWebPage.test_catalog handles an expected exception from the /catalog_objects page on the impalad. * test_pull_stats_profile: feature disabled with local catalog * test_hms_service_dies: invalidate the test table instead of the whole catalog. * test_compute_stats: Avro schema resolution behaviour changed with local catalog - IMPALA-7308 Some remaining issues: * IMPALA-8458 * IMPALA-8459 * IMPALA-7131 (data sources) * getTables() doesn't return comment Change-Id: I060f2076da74fbbe92ae26dbad51f09a3bd20169 Reviewed-on: http://gerrit.cloudera.org:8080/13122 Reviewed-by: Todd Lipcon <t...@apache.org> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> > Pick better default flags in containers > --------------------------------------- > > Key: IMPALA-8121 > URL: https://issues.apache.org/jira/browse/IMPALA-8121 > Project: IMPALA > Issue Type: Sub-task > Components: Infrastructure > Reporter: Tim Armstrong > Assignee: Tim Armstrong > Priority: Major > Labels: docker > > There are some new features of Impala that are done but disabled by default > because they are not strictly better than the previous versions. E.g. the > various metadata improvements. Containerised Impala is likely to be new > deployments, so it is easier to make potentially disruptive changes to > defaults now. > h2. Metadata V2 Flags > Catalogd: > --catalog_topic_mode=minimal > Impalad: > --use_local_catalog=true > I think we *may* also want to configure automatic invalidations of tables > from the catalogd so that changes to the underlying storage cluster are > eventually reflected in the compute cluster. There's a better solution in the > pipeline that uses HMS notifications > (https://issues.apache.org/jira/browse/IMPALA-7970), but in the meantime > invalidation is time-based. > Catalogd: > --invalidate_tables_timeout_s=3600 > Once IMPALA-7970 goes in, we probably also want automatic invalidation by > default (TBD - how to handle older HMS that doesn't support those APIs). > Catalogd: > --hms_event_polling_interval_s=??? > We probably want to enable HDFS preads for remote reads: -use_hdfs_pread > We may want to have an I/O cache enabled -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org