[ https://issues.apache.org/jira/browse/KUDU-1802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17297629#comment-17297629 ]
ASF subversion and git services commented on KUDU-1802: ------------------------------------------------------- Commit 074e7e9a08d3aa392df068e23c425326161cf38b in kudu's branch refs/heads/master from Alexey Serbin [ https://gitbox.apache.org/repos/asf?p=kudu.git;h=074e7e9 ] KUDU-3254 fix bug in meta-cache exposed by KUDU-1802 This patch fixes an issue resulting in a SIGABRT crash in Kudu client when working with stale scan tokens which contain information about tablet locations for a table (see KUDU-1802) whose range partition was dropped. The patch also adds a test scenario reproducing the crash; now it passes and can catch future regressions. This patch is a follow-up to d23ee5d38ddc4317f431dd65df0c825c00cc968a. Prior the change in src/kudu/client/meta_cache.cc was back-ported from Kudu 1.14 as part of this fix, the scenario crashed with SIGABRT when running with the stack trace similar to the following (this one below was captured on macOS): * frame #0: 0x00007fff7035833a libsystem_kernel.dylib`__pthread_kill + 10 frame #1: 0x00007fff70414e60 libsystem_pthread.dylib`pthread_kill + 430 frame #2: 0x00007fff702df808 libsystem_c.dylib`abort + 120 frame #3: 0x000000010ca1a259 libglog.0.dylib`google::logging_fail() at logging.cc:1474:3 frame #4: 0x000000010ca19121 libglog.0.dylib`google::LogMessage::SendToLog() [inlined] google::LogMessage::Fail() at logging.cc:1488:3 frame #5: 0x000000010ca1911b libglog.0.dylib`google::LogMessage::SendToLog() at logging.cc:1442 frame #6: 0x000000010ca19815 libglog.0.dylib`google::LogMessage::Flush() at logging.cc:1311:5 frame #7: 0x000000010ca1d76f libglog.0.dylib`google::LogMessageFatal::~LogMessageFatal() at logging.cc:2023:5 frame #8: 0x000000010ca1a5f9 libglog.0.dylib`google::LogMessageFatal::~LogMessageFatal() at logging.cc:2022:37 frame #9: 0x0000000103e365e3 libkudu_client.dylib`std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, kudu::client::internal::MetaCacheEntry, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, kudu::client::internal::MetaCacheEntry> > >::mapped_type& FindOrDie<std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, kudu::client::internal::MetaCacheEntry, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, kudu::client::internal::MetaCacheEntry> > > >() at map-util.h:109:3 frame #10: 0x0000000103e34cbb libkudu_client.dylib`kudu::client::internal::MetaCache::ProcessGetTableLocationsResponse() at meta_cache.cc:943:23 frame #11: 0x0000000103e86166 libkudu_client.dylib`kudu::client::KuduScanToken::Data::PBIntoScanner() at scan_token-internal.cc:192:35 frame #12: 0x0000000103e88051 libkudu_client.dylib`kudu::client::KuduScanToken::Data::DeserializeIntoScanner() at scan_token-internal.cc:111:10 frame #13: 0x0000000103d55d3c libkudu_client.dylib`kudu::client::KuduScanToken::DeserializeIntoScanner() at client.cc:1879:10 Change-Id: I5b8370290c13b1e496f461ed5bc2e0193bdf4b19 Reviewed-on: http://gerrit.cloudera.org:8080/17152 Tested-by: Alexey Serbin <aser...@cloudera.com> Reviewed-by: Andrew Wong <aw...@cloudera.com> (cherry picked from commit 7c8dca60d15b560017ef7e726a379788727502ba) Conflicts: src/kudu/client/meta_cache.cc src/kudu/client/scan_token-test.cc Reviewed-on: http://gerrit.cloudera.org:8080/17158 Tested-by: Kudu Jenkins Reviewed-by: Grant Henke <granthe...@apache.org> > Deserializing scan tokens should avoid round-trip to master > ----------------------------------------------------------- > > Key: KUDU-1802 > URL: https://issues.apache.org/jira/browse/KUDU-1802 > Project: Kudu > Issue Type: Improvement > Components: client, perf > Affects Versions: 1.2.0 > Reporter: Todd Lipcon > Assignee: Grant Henke > Priority: Major > Labels: impala, ramp-up > Fix For: 1.13.0 > > > Currently, KuduScanToken::DeserializeIntoScanner calls > KuduClient::OpenTable() which makes a GetTableSchema call to the master. This > round trip is a bit expensive because it's always a "thundering herd" for an > Impala query or Spark job -- every host deserializes a bunch of scan tokens > at the same time and ends up having to back off. > We should consider some ways to avoid this. -- This message was sent by Atlassian Jira (v8.3.4#803005)