[jira] [Created] (KUDU-2583) LeakSanitizer failure in kudu-admin-test
Mike Percy created KUDU-2583:
Summary: LeakSanitizer failure in kudu-admin-test
Key: KUDU-2583
URL: https://issues.apache.org/jira/browse/KUDU-2583
Project: Kudu
Issue Type: Improvement
Reporter: Mike Percy
Saw this error in an automated test run from kudu-admin-test in
DDLDuringRebalancingTest.TablesCreatedAndDeletedDuringRebalancing/0:
{code:java}
==27773==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 50 byte(s) in 1 object(s) allocated from:
#0 0x531928 in operator new(unsigned long)
/data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cc:92
#1 0x377b29c3c8 in std::string::_Rep::_S_create(unsigned long, unsigned long,
std::allocator const&) (/usr/lib64/libstdc++.so.6+0x377b29c3c8)
Direct leak of 40 byte(s) in 1 object(s) allocated from:
#0 0x531928 in operator new(unsigned long)
/data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-6.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cc:92
#1 0x7fe3255f5ccf in
_ZNSt14__shared_countILN9__gnu_cxx12_Lock_policyE2EEC2IN4kudu15ClosureRunnableESaIS5_EJNS4_8CallbackIFvvEESt19_Sp_make_shared_tagPT_RKT0_DpOT1_
../../../include/c++/4.9.2/bits/shared_ptr_base.h:616:25
#2 0x7fe3255f5b7a in
_ZNSt12__shared_ptrIN4kudu15ClosureRunnableELN9__gnu_cxx12_Lock_policyE2EEC2ISaIS1_EJNS0_8CallbackIFvvEESt19_Sp_make_shared_tagRKT_DpOT0_
../../../include/c++/4.9.2/bits/shared_ptr_base.h:1089:14
#3 0x7fe3255f5a5f in
_ZSt15allocate_sharedIN4kudu15ClosureRunnableESaIS1_EJNS0_8CallbackIFvvESt10shared_ptrIT_ERKT0_DpOT1_
../../../include/c++/4.9.2/bits/shared_ptr.h:587:14
#4 0x7fe3255ed9c0 in
_ZSt11make_sharedIN4kudu15ClosureRunnableEJNS0_8CallbackIFvvESt10shared_ptrIT_EDpOT0_
../../../include/c++/4.9.2/bits/shared_ptr.h:603:14
#5 0x7fe3255ea383 in kudu::ThreadPool::SubmitClosure(kudu::Callback)
/data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/threadpool.cc:450:17
#6 0x7fe32e4a42ff in kudu::log::Log::AppendThread::Wake()
/data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/consensus/log.cc:289:5
#7 0x7fe32e4af94f in
kudu::log::Log::AsyncAppend(std::unique_ptr >, kudu::Callback const&)
/data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/consensus/log.cc:602:19
#8 0x7fe32e4affbf in
kudu::log::Log::AsyncAppendReplicates(std::vector,
std::allocator > > const&,
kudu::Callback const&)
/data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/consensus/log.cc:614:10
#9 0x7fe32eb67994 in
kudu::consensus::LogCache::AppendOperations(std::vector,
std::allocator > > const&,
kudu::Callback const&)
/data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/consensus/log_cache.cc:213:29
#10 0x7fe32eb0b99e in
kudu::consensus::PeerMessageQueue::AppendOperations(std::vector,
std::allocator > > const&,
kudu::Callback const&)
/data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/consensus/consensus_queue.cc:403:3
#11 0x7fe32ebc8df0 in
kudu::consensus::RaftConsensus::UpdateReplica(kudu::consensus::ConsensusRequestPB
const*, kudu::consensus::ConsensusResponsePB*)
/data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/consensus/raft_consensus.cc:1451:7
#12 0x7fe32ebc52bf in
kudu::consensus::RaftConsensus::Update(kudu::consensus::ConsensusRequestPB
const*, kudu::consensus::ConsensusResponsePB*)
/data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/consensus/raft_consensus.cc:914:14
#13 0x7fe331bbb369 in
kudu::tserver::ConsensusServiceImpl::UpdateConsensus(kudu::consensus::ConsensusRequestPB
const*, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcContext*)
/data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/tablet_service.cc:946:25
#14 0x7fe3293f5cb9 in std::_Function_handler
const&, scoped_refptr
const&)::$_1>::_M_invoke(std::_Any_data const&, google::protobuf::Message
const*, google::protobuf::Message*, kudu::rpc::RpcContext*)
../../../include/c++/4.9.2/functional:2039:2
#15 0x7fe32841e2fb in std::function::operator()(google::protobuf::Message const*,
google::protobuf::Message*, kudu::rpc::RpcContext*) const
../../../include/c++/4.9.2/functional:2439:14
#16 0x7fe32841cd6a in
kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*)
/data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_if.cc:139:3
#17 0x7fe328420d87 in kudu::rpc::ServicePool::RunThread()
/data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/rpc/service_pool.cc:225:15
#18 0x7fe328426612 in boost::_bi::bind_t,
boost::_bi::list1 > >::operator()()
/data/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/uninstrumented/include/boost/bind/bind.hpp:1222:16
#19 0x7fe32837bf1b in boost::function0::operator()() const
[jira] [Resolved] (KUDU-2559) kudu-tool-test TestLoadgenDatabaseName fails with a memory leak
[ https://issues.apache.org/jira/browse/KUDU-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Percy resolved KUDU-2559. -- Resolution: Cannot Reproduce Fix Version/s: n/a Resolving this as cannot reproduce because there is no log file attached; Please reopen if you have the file! > kudu-tool-test TestLoadgenDatabaseName fails with a memory leak > --- > > Key: KUDU-2559 > URL: https://issues.apache.org/jira/browse/KUDU-2559 > Project: Kudu > Issue Type: Bug > Components: ksck >Reporter: Andrew Wong >Priority: Major > Fix For: n/a > > Attachments: kudu-tool-test.2.xml > > > I've attached a log with the LeakSanitizer error, though looking at the test > itself and the error, it isn't clear to me why the issue would be specific to > this test. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (KUDU-2582) Locks are acquired to cost much time in transactions
[ https://issues.apache.org/jira/browse/KUDU-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adar Dembo resolved KUDU-2582. -- Resolution: Information Provided Fix Version/s: NA This is indeed a known issue in Kudu, and one of several reasons why Kudu performs suboptimally for UPDATE-heavy workloads. In any case, Jira isn't the right medium for open-ended discussion like this; please e-mail [u...@kudu.apache.org|mailto:u...@kudu.apache.org], or join the Kudu Slack community. > Locks are acquired to cost much time in transactions > > > Key: KUDU-2582 > URL: https://issues.apache.org/jira/browse/KUDU-2582 > Project: Kudu > Issue Type: Bug >Reporter: xiaokai.wang >Priority: Major > Fix For: NA > > > Hi guys, I met a problem about the keys locks that almost impacts the service > normal writing. > As we all know, a transaction which get all row_key locks will go on next > step in kudu. Everything looks good, if keys are not concurrent updated. But > when keys are updated by more than one client at the same time, locks are > acquired to wait much time. The cases are often in my product environment. > Does anybody meet the problem? Has any good ideal for this? > Thanks. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (KUDU-2580) C++ client does not re-acquire authn token in some scenarios in case of multi-master Kudu cluster
[
https://issues.apache.org/jira/browse/KUDU-2580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Serbin updated KUDU-2580:
Description:
In case of multi-master Kudu cluster, the C++ client fails to re-acquire authn
token in the following scenario:
# Client is running against a multi-master cluster.
# Client successfully authenticates and gets an authn token by calling
ConnectToCluster.
# Client keeps the connection to leader master open, but follower masters close
connections to the client due to inactivity.
# After the authn token expires, a change in the master leadership happens.
# Client tries to open a table, first making a request to the former leader
master. However, the former leader returns NOT_THE_LEADER error.
Eventually, the latter operation fails out with the following error:
{{Timed out: GetTableSchema timed out after deadline expired}}
In the case of that error, along with the message above, the client is also
logging messages like:
{{Unable to determine the new leader Master: Not authorized: Client connection
negotiation failed: client connection to IP:port:
FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: authentication token
expired}}
was:
In case of multi-master Kudu cluster, the C++ client fails to re-acquire authn
token in the following scenario:
# Client is running against a multi-master cluster.
# Client successfully authenticates and gets an authn token by calling
ConnectToCluster.
# Client keeps the connection to leader master open, but follower masters close
connections to the client due to inactivity.
# After the authn token expires, a change in the master leadership happens.
# Client tries to open a table, first making a request to the former leader
master. However, the former leader returns NOT_THE_LEADER error.
Eventually, the latter operation fails out with the following error:
{{Timed out: GetTableSchema timed out after deadline expired}}
> C++ client does not re-acquire authn token in some scenarios in case of
> multi-master Kudu cluster
> -
>
> Key: KUDU-2580
> URL: https://issues.apache.org/jira/browse/KUDU-2580
> Project: Kudu
> Issue Type: Bug
> Components: master
>Affects Versions: 1.4.0, 1.5.0, 1.6.0, 1.7.0, 1.7.1
>Reporter: Alexey Serbin
>Assignee: Alexey Serbin
>Priority: Major
>
> In case of multi-master Kudu cluster, the C++ client fails to re-acquire
> authn token in the following scenario:
> # Client is running against a multi-master cluster.
> # Client successfully authenticates and gets an authn token by calling
> ConnectToCluster.
> # Client keeps the connection to leader master open, but follower masters
> close connections to the client due to inactivity.
> # After the authn token expires, a change in the master leadership happens.
> # Client tries to open a table, first making a request to the former leader
> master. However, the former leader returns NOT_THE_LEADER error.
> Eventually, the latter operation fails out with the following error:
> {{Timed out: GetTableSchema timed out after deadline expired}}
> In the case of that error, along with the message above, the client is also
> logging messages like:
> {{Unable to determine the new leader Master: Not authorized: Client
> connection negotiation failed: client connection to IP:port:
> FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: authentication token
> expired}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (KUDU-2582) Locks are acquired to cost much time in transactions
[ https://issues.apache.org/jira/browse/KUDU-2582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xiaokai.wang updated KUDU-2582: --- Summary: Locks are acquired to cost much time in transactions (was: Lock are acquired to cost much time in transactions) > Locks are acquired to cost much time in transactions > > > Key: KUDU-2582 > URL: https://issues.apache.org/jira/browse/KUDU-2582 > Project: Kudu > Issue Type: Bug >Reporter: xiaokai.wang >Priority: Major > > Hi guys, I met a problem about the keys locks that almost impacts the service > normal writing. > As we all know, a transaction which get all row_key locks will go on next > step in kudu. Everything looks good, if keys are not concurrent updated. But > when keys are updated by more than one client at the same time, locks are > acquired to wait much time. The cases are often in my product environment. > Does anybody meet the problem? Has any good ideal for this? > Thanks. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KUDU-2582) Lock are acquired to cost much time in transactions
xiaokai.wang created KUDU-2582: -- Summary: Lock are acquired to cost much time in transactions Key: KUDU-2582 URL: https://issues.apache.org/jira/browse/KUDU-2582 Project: Kudu Issue Type: Bug Reporter: xiaokai.wang Hi guys, I met a problem about the keys locks that almost impacts the service normal writing. As we all know, a transaction which get all row_key locks will go on next step in kudu. Everything looks good, if keys are not concurrent updated. But when keys are updated by more than one client at the same time, locks are acquired to wait much time. The cases are often in my product environment. Does anybody meet the problem? Has any good ideal for this? Thanks. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (KUDU-2581) Reject the scan request if it has reached the num or memory threshold
Hexin created KUDU-2581: --- Summary: Reject the scan request if it has reached the num or memory threshold Key: KUDU-2581 URL: https://issues.apache.org/jira/browse/KUDU-2581 Project: Kudu Issue Type: New Feature Components: tserver Affects Versions: NA Reporter: Hexin Fix For: NA Sometimes we have a lot of scan, which maybe causes the soft memory limit. We plan to achieve a method that rejecting the scan request if it has reached the soft memory limit or the scanner num has reached the threshold. The threshold is estimated by the actual memory of the system running the kudu. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
