Hi list
I've managed to trigger one of the errors that were mentioned. It
seems like all of them get triggered on filesystem write operations
for some reason.
Here's the relevant part of the stack traces from all threads
Thread 1 (Thread 0xb7f6a700 (LWP 27256)):
#0 0x00a4b402 in __kernel_vsyscall ()
#1 0x0065e472 in pthread_cond_timedwait@@GLIBC_2.3.2 () from
/lib/tls/i686/nosegneg/libpthread.so.0
---Type <return> to continue, or q <return> to quit---
#2 0x084c69c2 in
boost::condition_variable_any::timed_wait<boost::unique_lock<boost::mutex>
> (this=0x949fbe0, m...@0xbfde0930, wait_unt...@0xbfde08d0)
at /usr/local/include/boost/thread/pthread/condition_variable.hpp:110
#3 0x084c6abc in
boost::condition_variable_any::timed_wait<boost::unique_lock<boost::mutex>
> (this=0x949fbe0, m...@0xbfde0930, wait_unt...@0xbfde0924)
at /usr/local/include/boost/thread/pthread/condition_variable.hpp:127
#4 0x084c6b31 in
Hypertable::TableMutatorCompletionCounter::wait_for_completion
(this=0x949fbc8, tim...@0xbfde0b88)
at
/home/mateusz/episcale-hypertable/src/cc/Hypertable/Lib/TableMutatorCompletionCounter.h:70
#5 0x084c0b04 in
Hypertable::TableMutatorScatterBuffer::wait_for_completion
(this=0x949fb78, tim...@0xbfde0b88)
at
/home/mateusz/episcale-hypertable/src/cc/Hypertable/Lib/TableMutatorScatterBuffer.cc:262
#6 0x084ba444 in Hypertable::TableMutator::wait_for_previous_buffer
(this=0x948a9f8, tim...@0xbfde0b88) at
/home/mateusz/episcale-hypertable/src/cc/Hypertable/Lib/TableMutator.cc:283
#7 0x084bc762 in Hypertable::TableMutator::flush (this=0x948a9f8) at
/home/mateusz/episcale-hypertable/src/cc/Hypertable/Lib/TableMutator.cc:233
#8 0x08410c9c in
Hypertable::ThriftBroker::ServerHandler::flush_mutator
(this=0x93b7c38, mutator=155757048, tok...@0xbfde2fdc)
at
/home/mateusz/episcale-hypertable/src/cc/ThriftBroker/ThriftBroker.cc:1395
#9 0x084165da in
Hypertable::ThriftBroker::ServerHandler::close_mutator
(this=0x93b7c38, mutator=155757048, tok...@0xbfde2fdc, flush=true)
at
/home/mateusz/episcale-hypertable/src/cc/ThriftBroker/ThriftBroker.cc:1417
#10 0x0843cb06 in
Hypertable::ThriftGen::ClientServiceProcessor::process_close_mutator
(this=0x93b78a8, seqid=0, iprot=0x93d27a8, oprot=0x941afe8)
at
/home/mateusz/episcale-hypertable/src/cc/ThriftBroker/gen-cpp/ClientService.cpp:7153
#11 0x08431a5c in
Hypertable::ThriftGen::ClientServiceProcessor::process_fn
(this=0x93b78a8, iprot=0x93d27a8, oprot=0x941afe8, fna...@0xbfde3164,
seqid=0)
at
/home/mateusz/episcale-hypertable/src/cc/ThriftBroker/gen-cpp/ClientService.cpp:6708
#12 0x0845de17 in
Hypertable::ThriftGen::HqlServiceProcessor::process_fn
(this=0x93b78a8, iprot=0x93d27a8, oprot=0x941afe8, fna...@0xbfde3164,
seqid=0)
at
/home/mateusz/episcale-hypertable/src/cc/ThriftBroker/gen-cpp/HqlService.cpp:582
#13 0x0845e4e5 in Hypertable::ThriftGen::HqlServiceProcessor::process
(this=0x93b78a8, piprot={px = 0xbfde31c0, pn = {pi_ = 0xbfde31b8}},
poprot={px = 0xbfde31b8, pn = {pi_ = 0x1}})
at
/home/mateusz/episcale-hypertable/src/cc/ThriftBroker/gen-cpp/HqlService.cpp:575
#14 0x00576551 in apache::thrift::server::TConnection::transition
(this=0x93b9118) at src/server/TNonblockingServer.cpp:303
#15 0x00576d89 in apache::thrift::server::TConnection::workSocket
(this=0x93b9118) at src/server/TNonblockingServer.cpp:147
#16 0x0057c5c5 in apache::thrift::server::TConnection::eventHandler
(fd=13, v=0x93b9118) at src/server/TNonblockingServer.h:418
#17 0x005875dc in event_base_loop () from /usr/lib/libevent-1.3e.so.1
#18 0x0057555b in apache::thrift::server::TNonblockingServer::serve
(this=0xbfde336c) at src/server/TNonblockingServer.cpp:747
#19 0x0839e647 in main (argc=Cannot access memory at address 0x0
) at /home/mateusz/episcale-hypertable/src/cc/ThriftBroker/ThriftBroker.cc:1818
(gdb) detach
It gets triggered once, then any subsequent call afterwards works for
some time. After some period of inactivity the problem re-appears.
Mateusz
On Wed, Jun 10, 2009 at 2:04 AM, Josh Adams<[email protected]> wrote:
>
> On Tue, Jun 9, 2009 at 4:51 PM, Luke<[email protected]> wrote:
>> Is there any request timeout errors in the thrift broker log after the
>> restart?
>
> There might be but I'm not sure because I think the log lines I sent
> were from before I restarted.
>
>> You can do me a favor by restarting with --log-api true and see which
>> queries are taking a long time, next time you need to do a restart.
>
> Will do, thanks!
>
> Josh
>
> >
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Hypertable Development" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/hypertable-dev?hl=en
-~----------~----~----~----~------~----~------~--~---
