thanks, it's ok 2013/1/8, Christoph Rupp <[email protected]>: > Hi, > > is this 0.9.6.5 and always reproducible after you created the empty hdfs > directory as you described in one of your previous mails? > > can you run the RangeServer in valgrind? > > ht start-rangeserver.sh --valgrind > > this creates a log file vg.rs.{pid} which may contain more information > about the invalid memory access. > > bye > Christoph > > 2013/1/7 richardman <[email protected]> > >> Hi,all >> we can got murderer , the member this->m_send_queue of >> Hypertable::IOHandlerData, rangeServer crash when m_send_queue >> be Destructor, thanks, Expecting a reply >> >> (gdb) p &this->m_send_queue >> $4 = ( >> std::list<boost::intrusive_ptr<Hypertable::CommBuf>, >> std::allocator<boost::intrusive_ptr<Hypertable::CommBuf> > > *) 0x516fd98 >> (gdb) >> >> >> 2013/1/7 richardman <[email protected]> >> >>> Hi,all >>> >>> the gdb debug log as follow: >>> ////////////////////////////////////begin//////////////////////// >>> 1357564314 INFO Hypertable.RangeServer : >>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/MaintenanceScheduler.cc:276) >>> Memory Allocation: BlockCache=5.42% BlockIndex=0.53% BloomFilter=6.75% >>> CellCache=64.58% ShadowCache=0.00% QueryCache=22.72% >>> 1357564314 INFO Hypertable.RangeServer : >>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/RangeServer.cc:3777) >>> Memory Usage: 172245191 bytes >>> 1357564329 INFO Hypertable.RangeServer : >>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/AsyncComm/IOHandler.h:95) >>> Event: type=CONNECTION_ESTABLISHED from=1.164.167.224:64707 >>> 1357564330 ERROR Hypertable.RangeServer : >>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/AsyncComm/IOHandlerData.cc:237) >>> socket read(37, len=1953718478) failure : Bad address >>> 1357564330 INFO Hypertable.RangeServer : >>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/ConnectionHandler.cc:215) >>> Event: type=DISCONNECT from=1.164.167.224:64707 >>> >>> Program received signal SIGSEGV, Segmentation fault. >>> [Switching to Thread 0x7ffff3a65700 (LWP 21510)] >>> 0x0000000000e18324 in >>> std::_List_base<boost::intrusive_ptr<Hypertable::CommBuf>, >>> std::allocator<boost::intrusive_ptr<Hypertable::CommBuf> > >::_M_clear >>> (this=0x516fd98) at /usr/include/c++/4.4/bits/list.tcc:72 >>> 72 __cur = static_cast<_Node*>(__cur->_M_next); >>> (gdb) where >>> #0 0x0000000000e18324 in >>> std::_List_base<boost::intrusive_ptr<Hypertable::CommBuf>, >>> std::allocator<boost::intrusive_ptr<Hypertable::CommBuf> > >::_M_clear >>> (this=0x516fd98) at /usr/include/c++/4.4/bits/list.tcc:72 >>> #1 0x0000000000e1674d in ~_List_base (this=0x516fd98, __in_chrg=<value >>> optimized out>) >>> at /usr/include/c++/4.4/bits/stl_list.h:360 >>> #2 0x0000000000e13200 in ~list (this=0x516fd98, __in_chrg=<value >>> optimized out>) >>> at /usr/include/c++/4.4/bits/stl_list.h:418 >>> #3 0x0000000000e29d4d in ~IOHandlerData (this=0x516fc60, >>> __in_chrg=<value optimized out>) >>> at >>> /home/hadoop/temp/hypertable-0.9.6.5/src/cc/AsyncComm/IOHandlerData.h:55 >>> #4 0x00000000009f05b0 in Hypertable::intrusive_ptr_release >>> (rc=0x516fc60) >>> at >>> /home/hadoop/temp/hypertable-0.9.6.5/src/cc/Common/ReferenceCount.h:73 >>> #5 0x0000000000e16f49 in ~intrusive_ptr (this=0x7ffff3a63f10, >>> __in_chrg=<value optimized out>) >>> at /usr/include/boost/smart_ptr/intrusive_ptr.hpp:101 >>> #6 0x0000000000e3eab7 in Hypertable::HandlerMap::purge_handler >>> (this=0x134e000, handler=0x516fc60) >>> at >>> /home/hadoop/temp/hypertable-0.9.6.5/src/cc/AsyncComm/HandlerMap.h:275 >>> #7 0x0000000000e3e760 in >>> Hypertable::ReactorRunner::cleanup_and_remove_handlers >>> (this=0x7fffec001060, >>> handlers=...) at >>> /home/hadoop/temp/hypertable-0.9.6.5/src/cc/AsyncComm/ReactorRunner.cc:326 >>> #8 0x0000000000e3deff in Hypertable::ReactorRunner::operator() >>> (this=0x7fffec001060) >>> at >>> /home/hadoop/temp/hypertable-0.9.6.5/src/cc/AsyncComm/ReactorRunner.cc:171 >>> #9 0x0000000000e3cd5a in >>> boost::detail::thread_data<Hypertable::ReactorRunner>::run >>> (this=0x7fffec000f30) >>> at /usr/include/boost/thread/detail/thread.hpp:56 >>> #10 0x00007ffff7324200 in thread_proxy () from >>> /usr/lib/libboost_thread.so.1.42.0 >>> #11 0x00007ffff64778ca in start_thread (arg=<value optimized out>) at >>> pthread_create.c:300 >>> #12 0x00007ffff5371b6d in clone () at >>> ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 >>> #13 0x0000000000000000 in ?? () >>> (gdb) >>> >>> ////////////////////////////////////end////////////////////////// >>> >>> >>> >>> 2013/1/7 richardman <[email protected]> >>> >>>> thanks, I should debug the RangeServer later and feedback to you ; >>>> >>>> >>>> 2013/1/6 Christoph Rupp <[email protected]> >>>> >>>>> Does it really crash? this is just a normal error message. You have to >>>>> figure out which client has this IP (could be another RangeServer, a >>>>> Hyperspace machine, the Master, a ThriftBroker or an application using >>>>> the >>>>> native C++ API). If it's an unknown IP then maybe setup a firewall to >>>>> protect your cluster from external requests. >>>>> >>>>> bye >>>>> Christoph >>>>> >>>>> 2013/1/6 richardman <[email protected]> >>>>> >>>>>> Hi,all >>>>>> Hypertable.RangeServer will crash, running for some time, the >>>>>> log is as follows: >>>>>> >>>>>> CPU cores count=4 >>>>>> CephBroker.MonAddr=10.0.1.245:6789 >>>>>> DfsBroker.Local.Root=fs/local >>>>>> DfsBroker.Port=38030 >>>>>> HdfsBroker.fs.default.name=hdfs://192.168.0.245:9000 >>>>>> Hyperspace.GracePeriod=200000 >>>>>> Hyperspace.KeepAlive.Interval=30000 >>>>>> Hyperspace.Lease.Interval=1000000 >>>>>> Hyperspace.Replica.Dir=hyperspace >>>>>> Hyperspace.Replica.Host=[192.168.0.245] >>>>>> Hyperspace.Replica.Port=38040 >>>>>> Hypertable.Master.Port=38050 >>>>>> Hypertable.RangeServer.CellStore.SkipNotFound=true >>>>>> Hypertable.RangeServer.LowMemoryLimit.Percentage=10 >>>>>> Hypertable.RangeServer.MaintenanceThreads=10 >>>>>> Hypertable.RangeServer.MemoryLimit=4096000000 >>>>>> Hypertable.RangeServer.MemoryLimit.Percentage=25 >>>>>> Hypertable.RangeServer.Port=38060 >>>>>> Hypertable.RangeServer.Range.SplitSize=512000000 >>>>>> Hypertable.RangeServer.Reactors=4 >>>>>> Hypertable.RangeServer.Workers=50 >>>>>> Hypertable.Verbose=true >>>>>> ThriftBroker.Port=38080 >>>>>> config=/opt/hypertable/0.9.6.5/conf/hypertable.cfg >>>>>> dfs-port=38030 >>>>>> grace-period=200000 >>>>>> hs-host=[192.168.0.245] >>>>>> hs-port=38040 >>>>>> keepalive=30000 >>>>>> lease-interval=1000000 >>>>>> master-port=38050 >>>>>> pidfile=/opt/hypertable/current/run/Hypertable.RangeServer.pid >>>>>> port=38060 >>>>>> reactors=4 >>>>>> verbose=true >>>>>> workers=50 >>>>>> 1357412412 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/HyperspaceSessionHandler.cc:31) >>>>>> Hyperspace session state change: SAFE >>>>>> drive count = 1 >>>>>> maintenance threads = 10 >>>>>> 1357412413 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/ConnectionHandler.cc:215) >>>>>> Event: type=CONNECTION_ESTABLISHED from=192.168.0.245:38050 >>>>>> 1357412413 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/RangeServer.cc:502) >>>>>> log_dir=/hypertable/servers/rs1/log >>>>>> 1357412413 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/AsyncComm/IOHandler.h:95) >>>>>> Event: type=CONNECTION_ESTABLISHED from=192.168.0.245:33108 >>>>>> 1357412413 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/AsyncComm/IOHandler.h:95) >>>>>> Event: type=CONNECTION_ESTABLISHED from=192.168.0.245:48072 >>>>>> 1357412413 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/AsyncComm/IOHandler.h:95) >>>>>> Event: type=CONNECTION_ESTABLISHED from=192.168.0.245:51749 >>>>>> 1357412414 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/AsyncComm/IOHandler.h:95) >>>>>> Event: type=CONNECTION_ESTABLISHED from=192.168.0.245:47547 >>>>>> 1357412414 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/ConnectionHandler.cc:215) >>>>>> Event: type=DISCONNECT from=192.168.0.245:47547 >>>>>> 1357412414 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/Range.cc:269) >>>>>> Loading CellStore 0/0/default/4MmZEeUI1pJuk6SA/cs12 >>>>>> 1357412414 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/Range.cc:269) >>>>>> Loading CellStore 0/0/location/4MmZEeUI1pJuk6SA/cs13 >>>>>> 1357412414 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/TableInfo.cc:252) >>>>>> Adding range 0/0[..0/0:ÿÿ] to TableInfo >>>>>> >>>>>> .....................skip......follow is the last log >>>>>> >>>>>> 1357420036 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/MaintenanceScheduler.cc:271) >>>>>> Memory Statistics (MB): VM=1099.50, RSS=167.14, tracked=142.00, >>>>>> computed=188.75 limit=3906.25 >>>>>> 1357420036 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/MaintenanceScheduler.cc:276) >>>>>> Memory Allocation: BlockCache=0.74% BlockIndex=0.35% >>>>>> BloomFilter=5.80% >>>>>> CellCache=67.84% ShadowCache=0.00% QueryCache=25.26% >>>>>> 1357420036 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/RangeServer.cc:3777) >>>>>> Memory Usage: 148895531 bytes >>>>>> 1357420045 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/AsyncComm/IOHandler.h:95) >>>>>> Event: type=CONNECTION_ESTABLISHED from=80.179.7.132:2564 >>>>>> 1357420056 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/RSStats.h:83) >>>>>> Maintenance stats scans=(0 0 0 0.000000) updates=(0 0 0 0.000000 0) >>>>>> 1357420056 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/MaintenanceScheduler.cc:271) >>>>>> Memory Statistics (MB): VM=1099.50, RSS=167.14, tracked=142.00, >>>>>> computed=188.75 limit=3906.25 >>>>>> 1357420056 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/MaintenanceScheduler.cc:276) >>>>>> Memory Allocation: BlockCache=0.74% BlockIndex=0.35% >>>>>> BloomFilter=5.80% >>>>>> CellCache=67.84% ShadowCache=0.00% QueryCache=25.26% >>>>>> 1357420056 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/RangeServer.cc:3777) >>>>>> Memory Usage: 148895531 bytes >>>>>> 1357420057 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/RangeServer.cc:3038) >>>>>> Entering get_statistics() >>>>>> 1357420057 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/RangeServer.cc:3283) >>>>>> Exiting get_statistics() >>>>>> 1357420067 ERROR Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/AsyncComm/IOHandlerData.cc:237) >>>>>> socket read(33, len=50331562) failure : Bad address >>>>>> 1357420067 INFO Hypertable.RangeServer : >>>>>> (/home/hadoop/temp/hypertable-0.9.6.5/src/cc/Hypertable/RangeServer/ConnectionHandler.cc:215) >>>>>> Event: type=DISCONNECT from=80.179.7.132:2564 >>>>>> >>>>>> ///////////////////////////End//////////////////////////////////// >>>>>> the rangeserver want to connect to a IP 80.179.7.132:2564 >>>>>> but my master server in 192.168.0.245,what happen? >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Hypertable Development" group. >>>>>> To post to this group, send email to [email protected]. >>>>>> To unsubscribe from this group, send email to >>>>>> [email protected]. >>>>>> For more options, visit this group at >>>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Hypertable Development" group. >>>>> To post to this group, send email to [email protected]. >>>>> To unsubscribe from this group, send email to >>>>> [email protected]. >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/hypertable-dev?hl=en. >>>>> >>>> >>>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Hypertable Development" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/hypertable-dev?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Hypertable Development" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/hypertable-dev?hl=en. > >
-- You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/hypertable-dev?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
