Hi guys. When I send an email from my yahoo account (from a PC/laptop), the hbase mail servers are treating it as spam.. if I send it from my cell using the same yahoo account it goes through (like this one).. my last email got marked as spam by hbase servers as you can read below..
----Forwarded Message---- From: mailer-dae...@yahoo.com To: prince_mithi...@yahoo.co.in Sent: Fri 1 Jun, 2012 8:18 PM IST Subject: Failure Notice Sorry, we were unable to deliver your message to the following address. <user@hbase.apache.org>: Remote host said: 552 spam score (5.4) exceeded threshold (FREEMAIL_FORGED_REPLYTO,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL ) [BODY] --- Below this line is a copy of the message. Received: from [106.10.166.116] by nm2.bullet.mail.sg3.yahoo.com with NNFMP; 01 Jun 2012 14:48:07 -0000 Received: from [106.10.151.251] by tm5.bullet.mail.sg3.yahoo.com with NNFMP; 01 Jun 2012 14:48:07 -0000 Received: from [127.0.0.1] by omp1022.mail.sg3.yahoo.com with NNFMP; 01 Jun 2012 14:48:07 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 151978.45173...@omp1022.mail.sg3.yahoo.com Received: (qmail 69489 invoked by uid 60001); 1 Jun 2012 14:48:07 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.co.in; s=s1024; t=1338562087; bh=4UyEMs/mYUWEloKWPm8TNHpf0XUq1hS6shr2jtC2png=; h=X-YMail-OSG:Received:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=Idtey84AMgbZ1B0L9sZFUeXVnhL4qrmUCjIVia5g6jKmTDtZ3QS5Qg8VbHpWAVzIORmWqOx1ia3u9WqCIwuMhHNDI9fXcHjJ0V6E5GX0dCXXarubdolBxCWAvodQr9ZXrj8JO+zsSDMFCDSW83ZmpPoFptt5NQWdNE1wNVUEKN8= DomainKey-Signature:a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.co.in; h=X-YMail-OSG:Received:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=yn52KmZA19Tre13iYbmt0H4NRfudP7x7xrlGehPDMUU7OXCOWfKtfyaNZ5e7x0lI1A4mjdEmeEwaNkEFV4MYDcBl8LmuV3HQsX4NZl15VuPYS7GbDGxDdeoR9CZinzJhWHlzPuhPi+g4MGDWbTev7FtayVKJrLQSCrQCmw5WAFI=; X-YMail-OSG: O1MD734VM1nUIwivrAyqVmVyGupyuKq4og1.Ui6RaHpyrCi oqTdLss0wdr4WW5pfNFAxC4oK36jwewpOMhWLixlxazByEySyIIPKzBdakrK z0IlUE4QclLH6g3LjSc_JYtH9M4tHgSCN4WTuBS0s34F7xcTAFPdXhm2v.xu dv.7wJ1N_3QHrLvZh9XeEQxm721CMic72Yk.PtcEg_aSljOiZVd_MdLbQkyq 80l2H98OCHLqfvgbs2qO40x5_RwJ.pzUtRmx_gs.GxsfCuIvQMiA7XkXbKAs ZFQSJR0EDVRFCc5QSjCKPkK25hgjzkAzQ8MqPpc7o44O1az8bQzWPQbqVHaS 89m8QcqJ.R43KIRDRrdausENT199M0HvqygTrcPhkzUhSW73RXiOQyxJP_BP lRx2245t8bkU4Rm34LqkkyKTtPnhK7VWHCi8V..yq0qUKMoN1_KN6Y1XhVID dEucmLKc3PRe2z_BEAEr_hsh7HB.xmkXNM6JCFE3hXh1k9NCtnNm.7EhdS8S 9mw-- Received: from [199.172.169.86] by web192504.mail.sg3.yahoo.com via HTTP; Fri, 01 Jun 2012 22:48:06 SGT X-Mailer: YahooMailWebService/0.8.118.349524 References: <4fc4e89f.3090...@free.fr> <4fc7c20c.4040...@free.fr> <CAGpTDNcDBGF=v+fx7vssvpdx91qg_4_vq0a8fxzee+70gue...@mail.gmail.com> <4fc7ddf1.8080...@free.fr> <CAGpTDNe94LkkvLcHP1nCucXzb6Dapb-X+tfQEBw+55Tnx=5e=a...@mail.gmail.com> <4fc8c2c6.7060...@free.fr> <4fc8d456.6060...@free.fr> Message-ID: <1338562086.60488.yahoomail...@web192504.mail.sg3.yahoo.com> Date: Fri, 1 Jun 2012 22:48:06 +0800 (SGT) From: Dhaval Shah <prince_mithi...@yahoo.co.in> Reply-To: Dhaval Shah <prince_mithi...@yahoo.co.in> Subject: Re: hosts unreachables To: "user@hbase.apache.org" <user@hbase.apache.org> In-Reply-To: <4fc8d456.6060...@free.fr> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="1766475574-631287525-1338562086=:60488" --1766475574-631287525-1338562086=:60488 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Can you try removing=A0-XX:+CMSIncrementalMode from your GC settings.. That= had caused a lot of pain a few months back=0A=A0=0ARegards,=0ADhaval=0A=0A= =0A________________________________=0A From: Cyril Scetbon <cyril.scetbon@f= ree.fr>=0ATo: user@hbase.apache.org =0ASent: Friday, 1 June 2012 10:40 AM= =0ASubject: Re: hosts unreachables=0A =0AI've another regionserver (hb-d2) = that crashed (I can easily reproduce the issue by continuing injections), a= nd as I see in master log, it gets information about hb-d2 every 5 minutes.= I suppose it's what helps him to note if a node is dead or not. However it= adds hb-d2 to the dead node list at 13:32:20, so before 5 minutes since th= e last time it got the server information. Is it normal ?=0A=0A2012-06-01 1= 3:02:36,309 DEBUG org.apache.hadoop.hbase.master.LoadBalancer: Server infor= mation: hb-d5,60020,1338553124247=3D47, hb-d4,60020,1338553126577=3D47, hb-= d7,60020,1338553124279=3D46, hb-d10,60020,1338553126695=3D47, hb-d6,60020,1= 33=0A8553124588=3D47, hb-d8,60020,1338553124113=3D47, hb-d2,60020,133855312= 6560=3D47, hb-d11,60020,1338553124329=3D47, hb-d12,60020,1338553126567=3D47= , hb-d1,60020,1338553126474=3D47, hb-d9,60020,1338553124179=3D47=0A..=0A201= 2-06-01 13:07:36,319 DEBUG org.apache.hadoop.hbase.master.LoadBalancer: Ser= ver information: hb-d5,60020,1338553124247=3D47, hb-d4,60020,1338553126577= =3D47, hb-d7,60020,1338553124279=3D46, hb-d10,60020,1338553126695=3D47, hb-= d6,60020,133=0A8553124588=3D47, hb-d8,60020,1338553124113=3D47, hb-d2,60020= ,1338553126560=3D47, hb-d11,60020,1338553124329=3D47, hb-d12,60020,13385531= 26567=3D47, hb-d1,60020,1338553126474=3D47, hb-d9,60020,1338553124179=3D47= =0A..=0A2012-06-01 13:12:36,328 DEBUG org.apache.hadoop.hbase.master.LoadBa= lancer: Server information: hb-d5,60020,1338553124247=3D47, hb-d4,60020,133= 8553126577=3D47, hb-d7,60020,1338553124279=3D46, hb-d10,60020,1338553126695= =3D47, hb-d6,60020,133=0A8553124588=3D47, hb-d8,60020,1338553124113=3D47, h= b-d2,60020,1338553126560=3D47, hb-d11,60020,1338553124329=3D47, hb-d12,6002= 0,1338553126567=3D47, hb-d1,60020,1338553126474=3D47, hb-d9,60020,133855312= 4179=3D47=0A..=0A2012-06-01 13:17:36,337 DEBUG org.apache.hadoop.hbase.mast= er.LoadBalancer: Server information: hb-d5,60020,1338553124247=3D47, hb-d4,= 60020,1338553126577=3D47, hb-d7,60020,1338553124279=3D46, hb-d10,60020,1338= 553126695=3D47, hb-d6,60020,133=0A8553124588=3D47, hb-d8,60020,133855312411= 3=3D47, hb-d2,60020,1338553126560=3D47, hb-d11,60020,1338553124329=3D47, hb= -d12,60020,1338553126567=3D47, hb-d1,60020,1338553126474=3D47, hb-d9,60020,= 1338553124179=3D47=0A..=0A2012-06-01 13:22:36,346 DEBUG org.apache.hadoop.h= base.master.LoadBalancer: Server information: hb-d5,60020,1338553124247=3D4= 7, hb-d4,60020,1338553126577=3D47, hb-d7,60020,1338553124279=3D46, hb-d10,6= 0020,1338553126695=3D47, hb-d6,60020,133=0A8553124588=3D47, hb-d8,60020,133= 8553124113=3D47, hb-d2,60020,1338553126560=3D47, hb-d11,60020,1338553124329= =3D47, hb-d12,60020,1338553126567=3D47, hb-d1,60020,1338553126474=3D47, hb-= d9,60020,1338553124179=3D47=0A..=0A2012-06-01 13:27:36,353 DEBUG org.apache= .hadoop.hbase.master.LoadBalancer: Server information: hb-d5,60020,13385531= 24247=3D47, hb-d4,60020,1338553126577=3D47, hb-d7,60020,1338553124279=3D46,= hb-d10,60020,1338553126695=3D47, hb-d6,60020,133=0A8553124588=3D47, hb-d8,= 60020,1338553124113=3D47, hb-d2,60020,1338553126560=3D47, hb-d11,60020,1338= 553124329=3D47, hb-d12,60020,1338553126567=3D47, hb-d1,60020,1338553126474= =3D47, hb-d9,60020,1338553124179=3D47=0A..=0A2012-06-01 13:32:20,048 INFO o= rg.apache.hadoop.hbase.zookeeper.RegionServerTracker: RegionServer ephemera= l node deleted, processing expiration [hb-d2,60020,1338553126560]=0A2012-06= -01 13:32:20,048 DEBUG org.apache.hadoop.hbase.master.ServerManager: Added= =3Dhb-d2,60020,1338553126560 to dead servers, submitted shutdown handler to= be executed, root=3Dfalse, meta=3Dfalse=0A2012-06-01 13:32:20,048 INFO org= .apache.hadoop.hbase.master.handler.ServerShutdownHandler: Splitting logs f= or hb-d2,60020,1338553126560=0A=0A=0AOn 6/1/12 3:25 PM, Cyril Scetbon wrote= :=0A> I've added hbase.hregion.memstore.mslab.enabled =3D true to the confi= guration of all regionservers and add flags -XX:+UseParNewGC -XX:+UseConcMa= rkSweepGC -XX:+CMSIncrementalMode -XX:CMSInitiatingOccupancyFraction=3D60 t= o the hbase environment=0A> However my regionservers are still crashing whe= n I load data into the cluster=0A> =0A> Here are the logs for the node hb-d= 3 that crashed at 12:56=0A> =0A> - GC logs : http://pastebin.com/T0d0y8pZ= =0A> - regionserver logs : http://pastebin.com/n6v9x3XM=0A> =0A> thanks=0A>= =0A> On 5/31/12 11:12 PM, Jean-Daniel Cryans wrote:=0A> Both, also you co= uld bigger log snippets (post them on something like=0A> pastebin.com) and= we could see more evidence of the issue.=0A> =0A> J-D=0A> =0A> On Thu,= May 31, 2012 at 2:09 PM, Cyril Scetbon<cyril.scet...@free.fr>=A0 wrote:=0A= >> On 5/31/12 11:00 PM, Jean-Daniel Cryans wrote:=0A>> What I'm seeing l= ooks more like GC issues. Start reading this:=0A>> http://hbase.apache.or= g/book.html#gc=0A>> =0A>> J-D=0A>> Hi,=0A>> =0A>> Really not sure ca= use I've enabled gcc's verbose option and I don't see=0A>> anything taking= a long time. Maybe I can check again on one node. On which=0A>> node do y= ou think I should check GC issue ?=0A>> =0A>> =0A>> -- Cyril SCETBON=0A>= > =0A> =0A> =0A=0A=0A-- Cyril SCETBON --1766475574-631287525-1338562086=:60488 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable <html><body><div style=3D"color:#000; background-color:#fff; font-family:ti= mes new roman, new york, times, serif;font-size:12pt"><div><span>Can you tr= y removing
