Did as you described. Removed Master WALs, restarted, still getting 80% of my regions stuck. I will keep playing a bit but I think I will "just" upgrade to the last 2.2 and retry... Right now 2.0.4 seems a bit unstable for me :(
Le sam. 16 févr. 2019 à 01:38, Stack <st...@duboce.net> a écrit : > Remove the Master WALs at least. The beta is damaged. > > If testing, trying to help out, I'd suggest move to branch-2.2. It is the > next minor release. 2.1.x has had some love and is in a good state. It is > superior to 2.0.x which we are trying to EOL (too many branches) so your > time would be better spent elsewhere. > > Thanks for the help JMS, > S > > > On Wed, Feb 13, 2019 at 5:03 AM Jean-Marc Spaggiari < > jean-m...@spaggiari.org> > wrote: > > > Hi Stack, > > > > Thanks for looking at it. So what's next? Do you want to me try to put > the > > latest 2.1 RC on top of it and see if it behaves well? Or just remove the > > Master WALs and stay on 2.0.4 to try the 2.2 upgrade path? I don' thave > an > > urgent need of this cluster, so I can play a bit with it if it helps. > > > > Any JIRA that I should open or update based on that? > > > > JMS > > > > Le lun. 11 févr. 2019 à 21:16, Stack <st...@duboce.net> a écrit : > > > > > Took a quick look. > > > > > > On startup, it has hundreds of AMv2 WALs to recover. Its backed up > unable > > > to let go of the early files because some old procedures have not > > > 'completed'.. This sort of backup problem has been addressed in later > > > hbase-2.0s. > > > > > > (Looks like oldest WAL is hdfs:// > > > > > > node2.distparser.com:8020/hbase/MasterProcWALs/pv2-00000000000000000016.log > > > ) > > > > > > It then finds corrupt procedures which is probably why we are retaining > > > logs in first place. Corruption was fixed in later hbase-2.0s (may be a > > new > > > instance of corruption raising its head on master branch... but that > > > doesn't apply here). > > > > > > There is then an issue assigning namespace -- it has a mangled hostname > > -- > > > which is probably why the fail... Would be interesting if we saw > similar > > in > > > a later hbase... but probably a result of the mess above. > > > > > > Thanks for letting me take a look JMS. > > > > > > S > > > > > > > > > > > > On Mon, Feb 11, 2019 at 4:24 PM Jean-Marc Spaggiari < > > > jean-m...@spaggiari.org> > > > wrote: > > > > > > > Doesn't seems to work very well. > > > > > > > > Can you try this one? > > > > > > > > > > > > > > > > > > https://drive.google.com/file/d/1PMqJz4LjkEx0U2jYVVSLXaSEqcdHq7w4/view?usp=sharing > > > > > > > > JMS > > > > > > > > Le lun. 11 févr. 2019 à 16:27, Stack <st...@duboce.net> a écrit : > > > > > > > > > On Fri, Feb 8, 2019 at 11:47 AM Jean-Marc Spaggiari < > > > > > jean-m...@spaggiari.org> > > > > > wrote: > > > > > > > > > > > Probably not the best way to host a file, but logs are there: > > > > > > https://pastebin.com/dl/JaETSZfh > > > > > > I had a running 2.0.0-beta but 2.0.4 can't start. > > > > > > Let me know if you can't see the file (.tar.bz2) and I will try > > > > something > > > > > > different. > > > > > > JMS > > > > > > > > > > > > > > > > > Says "Your request is blocked due to invalid referrer. If you are > > > trying > > > > to > > > > > hotlink this page, please use: https://pastebin.com/raw/JaETSZfh". > > > > > > > > > > I'm doing something wrong? > > > > > > > > > > Thanks JMS, > > > > > S > > > > > > > > > > > > > > > > Le ven. 8 févr. 2019 à 14:15, Stack <st...@duboce.net> a écrit : > > > > > > > > > > > > > On Wed, Feb 6, 2019 at 1:42 PM Jean-Marc Spaggiari < > > > > > > > jean-m...@spaggiari.org> > > > > > > > wrote: > > > > > > > > > > > > > > > Quick question here. Just upgraded from 2.0.0 BETA 2 to 2.0.4 > > and > > > > I'm > > > > > > > > having some issues to start HBase. I will probably be able to > > fix > > > > > that, > > > > > > > but > > > > > > > > I'm wondering if you want me to document those issues or if > > 2.0.x > > > > is > > > > > > too > > > > > > > > old so it's not needed? > > > > > > > > > > > > > > > > > > > > > > > Hey JMS. > > > > > > > > > > > > > > Would be interested in the issues you are seeing. Perhaps they > > > > pertain > > > > > to > > > > > > > branch-2.1+? > > > > > > > > > > > > > > Thanks, > > > > > > > S > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Le lun. 4 févr. 2019 à 21:11, 张铎(Duo Zhang) < > > > palomino...@gmail.com > > > > > > > > > > a > > > > > > > > écrit : > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > Andrew Purtell <apurt...@apache.org> 于2019年2月5日周二 > 上午3:24写道: > > > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Feb 4, 2019 at 10:29 AM Stack <st...@duboce.net> > > > > wrote: > > > > > > > > > > > > > > > > > > > > > I was going to put up a 2.0.5 RC to address > > > > > > > > > > > https://issues.apache.org/jira/browse/HBASE-21791.Its > > been > > > > > > about a > > > > > > > > > month > > > > > > > > > > > since 2.0.4. There are 50 odd fixes in 2.0.5 currently > > [1] > > > > > mostly > > > > > > > > > > > perfunctory backports. > > > > > > > > > > > > > > > > > > > > > > Stepping back though, I'd like to entertain our letting > > the > > > > > > > > > branch-2.0.x > > > > > > > > > > > releases go. After 2.0.5, let's not backport going > > forward > > > > and > > > > > > > > > encourage > > > > > > > > > > > users to move to branch-2.1. I can make a release > should > > a > > > > > > > > > super-critical > > > > > > > > > > > arise but otherwise, will let the branch go to seed. We > > > have > > > > > > enough > > > > > > > > > > > branches as it is (smile). > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > S > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 1. > > > > > > https://issues.apache.org/jira/projects/HBASE/versions/12344604 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Best regards, > > > > > > > > > > Andrew > > > > > > > > > > > > > > > > > > > > Words like orphans lost among the crosstalk, meaning torn > > > from > > > > > > > truth's > > > > > > > > > > decrepit hands > > > > > > > > > > - A23, Crosstalk > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >