Re: Upgrade to Hadoop 3

2018-03-19 Thread Sebastian Nagel
> If I remember correct;ly one could not run multiple Nutch instances from the > same user > because all those instances would write to the same TMP file or something > like this... Just make sure that every instance has it's own temp folder configured by setting -Dhadoop.tmp.dir=... That's

Re: Upgrade to Hadoop 3

2018-03-14 Thread BlackIce
I'm redoing everything on my end pretty much from scratch The question I had when I woke up this morning... Do I really need to configure VM's in order to run multiple nodes? Or does running multiple, Nutch-Solr nodes, sufice if they are under their own user space? If I remember

Re: Upgrade to Hadoop 3

2018-03-13 Thread Lewis John McGibbney
Hi Seb, On 2018/03/12 11:00:52, Sebastian Nagel wrote: > Hi, > > > seeing as we have just merged in the 'new' MR patch > > yep, but there's still something to do (NUTCH-2517, ACK, this needs more testing. > NUTCH-2518). I honestly didn't see this come through

Re: Upgrade to Hadoop 3

2018-03-13 Thread Lewis John McGibbney
Hi RRK, Response inline On 2018/03/08 01:46:18, BlackIce wrote: > > Why do you say "Is it too early"? Could you please elaborate on this, thnx. > What I mean is that maybe a lot of people have not upgraded existing infrastructure to Hadoop 3 yet. People don't usually

Re: Upgrade to Hadoop 3

2018-03-12 Thread Sebastian Nagel
Hi, > seeing as we have just merged in the 'new' MR patch yep, but there's still something to do (NUTCH-2517, NUTCH-2518). Better to address this before any upgrade of the Hadoop version. But since there seem to be no breaking MapReduce API changes

Re: Upgrade to Hadoop 3

2018-03-07 Thread BlackIce
It looks like that Hadoop 3 has some very interesting features. Sooner or later we will have to got there anyway. Why do you say "Is it too early"? Could you please elaborate on this, thnx. RRK On Thu, Mar 8, 2018 at 12:08 AM, lewis john mcgibbney wrote: > Hi Folks, >

Upgrade to Hadoop 3

2018-03-07 Thread lewis john mcgibbney
Hi Folks, Before we get started with GSoC again, and seeing as we have just merged in the 'new' MR patch, I wonder if folks are partial to migration to Hadoop 3? Is it too early? Comments? Lewis -- http://home.apache.org/~lewismc/ http://people.apache.org/keys/committer/lewismc