Thanks Talat, I run Nutch2.2.1 in two ways, one is run the CRAWL commond directly ,and there is "host" table in the Hbase after the exceution of the CRAWL commond;another is run the commond line step by step, I start from INJECT ,and then GENERATE,FETCH,PARSE,UPDATEDB,UPDATEHOSTDB. By watching the changes of hbase through the steps, I noticed that the "host" table first showed up after GENERATE, with no content. And it is not empty until the excution of UPDATEHOSTDB.
Compared to Nutch1.7, I think the "webpage" table in Nutch2.2.1 acts the same as "CrawlBD"+"LinkBD" in 1.7, am I right? what really confused me is the "host" table, so, as you said , I can neglect the "host" in the most case ,right? Best Reagrds, Edward From: Talat UYARER Date: 2013-11-07 14:55 To: user Subject: Re: whst does the "host" table do in nutch2.2.1? Hi Edward, Host table is using for Host Based configuration like maxThreads, crawlDelay, mincrawlDelay etc. But this tables is option. In normal usage Host table dont create. Can you explain how do you start your crawler ? Talat 07-11-2013 08:22 tarihinde, [email protected] yazdı: > > Hi,everyone, > I'm new to Nutch and using Nutch2.2.1 with Hbase as the datastore.When I > finished a whole round of crawing,I found "host","webtable" in the Hbase. As > to the "host" table,I am not quite sure about it's function, like in which > step(inject,generate,fetch,parse,updatedb,updatehostdb) is this "host" table > get involved in? And what does the data stored in the 'host" table really > mean? Can anyone share some information? Thank a lot! > > > Edward > .
