On Tue, Aug 25, 2009 at 00:58, MilleBii <[email protected]> wrote: > So far so could did have 2-3 fetch cycle no strange abort, of course I also > applied NUTCH-721 manually. >
Great, thanks for testing. I will commit NUTCH-721 then. > > 2009/8/21 Julien Nioche <[email protected]> > > > Could it be related to https://issues.apache.org/jira/browse/NUTCH-719 ? > > > > 2009/8/21 MilleBii <[email protected]> > > > > > Aborting again with a fetchqueue size of 1.... any idea what can I do. > > > > > > 2009/8/19 MilleBii <[email protected]> > > > > > > > Well in the segment there is nothing but _temporary... and then after > a > > > > number of spinwaiting for the last 6 elements it aborts and the > segment > > > is > > > > empty... > > > > I guess everything is left in the tmp files. > > > > If only it would abort gracefully and close the segment properly... > > > > basically lost 200k URL. > > > > > > > > > > > > 2009/8/19 Doğacan Güney <[email protected]> > > > > > > > > On Wed, Aug 19, 2009 at 09:40, MilleBii <[email protected]> wrote: > > > >> > > > >> > Quite frustrating problem , after 24h fetching properly > > > >> > > > > >> > fetcher is spinwaiting on a fetchqueue size of 6 and then goes in > > > >> aborting > > > >> > the fetch. > > > >> > I have this problem from time to time and I have no idea why. > > > >> > Can anyone help or suggest something to debug this issue. > > > >> > > > > >> > > > >> When this happens, every URL successfully fetched should be stored > in > > > the > > > >> segment. This part works for you right? Only those last 6 URLs are > not > > > >> fetched? > > > >> > > > >> > > > >> > > > > >> > -- > > > >> > -MilleBii- > > > >> > > > > >> > > > >> > > > >> > > > >> -- > > > >> Doğacan Güney > > > >> > > > > > > > > > > > > > > > > -- > > > > -MilleBii- > > > > > > > > > > > > > > > > -- > > > -MilleBii- > > > > > > > > > > > -- > > DigitalPebble Ltd > > http://www.digitalpebble.com > > > > > > -- > -MilleBii- > -- Doğacan Güney
