Could it be related to https://issues.apache.org/jira/browse/NUTCH-719 ?
2009/8/21 MilleBii <[email protected]> > Aborting again with a fetchqueue size of 1.... any idea what can I do. > > 2009/8/19 MilleBii <[email protected]> > > > Well in the segment there is nothing but _temporary... and then after a > > number of spinwaiting for the last 6 elements it aborts and the segment > is > > empty... > > I guess everything is left in the tmp files. > > If only it would abort gracefully and close the segment properly... > > basically lost 200k URL. > > > > > > 2009/8/19 Doğacan Güney <[email protected]> > > > > On Wed, Aug 19, 2009 at 09:40, MilleBii <[email protected]> wrote: > >> > >> > Quite frustrating problem , after 24h fetching properly > >> > > >> > fetcher is spinwaiting on a fetchqueue size of 6 and then goes in > >> aborting > >> > the fetch. > >> > I have this problem from time to time and I have no idea why. > >> > Can anyone help or suggest something to debug this issue. > >> > > >> > >> When this happens, every URL successfully fetched should be stored in > the > >> segment. This part works for you right? Only those last 6 URLs are not > >> fetched? > >> > >> > >> > > >> > -- > >> > -MilleBii- > >> > > >> > >> > >> > >> -- > >> Doğacan Güney > >> > > > > > > > > -- > > -MilleBii- > > > > > > -- > -MilleBii- > -- DigitalPebble Ltd http://www.digitalpebble.com
