[ http://issues.apache.org/jira/browse/NUTCH-211?page=comments#action_12366505 ]
Doug Cutting commented on NUTCH-211: ------------------------------------ The interfaces that FetchedSegments implements should have a close method. Moreover, these interfaces should extend a Closeable interface. JDK 1.5 has added such an interface, and, in the meantime, I can add one to Hadoop (org.apache.hadoop.io.Closeable) that Nutch can use until we upgrade to Java 1.5. So, once I've added Closeable to Hadoop, please submit a patch that makes HitContent and HitSummarizer extend Closeable, and FetchedSegments implement it. Does that sound reasonable? > FetchedSegments leave readers open > ---------------------------------- > > Key: NUTCH-211 > URL: http://issues.apache.org/jira/browse/NUTCH-211 > Project: Nutch > Type: Bug > Versions: 0.8-dev > Reporter: Stefan Groschupf > Priority: Critical > Fix For: 0.8-dev > > I have a case here where the NutchBean is instantiated more than once, > however I do cache the nutch bean, but in some situations the bean needs to > re created. The problem is the FetchedSegments leaves open all reads it > uses. So a nio Exception is thrown as soon I try to create the NutchBean > again. > I would suggest to add a close method to FetchedSegments and all involved > objects to be able cleanly shutting down the NutchBean. > Any comments? Would a patch be welcome? > Caused by: java.nio.channels.ClosedChannelException > at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:89) > at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:272) > at > org.apache.nutch.fs.LocalFileSystem$LocalNFSFileInputStream.seek(LocalFileSystem.java:83) > at > org.apache.nutch.fs.NFSDataInputStream$Checker.seek(NFSDataInputStream.java:66) > at > org.apache.nutch.fs.NFSDataInputStream$PositionCache.seek(NFSDataInputStream.java:162) > at > org.apache.nutch.fs.NFSDataInputStream$Buffer.seek(NFSDataInputStream.java:191) > at org.apache.nutch.fs.NFSDataInputStream.seek(NFSDataInputStream.java:241) > at org.apache.nutch.io.SequenceFile$Reader.seek(SequenceFile.java:403) > at org.apache.nutch.io.MapFile$Reader.seek(MapFile.java:329) > at org.apache.nutch.io.MapFile$Reader.get(MapFile.java:374) > at > org.apache.nutch.mapred.MapFileOutputFormat.getEntry(MapFileOutputFormat.java:76) > at > org.apache.nutch.searcher.FetchedSegments$Segment.getEntry(FetchedSegments.java:93) > at > org.apache.nutch.searcher.FetchedSegments$Segment.getParseText(FetchedSegments.java:84) > at > org.apache.nutch.searcher.FetchedSegments.getSummary(FetchedSegments.java:147) > at org.apache.nutch.searcher.NutchBean.getSummary(NutchBean.java:321) -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira ------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642 _______________________________________________ Nutch-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-developers
