I am using ROME in a modified version of the feedparse plugin. It is pretty straight forward and easy. We had issues with ROME 0.8 and ATOM or some dates. ROME 0.9 resolved that.
-----Original Message----- From: Nick Lothian (JIRA) [mailto:[EMAIL PROTECTED] Sent: Tuesday, February 13, 2007 2:35 PM To: [email protected] Subject: [jira] Commented: (NUTCH-444) Possibly use a different library toparse RSS feed for improved performance and compatibility [ https://issues.apache.org/jira/browse/NUTCH-444?page=com.atlassian.jira. plugin.system.issuetabpanels:comment-tabpanel#action_12472907 ] Nick Lothian commented on NUTCH-444: ------------------------------------ I'm a developer on the ROME project and I done some patches to FeedParser. I've also been a long-time lurker on the Nutch lists. To clear up a couple of misconceptions: The current version of FeedParser is Kevin Burton's one available from http://tailrank.com/code.php. It does have Atom 1.0 support. ROME only has a single dependency: JDom. Both FeedParser & ROME load the feed into a DOM before working on it. FeedParser exposes a SAX-like API, while ROME exposes objects. My tests (a while ago now, but probably still reasonable) showed little performance difference between the two libraries (See http://www.mackmo.com/syndbench/feedparserresults.html and http://www.mackmo.com/syndbench/romeresults.html). I don't understand nutch.newbie's comments about different Atom & RSS mappings. I'm not aware of any issues with the mapping of Author. There are some docs on mappings at http://rollerweblogger.org/roller/entry/rome_0_9_beta_is, http://wiki.java.net/bin/view/Javawsxml/Rome05DateMapping and http://wiki.java.net/bin/view/Javawsxml/Rome05URIMapping. I'd HIGHLY recommend not writing your own custom feed parser. It's a much bigger job than you'd expect. In particular the difficulties of dealing with the bizzare things seen in real-world feeds should not be underestimated. Apache Abdera (http://incubator.apache.org/abdera/) is another option if anyone is just interested in Atom parsing. > Possibly use a different library to parse RSS feed for improved > performance and compatibility > ---------------------------------------------------------------------- > ----------------------- > > Key: NUTCH-444 > URL: https://issues.apache.org/jira/browse/NUTCH-444 > Project: Nutch > Issue Type: Improvement > Components: fetcher > Affects Versions: 0.9.0 > Reporter: Renaud Richardet > Assigned To: Chris A. Mattmann > Priority: Minor > Fix For: 0.9.0 > > Attachments: parse-feed-v2.tar.bz2, parse-feed.tar.bz2 > > > As discussed by Nutch Newbie, Gal, and Chris on NUTCH-443, the current library (feedparser) has the following issues: > - OutOfMemory when parsing > 100k feeds, since it has to convert the > feed to jdom first > - no support for Atom 1.0 > - there has been no development in the last year Alternatives are: > - Rome > - Informa > - custom implementation based on Stax > - ?? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Nutch-developers mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-developers
