[ https://issues.apache.org/jira/browse/TIKA-1599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15048819#comment-15048819 ]
Ken Krugler commented on TIKA-1599: ----------------------------------- I think we'd be wanting to parse the raw crawl results (with both TagSoup & JSoup) to compare which one seems to do a better job with broken HTML. So I'm not seeing how the parser that CommonCrawl uses would factor into that. But I also don't have a good suggestion for how to automatically evaluate the TagSoup vs. JSoup parse-off, other than simple measures like how many failed, or maybe the amount of text extracted? > Switch from TagSoup to JSoup > ---------------------------- > > Key: TIKA-1599 > URL: https://issues.apache.org/jira/browse/TIKA-1599 > Project: Tika > Issue Type: Improvement > Components: parser > Affects Versions: 1.7, 1.8 > Reporter: Ken Krugler > Assignee: Ken Krugler > Priority: Minor > > There are several Tika issues related to how TagSoup cleans up HTML > ([TIKA-381], [TIKA-985], maybe [TIKA-715]), but TagSoup doesn't seem to be > under active development. > On the other hand I know of several projects that are now using > [JSoup|https://github.com/jhy/jsoup], which is an active project (albeit only > one main contributor) under the MIT license. > I haven't looked into how hard it would be to switch this dependency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)