[
https://issues.apache.org/jira/browse/ANY23-324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336518#comment-16336518
]
ASF GitHub Bot commented on ANY23-324:
--------------------------------------
Github user HansBrende commented on the issue:
https://github.com/apache/any23/pull/58
@lewismc Then I did the same test again, with the following modification:
instead of alternating between the two parsers, I ran the same parser over and
over again.
NekoHTML code:
```
public static void main(String[] args) throws IOException {
String url =
"https://en.wikipedia.org/wiki/List_of_compositions_by_Johann_Sebastian_Bach";;
String encoding = "utf-8";
byte[] webpage = IOUtils.toString(new URL(url)).getBytes(encoding);
TagSoupParsingConfiguration config = TagSoupParser.legacyConfig();
long totalTime = 0;
for (int i = 0; i < 6000; i++) {
ByteArrayInputStream input = new ByteArrayInputStream(webpage);
long start = System.currentTimeMillis();
config.parse(input, url, encoding);
long duration = System.currentTimeMillis() - start;
long timeToAdd = i < 1000 ? 0 : duration; //SKIP FIRST 1000
ITERATIONS
totalTime += timeToAdd;
if (i % 100 == 0) {
System.out.println(i);
}
}
System.out.println("total time neko: " + totalTime + " ms");
}
```
RESULT:
<pre>
total time neko: 522608 ms
</pre>
---------------------------------------------
Jsoup code (same as before, except I changed `config` to
`JsoupConfig.instance`):
```
public static void main(String[] args) throws IOException {
String url =
"https://en.wikipedia.org/wiki/List_of_compositions_by_Johann_Sebastian_Bach";;
String encoding = "utf-8";
byte[] webpage = IOUtils.toString(new URL(url)).getBytes(encoding);
TagSoupParsingConfiguration config = JsoupConfig.instance;
long totalTime = 0;
for (int i = 0; i < 6000; i++) {
ByteArrayInputStream input = new ByteArrayInputStream(webpage);
long start = System.currentTimeMillis();
config.parse(input, url, encoding);
long duration = System.currentTimeMillis() - start;
long timeToAdd = i < 1000 ? 0 : duration; //SKIP FIRST 1000
ITERATIONS
totalTime += timeToAdd;
if (i % 100 == 0) {
System.out.println(i);
}
}
System.out.println("total time jsoup: " + totalTime + " ms");
}
```
<pre>
total time jsoup: 437072 ms
</pre>
Obviously, this test isn't perfect, but I hope it helps!
> Replace net.sourceforge.nekohtml with jsoup
> --------------------------------------------
>
> Key: ANY23-324
> URL: https://issues.apache.org/jira/browse/ANY23-324
> Project: Apache Any23
> Issue Type: Improvement
> Components: core
> Reporter: Lewis John McGibbney
> Priority: Major
> Fix For: 2.2
>
>
> A long standing issue relates to the performance of the existing default
> [TagSoupParser.java|https://github.com/apache/any23/blob/master/core/src/main/java/org/apache/any23/extractor/html/TagSoupParser.java].
> There are a number of issues which now relate to limitations in the way
> nekohtml parses HTML5 for example
> [ANY23-317|https://issues.apache.org/jira/browse/ANY23-317],
> [ANY23-273|https://issues.apache.org/jira/browse/ANY23-273],
> [ANY23-267|https://issues.apache.org/jira/browse/ANY23-267]... there are
> several others.
> I propose to @Deprecate the TagSoupParser.java implementation for the next
> release (possibly making it configurable via
> default-configuration.properties). I also propose to replace it with
> https://jsoup.org/. AFAIK, Apache Tika also did this several years ago.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
