[ https://issues.apache.org/jira/browse/CONNECTORS-281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134662#comment-13134662 ]
Karl Wright commented on CONNECTORS-281: ---------------------------------------- Use of temporary files, plus carry-down data, seems to be what makes the RSS connector significantly slower than a file crawl. I'm still trying to assess whether the carrydown data is the issue later in the crawl. > RSS connector takes nearly a second to fetch a document even with no > throttling > ------------------------------------------------------------------------------- > > Key: CONNECTORS-281 > URL: https://issues.apache.org/jira/browse/CONNECTORS-281 > Project: ManifoldCF > Issue Type: Bug > Components: RSS connector > Affects Versions: ManifoldCF 0.4 > Reporter: Karl Wright > Assignee: Karl Wright > Fix For: ManifoldCF 0.4 > > > The RSS connector load test shows that the RSS connector is overthrottling, > for some reason. > 10-24-2011 05:30:50.423 fetch > http://localhost:8189/rss/gen.php?doc=4&feed=782&type=doc > 200 46 843 > ... Where 843 ms is taken to fetch a document of size 46 bytes. This is with > connection parameters as follows: > Parameters: Robots usage=none > Max fetches per minute=1000000 > Email address=someb...@somewhere.com > KB per second=1000000 > Max server connections=100 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira