[ https://issues.apache.org/jira/browse/CONNECTORS-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17773001#comment-17773001 ]
Mingchun Zhao edited comment on CONNECTORS-1748 at 10/11/23 7:56 PM: --------------------------------------------------------------------- Hi [~daddywri] , I've prepared a PR as below: https://github.com/apache/manifoldcf/pull/154 Could you confirm this please? Best regards, Mingchun was (Author: mingchun.zhao): Hi [~daddywri] , I've prepared a PR as below: [https://github.com/apache/manifoldcf/pull/153] Could you confirm this please? Best regards, Mingchun > Add "User-Agent platform" option for crawling mobile websites > ------------------------------------------------------------- > > Key: CONNECTORS-1748 > URL: https://issues.apache.org/jira/browse/CONNECTORS-1748 > Project: ManifoldCF > Issue Type: New Feature > Reporter: Mingchun Zhao > Priority: Major > > Hi [~daddywri] , > Allow me propose a new feature for crawling mobile sites which restrict > access to content based on User-Agent header. > Actually, Our customer's mobile website was failing to crawl because access > was restricted based on whether the User-Agent request header includes the > mobile info. For this reason, we added the "User-Agent platform" option to > the new "Request Headers" tab on the web repository connector page so that > this mobile website doesn't fail to crawl. -- This message was sent by Atlassian Jira (v8.20.10#820010)