[
https://issues.apache.org/jira/browse/CONNECTORS-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingchun Zhao reassigned CONNECTORS-1748:
-----------------------------------------
Assignee: (was: Mingchun Zhao)
> Add "User-Agent platform" option for crawling mobile websites
> -------------------------------------------------------------
>
> Key: CONNECTORS-1748
> URL: https://issues.apache.org/jira/browse/CONNECTORS-1748
> Project: ManifoldCF
> Issue Type: New Feature
> Reporter: Mingchun Zhao
> Priority: Major
>
> Hi [~daddywri] ,
> Allow me propose a new feature for crawling mobile sites which restrict
> access to content based on User-Agent header.
> Actually, Our customer's mobile website was failing to crawl because access
> was restricted based on whether the User-Agent request header includes the
> mobile info. For this reason, we added the "User-Agent platform" option to
> the new "Request Headers" tab on the web repository connector page so that
> this mobile website doesn't fail to crawl.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
