Hello: I have had no problem indexing the page https://plus.google.com/+projectglass (i made a test using this single url) following the instructions pointed by Tejas. I've just changed my nutch-site.xml removing protocol-http and adding protocol-httpclient within plugin includes node.
Check in your log file (default logs/hadoop.log) if the plugin httpclient is loaded: ........... *2013-04-03 02:03:19,489 INFO plugin.PluginRepository - Http / Https Protocol Plug-in (protocol-httpclient)* *..........* Regards On Wed, Apr 3, 2013 at 1:30 AM, Yves S. Garret <[email protected]>wrote: > I don't think anything has changed: > http://bin.cakephp.org/view/1498883341 > > This is my ${NUTCH_HOME}/conf/nutch-site.xml: > http://bin.cakephp.org/view/883727476 > > On Tue, Apr 2, 2013 at 6:51 PM, Tejas Patil <[email protected] > >wrote: > > > Looks like the required plugin aint present in the configuration file > > [0].Can you try adding "protocol-httpclient" to plugin.includes property > in > > conf/nutch-site.xml ? Like this > > > > <property> > > <name>plugin.includes</name> > > > > > <value>protocol-(http|httpclient)|urlfilter-regex|parse-(html|tika)|index-(basic|anchor)|indexer-solr|scoring-opic|urlnormalizer-(pass|regex|basic)</value> > > <description>Regular expression naming plugin directory names to > > include. Any plugin not matching this expression is excluded. > > In any case you need at least include the nutch-extensionpoints plugin. > > By > > default Nutch includes crawling just HTML and plain text via HTTP, > > and basic indexing and search plugins. In order to use HTTPS please > > enable > > protocol-httpclient, but be aware of possible intermittent problems > with > > the > > underlying commons-httpclient library. > > </description> > > </property> > > > > Let us know if that worked. > > > > [0] https://wiki.apache.org/nutch/HttpAuthenticationSchemes#Necessity > > > > > > On Tue, Apr 2, 2013 at 3:27 PM, Yves S. Garret > > <[email protected]>wrote: > > > > > Hi all, just tried crawling this site [ > > > https://plus.google.com/+projectglass ]: > > > > > > This is the output that nutch is showing me: > > > http://bin.cakephp.org/view/1701305116 > > > > > > It seems to be erroring out when it gets to here: > > > > > > fetching https://plus.google.com/+projectglass > > > Unexpected error for https://plus.google.com/+projectglass > > > org.apache.nutch.protocol.ProtocolNotFound: protocol not found for > > > url=https > > > at > > > > > > > > > org.apache.nutch.protocol.ProtocolFactory.getProtocol(ProtocolFactory.java:83) > > > at > > > > > > > > > org.apache.nutch.fetcher.FetcherReducer$FetcherThread.run(FetcherReducer.java:490) > > > Fetcher: throughput threshold: -1 > > > Fetcher: throughput threshold sequence: 5 > > > > > > > > > Why is this happening? > > > > > >
