I disagree with excluding older Java docs or refguide using robots! When I look for documentation of a class I generally enter class name and version number into google.
We can maybe handle this with priorities inside a sitemap.xml or custom http headers (X-Robots) using a htaccess rule. I can check this out. Uwe Am March 4, 2021 10:41:52 AM UTC schrieb "Jan Høydahl" <jan....@cominvent.com>: >Sure we could do robots. > >But I suspect that we put ourselves in this situation through >https://issues.apache.org/jira/browse/SOLR-10595 ourselves >Check out the attachment solr_redirects.conf on that JIRA (also here >https://gist.github.com/janhoy/a3149e1ed27df020194a2de1a7fa2c16) > >Here, we explicitly map all pages that used to be in Confluence (which >there are a ton of links to on the net), to version 6.6 of the guide. >Of course, if we change those to "latest", some of those links will >break, but perhaps it would still be better? >Or can we be more intelligent in the rewrite rules on Solr site - that >if you try a "/guide/foo.html" link and it is not found, that you >display a custom error page or go to front page of latest guide? > >Jan > >> 4. mar. 2021 kl. 10:21 skrev Ishan Chattopadhyaya ><ichattopadhy...@gmail.com>: >> >> We can add robots.txt to stop Google from indexing/showing in >results. >> >> On Thu, 4 Mar, 2021, 2:34 pm Jan Høydahl, <jan....@cominvent.com ><mailto:jan....@cominvent.com>> wrote: >> Hi, sending to this list since dev@solr list is not yet announced >properly. >> >> We have a few days of traffic to the new site and can see the most >visited pages at https://uls.apache.org/exports/solr.apache.org.yaml ><https://uls.apache.org/exports/solr.apache.org.yaml> (see copy below). >> When I search google for "solr query parser", I get the 6.6 guide on >top, which is probably why /guide/6_6/the-standard-query-parser.html >shows up, and the same for the other /guide/6_6/ links. >> Some questions: >> >> How can we make Google forget about version 6.6? I know we had a >bunch of redirects from Confluence to the 6.6 guide, are they still in >place? >> Why is /docs/6_6_0/solr-core/index.html the 2nd most visited page? >Anywhere that links to it? >> Why is /docs/4_8_1/solr-solrj/index.html so high? Ahywhere that links >to it? >> The /mirrors-solr-latest-redir.html redirect was not working. I just >pushed a fix >> >> >> Sheet3: >> Name: Most visited pages, past month >> Values: >> /index.html: 443 >> /docs/6_6_0/solr-core/index.html: 281 >> /guide/8_8/solr-tutorial.html: 104 >> /news.html: 92 >> /guide/solr-tutorial.html: 91 >> /resources.html: 75 >> /features.html: 69 >> /docs/8_7_0/solr-core/index.html: 68 >> /downloads.html: 68 >> /guide/6_6/the-standard-query-parser.html: 65 >> /docs/4_8_1/solr-solrj/index.html: 62 >> /guide/6_6/common-query-parameters.html: 50 >> /guide/index.html: 46 >> /docs/8_8_1/solr-solrj/index.html: 44 >> /docs/8_7_0/solr-solrj/index.html: 38 >> /docs/8_8_1/solr-core/index.html: 37 >> /community.html: 24 >> /guide/8_8/: 23 >> /guide/6_6/uploading-data-with-index-handlers.html: 22 >> /docs/8_6_3/solr-core/index.html: 21 >> /guide/6_6/filter-descriptions.html: 21 >> /guide/6_6/collections-api.html: 18 >> /docs/8_6_2/solr-solrj/overview-summary.html: 16 >> /guide/6_6/faceting.html: 16 >> /mirrors-solr-latest-redir.html: 15 >> /whoweare.html: 15 >> /guide/6_6/solrcloud.html: 14 >> /guide/6_6/tokenizers.html: 13 >> /guide/7_0/solr-configuration-files.html: 13 >> /guide/8_8/query-syntax-and-parsing.html: 13 >> /security.html: 13 >> /guide/6_6/introduction-to-solr-indexing.html: 12 >> /guide/8_8/solr-upgrade-notes.html: 12 >> /docs/8_0_0/solr-solrj/allclasses-frame.html: 11 >> /docs/8_6_3/solr-solrj/index.html: 11 >> /guide/6_6/the-dismax-query-parser.html: 11 >> /guide/8_8/getting-started.html: 11 >> /guide/solr-upgrade-notes.html: 11 >> /docs/8_0_0/solr-solrj/overview-summary.html: 10 >> /docs/8_6_2/solr-solrj/index.html: 10 >> /guide/6_6/running-solr.html: 10 >> /docs/7_2_1/solr-solrj/overview-summary.html: 9 >> /docs/8_0_0/solr-solrj/overview-frame.html: 9 >> /guide/6_6/format-of-solr-xml.html: 9 >> /guide/6_6/index.html: 9 >> /guide/6_6/learning-to-rank.html: 9 >> /guide/6_6/making-and-restoring-backups.html: 9 >> /guide/6_6/working-with-dates.html: 9 >> /guide/8_0/reindexing.html: 9 >> /docs/7_2_1/solr-solrj/allclasses-frame.html: 8 >> >> -- Uwe Schindler Achterdiek 19, 28357 Bremen https://www.thetaphi.de
