Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

[Zheng Lin Edwin Yeo]

Hi Jason,

Thanks for your reply.

I have tried to add the "forwardCredentials": true in the security.json,
but I still get the same error.

Regards,
Edwin

On Mon, 3 Jun 2019 at 22:19, Colvin Cowie 
wrote:

> Hi, thanks I'll give that a go when I get a chance.
>
> I was trying to reply to an older thread (
>
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201904.mbox/%3CCAF2DzVXeVZqnixnkbzw0La1ui5N5-RG9PwfMBHG9vmkfBSMzJA%40mail.gmail.com%3E
> ),
> which I don't have in my mailbox, so obviously didn't reply to the right
> address to get my response threaded so mine has appeared on its own. Oops.
>
> A JIRA issue was raised on that thread
> https://issues.apache.org/jira/browse/SOLR-13421 but it's not had any
> attention.
>
>
> On Mon, 3 Jun 2019 at 14:46, Jason Gerlowski 
> wrote:
>
> > Hi Colvin,
> >
> > We're still taking a look at fixing the bug, but as a workaround in
> > the meantime, you can look into adding a "forwardCredentials":true
> > property under the "authentication" section of security.json.  That
> > seems to fix the issue in my reproduction at least.
> >
> > e.g.
> >
> > {
> > "authentication": {
> > "blockUnknown": true,
> > "class": "solr.BasicAuthPlugin",
> > "credentials": {
> > "solradmin": ""
> > },
> > "forwardCredentials": true
> > },
> > ...
> > }
> >
> > Jason
> >
> > On Mon, Jun 3, 2019 at 9:31 AM Jason Gerlowski 
> > wrote:
> > >
> > > One last note: as far as I can tell, nothing about this issue is
> > > specific to JSON Faceting or the JSON request API.  It can be
> > > triggered just as easily with "/select?q=*:*".
> > >
> > > The bug created for this is: SOLR-13510
> > >
> > > On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski 
> > wrote:
> > > >
> > > > I'm also able to reproduce this bug on master.  A few more notes
> about
> > > > the bad behavior:
> > > >
> > > > - the behavior occurs regardless of the specific permissions
> > > > configured in security.json.  (i.e. whether the top permission is
> > > > "all", or "security-edit", or there are no permissions at all.)
> > > > - I tried looking for a pattern in which requests saw the 401s, but
> > > > didn't have any luck.  The 401 occurs when talking to the whole
> > > > collection or targeting individual cores directly.  It occurs when
> > > > curl hits a host containing a replica for the collection in question,
> > > > and when it doesnt. etc.  This distinguishes it from SOLR-13472,
> which
> > > > seems more specific to collection structure/layout.
> > > >
> > > > I'll create a bug for this in JIRA.
> > > >
> > > > On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie <
> > colvin.cowie@gmail.com> wrote:
> > > > >
> > > > > Hello. I encountered this issue too and wrote this up before I
> found
> > this
> > > > > thread, but I thought I might as well post it still, if it helps...
> > > > >
> > > > > Currently I'm trying to move our product on to Solr 8.1.1. We are
> > currently
> > > > > using 6.6.6, so things have definitely moved on.
> > > > >
> > > > > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock
> > down Solr
> > > > > (and we also secure our zookeeper). Here's an example for solradmin
> > as the
> > > > > user and password
> > > > >
> > > > > {
> > > > > "authentication": {
> > > > > "blockUnknown": true,
> > > > > "class": "solr.BasicAuthPlugin",
> > > > > "credentials": {
> > > > > "solradmin":
> > "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > > > > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> > > > > }
> > > > > },
> > > > > "authorization": {
> > > > > "class": "solr.RuleBasedAuthorizationPlugin",
> > > > > "permissions": [
> > > > > {
> > > > > "name": "all",
> > > > > "role": "admin"
> > > > > }
> > > > > ],
> > > > > "user-role": {
> > > > > "solradmin": "admin"
> > > > > }
> > > > > }
> > > > > }
> > > > >
> > > > >
> > > > > On Solr 8.1.1, using our previously working security.json, running
> > queries
> > > > > (through the admin UI currently) I non-deterministically get 401
> > responses
> > > > > on queries when a collection has more than 1 shard. Increasing the
> > number
> > > > > of shards in the collection makes the errors more likely.
> > > > >
> > > > > {
> > > > >   "responseHeader":{
> > > > > "zkConnected":true,
> > > > > "status":401,
> > > > > "QTime":30,
> > > > > "params":{
> > > > >   "q":"*:*",
> > > > >   "_":"1559474550365"}},
> > > > >   "error":{
> > > > > "metadata":[
> > > > >
> > > > >
> >
> "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
> > > > >
> > > > >
> >
> "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> > > > > "msg":"Error from server at null: Expected mime type
> > > > > application/octet-stre

not able to optimize

[Midas A]

Hi ,
 Index size is 400GB. we used master slave architecture .

commit is taking time while not able to perform optimize .

what should i do .

AW: WG: SolrException: Can't determine a Sort Order with Solr 6.6

[Schwank , Désirée]

Hmh, first thought was that I can't influence that cause my fellow colleauges 
are responsible for servers and hardware. But of course I will be able to tell 
them to change the network location. Thanks, I will give it a hard try. 

Greets 
Desiree

-Ursprüngliche Nachricht-
Von: Shawn Heisey  
Gesendet: Montag, 3. Juni 2019 15:01
An: solr-user@lucene.apache.org
Betreff: Re: WG: SolrException: Can't determine a Sort Order with Solr 6.6

On 6/3/2019 5:12 AM, Schwank, Désirée wrote:
> But how can it be protected. What can I do? What is to configure? Can you 
> help me with an example.

Place the Solr server in a network location such that only trusted systems and 
people can reach it.

Sanitize all input in your application before you send it to Solr.

Thanks,
Shawn

Re: Using Solr as a Database?

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Daniel,

On 6/3/19 16:26, Davis, Daniel (NIH/NLM) [C] wrote:
> I think the sweet spot of Cassandra and Solr should be mentioned
> in this discussion.   Cassandra is more scalable/clusterable than
> an RDBMS, without losing all of the structure that is desirable in
> an RDBMS.

Amusingly enough, there is also Solandra if you don't want to choose :)

https://github.com/tjake/Solandra

It's a lot like DataStax.

> In contrast, if you use a full document store such as MongoDB, you 
> lose some of the abilities to know what is in your schema.
> 
> DataStax markets a platform that combines Cassandra (as a
> distributed replacement for an RDBMS) that is integrated with Solr
> so that records in managed in Cassandra are indexed and
> up-to-date.
> 
> If your real problem with an RDBMS is the lack of scaling, but you 
> like the ability to specify columnar structure explicitly, then
> this combination might be a good fit.
> 
> Now, MongoDB is also a strong alternative to an RDBMS.
> 
> The other thing to recall though is that the power of sharding has 
> reached into the databases themselves, and databases such as 
> PostgreSQL can operate with some tables sharded and other tables 
> duplicated.   See
> https://pgdash.io/blog/postgres-11-sharding.html.

Even MySQL and MariaDB -- the most bare-bones solutions in the RDBMS
space -- now have clustering available to them, to it's hard to defend
an RDBMS solution at this point that does NOT provide clustering, or
something similar.

- -chris
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlz1kA8ACgkQHPApP6U8
pFif4w/+Ph5ZsQdEiVuK96ygWYJcq0x5RzBfrQhQ5oq7IvhdlLzdzwIPilwLZoaO
9/JcwQOUfVo5XNC72mpclg6J+1jhkBuvee7tMqvSA90PLoTmJLft/oeFoBBm374Z
9UAhJgHF/lhcyp00w4L1JjRH+jQzZia3cohi56oeLReKnyHY//EvqzHKNe2TbiPf
7m5jOIiscxmzAMaI2pEBE4gHWUL8rXVG0SVkUbMQYqR+dRj50sOKk3w2lO2akWV/
rLkYD175LAtpQ7qMXU+CAGro2UAIdTXJOtp7yhCquA6T6Vo4BcBsvQ2bGBMDpeld
MsnyxzM1hiOZ71DOhyFjfGN9Ivqr1/UijVNsZWazBYtYp9N9/H1l3hl6NlKUVGIF
c+pSVWleNAzsO4ShUGJrOkdfv64vRjfK1s/unggAnu/XtyTWKoNV6vxhXwceEYlD
1xVDk8O4ANErXxj4XvQvtgrvBeYOK5sJ5aqn0guN1UIX6Q2gE61bclYwJp9r4NO9
cJjTQedEPdVdRYAz+lDucmSESETQITghhSgub8558BmTSc1PF61f3nAKEYiWrhfN
NnxR0dLKY+QOQ5Mo9lX6RSsCYb9x5F8K1jAoy/GSllpnGc88oswquJT/7Vm6R0yX
9YvFI7JsUHfhIwSkV8uupBZ03KJpYgJvXwirBGzV4j7i4M4qr7o=
=9ZXf
-END PGP SIGNATURE-

Re: Using Solr as a Database?

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Ralph,

On 6/2/19 16:32, Ralph Soika wrote:
> The whole system is highly transactional as it runs on Java EE with
> JPA and Session EJBs.

And you write-through from your application -> RDBMS -> Lucene/Solr?

How are you handling commits (both soft and hard) and re-opening the
index?

> So, as far as I understand, you recommend to leave the data in the
> RDBMS?

I certainly would, even if it's just to allow a rebuild of the index
from a "trusted" source.

> The problem with RDBMS is that you can not easily scale over many
> nodes with a master less cluster.

That sounds like it's a problem with your choice of RDBMS, and not of
RDBMS's in general.

> This was why I thought Solr can solve this problem easily. On the 
> other hand my Lucene index also did not scale over multiple nodes.

If you want a clustered document-store[1], you might want to look at a
storage system designed for that purpose such as CouchDB or MongoDB.
Lucene/Solr is really best used as a distillation of data stored
elsewhere and not as a backing-store itself.

> Maybe Solr would be a solution to scale just the index?

That's exactly what Solr is for.

> Another solution I am working on is to store all my data in a HA 
> Cassandra cluster because I do not need the SQL-Core
> functionallity. But in this case I only replace the RDBMS with
> Cassandra and Lucene/Solr holds again only the index.

This seems like another plausible solution.

> So Solr can't improve my architecture, with the exception of the
> fact that the search index could be distributed across multiple
> nodes with Solr. Did I get that right?

Yes.

Hope that helps,
- -chris

[1]
https://en.wikipedia.org/wiki/Document-oriented_database#Implementations
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlz1jYYACgkQHPApP6U8
pFjw6w/9GGv4Z4FIoypv8XQrtIf5heT8yH0On6pQaFI313mglmzerTrD4W9Jz3y7
VWQHeMw5Q5LBg56KKMGSKv/PEnNmiA+69YTMdXB+R5gJnwtW0ZEZU0jP1uhPO+af
UO6ZpdbMnIuIyeZK8oeo99rL7nrb0CaPvzrVP7LoF+flX9gp5qt30841QPTVwNgZ
ryC+mrlWTidRpFF/uKCctDOwDJgw6pKNf352F+n/Oc85maBTySgIla1ZEqz+B+G3
tdgdTiDT/ueZY0BNFubnWlpjVTP+rwQjOrq1cD/Z53zV6APs4v7RQ0JBqDeJcadj
5xohEmZh47lKiNqsrSpB+CZy5mebxEalB3ptB+O7zexwLoixzJB4wmqfbP/hcO69
ijp58mhdoYDZqqwNJXoRNQ6OfQ9KlTyxtQwQGNcKCDiOOzZkhPInaYFnDo4AARG7
bI4z4eMpDuAm0VKi+b1voASSDxvIcT1gUZVVEtQWR5O3lzWDYmpKLsdMXQi34TKG
CXtpjgq5CR8x8kFhVQD8QijTG/zOsDf0pksF1AZx/6DQvN3JaFy3hy2dSW1Plbm6
n0WMDIkJ8w9IxofU+pFcu+tJuSRvKdcieK6dHSMHSrTvUAZc3VcCXWI4w25eODX2
985JoQF5tP6IizxBOv334VwizGu7GRyPmLLMSnQFuJXzjB52v2w=
=cN5b
-END PGP SIGNATURE-

Re: Solr Heap Usage

[Shawn Heisey]

On 6/2/2019 4:35 PM, John Davis wrote:

If we assume there is no query load then effectively this boils down to
most effective way for adding a large number of documents to the solr
index. I've looked through SolrJ, DIH and others -- is the bottomline
across all of them to "batch updates" and not commit as long as possible?


If you want the maximum indexing speed, you'll need to batch updates and 
send multiple batches in parallel.  I cannot tell you how much 
concurrency you need, you'll have to experiment.  I would probably start 
at the same number of threads as you have CPU cores in your Solr server, 
and then try 1.5 times that, and 2 times that, see which works better. 
I'd even try 3 or 4 times the CPU count, just to see how it behaves.


As long as commits are not happening in rapid succession, I wouldn't 
worry too much about those interfering with indexing speed.  Commits 
that don't open a searcher probably should be no more frequent than 
every minute or two, commits that DO open a new searcher should be less 
frequent than that.


Thanks,
Shawn

Re: where to see deleted document in Solr log

[Shawn Heisey]

On 6/3/2019 2:51 PM, Wendy2 wrote:

Hi,

I am using Solr 7.3.1 to index data via DIH.
Solr admin panel indicated that 152160 documents got indexed, while 3944
documents were deleted. But DIH indicated that added/update: 662059
documents. Deleted 0 documents.
I try to find the deleted documents, but I don't see anywhere in the solr
log. Where could I see it? Is there something I need to configure?  Why the
two status summaries were so different?   Thanks!


When there is an existing document that has the same value in the 
uniqueKey field as a new one that is being indexed, Solr deletes the 
current one before it indexes the new one.  That operation is silent.


So you may not see any actual deletions in the log.

Probably what's happening is that the 60 documents that were indexed 
included duplicates which deleted the previous version automatically. 
Through segment merging, only 3944 of those deletions remain when the 
dust settles.


Thanks,
Shawn

where to see deleted document in Solr log

[Wendy2]

Hi,

I am using Solr 7.3.1 to index data via DIH.
Solr admin panel indicated that 152160 documents got indexed, while 3944
documents were deleted. But DIH indicated that added/update: 662059
documents. Deleted 0 documents.
I try to find the deleted documents, but I don't see anywhere in the solr
log. Where could I see it? Is there something I need to configure?  Why the
two status summaries were so different?   Thanks!
 


 


 



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Solr Learning to Rank - Trailing slash in path behavior differences, expected?

[Doug Turnbull]

Hello everyone,

I encountered some surprising behavior that got be stuck on Solr LTR for a
good hour. I wanted to share it, and you can decide if its a bug (I suspect
it's a bug)

I wanted to list all the feature stores on my Solr 7.7.1 instance. So I
visited

GET http://localhost:8983/solr/tmdb/schema/feature-store/

And I scratched my head, because I thought I would find a list of
featureStores like:

{
  "responseHeader":{
"status":0,
"QTime":0},
  "featureStores":["title"]}


Instead, I got the response for a feature store with empty features,
which was quite perplexing:

  "responseHeader":{
"status":0,
"QTime":6},
  "features":[]}

Once I removed the trailing / from my URL, I got the expected response:

{
  "responseHeader":{
"status":0,
"QTime":0},
  "featureStores":["title"]}


Feels like this is a really easy mistake to make and lose a lot of time on.
But is there a reason this is actually somehow expected behavior? It seems
that with a / my request is interpreted as for a feature store (of name
space or /?!?), which I'm guessing would not be what someone wanted.

Thoughts?

-- 
*Doug Turnbull **| CTO* | OpenSource Connections
, LLC | 240.476.9983
Author: Relevant Search 
This e-mail and all contents, including attachments, is considered to be
Company Confidential unless explicitly stated otherwise, regardless
of whether attachments are marked as such.

RE: Using Solr as a Database?

[Davis, Daniel (NIH/NLM) [C]]

I think the sweet spot of Cassandra and Solr should be mentioned in this 
discussion.   Cassandra is more scalable/clusterable than an RDBMS, without 
losing all of the structure that is desirable in an RDBMS.   

In contrast, if you use a full document store such as MongoDB, you lose some of 
the abilities to know what is in your schema.

DataStax markets a platform that combines Cassandra (as a distributed 
replacement for an RDBMS) that is integrated with Solr so that records in 
managed in Cassandra are indexed and up-to-date.

If your real problem with an RDBMS is the lack of scaling, but you like the 
ability to specify columnar structure explicitly, then this combination might 
be a good fit.

Now, MongoDB is also a strong alternative to an RDBMS.

The other thing to recall though is that the power of sharding has reached into 
the databases themselves, and databases such as PostgreSQL can operate with 
some tables sharded and other tables duplicated.   See 
https://pgdash.io/blog/postgres-11-sharding.html.

Re: Using Solr as a Database?

[Shawn Heisey]

On 6/2/2019 7:28 AM, Ralph Soika wrote:

This is not intended to contradict the other replies you've gotten, only 
supplement them.


Now as far as I understand is solr a cluster enabled datastore which can 
be used to store also all the data form our document.
The problem with relational databases was always the lack of 
cloud/cluster support to get more stable data by using redundancy over 
serveral nodes.


At it's heart, Solr is using something you already understand -- Lucene. 
 Certain functionality is implemented above that in Solr -- facets 
being probably the primary example.  For the most part, if you wouldn't 
use Lucene for some purpose, you shouldn't use Solr for that purpose 
either -- because Solr is written with the Lucene API.


Search is Solr's primary function, and what it is optimized to do.  Any 
other use, even when it is possible, is probably going to be better 
handled by another piece of software.


We have done what we can to eliminate problems that lose data, but data 
retention in the face of all potential problems is not one of the design 
goals.  Things CAN go wrong that result in data loss ... while most 
database software is hardened against data loss from even unexpected 
problems.


What do you think? Is solr an alternative to store and index data 
instead of useing Lucene in combination with RDBMS?


In general, no.  There are things databases can do that Solr can't, and 
some things that a database is better at than Solr is.  Solr is good at 
search, and things related to search.


If you have the system resources, putting a complete copy of your data 
in Solr is not necessarily a bad thing.  Some amazing things can be done 
in the arena of data mining.  The facet feature that I mentioned above 
tends to be very usable for that.


Thanks,
Shawn

Re: alias read access impossible for anyone other than admin?

[Sotiris Fragkiskos]

it's 7.2.1. Thanks!

On Mon, Jun 3, 2019 at 6:26 PM Jason Gerlowski 
wrote:

> Hi Sotiris,
>
> What version of Solr are you running?  The behavior has changed some
> over time, both intentionally and due to bugs that have come and gone
> over time.  I (or someone else) can explain things and offer you
> better help once we know your Solr version.
>
> Jason
>
> On Mon, Jun 3, 2019 at 12:13 PM Sotiris Fragkiskos 
> wrote:
> >
> > Hi again,
> >
> > I moved the "all" permission to the bottom as suggested, but it still
> > doesn't work. Actually, i tried all possible combinations that I could
> > think of, but I just can't get it to work.
> > Could there be something else that I'm doing wrong? I'm a complete
> newbie,
> > so pretty much anything is a possibility at this point :(
> > Could it be because I use getfile/putfile commands to update the
> > security.json file? (it seems to be working, i.e. what i put with putfile
> > is later retrieved successfully with getfile)
> > Could there be some system update/refresh mechanism that I'm not aware of
> > and is currently not taking place?
> > Could someone please ELI5 going through the rules one by one? I can't
> > exactly understand the "narrative" that's going on,
> >
> > My security.json file's "authorization"  at this point looks like the
> > snippet below, and almost nothing is working (except admin, and userC
> who,
> > for some weird reason, can access  readCollC55b , which is tied to a role
> > that the userC is NOT tied to..
> > I'm completely lost any pointers, anyone?
> > Mind you, i'm testing whether it works either directly in the browser by
> > prepending a "username:password@" to the URL or from the cmdline with a
> > curl command like so:
> > *curl http://@IP/solr/collName/select?q=field:value*
> >
> > Many thanks!
> > Sotiri
> >
> > "authorization":{
> > "class":"solr.RuleBasedAuthorizationPlugin",
> > "permissions":[
> >   {
> > "name":"readCollA",
> > "collection":"CollA",
> > "path":"/select/*",
> > "role":"readCollA",
> > "index":1},
> >   {
> > "name":"readCollB",
> > "collection":"CollB",
> > "path":"/select/*",
> > "role":"readCollB",
> > "index":2},
> >   {
> > "name":"readCollC55b",
> > "collection":"CollC55b",
> > "path":"/select/*",
> > "role":"readCollC55b",
> > "index":3},
> >   {
> > "name":"readCollCProduction",
> > "collection":"CollCProd",
> > "path":"/select/*",
> > "role":"readCollCProduction",
> > "index":4},
> >   {
> > "name":"all",
> > "role":"admin",
> > "index":5}],
> > "user-role":{
> >   "admin":[
> > "admin",
> > "readCollB",
> > "readCollA",
> > "readCollC55b",
> > "readCollCProduction"],
> >   "userA":["readCollC55b"],
> >   "userB":["readCollC55b"],
> >   "userC":["readCollCProduction"],
> >   "userD":[
> > "readCollCProduction",
> > "readCollC55b",
> > "readCollB",
> > "readCollA"]},
> >
> >
> >
> > On Fri, May 31, 2019 at 9:07 PM Sotiris Fragkiskos 
> > wrote:
> >
> > > Terribly sorry about the duplicate post. It was just when i had first
> > > subscribed, i mustn't have verified my subscription because i never
> > > received any posts. I could also not find my post in the mailing list
> > > archive, so I thought it never arrived. It was only today that I tried
> > > subscribing again (+verifying) that I started receiving emails.
> > > Thanks for your explanation, I had read this in the manual but it
> didn't
> > > make much sense to me. I intepreted my order as: "first rule, the
> request
> > > is not from an admin so fail, check the next rule, it's from role
> readColl
> > > trying to access Coll, go ahead"
> > > I will try it as soon as I can. Thanks very much.
> > > I'm currently using 7.2.
> > >
> > > On Fri, May 31, 2019 at 8:27 PM Jason Gerlowski  >
> > > wrote:
> > >
> > >> Hi Sotiris,
> > >>
> > >> Is this your second time asking this question here, or is there a
> > >> subtle difference I'm missing?  You asked a very similar question a
> > >> week or so ago, and I replied with a few suggestions for changing your
> > >> security.json and with a few questions.  In case you missed it for
> > >> whatever reason, I'll include my original response below:
> > >>
> > >> -
> > >>
> > >> Hi Sotiris,
> > >>
> > >> First, what version of Solr are you running?  We've made some fixes
> > >> recently (esp. SOLR-13355) to RBAP, and they might affect the behavior
> > >> you're seeing or any fixes we can recommend.
> > >>
> > >> Second, the order of permissions in security.json has a huge effect on
> > >> how .  Solr always uses the first permission rule that matches a given
> > >> API...later rules are ignored if a match is found in earlier ones.
> > >> The first rule in your permissions block ({"name": "all", 

Re: Solr 8.1.1 / Zookeeper 3.5.5 problem

[Erick Erickson]

See: https://issues.apache.org/jira/browse/SOLR-8346. We haven’t released 
anything yet that even purports to run against ZK 3.5.5 since it’s so new, but 
8.2 should have the upgrade.

Meanwhile, if you were ambitious you could apply the patch at the JIRA above 
and try it, or wait a few days and download/compile after the patch is 
committed, which should be today or tomorrow.

IOW, you’re on the bleeding edge… I’d like to get all the mileage on this that 
we can before releasing 8.2, so all help welcome….

Best,
Erick

> On Jun 3, 2019, at 8:51 AM, Paul Isaac's  
> wrote:
> 
> Hi,
> when attempting to run the example to create a collection the client appears 
> to timeout but has partially created the collection.
> 
> [solr@vm-ckan-head ~]$ /opt/solr/bin/solr start -e cloud
> 
> Welcome to the SolrCloud example!
> 
> This interactive session will help you launch a SolrCloud cluster on your 
> local workstation.
> To begin, how many Solr nodes would you like to run in your local cluster? 
> (specify 1-4 nodes) [2]: 
> 
> Ok, let's start up 2 Solr nodes for your example SolrCloud cluster.
> Please enter the port for node1 [8983]: 
> 
> Please enter the port for node2 [7574]: 
> 
> Creating Solr home directory /opt/solr/example/cloud/node1/solr
> Cloning /opt/solr/example/cloud/node1 into
>   /opt/solr/example/cloud/node2
> 
> Starting up Solr on port 8983 using command:
> "/opt/solr/bin/solr" start -cloud -p 8983 -s 
> "/opt/solr/example/cloud/node1/solr"
> 
> Waiting up to 180 seconds to see Solr running on port 8983 [\]  
> Started Solr server on port 8983 (pid=4425). Happy searching!
> 
> 
> Starting up Solr on port 7574 using command:
> "/opt/solr/bin/solr" start -cloud -p 7574 -s 
> "/opt/solr/example/cloud/node2/solr" -z 
> 192.168.124.125:2181,192.168.124.126:2181,192.168.124.124:2181/solr
> 
> Waiting up to 180 seconds to see Solr running on port 7574 [\]  
> Started Solr server on port 7574 (pid=4605). Happy searching!
> 
> INFO  - 2019-06-03 16:25:53.367; 
> org.apache.solr.common.cloud.ConnectionManager; zkClient has connected
> INFO  - 2019-06-03 16:25:53.412; org.apache.solr.common.cloud.ZkStateReader; 
> Updated live nodes from ZooKeeper... (0) -> (2)
> INFO  - 2019-06-03 16:25:53.457; 
> org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider; Cluster at 
> 192.168.124.125:2181,192.168.124.126:2181,192.168.124.124:2181/solr ready
> 
> Now let's create a new collection for indexing documents in your 2-node 
> cluster.
> Please provide a name for your new collection: [gettingstarted] 
> 
> How many shards would you like to split gettingstarted into? [2]
> 
> How many replicas per shard would you like to create? [2] 
> 
> Please choose a configuration for the gettingstarted collection, available 
> options are:
> _default or sample_techproducts_configs [_default] 
> 
> sample_techWARN  - 2019-06-03 16:26:33.927; org.apache.zookeeper.ClientCnxn; 
> Client session timed out, have not heard from server in 26679ms for sessionid 
> 0x1003d370009
> WARN  - 2019-06-03 16:26:34.032; 
> org.apache.solr.common.cloud.ConnectionManager; Watcher 
> org.apache.solr.common.cloud.ConnectionManager@2a6dc6ad name: 
> ZooKeeperConnection 
> Watcher:192.168.124.125:2181,192.168.124.126:2181,192.168.124.124:2181/solr 
> got event WatchedEvent state:Disconnected type:None path:null path: null 
> type: None
> WARN  - 2019-06-03 16:26:34.032; 
> org.apache.solr.common.cloud.ConnectionManager; zkClient has disconnected
> products_configs
> WARN  - 2019-06-03 16:27:01.683; org.apache.zookeeper.ClientCnxn; Client 
> session timed out, have not heard from server in 26679ms for sessionid 
> 0x1003d370009
> WARN  - 2019-06-03 16:27:01.784; 
> org.apache.solr.common.cloud.ConnectionManager; Watcher 
> org.apache.solr.common.cloud.ConnectionManager@2a6dc6ad name: 
> ZooKeeperConnection 
> Watcher:192.168.124.125:2181,192.168.124.126:2181,192.168.124.124:2181/solr 
> got event WatchedEvent state:Disconnected type:None path:null path: null 
> type: None
> WARN  - 2019-06-03 16:27:01.785; 
> org.apache.solr.common.cloud.ConnectionManager; zkClient has disconnected
> ^C[solr@vm-ckan-head ~]$ /opt/solr/bin/solr status
> 
> Found 2 Solr nodes: 
> 
> Solr process 4425 running on port 8983
> {
>  "solr_home":"/opt/solr/example/cloud/node1/solr",
>  "version":"8.1.1 fcbe46c28cef11bc058779afba09521de1b19bef - ab - 2019-05-22 
> 15:20:01",
>  "startTime":"2019-06-03T15:25:43.816Z",
>  "uptime":"0 days, 0 hours, 1 minutes, 34 seconds",
>  "memory":"55.7 MB (%10.9) of 512 MB",
>  "cloud":{
>
> "ZooKeeper":"192.168.124.125:2181,192.168.124.126:2181,192.168.124.124:2181/solr",
>"liveNodes":"2",
>"collections":"0"}}
> 
> 
> Solr process 4605 running on port 7574
> {
>  "solr_home":"/opt/solr/example/cloud/node2/solr",
>  "version":"8.1.1 fcbe46c28cef11bc058779afba09521de1b19bef - ab - 2019-05-22 
> 15:20:01",
>  "startTime":"2019-06-03T15:25:48.925Z",
>  "uptime":"0 days, 0 hours, 1 minutes, 30 seconds",
>  "memory

Solr 8.1.1 / Zookeeper 3.5.5 problem

[Paul Isaac's]

Hi,
when attempting to run the example to create a collection the client appears to 
timeout but has partially created the collection.

[solr@vm-ckan-head ~]$ /opt/solr/bin/solr start -e cloud

Welcome to the SolrCloud example!

This interactive session will help you launch a SolrCloud cluster on your local 
workstation.
To begin, how many Solr nodes would you like to run in your local cluster? 
(specify 1-4 nodes) [2]: 

Ok, let's start up 2 Solr nodes for your example SolrCloud cluster.
Please enter the port for node1 [8983]: 

Please enter the port for node2 [7574]: 

Creating Solr home directory /opt/solr/example/cloud/node1/solr
Cloning /opt/solr/example/cloud/node1 into
   /opt/solr/example/cloud/node2

Starting up Solr on port 8983 using command:
"/opt/solr/bin/solr" start -cloud -p 8983 -s 
"/opt/solr/example/cloud/node1/solr"

Waiting up to 180 seconds to see Solr running on port 8983 [\]  
Started Solr server on port 8983 (pid=4425). Happy searching!


Starting up Solr on port 7574 using command:
"/opt/solr/bin/solr" start -cloud -p 7574 -s 
"/opt/solr/example/cloud/node2/solr" -z 
192.168.124.125:2181,192.168.124.126:2181,192.168.124.124:2181/solr

Waiting up to 180 seconds to see Solr running on port 7574 [\]  
Started Solr server on port 7574 (pid=4605). Happy searching!

INFO  - 2019-06-03 16:25:53.367; 
org.apache.solr.common.cloud.ConnectionManager; zkClient has connected
INFO  - 2019-06-03 16:25:53.412; org.apache.solr.common.cloud.ZkStateReader; 
Updated live nodes from ZooKeeper... (0) -> (2)
INFO  - 2019-06-03 16:25:53.457; 
org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider; Cluster at 
192.168.124.125:2181,192.168.124.126:2181,192.168.124.124:2181/solr ready

Now let's create a new collection for indexing documents in your 2-node cluster.
Please provide a name for your new collection: [gettingstarted] 

How many shards would you like to split gettingstarted into? [2]

How many replicas per shard would you like to create? [2] 

Please choose a configuration for the gettingstarted collection, available 
options are:
_default or sample_techproducts_configs [_default] 

sample_techWARN  - 2019-06-03 16:26:33.927; org.apache.zookeeper.ClientCnxn; 
Client session timed out, have not heard from server in 26679ms for sessionid 
0x1003d370009
WARN  - 2019-06-03 16:26:34.032; 
org.apache.solr.common.cloud.ConnectionManager; Watcher 
org.apache.solr.common.cloud.ConnectionManager@2a6dc6ad name: 
ZooKeeperConnection 
Watcher:192.168.124.125:2181,192.168.124.126:2181,192.168.124.124:2181/solr got 
event WatchedEvent state:Disconnected type:None path:null path: null type: None
WARN  - 2019-06-03 16:26:34.032; 
org.apache.solr.common.cloud.ConnectionManager; zkClient has disconnected
products_configs
WARN  - 2019-06-03 16:27:01.683; org.apache.zookeeper.ClientCnxn; Client 
session timed out, have not heard from server in 26679ms for sessionid 
0x1003d370009
WARN  - 2019-06-03 16:27:01.784; 
org.apache.solr.common.cloud.ConnectionManager; Watcher 
org.apache.solr.common.cloud.ConnectionManager@2a6dc6ad name: 
ZooKeeperConnection 
Watcher:192.168.124.125:2181,192.168.124.126:2181,192.168.124.124:2181/solr got 
event WatchedEvent state:Disconnected type:None path:null path: null type: None
WARN  - 2019-06-03 16:27:01.785; 
org.apache.solr.common.cloud.ConnectionManager; zkClient has disconnected
^C[solr@vm-ckan-head ~]$ /opt/solr/bin/solr status

Found 2 Solr nodes: 

Solr process 4425 running on port 8983
{
  "solr_home":"/opt/solr/example/cloud/node1/solr",
  "version":"8.1.1 fcbe46c28cef11bc058779afba09521de1b19bef - ab - 2019-05-22 
15:20:01",
  "startTime":"2019-06-03T15:25:43.816Z",
  "uptime":"0 days, 0 hours, 1 minutes, 34 seconds",
  "memory":"55.7 MB (%10.9) of 512 MB",
  "cloud":{

"ZooKeeper":"192.168.124.125:2181,192.168.124.126:2181,192.168.124.124:2181/solr",
"liveNodes":"2",
"collections":"0"}}


Solr process 4605 running on port 7574
{
  "solr_home":"/opt/solr/example/cloud/node2/solr",
  "version":"8.1.1 fcbe46c28cef11bc058779afba09521de1b19bef - ab - 2019-05-22 
15:20:01",
  "startTime":"2019-06-03T15:25:48.925Z",
  "uptime":"0 days, 0 hours, 1 minutes, 30 seconds",
  "memory":"223.7 MB (%43.7) of 512 MB",
  "cloud":{

"ZooKeeper":"192.168.124.125:2181,192.168.124.126:2181,192.168.124.124:2181/solr",
"liveNodes":"2",
"collections":"0"}}


[solr@vm-ckan-head solr]$ bin/solr create -c mycollection -d _default
WARNING: Using _default configset with data driven schema functionality. NOT 
RECOMMENDED for production use.
 To turn off: bin/solr config -c mycollection -p 7574 -action 
set-user-property -property update.autoCreateFields -value false
WARN  - 2019-06-03 16:43:05.394; org.apache.zookeeper.ClientCnxn; Client 
session timed out, have not heard from server in 26675ms for sessionid 
0x3007037
WARN  - 2019-06-03 16:43:05.513; 
org.apache.solr.common.cloud.ConnectionManager; Watcher 
org.apache.solr.common.cloud.Connectio

Re: SolrJ, CloudSolrClient and basic authentication

[Kevin Risden]

Chris - not sure if what you are seeing is related to basic auth
credentials not being sent until a 401. There was report of this behavior
with Apache Knox in front of Solr.

https://issues.apache.org/jira/browse/KNOX-1066

The jira above has an example of how to preemptively send basic auth
instead of waiting for the 401 from the server.

Kevin Risden


On Fri, May 31, 2019 at 4:28 PM Christopher Schultz <
ch...@christopherschultz.net> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Dimitris,
>
> On 6/1/18 02:46, Dimitris Kardarakos wrote:
> > Thanks a lot Shawn. I had tried with the documented approach, but
> > since I use SolrClient.add to add documents to the index, I could
> > not "port" the documented approach to my case (probably I do miss
> > something).
> >
> > The custom HttpClient suggestion worked as expected!
>
> Can you please explain how you did this?
>
> I'm facing a problem where the simplest possible solution is giving
> the error "org.apache.http.client.NonRepeatableRequestException:
> Cannot retry request with a non-repeatable request entity.".
>
> It seems that SolrClient is using something like BasicHttpEntity which
> isn't "repeatable" when using HTTP Basic auth (where the server is
> supposed to challenge the client and the client only then sends the
> credentials). I need to either make the client data repeatable (which
> is in SolrClient, which I'd prefer to avoid) or I need to make
> HttpClient use an "expectant" credential-sending technique, or I need
> to just stuff things into a header manually.
>
> What did you do to solve this problem? It seems like this should
> really probably come up more often than it does. Maybe nobody bothers
> to lock-down their Solr instances?
>
> Thanks,
> - -chris
>
> > On 31/05/2018 06:16 μμ, Shawn Heisey wrote:
> >> On 5/31/2018 8:03 AM, Dimitris Kardarakos wrote:
> >>> Following the feedback in the "Index protected zip" thread, I
> >>> am trying to add documents to the index using SolrJ API.
> >>>
> >>> The server is in SolrCloud mode with BasicAuthPlugin for
> >>> authentication.
> >>>
> >>> I have not managed to figure out how to pass username/password
> >>> to my client.
> >> There are two ways to approach this.
> >>
> >> One approach is to build a custom HttpClient object that uses
> >> credentials by default, and then use that custom HttpClient
> >> object to build your CloudSolrClient.  Exactly how to correctly
> >> build the HttpClient object will depend on exactly which
> >> HttpClient version you've included into your program.  If you go
> >> with SolrJ dependency defaults, then the HttpClient version will
> >> depend on the SolrJ version.
> >>
> >> The other approach is the method described in the documentation,
> >> where credentials are added to each request object:
> >>
> >> https://lucene.apache.org/solr/guide/6_6/basic-authentication-plugin.
> html#BasicAuthenticationPlugin-UsingBasicAuthwithSolrJ
> 
> >>
> >>
> >>
> >>
> There are several different kinds of request objects.  A few examples:
> >> UpdateRequest, QueryRequest, CollectionAdminRequest.
> >>
> >> Thanks, Shawn
> >>
> >
> -BEGIN PGP SIGNATURE-
> Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
>
> iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlzxjlEACgkQHPApP6U8
> pFhoeQ/7BzlhjGGE8tnMcrdmruP+N2rgvawfLcTdzDg3U4cQFNUVRoCclZcM8LiA
> iuZf+cAewTTQTjLpQuSv2WoknQgO/YRgaqTlo+b3hv9zR2awY8Mob/m5RYcYAwmn
> i+2SJrG7+u+qhpfDQGSjwppUKpm2WrfvGXL3lcRF48UXQ+z7J95o2g88SnP44FKH
> 87/X/iYX+xMsj0bkIEOkyppuXENQQwUZ7QWhgfAxSItJr2A0Ma6zkuuNPf4FvBJ1
> JQM/c33WWbAXK3B7tI5iQsstVi5CMOhRF0Z336/vZgq6aF9uEZvIOWEVAlM+E8Qp
> mYlZz7tERzUMs+QbcBcSdDIb8VSPwYy5kvKiJ9eEpjFGXmPBLOqiJ4M+4SOeGFq7
> BA5sbm6k4gwHc33MiKvnHE1K+k3r1OBPngjxvelsyIaqSnX3zpKPTFhkU2dvWMPt
> XPo/ICuiliGowD8xh5EhB6w0BuYZhK3dW7AKMCLbyoANwk7SLfHxC6O+rdmYyDQF
> UwiR65+3ImmeKJOZt7lFoR43BXoFuz6L1SILU8XRcclS5KwXHg3moBElU7jM9iKV
> 9vMwWkuPGUA2gq5K0oV4XFEOShxUxFiCL4FXjd/P7x9Evhio+itvaUlHzP8FGblh
> YyK+l2YqjKBnTJ0G4XE8UnJcmH8C23jJ05gwMgq92pXBQy5ly6s=
> =6kab
> -END PGP SIGNATURE-
>

Re: alias read access impossible for anyone other than admin?

[Jason Gerlowski]

Hi Sotiris,

What version of Solr are you running?  The behavior has changed some
over time, both intentionally and due to bugs that have come and gone
over time.  I (or someone else) can explain things and offer you
better help once we know your Solr version.

Jason

On Mon, Jun 3, 2019 at 12:13 PM Sotiris Fragkiskos  wrote:
>
> Hi again,
>
> I moved the "all" permission to the bottom as suggested, but it still
> doesn't work. Actually, i tried all possible combinations that I could
> think of, but I just can't get it to work.
> Could there be something else that I'm doing wrong? I'm a complete newbie,
> so pretty much anything is a possibility at this point :(
> Could it be because I use getfile/putfile commands to update the
> security.json file? (it seems to be working, i.e. what i put with putfile
> is later retrieved successfully with getfile)
> Could there be some system update/refresh mechanism that I'm not aware of
> and is currently not taking place?
> Could someone please ELI5 going through the rules one by one? I can't
> exactly understand the "narrative" that's going on,
>
> My security.json file's "authorization"  at this point looks like the
> snippet below, and almost nothing is working (except admin, and userC who,
> for some weird reason, can access  readCollC55b , which is tied to a role
> that the userC is NOT tied to..
> I'm completely lost any pointers, anyone?
> Mind you, i'm testing whether it works either directly in the browser by
> prepending a "username:password@" to the URL or from the cmdline with a
> curl command like so:
> *curl http://@IP/solr/collName/select?q=field:value*
>
> Many thanks!
> Sotiri
>
> "authorization":{
> "class":"solr.RuleBasedAuthorizationPlugin",
> "permissions":[
>   {
> "name":"readCollA",
> "collection":"CollA",
> "path":"/select/*",
> "role":"readCollA",
> "index":1},
>   {
> "name":"readCollB",
> "collection":"CollB",
> "path":"/select/*",
> "role":"readCollB",
> "index":2},
>   {
> "name":"readCollC55b",
> "collection":"CollC55b",
> "path":"/select/*",
> "role":"readCollC55b",
> "index":3},
>   {
> "name":"readCollCProduction",
> "collection":"CollCProd",
> "path":"/select/*",
> "role":"readCollCProduction",
> "index":4},
>   {
> "name":"all",
> "role":"admin",
> "index":5}],
> "user-role":{
>   "admin":[
> "admin",
> "readCollB",
> "readCollA",
> "readCollC55b",
> "readCollCProduction"],
>   "userA":["readCollC55b"],
>   "userB":["readCollC55b"],
>   "userC":["readCollCProduction"],
>   "userD":[
> "readCollCProduction",
> "readCollC55b",
> "readCollB",
> "readCollA"]},
>
>
>
> On Fri, May 31, 2019 at 9:07 PM Sotiris Fragkiskos 
> wrote:
>
> > Terribly sorry about the duplicate post. It was just when i had first
> > subscribed, i mustn't have verified my subscription because i never
> > received any posts. I could also not find my post in the mailing list
> > archive, so I thought it never arrived. It was only today that I tried
> > subscribing again (+verifying) that I started receiving emails.
> > Thanks for your explanation, I had read this in the manual but it didn't
> > make much sense to me. I intepreted my order as: "first rule, the request
> > is not from an admin so fail, check the next rule, it's from role readColl
> > trying to access Coll, go ahead"
> > I will try it as soon as I can. Thanks very much.
> > I'm currently using 7.2.
> >
> > On Fri, May 31, 2019 at 8:27 PM Jason Gerlowski 
> > wrote:
> >
> >> Hi Sotiris,
> >>
> >> Is this your second time asking this question here, or is there a
> >> subtle difference I'm missing?  You asked a very similar question a
> >> week or so ago, and I replied with a few suggestions for changing your
> >> security.json and with a few questions.  In case you missed it for
> >> whatever reason, I'll include my original response below:
> >>
> >> -
> >>
> >> Hi Sotiris,
> >>
> >> First, what version of Solr are you running?  We've made some fixes
> >> recently (esp. SOLR-13355) to RBAP, and they might affect the behavior
> >> you're seeing or any fixes we can recommend.
> >>
> >> Second, the order of permissions in security.json has a huge effect on
> >> how .  Solr always uses the first permission rule that matches a given
> >> API...later rules are ignored if a match is found in earlier ones.
> >> The first rule in your permissions block ({"name": "all", "role":
> >> "admin"}) will match all APIs and will only allow requests through if
> >> the requesting user has the "admin" role.  So "user" being unable to
> >> query an alias makes sense.  Usually "all" and other catchall
> >> permissions are best used at the very bottom of your permissions list.
> >> That way the catchall is the last r

Re: alias read access impossible for anyone other than admin?

[Sotiris Fragkiskos]

Hi again,

I moved the "all" permission to the bottom as suggested, but it still
doesn't work. Actually, i tried all possible combinations that I could
think of, but I just can't get it to work.
Could there be something else that I'm doing wrong? I'm a complete newbie,
so pretty much anything is a possibility at this point :(
Could it be because I use getfile/putfile commands to update the
security.json file? (it seems to be working, i.e. what i put with putfile
is later retrieved successfully with getfile)
Could there be some system update/refresh mechanism that I'm not aware of
and is currently not taking place?
Could someone please ELI5 going through the rules one by one? I can't
exactly understand the "narrative" that's going on,

My security.json file's "authorization"  at this point looks like the
snippet below, and almost nothing is working (except admin, and userC who,
for some weird reason, can access  readCollC55b , which is tied to a role
that the userC is NOT tied to..
I'm completely lost any pointers, anyone?
Mind you, i'm testing whether it works either directly in the browser by
prepending a "username:password@" to the URL or from the cmdline with a
curl command like so:
*curl http://@IP/solr/collName/select?q=field:value*

Many thanks!
Sotiri

"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"permissions":[
  {
"name":"readCollA",
"collection":"CollA",
"path":"/select/*",
"role":"readCollA",
"index":1},
  {
"name":"readCollB",
"collection":"CollB",
"path":"/select/*",
"role":"readCollB",
"index":2},
  {
"name":"readCollC55b",
"collection":"CollC55b",
"path":"/select/*",
"role":"readCollC55b",
"index":3},
  {
"name":"readCollCProduction",
"collection":"CollCProd",
"path":"/select/*",
"role":"readCollCProduction",
"index":4},
  {
"name":"all",
"role":"admin",
"index":5}],
"user-role":{
  "admin":[
"admin",
"readCollB",
"readCollA",
"readCollC55b",
"readCollCProduction"],
  "userA":["readCollC55b"],
  "userB":["readCollC55b"],
  "userC":["readCollCProduction"],
  "userD":[
"readCollCProduction",
"readCollC55b",
"readCollB",
"readCollA"]},



On Fri, May 31, 2019 at 9:07 PM Sotiris Fragkiskos 
wrote:

> Terribly sorry about the duplicate post. It was just when i had first
> subscribed, i mustn't have verified my subscription because i never
> received any posts. I could also not find my post in the mailing list
> archive, so I thought it never arrived. It was only today that I tried
> subscribing again (+verifying) that I started receiving emails.
> Thanks for your explanation, I had read this in the manual but it didn't
> make much sense to me. I intepreted my order as: "first rule, the request
> is not from an admin so fail, check the next rule, it's from role readColl
> trying to access Coll, go ahead"
> I will try it as soon as I can. Thanks very much.
> I'm currently using 7.2.
>
> On Fri, May 31, 2019 at 8:27 PM Jason Gerlowski 
> wrote:
>
>> Hi Sotiris,
>>
>> Is this your second time asking this question here, or is there a
>> subtle difference I'm missing?  You asked a very similar question a
>> week or so ago, and I replied with a few suggestions for changing your
>> security.json and with a few questions.  In case you missed it for
>> whatever reason, I'll include my original response below:
>>
>> -
>>
>> Hi Sotiris,
>>
>> First, what version of Solr are you running?  We've made some fixes
>> recently (esp. SOLR-13355) to RBAP, and they might affect the behavior
>> you're seeing or any fixes we can recommend.
>>
>> Second, the order of permissions in security.json has a huge effect on
>> how .  Solr always uses the first permission rule that matches a given
>> API...later rules are ignored if a match is found in earlier ones.
>> The first rule in your permissions block ({"name": "all", "role":
>> "admin"}) will match all APIs and will only allow requests through if
>> the requesting user has the "admin" role.  So "user" being unable to
>> query an alias makes sense.  Usually "all" and other catchall
>> permissions are best used at the very bottom of your permissions list.
>> That way the catchall is the last rule to be checked, giving other
>> rules a chance to match first.
>>
>> Hope that helps.
>>
>> On Fri, May 31, 2019 at 9:34 AM Sotiris Fragkiskos 
>> wrote:
>> >
>> > Hi everyone!
>> > I've been trying unsuccessfully to read an alias to a collection with a
>> > curl command.
>> > The command only works when I put in the admin credentials, although the
>> > user I want access for also has the required role for accessing.
>> > Is this perhaps built-in, or should anyone be able to access an alias
>> from
>> > the API?
>> >
>> > The command I'm using is:
>> > 

Re: Adding Multiple JSON Documents

[Alexandre Rafalovitch]

Hi John,

This may be useful:
https://www.slideshare.net/arafalov/json-in-solr-from-top-to-bottom
(there is the video of the session at the end too).

Basically, we have two ways to process JSON and sometimes they look
very similar and you have to be very deliberate in indicating which
one is the correct way.

Regards,
   Alex.

On Sun, 2 Jun 2019 at 22:50, John Davis  wrote:
>
> Hi there,
>
> I was looking at the solr documentation for indexing multiple documents via
> json and noticed inconsistency in the docs.
>
> Should the POST url be /update/*json/docs *instead of just /update. It does
> look like former does work, unless both will work just fine?
>
> https://lucene.apache.org/solr/guide/7_3/uploading-data-with-index-handlers.html#adding-multiple-json-documents
> Adding Multiple JSON Documents
> 
>
> Adding multiple documents at one time via JSON can be done via a JSON Array
> of JSON Objects, where each object represents a document:
>
> curl -X POST -H 'Content-Type: application/json'
> 'http://localhost:8983/solr/my_collection/*update*' --data-binary '[
> {"id": "1","title": "Doc 1"  },  {"id": "2","title":
> "Doc 2"  }]'

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

[Colvin Cowie]

Hi, thanks I'll give that a go when I get a chance.

I was trying to reply to an older thread (
http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201904.mbox/%3CCAF2DzVXeVZqnixnkbzw0La1ui5N5-RG9PwfMBHG9vmkfBSMzJA%40mail.gmail.com%3E),
which I don't have in my mailbox, so obviously didn't reply to the right
address to get my response threaded so mine has appeared on its own. Oops.

A JIRA issue was raised on that thread
https://issues.apache.org/jira/browse/SOLR-13421 but it's not had any
attention.


On Mon, 3 Jun 2019 at 14:46, Jason Gerlowski  wrote:

> Hi Colvin,
>
> We're still taking a look at fixing the bug, but as a workaround in
> the meantime, you can look into adding a "forwardCredentials":true
> property under the "authentication" section of security.json.  That
> seems to fix the issue in my reproduction at least.
>
> e.g.
>
> {
> "authentication": {
> "blockUnknown": true,
> "class": "solr.BasicAuthPlugin",
> "credentials": {
> "solradmin": ""
> },
> "forwardCredentials": true
> },
> ...
> }
>
> Jason
>
> On Mon, Jun 3, 2019 at 9:31 AM Jason Gerlowski 
> wrote:
> >
> > One last note: as far as I can tell, nothing about this issue is
> > specific to JSON Faceting or the JSON request API.  It can be
> > triggered just as easily with "/select?q=*:*".
> >
> > The bug created for this is: SOLR-13510
> >
> > On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski 
> wrote:
> > >
> > > I'm also able to reproduce this bug on master.  A few more notes about
> > > the bad behavior:
> > >
> > > - the behavior occurs regardless of the specific permissions
> > > configured in security.json.  (i.e. whether the top permission is
> > > "all", or "security-edit", or there are no permissions at all.)
> > > - I tried looking for a pattern in which requests saw the 401s, but
> > > didn't have any luck.  The 401 occurs when talking to the whole
> > > collection or targeting individual cores directly.  It occurs when
> > > curl hits a host containing a replica for the collection in question,
> > > and when it doesnt. etc.  This distinguishes it from SOLR-13472, which
> > > seems more specific to collection structure/layout.
> > >
> > > I'll create a bug for this in JIRA.
> > >
> > > On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie <
> colvin.cowie@gmail.com> wrote:
> > > >
> > > > Hello. I encountered this issue too and wrote this up before I found
> this
> > > > thread, but I thought I might as well post it still, if it helps...
> > > >
> > > > Currently I'm trying to move our product on to Solr 8.1.1. We are
> currently
> > > > using 6.6.6, so things have definitely moved on.
> > > >
> > > > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock
> down Solr
> > > > (and we also secure our zookeeper). Here's an example for solradmin
> as the
> > > > user and password
> > > >
> > > > {
> > > > "authentication": {
> > > > "blockUnknown": true,
> > > > "class": "solr.BasicAuthPlugin",
> > > > "credentials": {
> > > > "solradmin":
> "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > > > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> > > > }
> > > > },
> > > > "authorization": {
> > > > "class": "solr.RuleBasedAuthorizationPlugin",
> > > > "permissions": [
> > > > {
> > > > "name": "all",
> > > > "role": "admin"
> > > > }
> > > > ],
> > > > "user-role": {
> > > > "solradmin": "admin"
> > > > }
> > > > }
> > > > }
> > > >
> > > >
> > > > On Solr 8.1.1, using our previously working security.json, running
> queries
> > > > (through the admin UI currently) I non-deterministically get 401
> responses
> > > > on queries when a collection has more than 1 shard. Increasing the
> number
> > > > of shards in the collection makes the errors more likely.
> > > >
> > > > {
> > > >   "responseHeader":{
> > > > "zkConnected":true,
> > > > "status":401,
> > > > "QTime":30,
> > > > "params":{
> > > >   "q":"*:*",
> > > >   "_":"1559474550365"}},
> > > >   "error":{
> > > > "metadata":[
> > > >
> > > >
> "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
> > > >
> > > >
> "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> > > > "msg":"Error from server at null: Expected mime type
> > > > application/octet-stream but got text/html. \n\n > > > http-equiv=\"Content-Type\"
> > > > content=\"text/html;charset=utf-8\"/>\nError 401 require
> > > > authentication\n\nHTTP ERROR
> 401\nProblem
> > > > accessing /solr/gettingstarted_shard4_replica_n6/select.
> Reason:\n
> > > >  require authentication\n\n\n",
> > > > "code":401}}
> > > >
> > > > The security stats indicate this is happening because the requests
> do not
> > > > have credentials with them, e.g.
> > > >
> http://localhost:8983

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

[Jason Gerlowski]

Hi Colvin,

We're still taking a look at fixing the bug, but as a workaround in
the meantime, you can look into adding a "forwardCredentials":true
property under the "authentication" section of security.json.  That
seems to fix the issue in my reproduction at least.

e.g.

{
"authentication": {
"blockUnknown": true,
"class": "solr.BasicAuthPlugin",
"credentials": {
"solradmin": ""
},
"forwardCredentials": true
},
...
}

Jason

On Mon, Jun 3, 2019 at 9:31 AM Jason Gerlowski  wrote:
>
> One last note: as far as I can tell, nothing about this issue is
> specific to JSON Faceting or the JSON request API.  It can be
> triggered just as easily with "/select?q=*:*".
>
> The bug created for this is: SOLR-13510
>
> On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski  wrote:
> >
> > I'm also able to reproduce this bug on master.  A few more notes about
> > the bad behavior:
> >
> > - the behavior occurs regardless of the specific permissions
> > configured in security.json.  (i.e. whether the top permission is
> > "all", or "security-edit", or there are no permissions at all.)
> > - I tried looking for a pattern in which requests saw the 401s, but
> > didn't have any luck.  The 401 occurs when talking to the whole
> > collection or targeting individual cores directly.  It occurs when
> > curl hits a host containing a replica for the collection in question,
> > and when it doesnt. etc.  This distinguishes it from SOLR-13472, which
> > seems more specific to collection structure/layout.
> >
> > I'll create a bug for this in JIRA.
> >
> > On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie  
> > wrote:
> > >
> > > Hello. I encountered this issue too and wrote this up before I found this
> > > thread, but I thought I might as well post it still, if it helps...
> > >
> > > Currently I'm trying to move our product on to Solr 8.1.1. We are 
> > > currently
> > > using 6.6.6, so things have definitely moved on.
> > >
> > > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock down 
> > > Solr
> > > (and we also secure our zookeeper). Here's an example for solradmin as the
> > > user and password
> > >
> > > {
> > > "authentication": {
> > > "blockUnknown": true,
> > > "class": "solr.BasicAuthPlugin",
> > > "credentials": {
> > > "solradmin": "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> > > }
> > > },
> > > "authorization": {
> > > "class": "solr.RuleBasedAuthorizationPlugin",
> > > "permissions": [
> > > {
> > > "name": "all",
> > > "role": "admin"
> > > }
> > > ],
> > > "user-role": {
> > > "solradmin": "admin"
> > > }
> > > }
> > > }
> > >
> > >
> > > On Solr 8.1.1, using our previously working security.json, running queries
> > > (through the admin UI currently) I non-deterministically get 401 responses
> > > on queries when a collection has more than 1 shard. Increasing the number
> > > of shards in the collection makes the errors more likely.
> > >
> > > {
> > >   "responseHeader":{
> > > "zkConnected":true,
> > > "status":401,
> > > "QTime":30,
> > > "params":{
> > >   "q":"*:*",
> > >   "_":"1559474550365"}},
> > >   "error":{
> > > "metadata":[
> > >
> > > "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
> > >
> > > "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> > > "msg":"Error from server at null: Expected mime type
> > > application/octet-stream but got text/html. \n\n > > http-equiv=\"Content-Type\"
> > > content=\"text/html;charset=utf-8\"/>\nError 401 require
> > > authentication\n\nHTTP ERROR 401\nProblem
> > > accessing /solr/gettingstarted_shard4_replica_n6/select. Reason:\n
> > >  require authentication\n\n\n",
> > > "code":401}}
> > >
> > > The security stats indicate this is happening because the requests do not
> > > have credentials with them, e.g.
> > > http://localhost:8983/solr/#/gettingstarted_shard4_replica_n6/plugins?type=security&entry=org.apache.solr.security.BasicAuthPlugin
> > >
> > >  org.apache.solr.security.BasicAuthPlugin
> > > class:
> > > org.apache.solr.security.BasicAuthPlugin
> > > description:
> > > Authentication Plugin org.apache.solr.security.BasicAuthPlugin
> > > stats
> > > SECURITY./authentication.authenticated:
> > > 182
> > > SECURITY./authentication.errors.count:
> > > 0
> > > SECURITY./authentication.failMissingCredentials:
> > > 58
> > > SECURITY./authentication.failWrongCredentials:
> > > 0
> > > SECURITY./authentication.passThrough:
> > > 0
> > > SECURITY./authentication.requestTimes.meanRate:
> > > 0.4183414110946125

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

[Jason Gerlowski]

One last note: as far as I can tell, nothing about this issue is
specific to JSON Faceting or the JSON request API.  It can be
triggered just as easily with "/select?q=*:*".

The bug created for this is: SOLR-13510

On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski  wrote:
>
> I'm also able to reproduce this bug on master.  A few more notes about
> the bad behavior:
>
> - the behavior occurs regardless of the specific permissions
> configured in security.json.  (i.e. whether the top permission is
> "all", or "security-edit", or there are no permissions at all.)
> - I tried looking for a pattern in which requests saw the 401s, but
> didn't have any luck.  The 401 occurs when talking to the whole
> collection or targeting individual cores directly.  It occurs when
> curl hits a host containing a replica for the collection in question,
> and when it doesnt. etc.  This distinguishes it from SOLR-13472, which
> seems more specific to collection structure/layout.
>
> I'll create a bug for this in JIRA.
>
> On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie  
> wrote:
> >
> > Hello. I encountered this issue too and wrote this up before I found this
> > thread, but I thought I might as well post it still, if it helps...
> >
> > Currently I'm trying to move our product on to Solr 8.1.1. We are currently
> > using 6.6.6, so things have definitely moved on.
> >
> > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock down Solr
> > (and we also secure our zookeeper). Here's an example for solradmin as the
> > user and password
> >
> > {
> > "authentication": {
> > "blockUnknown": true,
> > "class": "solr.BasicAuthPlugin",
> > "credentials": {
> > "solradmin": "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> > }
> > },
> > "authorization": {
> > "class": "solr.RuleBasedAuthorizationPlugin",
> > "permissions": [
> > {
> > "name": "all",
> > "role": "admin"
> > }
> > ],
> > "user-role": {
> > "solradmin": "admin"
> > }
> > }
> > }
> >
> >
> > On Solr 8.1.1, using our previously working security.json, running queries
> > (through the admin UI currently) I non-deterministically get 401 responses
> > on queries when a collection has more than 1 shard. Increasing the number
> > of shards in the collection makes the errors more likely.
> >
> > {
> >   "responseHeader":{
> > "zkConnected":true,
> > "status":401,
> > "QTime":30,
> > "params":{
> >   "q":"*:*",
> >   "_":"1559474550365"}},
> >   "error":{
> > "metadata":[
> >
> > "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
> >
> > "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> > "msg":"Error from server at null: Expected mime type
> > application/octet-stream but got text/html. \n\n > http-equiv=\"Content-Type\"
> > content=\"text/html;charset=utf-8\"/>\nError 401 require
> > authentication\n\nHTTP ERROR 401\nProblem
> > accessing /solr/gettingstarted_shard4_replica_n6/select. Reason:\n
> >  require authentication\n\n\n",
> > "code":401}}
> >
> > The security stats indicate this is happening because the requests do not
> > have credentials with them, e.g.
> > http://localhost:8983/solr/#/gettingstarted_shard4_replica_n6/plugins?type=security&entry=org.apache.solr.security.BasicAuthPlugin
> >
> >  org.apache.solr.security.BasicAuthPlugin
> > class:
> > org.apache.solr.security.BasicAuthPlugin
> > description:
> > Authentication Plugin org.apache.solr.security.BasicAuthPlugin
> > stats
> > SECURITY./authentication.authenticated:
> > 182
> > SECURITY./authentication.errors.count:
> > 0
> > SECURITY./authentication.failMissingCredentials:
> > 58
> > SECURITY./authentication.failWrongCredentials:
> > 0
> > SECURITY./authentication.passThrough:
> > 0
> > SECURITY./authentication.requestTimes.meanRate:
> > 0.4183414110946125
> > SECURITY./authentication.requests:
> > 240
> > SECURITY./authentication.totalTime:
> > 117791100
> >
> > I assume that this is connected to the changes around
> > https://issues.apache.org/jira/browse/SOLR-7896 and
> > https://issues.apache.org/jira/browse/SOLR-13344 I've tested with Solr
> > 7.6.0 and it appears to be unaffected
> >
> > Repro steps:
> ># Extract solr 8.1.1.
> ># bin\solr start -e cloud
> > 1 node / [default port] / [default collection name] / 4 shards / 1
> > replica / [_default configuration]
> ># server\scripts\cloud-scripts\zkcli -zkhost localhost:9983 -cmd putfile
> > /security.json 
> >
> ># Execute repeated GETS to
> > http://localhost:8983/solr/gettingstarted/select?q=*%3A* - a lot 

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

[Jason Gerlowski]

I'm also able to reproduce this bug on master.  A few more notes about
the bad behavior:

- the behavior occurs regardless of the specific permissions
configured in security.json.  (i.e. whether the top permission is
"all", or "security-edit", or there are no permissions at all.)
- I tried looking for a pattern in which requests saw the 401s, but
didn't have any luck.  The 401 occurs when talking to the whole
collection or targeting individual cores directly.  It occurs when
curl hits a host containing a replica for the collection in question,
and when it doesnt. etc.  This distinguishes it from SOLR-13472, which
seems more specific to collection structure/layout.

I'll create a bug for this in JIRA.

On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie  wrote:
>
> Hello. I encountered this issue too and wrote this up before I found this
> thread, but I thought I might as well post it still, if it helps...
>
> Currently I'm trying to move our product on to Solr 8.1.1. We are currently
> using 6.6.6, so things have definitely moved on.
>
> We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock down Solr
> (and we also secure our zookeeper). Here's an example for solradmin as the
> user and password
>
> {
> "authentication": {
> "blockUnknown": true,
> "class": "solr.BasicAuthPlugin",
> "credentials": {
> "solradmin": "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> }
> },
> "authorization": {
> "class": "solr.RuleBasedAuthorizationPlugin",
> "permissions": [
> {
> "name": "all",
> "role": "admin"
> }
> ],
> "user-role": {
> "solradmin": "admin"
> }
> }
> }
>
>
> On Solr 8.1.1, using our previously working security.json, running queries
> (through the admin UI currently) I non-deterministically get 401 responses
> on queries when a collection has more than 1 shard. Increasing the number
> of shards in the collection makes the errors more likely.
>
> {
>   "responseHeader":{
> "zkConnected":true,
> "status":401,
> "QTime":30,
> "params":{
>   "q":"*:*",
>   "_":"1559474550365"}},
>   "error":{
> "metadata":[
>
> "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
>
> "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> "msg":"Error from server at null: Expected mime type
> application/octet-stream but got text/html. \n\n http-equiv=\"Content-Type\"
> content=\"text/html;charset=utf-8\"/>\nError 401 require
> authentication\n\nHTTP ERROR 401\nProblem
> accessing /solr/gettingstarted_shard4_replica_n6/select. Reason:\n
>  require authentication\n\n\n",
> "code":401}}
>
> The security stats indicate this is happening because the requests do not
> have credentials with them, e.g.
> http://localhost:8983/solr/#/gettingstarted_shard4_replica_n6/plugins?type=security&entry=org.apache.solr.security.BasicAuthPlugin
>
>  org.apache.solr.security.BasicAuthPlugin
> class:
> org.apache.solr.security.BasicAuthPlugin
> description:
> Authentication Plugin org.apache.solr.security.BasicAuthPlugin
> stats
> SECURITY./authentication.authenticated:
> 182
> SECURITY./authentication.errors.count:
> 0
> SECURITY./authentication.failMissingCredentials:
> 58
> SECURITY./authentication.failWrongCredentials:
> 0
> SECURITY./authentication.passThrough:
> 0
> SECURITY./authentication.requestTimes.meanRate:
> 0.4183414110946125
> SECURITY./authentication.requests:
> 240
> SECURITY./authentication.totalTime:
> 117791100
>
> I assume that this is connected to the changes around
> https://issues.apache.org/jira/browse/SOLR-7896 and
> https://issues.apache.org/jira/browse/SOLR-13344 I've tested with Solr
> 7.6.0 and it appears to be unaffected
>
> Repro steps:
># Extract solr 8.1.1.
># bin\solr start -e cloud
> 1 node / [default port] / [default collection name] / 4 shards / 1
> replica / [_default configuration]
># server\scripts\cloud-scripts\zkcli -zkhost localhost:9983 -cmd putfile
> /security.json 
>
># Execute repeated GETS to
> http://localhost:8983/solr/gettingstarted/select?q=*%3A* - a lot of them,
> but not all, will fail with 401s
>
>
> Also as a side note, because the authentication is now done through the
> form login rather than the browser basic auth, if you go directly to a non
> UI url (e.g. http://localhost:8983/solr/main_index/select?q=*%3A*) you have
> to authenticate to it using the browser's basic auth prompt. Which is
> slightly annoying since the query page in the Admin UI generates links to
> it for the queries you run, and you've already authenticated to get there.
> But it's no

Re: Please help on pdate type during indexing

[Shawn Heisey]

On 6/2/2019 11:34 PM, derrick cui wrote:

I spent whole day to indexing my data to solr(8.0), but there is one field 
which type is pdate always failed.
error adding field 
'UpdateDate'='org.apache.solr.common.SolrInputField:UpdateDate=2019-06-03T05:22:14.842Z'
 msg=Invalid Date in Date Math 
String:'org.apache.solr.common.SolrInputField:UpdateDate=2019-06-03T05:22:14.842Z',,
 retry=0 commError=false errorCode=400


If that whole string (including the org.apache.solr stuff) is the input, 
it will fail.  The value will need to be the ISO date format, starting 
with the year, including the T, and ending with Z.



I have put timezone in the string, please help,


As far as I know, you can't do that.  Solr only handles dates in UTC - 
no timezone.


There is one place where Solr has a timezone configurable -- that's for 
date math.  So that Solr knows what time a day starts when it is doing 
NOW/DAY, NOW/WEEK, etc.  Other than that, everything is ALWAYS in UTC.


Thanks,
Shawn

Re: WG: SolrException: Can't determine a Sort Order with Solr 6.6

[Shawn Heisey]

On 6/3/2019 5:12 AM, Schwank, Désirée wrote:

But how can it be protected. What can I do? What is to configure? Can you help 
me with an example.


Place the Solr server in a network location such that only trusted 
systems and people can reach it.


Sanitize all input in your application before you send it to Solr.

Thanks,
Shawn

Re: Adding Multiple JSON Documents

[Jason Gerlowski]

Hi John,

I believe the documentation there is correct.  That is: those are two
different "update" APIs.  /update takes a JSON array of potentially
multiple docs, /update/json/docs takes either a JSON array of multiple
docs, or a single document not wrapped in the JSON array syntax.

Best,

Jason

On Sun, Jun 2, 2019 at 10:50 PM John Davis  wrote:
>
> Hi there,
>
> I was looking at the solr documentation for indexing multiple documents via
> json and noticed inconsistency in the docs.
>
> Should the POST url be /update/*json/docs *instead of just /update. It does
> look like former does work, unless both will work just fine?
>
> https://lucene.apache.org/solr/guide/7_3/uploading-data-with-index-handlers.html#adding-multiple-json-documents
> Adding Multiple JSON Documents
> 
>
> Adding multiple documents at one time via JSON can be done via a JSON Array
> of JSON Objects, where each object represents a document:
>
> curl -X POST -H 'Content-Type: application/json'
> 'http://localhost:8983/solr/my_collection/*update*' --data-binary '[
> {"id": "1","title": "Doc 1"  },  {"id": "2","title":
> "Doc 2"  }]'

WG: SolrException: Can't determine a Sort Order with Solr 6.6

[Schwank , Désirée]

In fact I am sending nothing, no sort order. The URl only contains a parameter 
q with the searchterm. Sortorder is only configured in solconfig. 

I agree with Walter, it is not safe that a bot can send values directly to 
Solr. But how can it be protected. What can I do? What is to configure? Can you 
help me with an example. 

Should safety not be  business of solr rather than everyone configuring himself?

Thanks for all help.

Greets
Desiree

-Ursprüngliche Nachricht-
Von: Walter Underwood  
Gesendet: Dienstag, 28. Mai 2019 17:25
An: solr-user@lucene.apache.org
Betreff: Re: SolrException: Can't determine a Sort Order with Solr 6.6

The bigger problem is that a bot can send values directly to Solr. That is not 
safe. Everything sent to the front end or an API needs to be parsed, checked, 
then recreated to send to Solr. A bot should never be getting a sort parameter 
through to Solr.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On May 28, 2019, at 8:02 AM, Shawn Heisey  wrote:
> 
> On 5/28/2019 7:48 AM, Schwank, Désirée wrote:
>> At the end of April we realized lots of errors, "SolrException: Can't 
>> determine a Sort Order (asc or desc) in sort  spec 'score+desc,id+asc'" 
>> first appearance in logs about 2019-04-29, without apparent reason.
> 
> The problem here is that you are sending your "sort" parameter with plus 
> signs instead of spaces.
> 
> The plus sign is URL encoding for a space, but in this case, you are actually 
> sending plus signs, which means that what's actually on the URL is probably 
> "score%2Bdesc,id%2Basc" ... not "score+desc,id+asc".
> 
> I know this is the case because I tried the following URL:
> 
> http://localhost:8983/solr/foo/select?q=*:*&sort=drip+err
> 
> And this is the message I got back:
> 
> Can't determine a Sort Order (asc or desc) in sort spec 'drip err'
> 
> As you can see, the + has been converted to a space.
> 
> You will need to ensure that what your URL encoder is being fed has spaces, 
> not plus signs.
> 
> Thanks,
> Shawn

highlighting not working as expected

[Martin Frank Hansen (MHQ)]

Hi,

I am having some difficulties making highlighting work. For some reason the 
highlighting feature only works on some fields but not on other fields even 
though these fields are stored.

An example of a request looks like this: 
http://localhost/solr/mytest/select?fl=id,doc.Type,Journalnummer,Sagstitel&hl.fl=Sagstitel&hl.simple.post=%3C/b%3E&hl.simple.pre=%3Cb%3E&hl=on&q=rotte

It simply returns an empty set, for all documents even though I can see several 
documents which have “Sagstitel” containing the word “rotte” (rotte=rat).  What 
am I missing here?

I am using the standard highlighter as below.




  
  
  

  100

  

  
  

  
  70
  
  0.5
  
  [-\w ,/\n\"']{20,200}

  

  
  

  
  

  

  
  

  
  

  
  

  
 

  
  

  

  
  

  
  

  

  

  10
  .,!? 	


  

  

  
  WORD
  
  
  da

  

  

Hope that some one can help, thanks in advance.

Best regards
Martin



Internal - KMD A/S

Beskyttelse af dine personlige oplysninger er vigtig for os. Her finder du 
KMD’s Privatlivspolitik, der fortæller, 
hvordan vi behandler oplysninger om dig.

Protection of your personal data is important to us. Here you can read KMD’s 
Privacy Policy outlining how we process your 
personal data.

Vi gør opmærksom på, at denne e-mail kan indeholde fortrolig information. Hvis 
du ved en fejltagelse modtager e-mailen, beder vi dig venligst informere 
afsender om fejlen ved at bruge svarfunktionen. Samtidig beder vi dig slette 
e-mailen i dit system uden at videresende eller kopiere den. Selvom e-mailen og 
ethvert vedhæftet bilag efter vores overbevisning er fri for virus og andre 
fejl, som kan påvirke computeren eller it-systemet, hvori den modtages og 
læses, åbnes den på modtagerens eget ansvar. Vi påtager os ikke noget ansvar 
for tab og skade, som er opstået i forbindelse med at modtage og bruge e-mailen.

Please note that this message may contain confidential information. If you have 
received this message by mistake, please inform the sender of the mistake by 
sending a reply, then delete the message from your system without making, 
distributing or retaining any copies of it. Although we believe that the 
message and any attachments are free from viruses and other errors that might 
affect the computer or it-system where it is received and read, the recipient 
opens the message at his or her own risk. We assume no responsibility for any 
loss or damage arising from the receipt or use of this message.