I'm not quite getting it here. I'm guessing that you do not
allow fielded queries or you strictly control the fields a user
sees to pick from. Otherwise your security stuff goes out the
window, say you have a drop-down list of fields to choose from
or something.
Assuming you do NOT have such a thi
Hi,
I have been trying to create panels in the dashboard of banana similar to
ones that had been done by my colleagues in Kibana. I have attached an
image for your reference. I created a panel in a non-time series dashboard
in Banana using the configurations shown in images 1-4. But after saving
t
Yes. I am going to provide autocomplete with facet count as rank.i.e. when
yours input "owe a duty", the system will suggest "xxx owe a duty yyy" with
highest count.
Thanks.
Dave
> Date: Wed, 22 Jul 2015 14:35:40 +0100
> Subject: Re: Performance of facet contain search in 5.2.1
> From: benedetti.
Mikhail, you are completely right!
And your observation was not superficial or obvious !
Actually DocValues are an un-inverted index, but I was ignoring they can be
used to search !
In my observation I was not taking in consideration docValues ( as it was
not in the user schema), so actually mine w
Looks like this may be what I'm looking for
*SolrRequestInfo*
I have not tried this yet but looks promising.
Assuming this works, thinking about your suggestion I would need to rewrite
the users query with the appropriate fields, are there any utilities for
doing this? I'd be looking to rewrite
On Wed, Jul 22, 2015, at 09:21 PM, Shamik Bandopadhyay wrote:
> Just wondering if it's possible to combine to separate MLT queries (based
> on filtering condition) into a single one. I'm trying to combine the
> results of this two query:
>
> http://localhost:8983/solr/collection1/mlt?q=title:"AB
How many versions of Java do you have? I bet the bin/solr script is
using SolrJ (i.e. Java) to communicate with Solr itself (also Java), but
for some weird reason they are using incompatible Java versions.
SolrJ uses Java serialization to communicate with Solr, therefore it
must use the same Jars
Hi,
Is there a way to specify the max number of DB connections that the
dataimport will use in Solr? We have around 15 cores (with 1 entity and 16
sub-entities, btw) and I am seeing 190 connections being opened up at the
DB. Does anyone have any insights on what sort of parallelization Solr is
doi
I answered my own question, looks like the field infos are always read
within the IndexSearcher so that cost is already being paid.
I would potentially have to duplicate information in multiple fields if it
was present at multiple authorization levels, is there a limit to the
number of fields with
I had thought about this in the past, but thought it might be too
expensive. I guess in a search component I could look up all of the fields
that are in the index and only run queries against fields they should be
able to see once I know what is in the index (this is what you're
suggesting right?)
Just wondering if it's possible to combine to separate MLT queries (based
on filtering condition) into a single one. I'm trying to combine the
results of this two query:
http://localhost:8983/solr/collection1/mlt?q=title:"ABC"&fq=Source:("Test1"
OR "Test3" OR "Test4")
http://localhost:8983/solr/co
Why don't you handle it all at the app level? Here's what I mean:
I'm assuming that you're using edismax here, but the same principle
applies if not.
Your handler (say the "/select" handler) has a "qf" parameter which defines
the fields that are searched over in the absence of a field qualifier,
Seems like I'm missing something simple here. SolrCloud installs and seems
to work fine and the examples work as they should on each individual
machine. What I'm trying to do is setup 4 separate Solr machines (Ubuntu
14.04) with 3 external zookeepers (the ones currently being used by our
production
I'd be curious to see the parsed query that you get when adding
debugQuery=true to the URL. I bet that the clustering component is
extracting terms from the parsed query, and perhaps each of those
queries is parsed in some way differently?
Upayavira
On Wed, Jul 22, 2015, at 08:29 PM, Joseph Obern
Upon further investigation, it looks like it is either ignoring the
default field, or when the default field is specified the rest of the
query is ignored.
Example:
q=Field1:(term1 OR term2) AND (item1 OR item2)&df=Field2
that does not cluster correctly, but this does:
q=Field1:(term1 OR term2)
Hi - I'm using carrot2 inside of solr cloud and have noticed that
queries that involve parenthesis don't seem to work correctly. For
example if I have:
q=Field1:(term1 OR term2) AND Field2:(item1 OR item2)
The clustering seems to ignore the values in parenthesis. If instead I do:
q=(Field1:ter
I am looking for a way to prevent fields that users shouldn't be able to
know exist from contributing to the score. The goal is to provide a way to
essentially hide certain fields from requests based on an access level
provided on the query. I have managed to make terms that users shouldn't
be ab
Glad to hear it's working for you.
Erick
On Wed, Jul 22, 2015 at 10:56 AM, Vineeth Dasaraju
wrote:
> Hi Erick,
>
> As correctly pointed out by you, the main reason why documents were
> disappearing was that I was assigning same id to multiple documents. This
> got resolved after I used the UUID
Sorry, the "ID" mistake was pointed out by Upayavira. Thank you Upayavira!
On Wed, Jul 22, 2015 at 10:56 AM, Vineeth Dasaraju
wrote:
> Hi Erick,
>
> As correctly pointed out by you, the main reason why documents were
> disappearing was that I was assigning same id to multiple documents. This
> g
Hi Erick,
As correctly pointed out by you, the main reason why documents were
disappearing was that I was assigning same id to multiple documents. This
got resolved after I used the UUID as suggested by Mohsen. Thank you for
your inputs.
Regards,
Vineeth
On Wed, Jul 22, 2015 at 9:39 AM, Erick Er
That's really simple, as I said earlier, it could just require:
Upayavira
On Wed, Jul 22, 2015, at 11:59 AM, Sreekant Sreedharan wrote:
> Well I guess I oversimplified things. My goal is to transform a SOLR
> response
> that looks like:
>
>
> ...
>
>
>589587B2B1CA4C4683FC106967E7C3
Hi, I have some questions regarding basic auth and proper support in 5.3:
do you know when it would be released?
Are there any requirements of ZK authentication must be there as well?
Do we store the user/pass in ZK?
SOLR-7274 pluggable security is already in 5.2 (my requirement is to provide
Thanks; I don't know how the file path is getting into the id field. Must
be some Tika default?
On Wed, Jul 22, 2015 at 9:52 AM, Erick Erickson
wrote:
> the id field is absolutely NOT the thing you need to try to parse.
> Assuming you're stuffing the file path into that field, use a
> copyField
the id field is absolutely NOT the thing you need to try to parse.
Assuming you're stuffing the file path into that field, use a
copyField to copy the filepath into another text (not string)
field and do your work there.
As far as whether the filepath is in some other field, well, you have
to put
Take a look at TestCoreDiscovery.java, it creates a bunch of cores and
tests loading them. You probably want to ignore anything about
transient cores in that file.
But... how are you creating cores? 5.x doesn't support defining
multiple cores in solr.xml, so that's the first thing you need to be
a
Trying to figure out how to parse the file path, which when I run the
"cloud" instance becomes the "id" for each PDF document.
Is that "id" field the thing to parse with PatternReplaceFilterFactory in
the config? If not, is there a "file-path" field I can parse?
On Wed, Jul 22, 2015 at 9:42 AM,
Don't understand your question. If you're talking two different
fields, use copyField.
On Wed, Jul 22, 2015 at 8:55 AM, Andrew Musselman
wrote:
> Fwding to user..
>
> -- Forwarded message --
> From: Andrew Musselman
> Date: Wed, Jul 22, 2015 at 8:54 AM
> Subject: Re: Parsing and
The other classic error is to not send the batch at the end, but
at a glance that's not a problem for you, after the while loop
you send the batch that'll catch any docs left over.
solr.user, that might be your problem? Because I've never seen
this happen.
On Tue, Jul 21, 2015 at 1:47 PM, Fadi M
You have two problems
First, you're specifying fields in the q clause so the qf stuff
isn't respected.
My query params is something like below
q: person_full_name:louis OR address1:louis OR city:louis OR
state_code:louis
qf: person_full_name^5.0 address1^0.8 city^0.7 state_code^1.0
defType: ed
Frankly, I'm surprised it runs at all. 650M dos in 2G of memory is very,
very, very aggressive.
To get it to run at all I'm guessing that you have turned off things like
term vectors, position info
and the like.
Anyway...
Solr/Lucene cache various data as queries come in, so the scenario is this
I
bq: What happens if a shard(both leader and replica) goes down. If the
document on the "dead shard" is updated, will it forward the document to the
new shard. If so, when the "dead shard" comes up again, will this not be
considered for the same hask key range?
No. The index operation will just fa
Keep calm santhosh, if you read your parsed query it's easy to see that
it's not searching against the cat field ( which was not indexed ) :
"parsedquery":"text:software",
So it's returning a document because that document has the text field
indexed .
To answer your question :
"I have question
Fwding to user..
-- Forwarded message --
From: Andrew Musselman
Date: Wed, Jul 22, 2015 at 8:54 AM
Subject: Re: Parsing and indexing parts of the input file paths
To: d...@lucene.apache.org
Thanks, and tell it to index the "id" field, which eventually contains the
file path?
On
Hello - Depening on size differences between source data and indexed data, you
can gzip/bzip2 your source json/xml, then transfer it over WAN, and index it
locally. This is the fastest method in every case we encountered.
-Original message-
> From:Reitzel, Charles
> Sent: Wednesday 22n
Indexing over a WAN will be slow, limited by the bandwidth of the pipe.
I think you will be better served to move the data in bulk to the same LAN as
your target solr instances.I would suggest ZIP+scp ... or your favorite
file system replication/synchronization tool.
It's true, if you are u
Could not figure out actual reason why my configured Solr spell checker not
giving desire output. In my indexed data query: symphony+mobile has around
3.5K+ docs and spell checker detect it as correctly spelled. When i
miss-spell "symphony" in query: symphony+mobile it showing only results for
"mob
yes .. that line was commented out... and I have restarted the server ..
after updating the schema.xml .. and document was deleted and added back ..
On 22 July 2015 at 20:31, Mikhail Khludnev
wrote:
> did you removed
>
> from schema.xml ?
> did you restarted Solr or reloaded core, after that
I have question ... when we delete a document .. will all the indexes
generated for that document wont be get deleted? and when we add new
document , it wont be get checked against schema.xml ...
On 22 July 2015 at 20:04, Jack Krupansky wrote:
> Is this a feature or a bug of Solr? Seriously, ma
did you removed
from schema.xml ?
did you restarted Solr or reloaded core, after that?
did you reindex that document, after all?
On Wed, Jul 22, 2015 at 5:55 PM, santhosh kumar
wrote:
> but field 'cat' is not the 'copyFiled' not in the 'dynamicField'. :)
>
> On 22 July 2015 at 20:04, Mikhail
that will be an alternative option .. but what if the exiting field we need
to change?
On 22 July 2015 at 20:03, Alexandre Rafalovitch wrote:
> I would just reindex into a new core from scratch first. I think the
> suggestion that perhaps the content was evolving and did not get
> reindexed full
but field 'cat' is not the 'copyFiled' not in the 'dynamicField'. :)
On 22 July 2015 at 20:04, Mikhail Khludnev
wrote:
> it matches by text field
>
> On Wed, Jul 22, 2015 at 5:28 PM, santhosh kumar
> wrote:
>
> > Here is the query in debug mode ...
> >
> > {
> > "responseHeader": {
> > "s
Is this a feature or a bug of Solr? Seriously, maybe it is convenient to be
able to change the schema without reindexing documents that don't care
about the schema change, but it is definitely a support headache. I mean,
how many times have we had to ask that question on this list since the
current
It could be interesting to have a utility that will - for example -
compare schema definition with reverse-engineered definition from the
underlying indexes. That would catch the indexed leftovers. But it
probably still would not work for analyzer chain changes.
Solr Analyzers, Tokenizers, Fil
it matches by text field
On Wed, Jul 22, 2015 at 5:28 PM, santhosh kumar
wrote:
> Here is the query in debug mode ...
>
> {
> "responseHeader": {
> "status": 0,
> "QTime": 3,
> "params": {
> "debugQuery": "true",
> "indent": "true",
> "q": "software",
> "_":
some more info .. after updating schema.xml file I have restarted the Jetty
server and the document was deleted and added.
On 22 July 2015 at 19:58, santhosh kumar wrote:
> Here is the query in debug mode ...
>
> {
> "responseHeader": {
> "status": 0,
> "QTime": 3,
> "params": {
>
sorry .. I didnt get .. I am using default solr admin for query ...
http://localhost:8494/solr/myfirstcore/select?q=software&wt=json&indent=true&debugQuery=true
{
"responseHeader":{
"status":0,
"QTime":3,
"params":{
"debugQuery":"true",
"indent":"true",
"q":"soft
Hi Emir,
Yes, I'm trying to write some codes that will run as part of Solr.
I'll probably try out the Solr plugins.
Thanks for your information and advice.
Regards,
Edwin
On 22 July 2015 at 18:56, Emir Arnautovic
wrote:
> Hi Edwin,
> Not sure if I understood your case, but if I got it right
I would just reindex into a new core from scratch first. I think the
suggestion that perhaps the content was evolving and did not get
reindexed fully is the most likely cause.
Regards,
Alex.
Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/
On 22
Try adding the "start" call in your jetty.xml:
Realm Name
/etc/realm.properties
5
On Wed, Jul 22, 2015 at 2:53 PM, O. Klein wrote:
> Yeah I can't get it to work on Jetty 9 either on Linux.
>
> Just trying to password protect the admin pages.
>
>
excuse me. Schema Browser, I mean. Also, which query do you see at
debugQuery=true?
On Wed, Jul 22, 2015 at 5:10 PM, Mikhail Khludnev <
mkhlud...@griddynamics.com> wrote:
> What do you see at Session browser for this field?
>
> On Wed, Jul 22, 2015 at 4:49 PM, santhosh kumar
> wrote:
>
>> Hi,
>>
Here is the query in debug mode ...
{
"responseHeader": {
"status": 0,
"QTime": 3,
"params": {
"debugQuery": "true",
"indent": "true",
"q": "software",
"_": "1437575140328",
"wt": "json"
}
},
"response": {
"numFound": 1,
"start": 0,
"
I find so hard to believe you can search without the inverted index :)
Are you sure you didn't have in your index some documents with that field
indexed, before you did the change and put it as not indexed ?
Changes in the schema don't apply to already indexed documents ( if you
don't go with a r
What do you see at Session browser for this field?
On Wed, Jul 22, 2015 at 4:49 PM, santhosh kumar
wrote:
> Hi,
>
> I have started practicing Solr recently and my understanding on the field
> type properties "index=false", is that field is not searchable.
>
> But when I execute the below query I
Yeah I can't get it to work on Jetty 9 either on Linux.
Just trying to password protect the admin pages.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Basic-auth-tp4218053p4218565.html
Sent from the Solr - User mailing list archive at Nabble.com.
Hi,
I have started practicing Solr recently and my understanding on the field
type properties "index=false", is that field is not searchable.
But when I execute the below query I got the results.
http://localhost:8494/solr/myfirstcore/select?q=cat%3Asoftware&wt=json&indent=true
configured in sc
I think as usually Erick says, this is a X-Y problem.
I think the user was trying to solve the infix autocomplete problem with
faceting.
We should get from him the initial problem to try to suggest a better
solution.
Cheers
2015-07-22 14:01 GMT+01:00 Markus Jelsma :
> Hello - why not index the
Ok, now the situation is clearer !
Can you check the stored data size as Daniel correctly suggested ?
You are using a recent version of Solr, so your stored data should be
properly compressed.
An other idea that comes to my mind is related your merging policy.
Are you merging segments often or not
Alessandro,
Thanks.
see some confusion here.
*First of all you need a smart client that will load balance the docs to
index. Let's say the CloudSolrClient .
*
All these 5 shards are configured to load-balancer and requests are sent to
the load-balancer and whichever server is up, will accept t
please find [child] at
https://cwiki.apache.org/confluence/display/solr/Transforming+Result+Documents
On Wed, Jul 22, 2015 at 10:47 AM, Vineeth Dasaraju
wrote:
> Hi,
>
> I have been using json objects of the form shown below to index into solr.
> As you can see, there are nested JSON objects and
Hello - why not index the facet field as n-grams? It blows up the index but is
very fast!
Markus
-Original message-
> From:Erick Erickson
> Sent: Tuesday 21st July 2015 21:36
> To: solr-user@lucene.apache.org
> Subject: Re: Performance of facet contain search in 5.2.1
>
> "contains" has
SOLR-7692 is for ZK (or did I get it wrong?) In my case, I'm trying what's
documented here:
https://wiki.apache.org/solr/SolrSecurity#Jetty_realm_example and it won't
work (see my earlier email).
Steve
On Wed, Jul 22, 2015 at 8:33 AM, Noble Paul wrote:
> Solr 5.3 is coming with proper basic au
Solr 5.3 is coming with proper basic auth support
https://issues.apache.org/jira/browse/SOLR-7692
On Wed, Jul 22, 2015 at 5:28 PM, Peter Sturge wrote:
> if you're using Jetty you can use the standard realms mechanism for Basic
> Auth, and it works the same on Windows or UNIX. There's plenty of
if you're using Jetty you can use the standard realms mechanism for Basic
Auth, and it works the same on Windows or UNIX. There's plenty of docs on
the Jetty site about getting this working, although it does vary somewhat
depending on the version of Jetty you're running (N.B. I would suggest
using
Why are most of your fields stored but not indexed? That suggests to me
that you are using Solr as your primary data store, not as an index (which
is not Solr's ideal use case)
Secondly, I think there is confusion around the term "segments". You have
a field called segment in your schema, but se
I know some of the docValue APIs changed in 4.10, because we had to re-code
some custom stuff, looks like
https://issues.apache.org/jira/browse/LUCENE-5882 changed the format on
disk too. The comments on that ticket don't suggest an 8% increase in disk
space, so maybe you are hitting some kind of
Is this test index? Do you rewrite documents with same ids? Did you try
to optimize index?
Emir
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/
On 22.07.2015 13:10, Daniel Holmes wrote:
Upayavira number of docs in
Upayavira number of docs in that case is 140275. The solr memory is 30Gb.
Yes Emir I need most of them to be saved.
I don't know Alessandro is that usual to use disk for indexing more than 3x
of document size and presumably it will grow up in continue of crawl
exponentially... Its so suboptimal I
Well I guess I oversimplified things. My goal is to transform a SOLR response
that looks like:
...
589587B2B1CA4C4683FC106967E7C326
into something that looks like
Where the SOLR field 'id' is mapped to an attribute 'NewID' in the expected
result. Is there a simpler way
Dear Alessandro,
Thank you very much.
Yeah sure it is far better, I did not think of that ;)
Best regards.
On Wed, Jul 22, 2015 at 2:31 PM, Alessandro Benedetti <
benedetti.ale...@gmail.com> wrote:
> In addition to Erick answer :
> I agree 100% on your observations, but I would add that actually
Hi Edwin,
Not sure if I understood your case, but if I got it right you are trying
to write some code that will run as part of SOLR.
If that's the case, then you should take a look how to write SOLR
plugins (https://wiki.apache.org/solr/SolrPlugins). SolrJ is client side
library that simplifies
"In one case for instance my segments size is 8.4G while index size is
28G!!! It seems unusual…"
The index is a collection of index segments + few overhead .
So, do you simply mean you have 4 segments ?
Where is the problem anyway ?
You are also storing content which usually is a big part of the
Hi Daniel,
Do you need all fields stored in your index? Only field that is not
stored is host.
Thanks,
Emir
On 22.07.2015 12:27, Daniel Holmes wrote:
Hi All
I have problem with index size in solr 4.7.2. My OS is Ubuntu 14.10 64-bit.
my fields are :
In one case for instance my seg
How many documents do you have? What makes you think that a 28Gb index
is large? How much memory do you have in your Solr server?
Upayavira
On Wed, Jul 22, 2015, at 11:27 AM, Daniel Holmes wrote:
> Hi All
> I have problem with index size in solr 4.7.2. My OS is Ubuntu 14.10
> 64-bit.
> my fields
Hi,
Personally, in XSLT, I'd try to be less clever...
...
Dunno if that helps.
Upayavira
On Wed, Jul 22, 2015, at 11:17 AM, Sreekant Sreedharan wrote:
> Hi,
>I am using the SOLR XSLT response writer. My challenge is to convert
>some
> fields in the schema from
Hi All
I have problem with index size in solr 4.7.2. My OS is Ubuntu 14.10 64-bit.
my fields are :
In one case for instance my segments size is 8.4G while index size is
28G!!! It seems unusual...
What suggestions do you have to reduce index size?
Is there any way to check disk usage d
Steven White wrote
> Thanks for updating the wiki page. However, my issue remains, I cannot
> get
> Basic auth working. Has anyone got it working, on Windows?
Doesn't work for me on Linux either.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Basic-auth-tp4218053p4218519
Hi,
I am using the SOLR XSLT response writer. My challenge is to convert some
fields in the schema from one value to another. I am using a map to do this.
Here's the sample XSLT file
http://www.w3.org/1999/XSL/Transform";
xmlns:my="my:my">
In addition to Erick answer :
I agree 100% on your observations, but I would add that actually, DocValues
should be provided for all not tokenized fields instead of for all not
analysed fields.
In the end there will be not practical difference if you build the
docValues structures for fields that
Ali can I suggest you one thing ?
If your scope was to test your analysis chain, why simply not use :
1) analysis request handler and analysis tool in the adminUI, it's easy,
user friendly and really helpful for an administrator !
2) Schema Browser - to take a look to the current tokens , per fie
I'm preparing my data to migrate to 5.x from 4.6.1. One step referred in
documentation is upgrade the index to 4.10.4 format. To do this, I used the
command "java -cp lucene-core-4.10.4.jar
org.apache.lucene.index.IndexUpgrader".
The index size was 395 gigas and now is ~430 gigas. The only differen
https://docs.lucidworks.com/display/SiLK/Dashboard+Configuration
-Original message-
> From:Vineeth Dasaraju
> Sent: Wednesday 22nd July 2015 9:50
> To: solr-user@lucene.apache.org
> Subject: References for Banana
>
> Hi,
>
> Could anyone please direct me towards any online resources
Hi,
Could anyone please direct me towards any online resources that outline how
a dashboard can be created for Solr Queries in Banana?
Regards,
Vineeth
Hi,
I have been using json objects of the form shown below to index into solr.
As you can see, there are nested JSON objects and JSON arrays.
{
"RawEventMessage": "Lorem ipsum dolor sit amet, consectetur adipiscing
elit. Aliquam dolor orci, placerat ac pretium a, tincidunt consectetur
mauris.
83 matches
Mail list logo
