: My current setup does not use the ImageUrl field for the search (more
: specifically as the default search field). The ImageUrl field contains a URL
: to the image which is for most part a GUID, which is meaningless to users.
: However, I would like to note that the ImageUrl field is Indexed
: Thanks for the response, yes the way you describe I know it works and is
: how I get it to work but then what does mean the snippet of the
: documentation I see on the documentation about overriding the default
It means that there is implicitly a set of search components that have
default
: Thanks. There is definitely a field name=id .../ in each of the schemas.
:
: I am using 4.7.2.
if this conig is working for you when you don't use zookeeper/hdfs then
you must be using a newer version of Solr when you test w/ zk/hdfs
4.8.0 is when the fields and type section tags were
: out how to remove the configs from zookeeper through the cloud scripts, so I
: just manually removed the files in the zookeeper data directory.
https://cwiki.apache.org/confluence/display/solr/Using+ZooKeeper+to+Manage+Configuration+Files
: I think the documentation and example files for Solr 4.x need to be
: updated. If someone will let me know I'll be happy to fix the example
: and perhaps someone with edit rights could fix the reference guide.
I think you're correct - can you open a Jira with suggested improvements
for the
: second, and assuming your problem is really that you're looking at the
: _display_, you should get back exactly what you put in so I'm guessing
Not quite ... With the numeric types, the numeric value is both indexed
and stored so that there is no search/sort inconsistency between 1.1,
1.10,
: The error that the other server returned was:
: 552 spam score (6.2) exceeded threshold (HTML_MESSAGE,RCVD_IN_DNSWL_
: LOW,SPF_NEUTRAL,URIBL_SBL
HTML_MESSAGE: BAD - send plain text messages instead of HTML
RCVD_IN_DNSWL_LOW: GOOD - your domain is considered unlikely to send spam
: SCHEMA:
: field name=content type=string indexed=false stored=true
: required=true/
:
: LOGS:
: Caused by: java.lang.IllegalArgumentException: Document contains at least
: one immense term in field=content (whose UTF8 encoding is longer than the
I don't think you are using the schema.xml you
: http://e-mats.org/2011/01/solr-memory-usage-and-dynamic-fields/
:
: Just to confirm, the fieldCache doesn't store anything extra when we use
: dynamic fields and do sorting query on that...right?
the way the FieldCache works, is that it has to cache a value for every
document -- even if
if your Solr server is giving you the results you expect when you query it
directly, then it sounds like you have your fieldTypes and anaylsers
configured appropriately.
any problem you might be seeing when querying from your client via Spring
data source bindings sounds like it is probably a
: But it seems this has not fixed in 4.10, as the issue SOLR-6179
: https://issues.apache.org/jira/browse/SOLR-6179 is not in changes list
: http://lucene.apache.org/solr/4_10_0/changes/Changes.html
there was a jira glitch recording the commits, but tim added a comment
with the details.
it is
: But, To make it better, I would like to load this file only once and in the
: init() method of handler class. I am not sure how to get the access of
: SolrCore in the init method.
you can't access the SolrCore during hte init() method, because at the
time it's called the SolrCore itself is
: I'm trying to upgrade Solr from version 4.2 to 4.9, since then I'm
...
: haven't configured it. You can ignore this message. To get it to go
The fact that a WARN is logged at all was a bug in 4.9 that got fixed in
4.10...
https://issues.apache.org/jira/browse/SOLR-6179
-Hoss
: I'm trying to use a query with
fl=name_UK,name_FRA,itemDesc:def(name_UK,name_FRA)
: As you can see, the itemDesc field (builded by solr) is truncated :
functions get their values from the FieldCache (or DocValues if you've
enabled them) so that they can be efficient across a lot of docs.
: Is there a list of possible other statuscodes you can receive in case
: anything fails and what these errorcodes mean?
:
: I don't think we have a list of possible other status because Solr
: doen't return status other than 0. Instead of status code in XML,
: you should look at HTTP status
The Lucene PMC is pleased to announce that there is a new version of the
Solr Reference Guide for Solr 4.10.
The 511 page PDF serves as the definitive user's manual for Solr 4.10. It
can be downloaded from the Apache mirror network:
: change the highlight search component to use different search component
: based on a query parameter
What do you mean by use different search component ?
Differnet how?
What is the actual use case / goal you are trying to achieve?
https://people.apache.org/~hossman/#xyproblem
XY Problem
: I have solr installed on Debian and every time delta import takes place a
: file gets created in my root directory. The files that get created look
: like this
:
:
: dataimport?command=delta-import.1
that is exactly the output you would expect to see if you have a cron
somewhere, running
First question: ignoring the oiginal jira (which may be out of date
due to later improvements) have you seen the instructions?
https://cwiki.apache.org/confluence/display/solr/Enabling+SSL#EnablingSSL-SolrCloud
: I always get a message like this:
: Caused by:
: ./zkcli.sh -zkhost localhost:2181 -cmd put /clusterprops.json
: '{urlScheme:https}'
...
: Next I start Tomcat, I get this:
: 482 [localhost-startStop-1] ERROR org.apache.solr.core.SolrCore â
: null:org.noggit.JSONParser$ParseException: JSON Parse Error:
: char=',position=0 BEFORE='''
: Yes i'm just worried about load average reported by OS, because last week
: suddenly server can't accessed so we have to hard reboot. I'm still
: investigating what is the problem, because this server is dedicated to solr
ok - so here is the key bit.
basically, nothing else you've mentioend
: That's an interesting idea. If they aren't *independently* unique
: (which would make it impossible to treat them as a single unit
: together), that might work. Thanks for the idea! I'll chase it down on
if they are independently unique, check out the
SignatureUpdateProcessorFactory, but
: As you can see from the first image, the text field-type doesn't define the
: omitNorms flag, meaning it is set to false. Also on the first image you can
: see that the description field doesn't define the omitNorms flag, again
: meaning it is set to false. (Default for omitNorms is false).
: but am stuck with syntax with does not allow any concatenation of params to
: a prefix.
there's no inherent support for anything like that in the local param
parsing that would help you in building up facet params like this.
you'd either need to do this in some layer betwen your client and
You havne't given us a lot of information to go on (ie: full
solrconfig.xml, log messages arround the tim of your update, etc...) but
my best guess would be that you are seeing a delay between the time the
new searcher is opened and the time the newSearcher is made available to
requests due
: I'm not seeing any messages in the log with respect to cache warming at the
: time, but I will investigate that possibility. Thank you. In case it is
what logs *do* you see at the time you send the doc?
w/o details, we can't help you.
: helpful, I pasted the entire solrconfig.xml at
It's not clear to me from any of the comments you've made in this thread
wether you've ever confirmed *exactly* what you are getting back from
solr, ignoring the PHP completley. (ie: you refer to UTF-8 for all of the
web pages suggesting you are only looking at some web application which
is
: I was reviewing the httpclient code in HttpSolrServer and noticed that
: it sets a Content-Charset header. As far as I know this is not a real
: header and is not necessary. Anyone know a reason for this to be there?
: I'm guessing this was just a mistake when converting from httpclient3 to
: One of the specific pages that I've been looking at is the page on
: compiling Solr: https://wiki.apache.org/solr/HowToCompileSolr . I
go for it...
: haven't seen a corresponding page in Confluence. Could you help me
: understand the relationship between the wiki and the Confluence
:
: Thank you very much Erik. This is exactly what I was looking for. While at
: the moment I have no clue about these numbers, they ruby formatting makes it
: much more easier to understand.
Just to be clear, regardless of *which* response writer you use (xml,
ruby, json, etc...) the default
: I tried this poor mans cursor approach out ad-hoc, but I get OOM. Pretty
: sure this is because you need all uniqueKey-values in FieldCache in order to
: be able to sort on it. We do not have memory for that - and never will. Our
: uniqueKey field is not DocValue.
: Just out of curiosity
: *
: billions of documents (not enough memory). Please note that we are on 4.4,
: which does not contain the new CURSOR-feature. Please also note that speed is
: an important factor for us.
for situations where you know you will be processing every doc and order
doesn't matter you can use a poor
: So, I think a lot of people may be missing that power. I suspect it's
I've added some comments to the relevant pages as reminders to go back and
add more detail later so we don't lose track of the doc feedback expressed
in this thread ... when i get more time i'll try to go back and fill
: As Ahmet indicated, you must have a way to detect that deletions have
: happened. Marking rows as deleted with an active/inactive field is one
: way. Another way (the way that we use) is to have a delete trigger on
: the table that creates an entry in a delete tracking table.
If you have no
EmbeddedSolrServer is really just for dealing with embedded SolrCore
objects. It doesn't have ay of the other HTTP related plumbing
assocaited with it -- which is why it can't be used for replication and/or
cloud features, let alone the RESTLet based endpoints.
if you want to write a test
:
: Somebody (with more knowledge) should write up an in-depth article on
: this issue and whether the parent parser has to be default (lucene) or
: whatever.
It's a feature of Solr's standard query parser...
https://cwiki.apache.org/confluence/display/solr/Query+Syntax+and+Parsing
I think you are missunderstanding what Himanshu is suggesting to you.
You don't need to make lots of big changes ot the internals of solr's code
to get what you want -- instead you can leverage the Atomic Updates
Optimistic Concurrency features of Solr to get the existing internal Solr
to
: Now that I think about it, though, is there a way to use the Update Xml
: messages with something akin to the cloud solr server? I only see examples
: posting to actual Solr instances, but we really need to be able to take
: advantage of the zookeepers to send our updates to the appropriate
: Is it required for the schema.xml and solrconfig.xml to have those exact
: filenames?
It's an extremelely good idea ... but strictly speaking no...
https://cwiki.apache.org/confluence/display/solr/CoreAdminHandler+Parameters+and+Usage#CoreAdminHandlerParametersandUsage-CREATE
This smells
This is a long standing issue in solr, that has some suggested fixes (see
jira comments), but no one has been seriously afected by it enough for
anyone to invest time in trying to improve it...
https://issues.apache.org/jira/browse/SOLR-2357
In general, the fact that Solr is moving away from
: I mentioned id as string in schema.xml and i copied the csv into example docs
: folder. I used the below commaand to download the data Java
: -Dtype=application/csv -jar post.jar import.csv
:
: it's throwing the below error.Please help in this regard.
:
: ERROR - 2014-07-01 19:57:43.902;
: I want to run some query benchmarks, so I want to disable all type of caches
Just to be clear: disabling all internal caching because you want to run a
benchmark means you're probably going to wind up running a useless
benchmark.
Solr's internal caching is a key component of it's perormance
: We are getting the results for the query but the spellchecker component is
: returning 500. Please help us out.
:
: *query*: http://localhostt:8111/solr/srch/select?q=malerkotlaqt=search
what version of solr?
what does your solrconfig.xml show for /select the spellcheck config?
what does
: *ab:(system entity) OR ab:authorization* : Number of results returned 2
: which is not expected.
: It seems this query makes the previous terms as OR if the next term is
: introduced by an OR.
in general, that's they way the boolean operators like AND/OR work in
all of the various parser
I think you are correct -- deinitely looks like a bug to me...
https://issues.apache.org/jira/browse/LUCENE-5790
: Date: Fri, 13 Jun 2014 10:45:12 +
: From: 海老澤 志信 shinobu_ebis...@waku-2.com
: Reply-To: solr-user@lucene.apache.org
: To: solr-user@lucene.apache.org
: I repo'd using the example config (with sharding). I was missing one
: necessary condition: the schema needs a * dynamic field.
: It looks like serializeSearchGroup matches the sort expression as the
: * field, thus marshalling the double as TextField.
:
: Should I enter a ticket with the
FYI: The current plan is to call a vote for the 4.9 Solr Ref Guide
sometime tomorrow (2014-06-26) morning (~11AM UTC-0500 maybe?)
The main thing we are currently waiting on is that sarowe is working on a
simple page to document using Solr with SSL -- but now would be a great
time for folks to
: I see that result is affected by sorting order (ASC/DESC change order) but
: result is not precise. For example for query
:
:
params={mm=2pf=tags^10+title^5sort=created+ascq=queryqf=tags^10+title^5wt=javabinversion=2defType=edismaxrows=10}
those results don't really make sense -- can you
: Let's take this query sample:
: XXX OR AAA AND {!frange ...}
:
: For my use case:
: AAA returns a subset of 100k documents.
: frange returns 5k documents, all part of these 100k documents.
:
: Therefore, frange skips the most documents. From what you are saying,
: frange is going to be
: I don't believe however that the two queries (the one with the post filter
: and the one without one) are equivalent.
:
: Suppose out of the whole document set:
: XXX returns documents 1,2,3.
: AAA returns documents 6,7,8.
: {!frange}customfunction returns documents 7,8.
:
: Running this
: I am upgrading an index from Solr 3.6 to 4.2.0.
: Everything has been picked up except for the old DateFields.
Just to be crystal clear:
1) 4.2 is alreayd over a year old. the current rleease of Solr is 4.8,
and 4.9 will most likeley be available within a day or two
2) Even in 4.9,
: I recently tried upgrading our setup from 4.5.1 to 4.7+, and I'm
: seeing an exception when I use (1) a function to sort and (2) result
: grouping. The same query works fine with either (1) or (2) alone.
: Example below.
Did you modify your schema in any way when upgrading?
Can you provide
: record counts by faceted date. I also want to get the total of an integer
: field by faceted date but am having difficulty.
Unfortunately, what you are asking about isn't currently possible.
FWIW: lately i've been thinking a lot lately about stats and
accumulating stats over facets, and i
: Now, if I want to make a query that also contains some OR, it is impossible
: to do so with this approach. This is because fq with OR operator is not
: supported (SOLR-1223). As an alternative I've tried these queries:
:
: county='New York' AND (location:Maylands OR location:Holliscort or
:
: I want to understand why 13 segments are created in my index??
: Could appreciate if i can get response ASAP
: Imagine the following scenario. You start from a clean index and do
FWIW: the TL;DR of Shawn's response can be seen in this animation of how
Log based MergePolicy's work in the
if by old behavior you mean incremending the start param then the the
order of results when doing concurrent indexing was always dependent on
what exactly your sort was.
when using a cursor, the impacts of concurrent indexing are also dependent
on what your sort clause looks like -- but in
: Can you please share the Java code for Plural Only Porter Stemmer for English
if you don't mind?
The Porter stemmer algorithm, by definition, does more then just stip
plurals.
If you are interested in a lighter weight stemmer for english, this is
exactly what the
: WARN o.a.s.r.ManagedResource- No stored data found for
: /schema/analysis/stopwords/english
: WARN o.a.s.r.ManagedResource- No stored data found for
: /schema/analysis/synonyms/english
:
: I fixed these by commenting out the managed_en field type in my
: schema, see
All index wide statistics (like the docFreq of each term) are over the
entire index, which includes deleted docs -- because it's an *inverted*
index, it's not feasible to update those statistics to account for deleted
docs (that would basically kill all the performance advantages thatcome
: Currently we are not using SolrJ but are simply interacting with solr with
: json over http, this will change in a couple of months but currently not
: there. As of now we are putting all the logic in query building, using it
: to query solr and then passing on the json returned by it to front
: text in it, query is of the type keywords:(word1 OR word2 ... OR wordN).
: The client is relying on default relevancy based sort returned by solr.
: Some documents can get penalised because of some other documents which were
: deleted. Is this functionality correct?
yes, because term stats are
: I think you'd have to get creative with function queries. The trick is
You don't have to get *very* creative...
: I want to retrieve all docs or records from solr where updateDate =
: appliedDate OR appliedDate == null
Pretty sure all you need is...
fq={!frange
(NOTE: cross-posted announcement, please confine any replies to
general@lucene)
As you may be aware, ApacheCon will be held this year in Budapest, on
November 17-23. (See http://apachecon.eu for more info.)
### ### 1 - Call For Papers - June 25
The CFP for the conference is still open, but
: set with the latest solr version. (Now we are running version 4.8 - the
: current schema has a uniqueid field set, while it wasn't present in the
: earlier versions. This unique field is unsurprisingly called id but not
: all the documents have it.)
this is going to be the source of a lot
Ronald: I'm having a little trouble understading the steps o reproduce
that you are describing -- in particular Step 1 f ii because i'm not
really sure i understand what exactly you are putting in mem2.xml
Also: Since you don't appera to be using implicit routing, i'm not clear
on why you
You need to explain what you mean here -- you're using hte term
multiplicitive boost but the context in which you are asking the
question suggests that isn't really applicable.
what does that expression mean to you?
Or to step back and rephrase: What is your usecase? What does your data
look
: My guess is that the customized toolkit AMI (ami-1e6b9d76) at AWS is not
: accessible by my AWS credentials. Is this an AMI permissioning issue or is
: it a problem with my particular account or how it is configured at AWS. I
: did not experience this specific problem when working with the
What you are asking for is comparable to saying:
In my index I have a TrieIntField called severity when i search for
severity:8765432 I get results, but when i search for severity:8* i get no
results -- what do i need to change so that this type of query matches
8765432 and 8978 and 84356
: edismax was developed before the lucene query parser syntax was
: changed to include regex, so maybe that's the issue.
edismax has never been explicilty updated to support regex queries...
https://issues.apache.org/jira/browse/SOLR-6009
-Hoss
http://www.lucidworks.com/
: I have an external system that indexes the data.
: One field has a closed set of values, the values are sent as integer but
: should be represented as String. EnumField is perfect for it. The
: problem is that the values are predefined and not order sequentially
: (the values were spaced
: Try the TZ parameter on the query, as blahTZ=GMT-4
Docs...
https://cwiki.apache.org/confluence/display/solr/Working+with+Dates
: There's a good discussion of why PDT is ambiguous here:
: https://issues.apache.org/jira/browse/SOLR-2690.
-Hoss
http://www.lucidworks.com/
: The presence of the {!tag} entry changes the filter query generated by
: the {!field...} tag. Note below that in one case the filter query is a
: phrase query, and in the other it's parsed with one term against the
: specified field and the other against the default field.
I think you are
: when local params are embedded in a query being parsed by the
: LuceneQParser, it applies them using the same scoping as other query
: operators
:
: : fq: {!tag=name_name}{!field f=name}United States
Think of that example in the context of this one -- the basics of
: So I think when I was paging through the results, if the query for page N
: was handled by replica1 and page N+1 handled by replica2, and the page
: boundary happened to be where the reversed rows were, this would produce
: the behavior I was seeing where the last row from the previous page was
: Using Solr 4.6.1 and in my schema I have a date field storing the time a
: document was added to Solr.
what *exactly* does your schema look like? are you using solr.DateField
or solr.TrieDateField ? what field options do you have specified?
: I have a utility program which:
: - queries for
: me incorporate these config files as before. I'm (naively?) trying the
: following:
:
: final StandardQueryParser parser = new StandardQueryParser();
: final Query luceneQuery = parser.parse(query, text);
: luceneIndex.getIndexSearcher().search(luceneQuery, collector);
As noted by others: you should definitely look into sharding your index --
fundementally there is no way to have that many documents in a single
Lucene index.
However: this is a terrible error for you to get, something in the stack
should have really given you an error when you tried to add
: 'query' is a function returning a number.
: You can't use it as a query.
Well ... you can, you just have to use the correct query parser.
since there is nothing to make it clear to solr that you want it to parse
the q paramater as a function, it's using hte default parser, and
probably
Shot in the dark: perhaps you have a doc w/o a value in the description
field, which means the xsl:variable's select doesn't match anything; which
perhaps means that your XSLT engine then leaves the variable undefined.
: Solr 4.7.2 (and 4.6.1)
: Tomcat 7.0.52
: Java 1.7.0_45 (and _55)
:
:
: Hi everybody
: can anyone give me a suitable interpretation for cat_rank in
: http://people.apache.org/~hossman/ac2012eu/ slide 15
Have you seen the video?
http://vimeopro.com/user11514798/apache-lucene-eurocon-2012/video/55822630
That slide starts ~ 23:00 and i go through a
The full details are farther down in the stack...
: null:org.apache.solr.common.SolrException: SolrCore 'master' is not
: available due to init failure: Error initializing QueryElevationComponent.
...
: Caused by: org.apache.solr.common.SolrException: Error initializing
:
: My understanding is that DynamicField can do something like
: FOO_BAR_TEXT_* but what I really need is *_TEXT_* as I might have
: FOO_BAR_TEXT_1 but I also might have WIDGET_BAR_TEXT_2. Both of those
: field names need to map to a field type of 'fullText'.
I'm pretty sure you can get what you
The Lucene PMC is pleased to announce that there is a new version of the
Solr Reference Guide available for Solr 4.8.
The 396 page PDF serves as the definitive user's manual for Solr 4.8. It
can be downloaded from the Apache mirror network:
Shamik:
I'm not sure what the cause of this is, but it definitely seems like a bug
to me. I've opened SOLR-6039 and noted a workarround for folks who don't
care about the new track debug info and just want the same debug info
that was available before 4.7...
https://people.apache.org/~hossman/#threadhijack
Thread Hijacking on Mailing Lists
When starting a new discussion on a mailing list, please do not reply to
an existing message, instead start a fresh email. Even if you change the
subject line of your email, other mail headers still track which
: Anyone knows why we can`t have an analysis chain on a numeric field ?
: Looks to me like it would be very useful to be able to
: manipulate/transform a value without an external resources.
Analysis only affects *indexed* terms -- it has no impact on the stored
values (or things like
: here is the query:
:
http://localhost:7001/solr/collection1/select?q=*%3A*rows=5fl=*%2Cscorewt=jsonindent=truedebugQuery=true
:
:
: and here the response:
that's bizare.
Do me a favor, and:
* post the results of
.../select?q=*%3A*rows=1fl=scorewt=jsonindent=trueechoParams=true
* show
: So, is Overseer really only an implementation detail or something that Solr
: Ops guys need to be very aware of?
Most people don't ever need to worry about the overseer - it's magic and
it will take care of itself.
The recent work on adding support for an overseer role in 4.7 was
: we tried another commands to delete the document ID:
:
: 1 For Deletion:
:
: curl http://localhost:8983/solr/update -H 'Content-type:application/json' -d
: '
: [
You're use of square brackets here is triggering the syntax-sugar that
let's you add documents as objects w/o needing the add
Mark: first off, the details matter.
Nothing in your first email mae it clear that the {!join} query you were
refering to was not the entirety of your query param -- which is part of
the confusion and was a significant piece of Shawn's answer. Had you
posted the *exact* request you were
: field : // this is the field that I want to learn which document has
: it.
How you (can) query for a field value like that is going to depend
entirely on the FieldTYpe/Analyzer ... if it's a string field, of uses
KeywordTokenizer then q=field: should find it -- if you use a more
: I am trying to build lucene 4.7.1 from the sources. I can compile without
: any issues but when I try to build the dist, lucene gives me
: Cannot run program svnversion ... The system cannot find the specified
: file.
:
: I am compiling on Windows 7 64-bit using java version 1.7.0.45
: What should I be doing to fix them? Is there a replacement for those
: classes? Do I just need to change the luceneMatchVersion to be LUCENE_461
: or something?
that's pretty much exactly what that warning message is trying to tell you
-- your config ways to use LUCENE_33 mode, but that
: Thanks for your response. Here is an example of what I'm trying to do. If I
: had the following documents:
what you are attempting is fairly trivial -- you want to query for all
parent documents, then kapply 3 filters:
* parent of a child matching item1
* parent of a child matching item2
: I'm not sure if I am missing something of if this is a bug but I am facing an
: issue with the following scenario.
The specific scnerios you are descirbing is an interesting edge case --
but i believe it's working as designed.
basically the range generation logic that computes the set of
: I have a search that sorts on a boolean field. This search is pulling
: the following error: java.lang.String cannot be cast to
: org.apache.lucene.util.BytesRef.
This is almost certainly another manifestation of SOLR-5920...
https://issues.apache.org/jira/browse/SOLR-5920
-Hoss
: I have a date field in my Solr schema defined as described below
: When I'm trying to query fields stats, the max value for that date field is
: not constant, it changes between two distinct date values as I retry query.
: Any ideas as to why this is happening?
smells like ou might have
: What is the default value for the required attribute of a field element
: in a schema? I've just looked everywhere I can think of in the wiki, the
: reference manual, and the JavaDoc. Most of the documentation doesn't
: even mention that attribute.
Good catch, fixed...
: Thank-you, that all sounds great. My assumption about documents being
: missed was something like this:
...
: In that situation D would always be missed, whether the cursorMark 'C or
: greater' or 'greater than B' (I'm not sure which it is in practice), simply
: because the cursorMark
: Reason: In an index with millions of documents I don't want to know that a
: certain query matched 1 million docs (of course it will take time to
: calculate that). Why don't just stop looking for more results lets say
: after it finds 100 docs? Possible??
but if you care about sorting, ie:
501 - 600 of 4495 matches
Mail list logo