Re: Solr 4.2.1 SSLInitializationException
You have to add two new Java options to your Glassfish config (example if you use the standard keystore and truststore): asadmin create-jvm-options -- -Djavax.net.ssl.keyStorePassword=changeit asadmin create-jvm-options -- -Djavax.net.ssl.trustStorePassword=changeit /Uwe On 10 April 2013 03:59, Sarita Nair sarita...@yahoo.com wrote: Hi Chris, Thanks for your response. My understanding is that GlassFish specifies the keystore as a system property, but does not specify the password in order to protect it from snooping. There's a keychain that requires a password to be passed from the DAS in order to unlock the key for the keystore. Is there some way to specify a different HttpClient implementation (e.g. DefaultHttpClient rather than SystemDefaultHttpClient), as we don't want the application to have access to the keystore? I have also pasted the entire stack trace below: 2013-04-09 10:45:06,144 [main] ERROR org.apache.solr.servlet.SolrDispatchFilter - Could not start Solr. Check solr/home property and the logs 2013-04-09 10:45:06,224 [main] ERROR org.apache.solr.core.SolrCore - null:org.apache.http.conn.ssl.SSLInitializationException: Failure initializing default system SSL context at org.apache.http.conn.ssl.SSLSocketFactory.createSystemSSLContext(SSLSocketFactory.java:368) at org.apache.http.conn.ssl.SSLSocketFactory.getSystemSocketFactory(SSLSocketFactory.java:204) at org.apache.http.impl.conn.SchemeRegistryFactory.createSystemDefault(SchemeRegistryFactory.java:82) at org.apache.http.impl.client.SystemDefaultHttpClient.createClientConnectionManager(SystemDefaultHttpClient.java:118) at org.apache.http.impl.client.AbstractHttpClient.getConnectionManager(AbstractHttpClient.java:466) at org.apache.solr.client.solrj.impl.HttpClientUtil.setMaxConnections(HttpClientUtil.java:179) at org.apache.solr.client.solrj.impl.HttpClientConfigurer.configure(HttpClientConfigurer.java:33) at org.apache.solr.client.solrj.impl.HttpClientUtil.configureClient(HttpClientUtil.java:115) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:105) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:134) at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.start(GlassFishImpl.java:79) at com.sun.enterprise.glassfish.bootstrap.GlassFishDecorator.start(GlassFishDecorator.java:63) at com.sun.enterprise.glassfish.bootstrap.osgi.OSGiGlassFishImpl.start(OSGiGlassFishImpl.java:69) at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher.launch(GlassFishMain.java:117) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.sun.enterprise.glassfish.bootstrap.GlassFishMain.main(GlassFishMain.java:97) at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:55) Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:772) at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:55) at java.security.KeyStore.load(KeyStore.java:1214) at org.apache.http.conn.ssl.SSLSocketFactory.createSystemSSLContext(SSLSocketFactory.java:281) at org.apache.http.conn.ssl.SSLSocketFactory.createSystemSSLContext(SSLSocketFactory.java:366) ... 50 more Caused by: java.security.UnrecoverableKeyException: Password verification failed at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:770) ... 54 more From: Chris Hostetter hossman_luc...@fucit.org To: solr-user@lucene.apache.org solr-user@lucene.apache.org; Sarita Nair sarita...@yahoo.com Sent: Tuesday, April 9, 2013 1:31 PM Subject: Re: Solr 4.2.1 SSLInitializationException : Deploying Solr 4.2.1 to GlassFish 3.1.1 results in the error below. I : have seen similar problems being reported with Solr 4.2 Are you trying to use server SSL with glassfish? can you please post the full stack trace so we can see where this error is coming from. My best guess is that this is coming from the changes made in SOLR-4451 to use system defaults correctly when initializing HttpClient, which suggets that your problem is exactly what the error message says... Keystore was tampered with, or password was incorrect Is it possible that the default keystore password for your JVM (or as overridden by glassfish defaults - possibly using the javax.net.ssl.keyStore sysprop) has a password set on it? If so you need to confiure your JVM with the standard java system properties to specify what that password is.
Re: Indexed data not searchable
Thanks to this! No I have another problem. I tried to give the XML file the right format so I made this ?xml version=1.0 encoding=UTF-8? adddoc field name=id455HHS-2232/field field name=titleT0072-00031-DOWNLOAD - Blatt 12v/field field name=formatapplication/pdf/field field name=created2012-11-07T11:15:19.887+01:00/field field name=lastModified2012-11-07T11:15:19.887+01:00/field field name=issued2012-11-07T11:15:19.887+01:00/field field name=revision0/field field name=pidhdl:11858/00-1734--0008-12C5-2/field field name=extent1131033/field field name=projectSt. Matthias Test 07/field field name=availabilitypublic/field field name=rightsHolderStadtbibliothek und Stadtarchiv Trier/field /doc/add I also made the changes in the schema.xml I added this fields: field name=identifier type=text_general indexed=true stored=true/ field name=format type=text_general indexed=true stored=true/ field name=created type=date indexed=true stored=true/ field name=issued type=date indexed=true stored=true/ field name=revision type=int indexed=true stored=true/ field name=pid type=text_general indexed=true stored=true/ field name=extent type=int indexed=true stored=true/ field name=dataContributor type=text_general indexed=true stored=true/ field name=project type=text_general indexed=true stored=true/ field name=availability type=text_general indexed=true stored=true/ field name=rightsholder type=text_general indexed=true stored=true/ Did I made anything wrong? -- View this message in context: http://lucene.472066.n3.nabble.com/Indexed-data-not-searchable-tp4054473p4054960.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Empty Solr 4.2.1 can not create Collection
Hi Joel, I followed your steps, the cores and collection get created, but there is no leader elected so I can not query the collection... Do I miss something ? Kind Regards Alexander Am 2013-04-09 10:21, schrieb A.Eibner: Hi, thanks for your faster answer. You don't use the Collection API - may I ask you why ? Therefore you have to setup everything (replicas, ...) manually..., which I would like to avoid. Also what I don't understand, why my steps work in 4.0 but won't in 4.2.1... Any clues ? Kind Regards Alexander Am 2013-04-08 19:12, schrieb Joel Bernstein: The steps that I use to setup the collection are slightly different: 1) Start zk and upconfig the config set. Your approach is same. 2) Start appservers with Solr zkHost set to the zk started in step 1. 3) Use a core admin command to spin up a new core and collection. http://app01/solr/admin/cores?action=CREATEname=storage-corecollection=storagenumShards=1collection.configName=storage-confhttp://app03/solr/admin/collections?action=CREATEname=storagenumShards=1replicationFactor=2collection.configName=storage-conf shard=shard1 This will spin up the new collection and initial core. I'm not using a replication factor because the following commands manually bind the replicas. 4) Spin up replica with a core admin command: http://app02/solr/admin/cores?action=CREATEname=storage-corecollection=storage;http://app03/solr/admin/collections?action=CREATEname=storagenumShards=1replicationFactor=2collection.configName=storage-conf shard=shard1 5) Same command as above on the 3rd server to spin up another replica. This will spin up a new core and bind it to shard1 of the storage collection. On Mon, Apr 8, 2013 at 9:34 AM, A.Eibner a_eib...@yahoo.de wrote: Hi, I have a problem with setting up my solr cloud environment (on three machines). If I want to create my collections from scratch I do the following: *) Start ZooKeeper on all machines. *) Upload the configuration (on app02) for the collection via the following command: zkcli.sh -cmd upconfig --zkhost app01:4181,app02:4181,app03:**4181 --confdir config/solr/storage/conf/ --confname storage-conf *) Linking the configuration (on app02) via the following command: zkcli.sh -cmd linkconfig --collection storage --confname storage-conf --zkhost app01:4181,app02:4181,app03:**4181 *) Start Tomcats (containing Solr) on app02,app03 *) Create Collection via: http://app03/solr/admin/**collections?action=CREATE** name=storagenumShards=1**replicationFactor=2** collection.configName=storage-**confhttp://app03/solr/admin/collections?action=CREATEname=storagenumShards=1replicationFactor=2collection.configName=storage-conf This creates the replication of the shard on app02 and app03, but neither of them is marked as leader, both are marked as DOWN. And after wards I can not access the collection. In the browser I get: SEVERE: org.apache.solr.common.**SolrException: no servers hosting shard: In the log files the following error is present: SEVERE: Error from shard: app02:9985/solr org.apache.solr.common.**SolrException: Error CREATEing SolrCore 'storage_shard1_replica1': at org.apache.solr.client.solrj.**impl.HttpSolrServer.request(** HttpSolrServer.java:404) at org.apache.solr.client.solrj.**impl.HttpSolrServer.request(** HttpSolrServer.java:181) at org.apache.solr.handler.**component.HttpShardHandler$1.** call(HttpShardHandler.java:**172) at org.apache.solr.handler.**component.HttpShardHandler$1.** call(HttpShardHandler.java:**135) at java.util.concurrent.**FutureTask$Sync.innerRun(** FutureTask.java:334) at java.util.concurrent.**FutureTask.run(FutureTask.**java:166) at java.util.concurrent.**Executors$RunnableAdapter.** call(Executors.java:471) at java.util.concurrent.**FutureTask$Sync.innerRun(** FutureTask.java:334) at java.util.concurrent.**FutureTask.run(FutureTask.**java:166) at java.util.concurrent.**ThreadPoolExecutor.runWorker(** ThreadPoolExecutor.java:1110) at java.util.concurrent.**ThreadPoolExecutor$Worker.run(** ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.**java:722) Caused by: org.apache.solr.common.cloud.**ZooKeeperException: at org.apache.solr.core.**CoreContainer.registerInZk(** CoreContainer.java:922) at org.apache.solr.core.**CoreContainer.registerCore(** CoreContainer.java:892) at org.apache.solr.core.**CoreContainer.register(** CoreContainer.java:841) at org.apache.solr.handler.admin.**CoreAdminHandler.** handleCreateAction(**CoreAdminHandler.java:479) ... 19 more Caused by: org.apache.solr.common.**SolrException: Error getting leader from zk for shard shard1 at org.apache.solr.cloud.**ZkController.getLeader(** ZkController.java:864) at org.apache.solr.cloud.**ZkController.register(** ZkController.java:776) at org.apache.solr.cloud.**ZkController.register(**
Re: How can I set configuration options?
Thanks Hoss, those are some really useful clarifications. Since what I'm working on is currently at POC stage I'll go with the system properties and will refactor them out as I move towards having a standalone ZooKeeper ensemble. Thanks again. Edd On 10 April 2013 01:41, Chris Hostetter hossman_luc...@fucit.org wrote: : Thanks for the replies. The problem I have is that setting them at the JVM : level would mean that all instances of Solr deployed in the Tomcat instance : are forced to use the same settings. I actually want to set the properties : at the application level (e.g. in solr.xml, zoo.conf or maybe an : application level Tomcat Context.xml file). the thing to keep in mind is that most of the params you refered to are things you would not typically want in a deployed setup. others are just ways of specifying defaults that are substituted into configs... : java -Dbootstrap_confdir=./solr/collection1/conf you don't wnat this option for a normal setup, it's just for boostratping (hence it's only a system property). in a production setup you would use the zookeeper tools to load the configs into your zk quorum. https://wiki.apache.org/solr/SolrCloud#Config_Startup_Bootstrap_Params ...vs... https://wiki.apache.org/solr/SolrCloud#Command_Line_Util : -Dcollection.configName=myconf -DzkRun ditto for collection.configName -- it's only for boostraping zkRun is something you only use in trivial setups like the examples in the SolrCloud tutorial to run zookeeper embedded in Solr. if you are running a production cluster where you want to be able to add/remove solr nodes on the fly, then you are going to want to set of specific machines running standalone zookeper. : -DzkHost=localhost:9983,localhost:8574,localhost:9900 -DnumShards=2 zkHost can be specified in solr.xml (allthough i'm not sure why the example solr.xml doesn't include it, i'll update SOLR-4622 to address this), or it can be overridden by a system property. -Hoss -- Web: http://www.eddgrant.com Email: e...@eddgrant.com Mobile: +44 (0) 7861 394 543
Solrstrap
Greetings Solrians This is just a reminder that Solrstrap is a thing, and that it might help you out with your Solr project. http://fergiemcdowall.github.io/solrstrap/ Solrstap is wondering which new features it needs. Solrstrap would like to hear your suggestions. Feel free to post here or open an issue on Github. Regards Solrstrap
Re: Indexed data not searchable
Just for information: I indicate that the problem occurs when I try to add the fields, created, last_modified, issued (all three have the type date) and the field rightsholder. Maybe it is helpful! -- View this message in context: http://lucene.472066.n3.nabble.com/Indexed-data-not-searchable-tp4054473p4054977.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Indexing and searching documents in different languages
Thx, I'll try this approach. Zitat von Alexandre Rafalovitch arafa...@gmail.com: Have you looked at edismax and the 'qf' fields parameter? It allows you to define the fields to search. Also, you can define those parameters in solrconfig.xml and not have to send them down the wire. Finally, you can define several different request handlers (e.g. /ensearch, /frsearch) and have each of them use different 'qf' values, possibly with 'fl' field also defined and with field name aliasing from language-specific to generic names. Regards, Alex. Personal blog: http://blog.outerthoughts.com/ LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch - Time is the quality of nature that keeps events from happening all at once. Lately, it doesn't seem to be working. (Anonymous - via GTD book) On Tue, Apr 9, 2013 at 2:32 PM, d...@geschan.de wrote: Hello, I'm trying to index a large number of documents in different languages. I don't know the language of the document, so I'm using TikaLanguageIdentifierUpdatePr**ocessorFactory to identify it. So, this is my configuration in solrconfig.xml updateRequestProcessorChain name=langid processor class=org.apache.solr.update.**processor.** TikaLanguageIdentifierUpdatePr**ocessorFactory bool name=langidtrue/bool str name=langid.fltitle,**subtitle,content/str str name=langid.langField**language_s/str str name=langid.threshold0.3/**str str name=langid.fallback**general/str str name=langid.whitelisten,fr,**de,it,es/str bool name=langid.maptrue/bool bool name=langid.map.keepOrig**true/bool /processor processor class=solr.**LogUpdateProcessorFactory / processor class=solr.**RunUpdateProcessorFactory / /updateRequestProcessorChain So, the detection works fine and I put some dynamic fields in schema.xml to store the results: dynamicField name=*_en type=text_enindexed=true stored=true multiValued=true/ dynamicField name=*_fr type=text_frindexed=true stored=true multiValued=true/ dynamicField name=*_de type=text_deindexed=true stored=true multiValued=true/ dynamicField name=*_it type=text_itindexed=true stored=true multiValued=true/ dynamicField name=*_es type=text_esindexed=true stored=true multiValued=true/ My main problem now is how to search the document without knowing the language of the searched document. I don't want to have a huge querystring like ?q=title_en:+term+subtitle_en:**+term+title_de:+term... Okay, using copyField and copy all fields into the text field...but text has the type text_general, so the language specific indexing is not working. I could use at least a combined field for every language (like text_en, text_fr...) but still, my querystring gets very long and to add new languages is terribly uncomfortable. So, what can I do? Is there a better solution to index and search documents in many languages without knowing the language of the document and the query before? - Geschan
Re: Indexed data not searchable
On Wed, Apr 10, 2013 at 10:35 AM, Max Bo maximilian.brod...@gmail.comwrote: Just for information: I indicate that the problem occurs when I try to add the fields, created, last_modified, issued (all three have the type date) and the field rightsholder. Maybe it is helpful! From the example you sent earlier, it appears that the format of your timestamps does not conform with SOLR's expectations; they should be something like 1995-12-31T23:59:59.999Z (i.e, zero timezone offset, designated with a trailing Z).
Re: Indexed data not searchable
Thank you. I changed it and now it works. But is there any possibility to make the given timestamp acceptable for solr? -- View this message in context: http://lucene.472066.n3.nabble.com/Indexed-data-not-searchable-tp4054473p4054985.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Collection's Size
number of documents found can be found in a field called numFound in
the response.
If you do use SolrJ you will likely have a QueryResponse qr and can just
do a qr.setNumFound().
If you use do not use SolrJ try to add e.g. wt=json to your search query
to get the response in JSON. Find the numFound field in the readable
JSON response - it should be at response.numFound. If in javascript
with jQuery something like this should work:
$.getJSON(search_url,
function(data) {
... data.response.numFound ...
}
)
Go figure who to extract it in javascript without jQuery
Regards, Per Steffensen
On 4/5/13 3:20 PM, Alexandre Rafalovitch wrote:
I'd add rows=0, just to avoid the actual records serialization if size is
all that matters.
Regards,
Alex.
Personal blog: http://blog.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is the quality of nature that keeps events from happening all at
once. Lately, it doesn't seem to be working. (Anonymous - via GTD book)
On Fri, Apr 5, 2013 at 8:26 AM, Jack Krupansky j...@basetechnology.comwrote:
Query for *:* and look at the number of documents found.
-- Jack Krupansky
-Original Message- From: Ranjith Venkatesan
Sent: Friday, April 05, 2013 2:06 AM
To: solr-user@lucene.apache.org
Subject: Solr Collection's Size
Hi,
I am new to solr. I want to find size of collection dynamically via solrj.
I
tried many ways but i couldnt succeed in any of those. Pls help me with
this
issue.
Solr : Search with special character
We need to make Solr Search like Success Failure Working 50% but Solr query parser eliminates all special characters from search. my search query is as mentioned below http://localhost:8080/solr/core/select?q=%22Success%20%26%20Failure%22hl=onhl.snippets=99debugQuery=on below is debugQuery for it. lst name=debug str name=rawquerystringSuccess Failure/str str name=querystringSuccess Failure/str str name=parsedqueryPhraseQuery(text:success failure)/str str name=parsedquery_toStringtext:success failure/str lst name=explain/ str name=QParserLuceneQParser/str lst name=timing/lst /lst We want to make, solr should search with success failure , and should not eliminate special character. anybody have any idea, how to do this?? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Search-with-special-character-tp4054994.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Collection's Size
On 4/10/13 12:17 PM, Per Steffensen wrote:
number of documents found can be found in a field called numFound
in the response.
If you do use SolrJ you will likely have a QueryResponse qr and can
just do a qr.setNumFound().
qr.getResults().getNumFound() :-)
If you use do not use SolrJ try to add e.g. wt=json to your search
query to get the response in JSON. Find the numFound field in the
readable JSON response - it should be at response.numFound. If in
javascript with jQuery something like this should work:
$.getJSON(search_url,
function(data) {
... data.response.numFound ...
}
)
Go figure who to extract it in javascript without jQuery
Regards, Per Steffensen
On 4/5/13 3:20 PM, Alexandre Rafalovitch wrote:
I'd add rows=0, just to avoid the actual records serialization if
size is
all that matters.
Regards,
Alex.
Personal blog: http://blog.outerthoughts.com/
LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
- Time is the quality of nature that keeps events from happening all at
once. Lately, it doesn't seem to be working. (Anonymous - via GTD
book)
On Fri, Apr 5, 2013 at 8:26 AM, Jack Krupansky
j...@basetechnology.comwrote:
Query for *:* and look at the number of documents found.
-- Jack Krupansky
-Original Message- From: Ranjith Venkatesan
Sent: Friday, April 05, 2013 2:06 AM
To: solr-user@lucene.apache.org
Subject: Solr Collection's Size
Hi,
I am new to solr. I want to find size of collection dynamically via
solrj.
I
tried many ways but i couldnt succeed in any of those. Pls help me with
this
issue.
Re: Approximately needed RAM for 5000 query/second at a Solr machine?
Thank you for your explanations, this will help me to figure out my system. 2013/4/10 Shawn Heisey s...@elyograg.org On 4/9/2013 9:12 PM, Furkan KAMACI wrote: I am sorry but you said: *you need enough free RAM for the OS to cache the maximum amount of disk space all your indexes will ever use* I have made an assumption my indexes at my machine. Let's assume that it is 5 GB. So it is better to have at least 5 GB RAM? OK, Solr will use RAM up to how much I define it as a Java processes. When we think about the indexes at storage and caching them at RAM by OS, is that what you talk about: having more than 5 GB - or - 10 GB RAM for my machine? If your index is 5GB, and you give 3GB of RAM to the Solr JVM, then you would want at least 8GB of total RAM for that machine - the 3GB of RAM given to Solr, plus the rest so the OS can cache the index in RAM. If you plan for double the cache memory, you'd need 13 to 14GB. Thanks, Shawn
Combining join queries
I'm sure the best way for me to solve this issue myself is to ask it
publicly, so...
If I have two {!join} queries that select a collection of documents
each, how do I create a filter query that combines their results?
If I do fq={!join} {!join...} it only considers the first.
From what I understand, the Lucene query parser should combine these two
with an implicit OR, no?
Upayavira
Re: Indexed data not searchable
Solr assumes you are using UTC. It is your job to do a conversion. If you want Solr to do it, you could use an UpdateProcessor to do it, either using RegExp, or perhaps a ScriptUpdateProcessor. In fact, if you're comfortable with XSLT, you can make Solr accept your old format of XML by posting it to Solr providing a tr= parameter that points to an XSLT stylesheet which transforms your XML into an adddoc format. Upayavira On Wed, Apr 10, 2013, at 10:36 AM, Max Bo wrote: Thank you. I changed it and now it works. But is there any possibility to make the given timestamp acceptable for solr? -- View this message in context: http://lucene.472066.n3.nabble.com/Indexed-data-not-searchable-tp4054473p4054985.html Sent from the Solr - User mailing list archive at Nabble.com.
Troubles with solr replication
Hi guys, I have some problems with Solr replication and can see some unexpected behavior. Would be nice to have some answers where am I wrong, or what is the best way to solve the problem. I have a replication master-slave. http://192.168.2.204:8080/solr/ is master and http://192.168.2.174:8080/solr/ is slave. With quite simple config. requestHandler name=/replication class=solr.ReplicationHandler lst name=master str name=enable false /str str name=replicateAfter commit /str str name=replicateAfter startup /str str name=confFiles schema.xml,boosting.txt /str /lst lst name=slave str name=enable true /str str name=masterUrlhttp://192.168.2.204:8080/solr/replication/str str name=pollInterval00:00:60/str str name=httpConnTimeout 5000 /str str name=httpReadTimeout 1 /str /lst /requestHandler The main idea when I started playing around Solr is to replicate some boosting values. I wanted to use confFile option for it. An here is my first problem. I wasn't able to replicate files from master. On slave I was able to see only schema.xml. I wanted to check, do I actually have the files and everything correct in solr config. So I checked on master the file list and it returns the list of all files http://192.168.2.204:8080/solr/replication?command=filelistindexversion=1341328964983 but for slave I can't see anything http://192.168.2.174:8080/solr/replication?command=filelistindexversion=1341328964983 returns str name=statusinvalid indexversion/str Seems like we don't have this index version. After I tried to find what is wrong with that. On slave http://192.168.2.174:8080/solr/replication?command=indexversion returns only 0 long name=indexversion0/long long name=generation0/long on master I could see the version of current index long name=indexversion1341328964983/long long name=generation3/long but slave's http://192.168.2.174:8080/solr/admin/stats.jsp I can the the right version indexVersion : 1341328964983 generation : 3 Also when I checked the solr log. [org.apache.solr.handler.SnapPuller] Master at: http://192.168.2.204:8080/solr/replication is not available. Index fetch failed. Exception: Connection refused BTW, I was able to fetch the replication file with wget directly. So my question is: What is wrong with my replication or Solr? About version, I use some legacy version of Solr: Solr Specification Version: 3.5.0.2011.11.22.14.54.38, because we have some legacy systems here. And another question what is the best way to migrate to the latest version. I mean to keep alive all the boosting infrastructure based on ExternalFileField options. Thank you in advance for your time and help you can provide, Sergii
Re: Combining join queries
On Wed, Apr 10, 2013, at 12:22 PM, Upayavira wrote:
I'm sure the best way for me to solve this issue myself is to ask it
publicly, so...
If I have two {!join} queries that select a collection of documents
each, how do I create a filter query that combines their results?
If I do fq={!join} {!join...} it only considers the first.
From what I understand, the Lucene query parser should combine these two
with an implicit OR, no?
Yup. My theory worked, I did solve it myself.
If I instead do fq=_query_:{!join...} _query_:{!join...} it works
fine, but the shortened syntax above, which I believe is supposed to
work, doesn't. This is using Solr 4.0.
Upayavira
Highlighting multivalued fields using fvh
Hi all, accoridng to this ticket: https://issues.apache.org/jira/browse/SOLR-2998 Are there any plans to fix this bug? Is there antoher way to usign fvh and still having proper results (without concatenation)? -- Karol Sikora +48 781 493 788 Laboratorium EE ul. Mokotowska 46A/23 | 00-543 Warszawa | www.laboratorium.ee | www.laboratorium.ee/facebook
Re: Solr : Search with special character
Switch the field types from the standard tokenizer to the white space tokenizer and don't use the word delimiter filter. Or, you can sometimes add custom character mapping tables to some filters and indicate that your desired special characters should be mapped to type ALPHA. -- Jack Krupansky -Original Message- From: meghana Sent: Wednesday, April 10, 2013 6:25 AM To: solr-user@lucene.apache.org Subject: Solr : Search with special character We need to make Solr Search like Success Failure Working 50% but Solr query parser eliminates all special characters from search. my search query is as mentioned below http://localhost:8080/solr/core/select?q=%22Success%20%26%20Failure%22hl=onhl.snippets=99debugQuery=on below is debugQuery for it. lst name=debug str name=rawquerystringSuccess Failure/str str name=querystringSuccess Failure/str str name=parsedqueryPhraseQuery(text:success failure)/str str name=parsedquery_toStringtext:success failure/str lst name=explain/ str name=QParserLuceneQParser/str lst name=timing/lst /lst We want to make, solr should search with success failure , and should not eliminate special character. anybody have any idea, how to do this?? -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Search-with-special-character-tp4054994.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Pushing a whole set of pdf-files to solr
Jack - I apologize for my ignorance here, but when you keep emphasizing 'new' - does that mean that there is ANOTHER version of this tool than the one that is built into solr-4.2.1? And on the encoding issue - I thought pdf was platform-agnostic? Or is the problem on my windows system - i.e. that it extracts the (correctly encoded) text into Win-1251, which solr then has a problem with? But can't I change that somewhere then? -- View this message in context: http://lucene.472066.n3.nabble.com/Pushing-a-whole-set-of-pdf-files-to-solr-tp4025256p4055010.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Pushing a whole set of pdf-files to solr
Yes, there is the version that comes with Solr 3.x. I'm not aware of an encoding issue. -- Jack Krupansky -Original Message- From: sdspieg Sent: Wednesday, April 10, 2013 8:11 AM To: solr-user@lucene.apache.org Subject: Re: Pushing a whole set of pdf-files to solr Jack - I apologize for my ignorance here, but when you keep emphasizing 'new' - does that mean that there is ANOTHER version of this tool than the one that is built into solr-4.2.1? And on the encoding issue - I thought pdf was platform-agnostic? Or is the problem on my windows system - i.e. that it extracts the (correctly encoded) text into Win-1251, which solr then has a problem with? But can't I change that somewhere then? -- View this message in context: http://lucene.472066.n3.nabble.com/Pushing-a-whole-set-of-pdf-files-to-solr-tp4025256p4055010.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Approximately needed RAM for 5000 query/second at a Solr machine?
On 4/9/2013 7:03 PM, Furkan KAMACI wrote: These are really good metrics for me: You say that RAM size should be at least index size, and it is better to have a RAM size twice the index size (because of worst case scenario). On the other hand let's assume that I have a RAM size that is bigger than twice of indexes at machine. Can Solr use that extra RAM or is it a approximately maximum limit (to have twice size of indexes at machine)? What we have been discussing is the OS cache, which is memory that is not used by programs. The OS uses that memory to make everything run faster. The OS will instantly give that memory up if a program requests it. Solr is a java program, and java uses memory a little differently, so Solr most likely will NOT use more memory when it is available. In a normal directly executable program, memory can be allocated at any time, and given back to the system at any time. With Java, you tell it the maximum amount of memory the program is ever allowed to use. Because of how memory is used inside Java, most long-running Java programs (like Solr) will allocate up to the configured maximum even if they don't really need that much memory. Most Java virtual machines will never give the memory back to the system even if it is not required. Thanks, Shawn Furkan KAMACI furkankam...@gmail.com writes: I am sorry but you said: *you need enough free RAM for the OS to cache the maximum amount of disk space all your indexes will ever use* I have made an assumption my indexes at my machine. Let's assume that it is 5 GB. So it is better to have at least 5 GB RAM? OK, Solr will use RAM up to how much I define it as a Java processes. When we think about the indexes at storage and caching them at RAM by OS, is that what you talk about: having more than 5 GB - or - 10 GB RAM for my machine? 2013/4/10 Shawn Heisey s...@elyograg.org 10 GB. Because when Solr shuffles the data around, it could use up to twice the size of the index in order to optimize the index on disk. -- Justin
Troubles with solr replication
Hi guys, I have some problems with Solr replication and can see some unexpected behavior. Would be nice to have some answers where am I wrong, or what is the best way to solve the problem. I have a replication master-slave. http://192.168.2.204:8080/solr/ is master and http://192.168.2.174:8080/solr/ is slave. With quite simple config. requestHandler name=/replication class=solr.ReplicationHandler lst name=master str name=enable false /str str name=replicateAfter commit /str str name=replicateAfter startup /str str name=confFiles schema.xml,boosting.txt /str /lst lst name=slave str name=enable true /str str name=masterUrlhttp://192.168.2.204:8080/solr/replication/str str name=pollInterval00:00:60/str str name=httpConnTimeout 5000 /str str name=httpReadTimeout 1 /str /lst /requestHandler The main idea when I started playing around Solr is to replicate some boosting values. I wanted to use confFile option for it. An here is my first problem. I wasn't able to replicate files from master. On slave I was able to see only schema.xml. I wanted to check, do I actually have the files and everything correct in solr config. So I checked on master the file list and it returns the list of all files http://192.168.2.204:8080/solr/replication?command=filelistindexversion=1341328964983 but for slave I can't see anything http://192.168.2.174:8080/solr/replication?command=filelistindexversion=1341328964983 returns str name=statusinvalid indexversion/str Seems like we don't have this index version. After I tried to find what is wrong with that. On slave http://192.168.2.174:8080/solr/replication?command=indexversion returns only 0 long name=indexversion0/long long name=generation0/long on master I could see the version of current index long name=indexversion1341328964983/long long name=generation3/long but slave's http://192.168.2.174:8080/solr/admin/stats.jsp I can the the right version indexVersion : 1341328964983 generation : 3 Also when I checked the solr log. [org.apache.solr.handler.SnapPuller] Master at: http://192.168.2.204:8080/solr/replication is not available. Index fetch failed. Exception: Connection refused BTW, I was able to fetch the replication file with wget directly. So my question is: What is wrong with my replication or Solr? About version, I use some legacy version of Solr: Solr Specification Version: 3.5.0.2011.11.22.14.54.38, because we have some legacy systems here. And another question what is the best way to migrate to the latest version. I mean to keep alive all the boosting infrastructure based on ExternalFileField options. Thank you in advance for your time and help you can provide, Sergii
Re: Approximately needed RAM for 5000 query/second at a Solr machine?
You're mixing up disk and RAM requirements when you talk about having twice the disk size. Solr does _NOT_ require twice the index size of RAM to optimize, it requires twice the size on _DISK_. In terms of RAM requirements, you need to create an index, run realistic queries at the installation and measure. Best Erick On Tue, Apr 9, 2013 at 10:32 PM, bigjust bigj...@lambdaphil.es wrote: On 4/9/2013 7:03 PM, Furkan KAMACI wrote: These are really good metrics for me: You say that RAM size should be at least index size, and it is better to have a RAM size twice the index size (because of worst case scenario). On the other hand let's assume that I have a RAM size that is bigger than twice of indexes at machine. Can Solr use that extra RAM or is it a approximately maximum limit (to have twice size of indexes at machine)? What we have been discussing is the OS cache, which is memory that is not used by programs. The OS uses that memory to make everything run faster. The OS will instantly give that memory up if a program requests it. Solr is a java program, and java uses memory a little differently, so Solr most likely will NOT use more memory when it is available. In a normal directly executable program, memory can be allocated at any time, and given back to the system at any time. With Java, you tell it the maximum amount of memory the program is ever allowed to use. Because of how memory is used inside Java, most long-running Java programs (like Solr) will allocate up to the configured maximum even if they don't really need that much memory. Most Java virtual machines will never give the memory back to the system even if it is not required. Thanks, Shawn Furkan KAMACI furkankam...@gmail.com writes: I am sorry but you said: *you need enough free RAM for the OS to cache the maximum amount of disk space all your indexes will ever use* I have made an assumption my indexes at my machine. Let's assume that it is 5 GB. So it is better to have at least 5 GB RAM? OK, Solr will use RAM up to how much I define it as a Java processes. When we think about the indexes at storage and caching them at RAM by OS, is that what you talk about: having more than 5 GB - or - 10 GB RAM for my machine? 2013/4/10 Shawn Heisey s...@elyograg.org 10 GB. Because when Solr shuffles the data around, it could use up to twice the size of the index in order to optimize the index on disk. -- Justin
Re: Solr 4.2.1 SSLInitializationException
Hi Uwe, Thanks for your response. As I mentioned in my email, I would prefer the application to not have access to the keystore. Do you know if there is a way of specifying a different HttpClient implementation (e.g. DefaultHttpClient rather than SystemDefaultHttpClient) ? From: Uwe Klosa uwe.kl...@gmail.com To: solr-user@lucene.apache.org; Sarita Nair sarita...@yahoo.com Sent: Wednesday, April 10, 2013 2:58 AM Subject: Re: Solr 4.2.1 SSLInitializationException You have to add two new Java options to your Glassfish config (example if you use the standard keystore and truststore): asadmin create-jvm-options -- -Djavax.net.ssl.keyStorePassword=changeit asadmin create-jvm-options -- -Djavax.net.ssl.trustStorePassword=changeit /Uwe On 10 April 2013 03:59, Sarita Nair sarita...@yahoo.com wrote: Hi Chris, Thanks for your response. My understanding is that GlassFish specifies the keystore as a system property, but does not specify the password in order to protect it from snooping. There's a keychain that requires a password to be passed from the DAS in order to unlock the key for the keystore. Is there some way to specify a different HttpClient implementation (e.g. DefaultHttpClient rather than SystemDefaultHttpClient), as we don't want the application to have access to the keystore? I have also pasted the entire stack trace below: 2013-04-09 10:45:06,144 [main] ERROR org.apache.solr.servlet.SolrDispatchFilter - Could not start Solr. Check solr/home property and the logs 2013-04-09 10:45:06,224 [main] ERROR org.apache.solr.core.SolrCore - null:org.apache.http.conn.ssl.SSLInitializationException: Failure initializing default system SSL context at org.apache.http.conn.ssl.SSLSocketFactory.createSystemSSLContext(SSLSocketFactory.java:368) at org.apache.http.conn.ssl.SSLSocketFactory.getSystemSocketFactory(SSLSocketFactory.java:204) at org.apache.http.impl.conn.SchemeRegistryFactory.createSystemDefault(SchemeRegistryFactory.java:82) at org.apache.http.impl.client.SystemDefaultHttpClient.createClientConnectionManager(SystemDefaultHttpClient.java:118) at org.apache.http.impl.client.AbstractHttpClient.getConnectionManager(AbstractHttpClient.java:466) at org.apache.solr.client.solrj.impl.HttpClientUtil.setMaxConnections(HttpClientUtil.java:179) at org.apache.solr.client.solrj.impl.HttpClientConfigurer.configure(HttpClientConfigurer.java:33) at org.apache.solr.client.solrj.impl.HttpClientUtil.configureClient(HttpClientUtil.java:115) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:105) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:134) at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.start(GlassFishImpl.java:79) at com.sun.enterprise.glassfish.bootstrap.GlassFishDecorator.start(GlassFishDecorator.java:63) at com.sun.enterprise.glassfish.bootstrap.osgi.OSGiGlassFishImpl.start(OSGiGlassFishImpl.java:69) at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher.launch(GlassFishMain.java:117) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.sun.enterprise.glassfish.bootstrap.GlassFishMain.main(GlassFishMain.java:97) at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:55) Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:772) at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:55) at java.security.KeyStore.load(KeyStore.java:1214) at org.apache.http.conn.ssl.SSLSocketFactory.createSystemSSLContext(SSLSocketFactory.java:281) at org.apache.http.conn.ssl.SSLSocketFactory.createSystemSSLContext(SSLSocketFactory.java:366) ... 50 more Caused by: java.security.UnrecoverableKeyException: Password verification failed at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:770) ... 54 more From: Chris Hostetter hossman_luc...@fucit.org To: solr-user@lucene.apache.org solr-user@lucene.apache.org; Sarita Nair sarita...@yahoo.com Sent: Tuesday, April 9, 2013 1:31 PM Subject: Re: Solr 4.2.1 SSLInitializationException : Deploying Solr 4.2.1 to GlassFish 3.1.1 results in the error below. I : have seen similar problems being reported with Solr 4.2 Are you trying to use server SSL with glassfish? can you please post the full stack trace so we can see where this error is coming from. My best guess is that this is coming from the changes made in SOLR-4451 to
Re: Solr 4.2.1 SSLInitializationException
Have you tried to create a HttpSolrServer with this constructor: * HttpSolrServerhttp://lucene.apache.org/solr/4_2_0/solr-solrj/org/apache/solr/client/solrj/impl/HttpSolrServer.html#HttpSolrServer(java.lang.String, org.apache.http.client.HttpClient)*(Stringhttp://download.oracle.com/javase/6/docs/api/java/lang/String.html?is-external=true baseURL, org.apache.http.client.HttpClient client) ? /Uwe On 10 April 2013 15:35, Sarita Nair sarita...@yahoo.com wrote: Hi Uwe, Thanks for your response. As I mentioned in my email, I would prefer the application to not have access to the keystore. Do you know if there is a way of specifying a different HttpClient implementation (e.g. DefaultHttpClient rather than SystemDefaultHttpClient) ? From: Uwe Klosa uwe.kl...@gmail.com To: solr-user@lucene.apache.org; Sarita Nair sarita...@yahoo.com Sent: Wednesday, April 10, 2013 2:58 AM Subject: Re: Solr 4.2.1 SSLInitializationException You have to add two new Java options to your Glassfish config (example if you use the standard keystore and truststore): asadmin create-jvm-options -- -Djavax.net.ssl.keyStorePassword=changeit asadmin create-jvm-options -- -Djavax.net.ssl.trustStorePassword=changeit /Uwe On 10 April 2013 03:59, Sarita Nair sarita...@yahoo.com wrote: Hi Chris, Thanks for your response. My understanding is that GlassFish specifies the keystore as a system property, but does not specify the password in order to protect it from snooping. There's a keychain that requires a password to be passed from the DAS in order to unlock the key for the keystore. Is there some way to specify a different HttpClient implementation (e.g. DefaultHttpClient rather than SystemDefaultHttpClient), as we don't want the application to have access to the keystore? I have also pasted the entire stack trace below: 2013-04-09 10:45:06,144 [main] ERROR org.apache.solr.servlet.SolrDispatchFilter - Could not start Solr. Check solr/home property and the logs 2013-04-09 10:45:06,224 [main] ERROR org.apache.solr.core.SolrCore - null:org.apache.http.conn.ssl.SSLInitializationException: Failure initializing default system SSL context at org.apache.http.conn.ssl.SSLSocketFactory.createSystemSSLContext(SSLSocketFactory.java:368) at org.apache.http.conn.ssl.SSLSocketFactory.getSystemSocketFactory(SSLSocketFactory.java:204) at org.apache.http.impl.conn.SchemeRegistryFactory.createSystemDefault(SchemeRegistryFactory.java:82) at org.apache.http.impl.client.SystemDefaultHttpClient.createClientConnectionManager(SystemDefaultHttpClient.java:118) at org.apache.http.impl.client.AbstractHttpClient.getConnectionManager(AbstractHttpClient.java:466) at org.apache.solr.client.solrj.impl.HttpClientUtil.setMaxConnections(HttpClientUtil.java:179) at org.apache.solr.client.solrj.impl.HttpClientConfigurer.configure(HttpClientConfigurer.java:33) at org.apache.solr.client.solrj.impl.HttpClientUtil.configureClient(HttpClientUtil.java:115) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:105) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:134) at com.sun.enterprise.glassfish.bootstrap.GlassFishImpl.start(GlassFishImpl.java:79) at com.sun.enterprise.glassfish.bootstrap.GlassFishDecorator.start(GlassFishDecorator.java:63) at com.sun.enterprise.glassfish.bootstrap.osgi.OSGiGlassFishImpl.start(OSGiGlassFishImpl.java:69) at com.sun.enterprise.glassfish.bootstrap.GlassFishMain$Launcher.launch(GlassFishMain.java:117) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.sun.enterprise.glassfish.bootstrap.GlassFishMain.main(GlassFishMain.java:97) at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:55) Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:772) at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:55) at java.security.KeyStore.load(KeyStore.java:1214) at org.apache.http.conn.ssl.SSLSocketFactory.createSystemSSLContext(SSLSocketFactory.java:281) at org.apache.http.conn.ssl.SSLSocketFactory.createSystemSSLContext(SSLSocketFactory.java:366) ... 50 more Caused by: java.security.UnrecoverableKeyException: Password verification failed at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:770) ... 54 more
eDisMax handler but with main query not allowing fielded/non-localparam values
Hello, I am building a Search Interface in front of Solr. I am using facets and other approaches to do fielded restrictions (via fq queries). I am also providing a free-form search field to the user. I would like that free-form field to search against eDisMax rules (multiple source fields, phrase boosting, mm param, etc). But I don't want (for security reasons) to let the user to enter Solr-specific features like localparams or fielded search. Is that possible? Am I missing something obvious? Regards, Alex. Personal blog: http://blog.outerthoughts.com/ LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch - Time is the quality of nature that keeps events from happening all at once. Lately, it doesn't seem to be working. (Anonymous - via GTD book)
Solr Sorting Not working properly
I'm using this sample query to group the result set by category:
q=testgroup=truegroup.field=category
This works as expected and I get this sample response:
response:
{numFound:1,start:0,docs:[
{
...
}
{numFound:6,start:0,docs:[
{
...
}
{numFound:3,start:0,docs:[
{
...
}
However, I can't find a way to specify the sort order of the groups by
number of docs each group has (numFound field). I think the sort param
has something to do with it, but I don't know how to use it.
Any help will be greatly appreciated!
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-Sorting-Not-working-properly-tp4055052.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Approximately needed RAM for 5000 query/second at a Solr machine?
Correct, except the worst case maximum for disk space is three times. --wunder On Apr 10, 2013, at 6:04 AM, Erick Erickson wrote: You're mixing up disk and RAM requirements when you talk about having twice the disk size. Solr does _NOT_ require twice the index size of RAM to optimize, it requires twice the size on _DISK_. In terms of RAM requirements, you need to create an index, run realistic queries at the installation and measure. Best Erick On Tue, Apr 9, 2013 at 10:32 PM, bigjust bigj...@lambdaphil.es wrote: On 4/9/2013 7:03 PM, Furkan KAMACI wrote: These are really good metrics for me: You say that RAM size should be at least index size, and it is better to have a RAM size twice the index size (because of worst case scenario). On the other hand let's assume that I have a RAM size that is bigger than twice of indexes at machine. Can Solr use that extra RAM or is it a approximately maximum limit (to have twice size of indexes at machine)? What we have been discussing is the OS cache, which is memory that is not used by programs. The OS uses that memory to make everything run faster. The OS will instantly give that memory up if a program requests it. Solr is a java program, and java uses memory a little differently, so Solr most likely will NOT use more memory when it is available. In a normal directly executable program, memory can be allocated at any time, and given back to the system at any time. With Java, you tell it the maximum amount of memory the program is ever allowed to use. Because of how memory is used inside Java, most long-running Java programs (like Solr) will allocate up to the configured maximum even if they don't really need that much memory. Most Java virtual machines will never give the memory back to the system even if it is not required. Thanks, Shawn Furkan KAMACI furkankam...@gmail.com writes: I am sorry but you said: *you need enough free RAM for the OS to cache the maximum amount of disk space all your indexes will ever use* I have made an assumption my indexes at my machine. Let's assume that it is 5 GB. So it is better to have at least 5 GB RAM? OK, Solr will use RAM up to how much I define it as a Java processes. When we think about the indexes at storage and caching them at RAM by OS, is that what you talk about: having more than 5 GB - or - 10 GB RAM for my machine? 2013/4/10 Shawn Heisey s...@elyograg.org 10 GB. Because when Solr shuffles the data around, it could use up to twice the size of the index in order to optimize the index on disk. -- Justin -- Walter Underwood wun...@wunderwood.org
Re: How to configure shards with SSL?
Ok, We figured it out: The cert wasn't in the trusted CA keystore. I know we put it in there earlier; I don't know why it was missing. But we added it in again and everything works as before. Thanks, -- View this message in context: http://lucene.472066.n3.nabble.com/How-to-configure-shards-with-SSL-tp4054735p4055064.html Sent from the Solr - User mailing list archive at Nabble.com.
Update 4.0 to 4.1 (4.2.1): No slice servicing hash code
Hello,
I tried updating our solrcloud from 4.0.0 to 4.1.0.
So I set up a cloud on my local machine with a standalone zookeeper
(3.4.5), 3 collections and 6 Solr servers (4.0.0).
I added some documents via SolrJ, and stopped the servers. After that I
restarted the nodes with the newer version (4.1.0).
After restarting the nodes everything looks fine - all nodes are active,
but when I started to add documents via SolrJ, the following exception
occured:
org.apache.solr.common.SolrException: No slice servicing hash code
8330c664 in DocCollection(anschriften)={shards:{shard1:{
replicas:{
hostname:8001_solr_anschriften:{
shard:shard1,
state:active,
core:anschriften,
collection:anschriften,
node_name:hostname:8001_solr,
base_url:http://hostname:8001/solr,
leader:true},
hostname:8002_solr_anschriften:{
shard:shard1,
state:active,
core:anschriften,
collection:anschriften,
node_name:hostname:8002_solr,
base_url:http://hostname:8002/solr}},
state:active}}}
at
org.apache.solr.common.cloud.HashBasedRouter.hashToSlice(HashBasedRouter.java:52)
at
org.apache.solr.common.cloud.HashBasedRouter.getTargetSlice(HashBasedRouter.java:34)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.setupRequest(DistributedUpdateProcessor.java:200)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:339)
at
org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
at
org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:246)
at
org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:173)
at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1817)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:560)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:365)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998)
at
org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:948)
at
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Unknown Source)
There is only one shard, and it should contain all documents. Do you have
any idea what's going wrong?
Thanks,
Norman Lenzner
migration solr 3.5 to 4.1 - JVM GC problems
Hi, I run multiple solr indexes in 1 single tomcat (1 webapp per index). All the indexes are solr 3.5 and I have upgraded few of them to solr 4.1 (about half of them). The JVM behavior is now radically different and doesn't seem to make sense. I was using ConcMarkSweepGC. I am now trying the G1 collector. The perm gen went from 410Mb to 600Mb. The eden space usage is a lot bigger and the survivor space usage is 100% all the time. I don't really understand what is happening. GC behavior really doesn't seem right. My jvm settings: -d64 -server -Xms40g -Xmx40g -XX:+UseG1GC -XX:NewRatio=1 -XX:SurvivorRatio=3 -XX:PermSize=728m -XX:MaxPermSize=728m I have tried NewRatio=1 and SurvivorRatio=3 hoping to get the Survivor space to not be 100% full all the time without success. Here is what jmap is giving me: Heap Configuration: MinHeapFreeRatio = 40 MaxHeapFreeRatio = 70 MaxHeapSize = 42949672960 (40960.0MB) NewSize = 1363144 (1.254223632812MB) MaxNewSize = 17592186044415 MB OldSize = 5452592 (5.169482421875MB) NewRatio = 1 SurvivorRatio= 3 PermSize = 754974720 (720.0MB) MaxPermSize = 763363328 (728.0MB) G1HeapRegionSize = 16777216 (16.0MB) Heap Usage: G1 Heap: regions = 2560 capacity = 42949672960 (40960.0MB) used = 23786449912 (22684.526359558105MB) free = 19163223048 (18275.473640441895MB) 55.382144432514906% used G1 Young Generation: Eden Space: regions = 674 capacity = 20619198464 (19664.0MB) used = 11307843584 (10784.0MB) free = 9311354880 (8880.0MB) 54.841334418226204% used Survivor Space: regions = 115 capacity = 1929379840 (1840.0MB) used = 1929379840 (1840.0MB) free = 0 (0.0MB) 100.0% used G1 Old Generation: regions = 732 capacity = 20401094656 (19456.0MB) used = 10549226488 (10060.526359558105MB) free = 9851868168 (9395.473640441895MB) 51.70911985792612% used Perm Generation: capacity = 754974720 (720.0MB) used = 514956504 (491.10079193115234MB) free = 240018216 (228.89920806884766MB) 68.20844332377116% used The Survivor space even went up to 3.6Gb but was still 100% used. I have disabled all caches. Obviously I am getting very bad GC performance. Any idea as to what could be wrong and why this could be happening? Thanks, Marc This transmission is strictly confidential, possibly legally privileged, and intended solely for the addressee. Any views or opinions expressed within it are those of the author and do not necessarily represent those of 192.com Ltd or any of its subsidiary companies. If you are not the intended recipient then you must not disclose, copy or take any action in reliance of this transmission. If you have received this transmission in error, please notify the sender as soon as possible. No employee or agent is authorised to conclude any binding agreement on behalf 192.com Ltd with another party by email without express written confirmation by an authorised employee of the company. http://www.192.com (Tel: 08000 192 192). 192.com Ltd is incorporated in England and Wales, company number 07180348, VAT No. GB 103226273.
Re: eDisMax handler but with main query not allowing fielded/non-localparam values
Either preprocess the query in your application layer, add a query preprocessor custom search component, or propose some additional options to Solr to disable certain features like local params nested queries, etc. Oh, with fielded search, maybe you can just set uf (user fields) to empty. I haven't checked if that restricts qf as well. -- Jack Krupansky -Original Message- From: Alexandre Rafalovitch Sent: Wednesday, April 10, 2013 10:14 AM To: solr-user@lucene.apache.org Subject: eDisMax handler but with main query not allowing fielded/non-localparam values Hello, I am building a Search Interface in front of Solr. I am using facets and other approaches to do fielded restrictions (via fq queries). I am also providing a free-form search field to the user. I would like that free-form field to search against eDisMax rules (multiple source fields, phrase boosting, mm param, etc). But I don't want (for security reasons) to let the user to enter Solr-specific features like localparams or fielded search. Is that possible? Am I missing something obvious? Regards, Alex. Personal blog: http://blog.outerthoughts.com/ LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch - Time is the quality of nature that keeps events from happening all at once. Lately, it doesn't seem to be working. (Anonymous - via GTD book)
Re: eDisMax handler but with main query not allowing fielded/non-localparam values
On Wed, Apr 10, 2013 at 11:59 AM, Jack Krupansky j...@basetechnology.comwrote: Oh, with fielded search, maybe you can just set uf (user fields) to empty. I haven't checked if that restricts qf as well. I just tested and UF seems to affect FQ (Filter Query). So, that would have been a cool hack but unfortunately it is not. Other options are possible, but more mid-term solution. I will think of those if there is no other way. Thank you, Alex. Personal blog: http://blog.outerthoughts.com/ LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch - Time is the quality of nature that keeps events from happening all at once. Lately, it doesn't seem to be working. (Anonymous - via GTD book)
Score field statistics
In our application we are using Solr 4.1. And we wanna filter results by score relevance. I had the idea to use statistic data (i.e. standard deviation, mean) for score field. Is it exists workaround of using …stats=truestats.field=score... ? Thanks in advance -- View this message in context: http://lucene.472066.n3.nabble.com/Score-field-statistics-tp4055089.html Sent from the Solr - User mailing list archive at Nabble.com.
Solr admin url hit is not taking to the correct page
Hi, When we try to access Solr(3.6) admin page some times it is not taking us to the right page instead it is showing the below message. Directory: /solr/admin/ Parent Directory Replication 4096 bytes Mar 25 2013 9:34:06 AM When we click on parent Directory it displays the below one and clicking on META-INF, WEB-INF says not found. META-INF/ WEB-INF/ admin/ How to resolve this? Could any one please suggest?. Thanks. -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-admin-url-hit-is-not-taking-to-the-correct-page-tp4055091.html Sent from the Solr - User mailing list archive at Nabble.com.
Cloud and Master slave replcation
Is it possible to have a Solr cloud in a master/slave configuration with another solr server where the cloud is the slave? -- To *know* is one thing, and to know for certain *that* we know is another. --William James
Re: migration solr 3.5 to 4.1 - JVM GC problems
Hi Marc, Why such a big heap? Do you really need it? You disabled all caches, so the JVM really shouldn't need much memory. Have you tried with -Xmx20g or even -Xmx8g? Aha, survivor is getting to 100% so you kept increasing -Xmx? Have you tried just not using any of these: -XX:+UseG1GC -XX:NewRatio=1 -XX:SurvivorRatio=3 -XX:PermSize=728m -XX:MaxPermSize=728m ? My hunch is that there is a leak somewhere, because without caches you shouldn't eed 40GB heap. Otis -- SOLR Performance Monitoring - http://sematext.com/spm/index.html Solr ElasticSearch Support http://sematext.com/ On Wed, Apr 10, 2013 at 11:48 AM, Marc Des Garets marc.desgar...@192.com wrote: Hi, I run multiple solr indexes in 1 single tomcat (1 webapp per index). All the indexes are solr 3.5 and I have upgraded few of them to solr 4.1 (about half of them). The JVM behavior is now radically different and doesn't seem to make sense. I was using ConcMarkSweepGC. I am now trying the G1 collector. The perm gen went from 410Mb to 600Mb. The eden space usage is a lot bigger and the survivor space usage is 100% all the time. I don't really understand what is happening. GC behavior really doesn't seem right. My jvm settings: -d64 -server -Xms40g -Xmx40g -XX:+UseG1GC -XX:NewRatio=1 -XX:SurvivorRatio=3 -XX:PermSize=728m -XX:MaxPermSize=728m I have tried NewRatio=1 and SurvivorRatio=3 hoping to get the Survivor space to not be 100% full all the time without success. Here is what jmap is giving me: Heap Configuration: MinHeapFreeRatio = 40 MaxHeapFreeRatio = 70 MaxHeapSize = 42949672960 (40960.0MB) NewSize = 1363144 (1.254223632812MB) MaxNewSize = 17592186044415 MB OldSize = 5452592 (5.169482421875MB) NewRatio = 1 SurvivorRatio= 3 PermSize = 754974720 (720.0MB) MaxPermSize = 763363328 (728.0MB) G1HeapRegionSize = 16777216 (16.0MB) Heap Usage: G1 Heap: regions = 2560 capacity = 42949672960 (40960.0MB) used = 23786449912 (22684.526359558105MB) free = 19163223048 (18275.473640441895MB) 55.382144432514906% used G1 Young Generation: Eden Space: regions = 674 capacity = 20619198464 (19664.0MB) used = 11307843584 (10784.0MB) free = 9311354880 (8880.0MB) 54.841334418226204% used Survivor Space: regions = 115 capacity = 1929379840 (1840.0MB) used = 1929379840 (1840.0MB) free = 0 (0.0MB) 100.0% used G1 Old Generation: regions = 732 capacity = 20401094656 (19456.0MB) used = 10549226488 (10060.526359558105MB) free = 9851868168 (9395.473640441895MB) 51.70911985792612% used Perm Generation: capacity = 754974720 (720.0MB) used = 514956504 (491.10079193115234MB) free = 240018216 (228.89920806884766MB) 68.20844332377116% used The Survivor space even went up to 3.6Gb but was still 100% used. I have disabled all caches. Obviously I am getting very bad GC performance. Any idea as to what could be wrong and why this could be happening? Thanks, Marc This transmission is strictly confidential, possibly legally privileged, and intended solely for the addressee. Any views or opinions expressed within it are those of the author and do not necessarily represent those of 192.com Ltd or any of its subsidiary companies. If you are not the intended recipient then you must not disclose, copy or take any action in reliance of this transmission. If you have received this transmission in error, please notify the sender as soon as possible. No employee or agent is authorised to conclude any binding agreement on behalf 192.com Ltd with another party by email without express written confirmation by an authorised employee of the company. http://www.192.com (Tel: 08000 192 192). 192.com Ltd is incorporated in England and Wales, company number 07180348, VAT No. GB 103226273.
Re: Empty Solr 4.2.1 can not create Collection
Can you post what your clusterstate.json? After you spin up the initial core, it will automatically become leader for that shard. On Wed, Apr 10, 2013 at 3:43 AM, A.Eibner a_eib...@yahoo.de wrote: Hi Joel, I followed your steps, the cores and collection get created, but there is no leader elected so I can not query the collection... Do I miss something ? Kind Regards Alexander Am 2013-04-09 10:21, schrieb A.Eibner: Hi, thanks for your faster answer. You don't use the Collection API - may I ask you why ? Therefore you have to setup everything (replicas, ...) manually..., which I would like to avoid. Also what I don't understand, why my steps work in 4.0 but won't in 4.2.1... Any clues ? Kind Regards Alexander Am 2013-04-08 19:12, schrieb Joel Bernstein: The steps that I use to setup the collection are slightly different: 1) Start zk and upconfig the config set. Your approach is same. 2) Start appservers with Solr zkHost set to the zk started in step 1. 3) Use a core admin command to spin up a new core and collection. http://app01/solr/admin/cores?**action=CREATEname=storage-** corecollection=storage**numShards=1collection.** configName=storage-confhttp://app01/solr/admin/cores?action=CREATEname=storage-corecollection=storagenumShards=1collection.configName=storage-conf http:/**/app03/solr/admin/collections?**action=CREATEname=storage** numShards=1replicationFactor=**2collection.configName=**storage-confhttp://app03/solr/admin/collections?action=CREATEname=storagenumShards=1replicationFactor=2collection.configName=storage-conf shard=shard1 This will spin up the new collection and initial core. I'm not using a replication factor because the following commands manually bind the replicas. 4) Spin up replica with a core admin command: http://app02/solr/admin/cores?**action=CREATEname=storage-** corecollection=storagehttp://app02/solr/admin/cores?action=CREATEname=storage-corecollection=storage; http:**//app03/solr/admin/**collections?action=CREATE** name=storagenumShards=1**replicationFactor=2** collection.configName=storage-**confhttp://app03/solr/admin/collections?action=CREATEname=storagenumShards=1replicationFactor=2collection.configName=storage-conf shard=shard1 5) Same command as above on the 3rd server to spin up another replica. This will spin up a new core and bind it to shard1 of the storage collection. On Mon, Apr 8, 2013 at 9:34 AM, A.Eibner a_eib...@yahoo.de wrote: Hi, I have a problem with setting up my solr cloud environment (on three machines). If I want to create my collections from scratch I do the following: *) Start ZooKeeper on all machines. *) Upload the configuration (on app02) for the collection via the following command: zkcli.sh -cmd upconfig --zkhost app01:4181,app02:4181,app03: 4181 --confdir config/solr/storage/conf/ --confname storage-conf *) Linking the configuration (on app02) via the following command: zkcli.sh -cmd linkconfig --collection storage --confname storage-conf --zkhost app01:4181,app02:4181,app03:4181 *) Start Tomcats (containing Solr) on app02,app03 *) Create Collection via: http://app03/solr/admin/collections?action=CREATE**http://app03/solr/admin/**collections?action=CREATE** name=storagenumShards=1replicationFactor=2** collection.configName=storage-confhttp://app03/solr/** admin/collections?action=**CREATEname=storagenumShards=** 1replicationFactor=2**collection.configName=storage-**confhttp://app03/solr/admin/collections?action=CREATEname=storagenumShards=1replicationFactor=2collection.configName=storage-conf This creates the replication of the shard on app02 and app03, but neither of them is marked as leader, both are marked as DOWN. And after wards I can not access the collection. In the browser I get: SEVERE: org.apache.solr.common.SolrException: no servers hosting shard: In the log files the following error is present: SEVERE: Error from shard: app02:9985/solr org.apache.solr.common.SolrException: Error CREATEing SolrCore 'storage_shard1_replica1': at org.apache.solr.client.solrj.impl.HttpSolrServer.request( HttpSolrServer.java:404) at org.apache.solr.client.solrj.impl.HttpSolrServer.request( HttpSolrServer.java:181) at org.apache.solr.handler.component.HttpShardHandler$1.** ** call(HttpShardHandler.java:172) at org.apache.solr.handler.component.HttpShardHandler$1.** ** call(HttpShardHandler.java:135) at java.util.concurrent.FutureTask$Sync.innerRun(** FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask. java:166) at java.util.concurrent.Executors$RunnableAdapter.** call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(** FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask. java:166)
Re: migration solr 3.5 to 4.1 - JVM GC problems
On 4/10/2013 9:48 AM, Marc Des Garets wrote: The JVM behavior is now radically different and doesn't seem to make sense. I was using ConcMarkSweepGC. I am now trying the G1 collector. The perm gen went from 410Mb to 600Mb. The eden space usage is a lot bigger and the survivor space usage is 100% all the time. I don't really understand what is happening. GC behavior really doesn't seem right. My jvm settings: -d64 -server -Xms40g -Xmx40g -XX:+UseG1GC -XX:NewRatio=1 -XX:SurvivorRatio=3 -XX:PermSize=728m -XX:MaxPermSize=728m As Otis has already asked, why do you have a 40GB heap? The only way I can imagine that you would actually NEED a heap that big is if your index size is measured in hundreds of gigabytes. If you really do need a heap that big, you will probably need to go with a JVM like Zing. I don't know how much Zing costs, but they claim to be able to make any heap size perform well under any load. It is Linux-only. I was running into extreme problems with GC pauses with my own setup, and that was only with an 8GB heap. I was using the CMS collector and NewRatio=1. Switching to G1 didn't help at all - it might have even made the problem worse. I never did try the Zing JVM. After a lot of experimentation (which I will admit was not done very methodically) I found JVM options that have reduced the GC pause problem greatly. Below is what I am using now on Solr 4.2.1 with a total per-server index size of about 45GB. This works properly on CentOS 6 with Oracle Java 7u17, UseLargePages may require special kernel tuning on other operating systems: -Xmx6144M -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:NewRatio=3 -XX:MaxTenuringThreshold=8 -XX:+CMSParallelRemarkEnabled -XX:+ParallelRefProcEnabled -XX:+UseLargePages -XX:+AggressiveOpts These options could probably use further tuning, but I haven't had time for the kind of testing that will be required. If you decide to pay someone to make the problem going away instead: http://www.azulsystems.com/products/zing/whatisit Thanks, Shawn
/admin/stats.jsp in SolrCloud
Hey guys, This feels like a silly question already, here goes: In SolrCloud it doesn't seem obvious to me where one can grab stats regarding caches for a given core using an http call (JSON/XML). Those values are available in the web-based app, but I am looking for a http call that would return this same data. In 3.x this was located at /admin/stats.php, and I used a script to grab the data, but in SolrCloud I am unclear and would like to add that to the docs below: http://wiki.apache.org/solr/SolrCaching#Overview http://wiki.apache.org/solr/SolrAdminStats Thanks! Tim
Re: /admin/stats.jsp in SolrCloud
Hey Tim SolrCloud-Mode or not does not really matter for this fact .. in 4.x (and afaik as well in 3.x) you can find the stats here: http://host:port/solr/admin/mbeans?stats=true in xml or json (setting the responsewriter with wt=json) - as you like HTH Stefan On Wednesday, April 10, 2013 at 9:53 PM, Tim Vaillancourt wrote: Hey guys, This feels like a silly question already, here goes: In SolrCloud it doesn't seem obvious to me where one can grab stats regarding caches for a given core using an http call (JSON/XML). Those values are available in the web-based app, but I am looking for a http call that would return this same data. In 3.x this was located at /admin/stats.php, and I used a script to grab the data, but in SolrCloud I am unclear and would like to add that to the docs below: http://wiki.apache.org/solr/SolrCaching#Overview http://wiki.apache.org/solr/SolrAdminStats Thanks! Tim
Re: /admin/stats.jsp in SolrCloud
It's under /admin/mbeans. Alan Woodward www.flax.co.uk On 10 Apr 2013, at 20:53, Tim Vaillancourt wrote: Hey guys, This feels like a silly question already, here goes: In SolrCloud it doesn't seem obvious to me where one can grab stats regarding caches for a given core using an http call (JSON/XML). Those values are available in the web-based app, but I am looking for a http call that would return this same data. In 3.x this was located at /admin/stats.php, and I used a script to grab the data, but in SolrCloud I am unclear and would like to add that to the docs below: http://wiki.apache.org/solr/SolrCaching#Overview http://wiki.apache.org/solr/SolrAdminStats Thanks! Tim
Re: /admin/stats.jsp in SolrCloud
There we go, Thanks Stefan! You're right, 3.x has this as well, I guess I missed it. I'll add this to the docs for SolrCaching. Cheers! Tim On 10 April 2013 13:19, Stefan Matheis matheis.ste...@gmail.com wrote: Hey Tim SolrCloud-Mode or not does not really matter for this fact .. in 4.x (and afaik as well in 3.x) you can find the stats here: http://host:port/solr/admin/mbeans?stats=true in xml or json (setting the responsewriter with wt=json) - as you like HTH Stefan On Wednesday, April 10, 2013 at 9:53 PM, Tim Vaillancourt wrote: Hey guys, This feels like a silly question already, here goes: In SolrCloud it doesn't seem obvious to me where one can grab stats regarding caches for a given core using an http call (JSON/XML). Those values are available in the web-based app, but I am looking for a http call that would return this same data. In 3.x this was located at /admin/stats.php, and I used a script to grab the data, but in SolrCloud I am unclear and would like to add that to the docs below: http://wiki.apache.org/solr/SolrCaching#Overview http://wiki.apache.org/solr/SolrAdminStats Thanks! Tim
Re: /admin/stats.jsp in SolrCloud
To complete my as well in 3.x phrase - what i wanted to say is: it was already there in the times of 3.x - but because there was stats.jsp .. you know :) On Wednesday, April 10, 2013 at 10:19 PM, Stefan Matheis wrote: Hey Tim SolrCloud-Mode or not does not really matter for this fact .. in 4.x (and afaik as well in 3.x) you can find the stats here: http://host:port/solr/admin/mbeans?stats=true in xml or json (setting the responsewriter with wt=json) - as you like HTH Stefan On Wednesday, April 10, 2013 at 9:53 PM, Tim Vaillancourt wrote: Hey guys, This feels like a silly question already, here goes: In SolrCloud it doesn't seem obvious to me where one can grab stats regarding caches for a given core using an http call (JSON/XML). Those values are available in the web-based app, but I am looking for a http call that would return this same data. In 3.x this was located at /admin/stats.php, and I used a script to grab the data, but in SolrCloud I am unclear and would like to add that to the docs below: http://wiki.apache.org/solr/SolrCaching#Overview http://wiki.apache.org/solr/SolrAdminStats Thanks! Tim
Re: Empty Solr 4.2.1 can not create Collection
Hi,
here the clusterstate.json (from zookeeper) after creating the core:
{storage:{
shards:{shard1:{
range:8000-7fff,
state:active,
replicas:{app02:9985_solr_storage-core:{
shard:shard1,
state:down,
core:storage-core,
collection:storage,
node_name:app02:9985_solr,
base_url:http://app02:9985/solr,
router:compositeId}}
cZxid = 0x10024
ctime = Wed Apr 10 22:18:13 CEST 2013
mZxid = 0x1003d
mtime = Wed Apr 10 22:21:26 CEST 2013
pZxid = 0x10024
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 467
numChildren = 0
But looking in the log files I found the following error (this also
occures with the collection api)
SEVERE: org.apache.solr.common.SolrException: Error CREATEing SolrCore
'storage_shard1_replica1':
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:483)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:140)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:591)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:192)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:999)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.solr.common.cloud.ZooKeeperException:
at
org.apache.solr.core.CoreContainer.registerInZk(CoreContainer.java:931)
at
org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:892)
at
org.apache.solr.core.CoreContainer.register(CoreContainer.java:841)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleCreateAction(CoreAdminHandler.java:479)
... 19 more
Caused by: java.lang.NullPointerException
at
org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:190)
at
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
at
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
at
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:266)
at
org.apache.solr.cloud.ZkController.joinElection(ZkController.java:935)
at
org.apache.solr.cloud.ZkController.register(ZkController.java:761)
at
org.apache.solr.cloud.ZkController.register(ZkController.java:727)
at
org.apache.solr.core.CoreContainer.registerInZk(CoreContainer.java:908)
... 22 more
Kind regards
Alexander
Am 2013-04-10 19:12, schrieb Joel Bernstein:
Can you post what your clusterstate.json?
After you spin up the initial core, it will automatically become leader for
that shard.
On Wed, Apr 10, 2013 at 3:43 AM, A.Eibner a_eib...@yahoo.de wrote:
Hi Joel,
I followed your steps, the cores and collection get created, but there is
no leader elected so I can not query the collection...
Do I miss something ?
Kind Regards
Alexander
Am 2013-04-09 10:21, schrieb A.Eibner:
Hi,
thanks for your faster answer.
You don't use the Collection API - may I ask you why ?
Therefore you have to setup everything (replicas, ...) manually...,
which I would like to avoid.
Also what I don't understand, why my steps work in 4.0 but won't in
4.2.1...
Any clues ?
Kind Regards
Alexander
Am 2013-04-08 19:12, schrieb Joel Bernstein:
The steps that I use to setup the collection are slightly different:
1) Start zk and upconfig the config set. Your approach is same.
2) Start appservers with Solr zkHost set to
NPE when faceting TEXTfield in a distributed search query
Hi, I have a nullable TextField with field type as follows(field name=fun_group):- fieldType name=lowercase_sort_missing_first class=solr.TextField sortMissingFirst=true positionIncrementGap=100 analyzertokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory//analyzer /fieldType field definition is:- field name=fun_group type=lowercase_sort_missing_first indexed=true stored=true multiValued=false/ I am running a distributed search(using shards= param) across 13 shards while faceting on fun_group with the following params:- http://shard1/solr/mycore/select?facet=truefacet.field=fun_groupfacet.mincount=1rows=0facet.limit=1facet.offset=0q=(((my_id:4024 AND ((flag:1 OR flag:0 AND class_id:2604)shards=shard1/solr/mycore,shard2/solr/mycore,shard3/solr/mycore,shard4/solr/mycore,shard5/solr/mycore,shard6/solr/mycore,shard7/solr/mycore,shard8/solr/mycore,shard9/solr/mycore,shard10/solr/mycore,shard11/solr/mycore,shard12/solr/mycore,shard13/solr/mycore I get Internal Server Error on shard6. In the logs, I see the NPE from org.apache.solr.servlet.SolrDispatchFilter. If I change the facet.limit from 1 to 5000 keeping the facet.offset=0,there are no exceptions. Now again if try facet.limit=5000 and facet.offset=5000 , I get the same exception. I tried facet.limit=4000 and facet.offset=5000, it worked fine. Please help -- View this message in context: http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: NPE when faceting TEXTfield in a distributed search query
Are you sure you want to facet on a text field??? That will facet on the individual terms, which isn't usually very useful. Usually, people want to facet on full phrases or entire strings, so they do a schema copyField from the text field to a string field and then facet on the string field. -- Jack Krupansky -Original Message- From: coolpriya5 Sent: Wednesday, April 10, 2013 6:19 PM To: solr-user@lucene.apache.org Subject: NPE when faceting TEXTfield in a distributed search query Hi, I have a nullable TextField with field type as follows(field name=fun_group):- fieldType name=lowercase_sort_missing_first class=solr.TextField sortMissingFirst=true positionIncrementGap=100 analyzertokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory//analyzer /fieldType field definition is:- field name=fun_group type=lowercase_sort_missing_first indexed=true stored=true multiValued=false/ I am running a distributed search(using shards= param) across 13 shards while faceting on fun_group with the following params:- http://shard1/solr/mycore/select?facet=truefacet.field=fun_groupfacet.mincount=1rows=0facet.limit=1facet.offset=0q=(((my_id:4024 AND ((flag:1 OR flag:0 AND class_id:2604)shards=shard1/solr/mycore,shard2/solr/mycore,shard3/solr/mycore,shard4/solr/mycore,shard5/solr/mycore,shard6/solr/mycore,shard7/solr/mycore,shard8/solr/mycore,shard9/solr/mycore,shard10/solr/mycore,shard11/solr/mycore,shard12/solr/mycore,shard13/solr/mycore I get Internal Server Error on shard6. In the logs, I see the NPE from org.apache.solr.servlet.SolrDispatchFilter. If I change the facet.limit from 1 to 5000 keeping the facet.offset=0,there are no exceptions. Now again if try facet.limit=5000 and facet.offset=5000 , I get the same exception. I tried facet.limit=4000 and facet.offset=5000, it worked fine. Please help -- View this message in context: http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: NPE when faceting TEXTfield in a distributed search query
Ah... I see now that you are using the keyword tokenizer that should preserve the phrase structure of the text. You haven't detailed the exception stack trace. What are the numbers in terms of number of values and average length of each value? -- Jack Krupansky -Original Message- From: Jack Krupansky Sent: Wednesday, April 10, 2013 6:30 PM To: solr-user@lucene.apache.org Subject: Re: NPE when faceting TEXTfield in a distributed search query Are you sure you want to facet on a text field??? That will facet on the individual terms, which isn't usually very useful. Usually, people want to facet on full phrases or entire strings, so they do a schema copyField from the text field to a string field and then facet on the string field. -- Jack Krupansky -Original Message- From: coolpriya5 Sent: Wednesday, April 10, 2013 6:19 PM To: solr-user@lucene.apache.org Subject: NPE when faceting TEXTfield in a distributed search query Hi, I have a nullable TextField with field type as follows(field name=fun_group):- fieldType name=lowercase_sort_missing_first class=solr.TextField sortMissingFirst=true positionIncrementGap=100 analyzertokenizer class=solr.KeywordTokenizerFactory/ filter class=solr.LowerCaseFilterFactory//analyzer /fieldType field definition is:- field name=fun_group type=lowercase_sort_missing_first indexed=true stored=true multiValued=false/ I am running a distributed search(using shards= param) across 13 shards while faceting on fun_group with the following params:- http://shard1/solr/mycore/select?facet=truefacet.field=fun_groupfacet.mincount=1rows=0facet.limit=1facet.offset=0q=(((my_id:4024 AND ((flag:1 OR flag:0 AND class_id:2604)shards=shard1/solr/mycore,shard2/solr/mycore,shard3/solr/mycore,shard4/solr/mycore,shard5/solr/mycore,shard6/solr/mycore,shard7/solr/mycore,shard8/solr/mycore,shard9/solr/mycore,shard10/solr/mycore,shard11/solr/mycore,shard12/solr/mycore,shard13/solr/mycore I get Internal Server Error on shard6. In the logs, I see the NPE from org.apache.solr.servlet.SolrDispatchFilter. If I change the facet.limit from 1 to 5000 keeping the facet.offset=0,there are no exceptions. Now again if try facet.limit=5000 and facet.offset=5000 , I get the same exception. I tried facet.limit=4000 and facet.offset=5000, it worked fine. Please help -- View this message in context: http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: NPE when faceting TEXTfield in a distributed search query
Number of values fun_group on shard 6 = 48000 Max length of fun_group is 20 chars If I run the facet on just shard6 it doesn't error out no matter the facet.limit. Also this query returns results only from shard 6 since the my_id:4024 belongs to shard 6. -- View this message in context: http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172p4055185.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: NPE when faceting TEXTfield in a distributed search query
If the NPE was in SolrDispatchFilter, it could relate to some limit on the HTTP request or response size. Again, we need the full stack trace, and the Solr release. -- Jack Krupansky -Original Message- From: coolpriya5 Sent: Wednesday, April 10, 2013 7:06 PM To: solr-user@lucene.apache.org Subject: Re: NPE when faceting TEXTfield in a distributed search query Number of values fun_group on shard 6 = 48000 Max length of fun_group is 20 chars If I run the facet on just shard6 it doesn't error out no matter the facet.limit. Also this query returns results only from shard 6 since the my_id:4024 belongs to shard 6. -- View this message in context: http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172p4055185.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: NPE when faceting TEXTfield in a distributed search query
Solr Version is 3.4. As for stacktrace, I tried setting logger level to
FINEST on the solr admin logging page and it still doesn't print the
stacktrace. All I get are one liners:-
2013-04-10 17:09:59,889 [http--18] ERROR [Marker: ]
org.apache.solr.core.SolrCore :
java.lang.NullPointerException
2013-04-10 17:09:59,890 [http--18] INFO [Marker: ]
org.apache.solr.core.SolrCore : [catalog] webapp=/solr
path=/select params={} status=500 QTime=1
2013-04-10 17:09:59,890 [http-xxx-18] ERROR [Marker: ]
org.apache.solr.servlet.SolrDispatchFilter :
java.lang.NullPointerException
--
View this message in context:
http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172p4055189.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: spell suggestions help
Try URL encoding it and/or escaping the On Tue, Apr 9, 2013 at 2:32 AM, Rohan Thakur rohan.i...@gmail.com wrote: hi all one thing I wanted to clear is for every other query I have got correct suggestions but these 2 cases I am not getting what suppose to be the suggestions: 1) I have kettle(doc frequency =5) and cable(doc frequecy=1) word indexed in direct solr spell cheker..but when I query for cattle I get cable as only suggestion and not kettle why is this happening i want to get kettle in suggestion as well im using jarowinkler distance according to which score for cattle = cable which is coming out to be 0.857 and for cattle = kettle which is coming out to be 0.777 kettle should also come in suggestions but its not how can I correct this any one. 2) how to query for sentence like hand blandar chopper as is delimiter for solr query and thus this query is returning error. thanks in advance regards Rohan
Re: Doc Transformer with SolrDocumentList object
I think a lot of your e-mail failed to make it through various filters, can you try sending in a simpler format? Best Erick On Tue, Apr 9, 2013 at 8:19 AM, neha yadav nehayadav...@gmail.com wrote: I am trying to modify the results of solr output . basically I need to change the ranking of the output of solr for a query. So please can anyone help. I wrote a java code that returns the SolrDocumentList object which is a union of the results I want this object to be displayed on solr. hats is once the query is hit. The solr runs the java code i wrote and the output returned in the java code gets as a output to the screen . I have tried to use the code as a data transformer. But I am getting this error: org.apache.solr.handler.dataimport.SolrWriter upload WARNING: Error creating document : SolrInputDocument[id=44, category=Apparel Fash Accessories, _version_=1431753044032225280, price=ERROR:SCHEMA-INDEX-MISMATC H,stringValue=1400, description=for girls, brand=Wrangler, price_c=1400,USD, siz e=ERROR:SCHEMA-INDEX-MISMATCH,stringValue=12] org.apache.solr.common.SolrException: version conflict for 44 expected=143175304 4032225280 actual=-1 Please can anyone help ?
RE: NPE when faceting TEXTfield in a distributed search query
Large facet.limit values cause a very large amount of form data to be sent to
the shards, though I'm not sure why this would cause a NullPointerException.
Perhaps the web server you are using is truncating the data instead of
returning a form too large error, which is somehow causing an NPE. Are you
using Jetty, Tomcat, or something else?
-Michael
-Original Message-
From: coolpriya5 [mailto:coolpri...@gmail.com]
Sent: Wednesday, April 10, 2013 7:17 PM
To: solr-user@lucene.apache.org
Subject: Re: NPE when faceting TEXTfield in a distributed search query
Solr Version is 3.4. As for stacktrace, I tried setting logger level to FINEST
on the solr admin logging page and it still doesn't print the stacktrace. All I
get are one liners:-
2013-04-10 17:09:59,889 [http--18] ERROR [Marker: ]
org.apache.solr.core.SolrCore :
java.lang.NullPointerException
2013-04-10 17:09:59,890 [http--18] INFO [Marker: ]
org.apache.solr.core.SolrCore : [catalog] webapp=/solr
path=/select params={} status=500 QTime=1
2013-04-10 17:09:59,890 [http-xxx-18] ERROR [Marker: ]
org.apache.solr.servlet.SolrDispatchFilter :
java.lang.NullPointerException
--
View this message in context:
http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172p4055189.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: NPE when faceting TEXTfield in a distributed search query
I'm using tomcat. Also in such a case, why wouldn't the same error occur when I run the same query on shard 6 alone? Is this a limitation of distributed search? Shard 6 is the only shard that has data for this query. -- View this message in context: http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172p4055208.html Sent from the Solr - User mailing list archive at Nabble.com.
RE: NPE when faceting TEXTfield in a distributed search query
Yes, this is a distributed search thing. In a distributed search, it will first make a somewhat normal facet request to all of the shards, get back the facet values, then make a second request in order to get the full counts of the facet values - this second request contains a list of facet terms that were not returned in the first response but were found on the other shards. Though, if only one shard even has this facet data, I would kind of expect you to not see this... If Tomcat is truncating the request, the NPE could be due to https://issues.apache.org/jira/browse/SOLR-435. Are you able to see the query in the shard 6 log for the working facet.limit=5000facet.offset=0 query? -Michael -Original Message- From: coolpriya5 [mailto:coolpri...@gmail.com] Sent: Wednesday, April 10, 2013 9:16 PM To: solr-user@lucene.apache.org Subject: RE: NPE when faceting TEXTfield in a distributed search query I'm using tomcat. Also in such a case, why wouldn't the same error occur when I run the same query on shard 6 alone? Is this a limitation of distributed search? Shard 6 is the only shard that has data for this query. -- View this message in context: http://lucene.472066.n3.nabble.com/NPE-when-faceting-TEXTfield-in-a-distributed-search-query-tp4055172p4055208.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Index Replication Failing in Solr 4.2.1
Root caused the Issue to a Code Bug / Contract Violation in SnapPuller in
solr 4.2.1 (impacts trunk as well) and Fixed by Patching the SnapPuller
locally.
fetchfilelist API expects indexversion to be specified as param.
So Call to Master should of be Form :
/solr/phcare/replication?command=filelistgen=108213wt=jsonindexversion=1323961125908
Instead Slave Calls the Master as :
/solr/phcare/replication?command=filelistgen=108213wt=json
Code bug lies in SnapPuller.fetchFileList(long gen) which gets called by
SnapPuller.fetchLatestIndex(final SolrCore core, boolean forceReplication)
The fix is pass along the version to fetchFileList and populate it.
A Patch is attached for trunk.
Thanks Regards
Umesh Prasad
Search Engineer @ Flipkart : India's Online Megastore
-
Empowering Consumers Find Products ..
On Tue, Apr 9, 2013 at 9:28 PM, Umesh Prasad umesh.i...@gmail.com wrote:
Hi All,
I am migrating from Solr 3.5.0 to Solr 4.2.1. And everything is running
fine and set to go, except the master slave replication.
We use master slave replication with multi cores ( 1 master, 10 slaves and
20 plus cores).
My Configuration is :
Master : Solr 3.5.0, Has existing index, and delta import running using
DIH.
Slave : Solr 4.2.1 , Has no startup index
Apr 9, 2013 9:18:40 PM org.apache.solr.core.SolrCore execute
INFO: [phcare] webapp= path=/replication
params={command=fetchindex_=1365522520521wt=json} status=0 QTime=1
Apr 9, 2013 9:18:40 PM org.apache.solr.handler.SnapPuller fetchLatestIndex
*INFO: Master's generation: 107876
*Apr 9, 2013 9:18:40 PM org.apache.solr.handler.SnapPuller
fetchLatestIndex
*INFO: Slave's generation: 79248
*Apr 9, 2013 9:18:40 PM org.apache.solr.handler.SnapPuller
fetchLatestIndex
INFO: Starting replication process
*Apr 9, 2013 9:18:40 PM org.apache.solr.handler.SnapPuller fetchFileList
SEVERE: No files to download for index generation: 107876
*Apr 9, 2013 9:18:40 PM org.apache.solr.core.SolrCore execute
INFO: [phcare] webapp= path=/replication
params={command=details_=1365522520556wt=json} status=0 QTime=7
In Both Master and Slave The File list for replicable version is correct.
*on Slave *
{
- masterDetails: {
- indexSize: 4.31 MB,
- indexPath:
/var/lib/fk-w3-sherlock/cores/phcare/data/index.20130124235012,
- commits: [
- [
- indexVersion,
- 1323961124638,
- generation,
- 107856,
- filelist,
- [
- _45e1.tii,
- _45e1.nrm,
-
..
*ON Master
*
[
- indexVersion,
- 1323961124638,
- generation,
- 107856,
- filelist,
- [
- _45e1.tii,
- _45e1.nrm,
- _45e2_1.del,
- _45e2.frq,
- _45e1_3.del,
- _45e1.tis,
- ..
Can someone help. Our whole Migration to Solr 4.2 is blocked on
Replication issue.
---
Thanks Regards
Umesh Prasad
--
---
Thanks Regards
Umesh Prasad
Re: Score field statistics
On 10 April 2013 22:03, lexus a...@scalepoint.com wrote: In our application we are using Solr 4.1. And we wanna filter results by score relevance. I had the idea to use statistic data (i.e. standard deviation, mean) for score field. Is it exists workaround of using …stats=truestats.field=score... ? Even simpler: You can just add fl=*,score to get the score returned in the search results along with all other fields. Please see http://wiki.apache.org/solr/SolrRelevancyFAQ#How_can_I_see_the_relevancy_scores_for_search_results Regards, Gora
