Setting up Solr Replica on different machine
Hi, I am planning to set up Solr with replica on different machine. How should I go about configuring the setup? Like for example, should the replica node be started on the host machine, or on the replica machine? I will be setting this in Solr 7.3.1. Regards, Edwin
Re: Replicate managed-schema in Solr Master/Slave Configuration
https://issues.apache.org/jira/secure/Dashboard.jspa You have to create a login, but then you can raise a Solr JIRA. Please put in all the detail you can to help someone reproduce. One more thing to try: Rename your managed-schema to something like testschema.xml, and change your solrconfig as I mentioned. Then verify that changing the schema (and afterwards adding some docs to the index and committing) _does_ replicate the schema. Mostly just insuring that the only difference between successfully replicating the schema and not is the name and not something else. Best, Erick On Thu, May 31, 2018 at 7:07 PM, Kelly Rusk wrote: > Thanks Erick, > > I have made changes on the Master/indexing that are replicated to the Slave > and the managed-schema does not come over. Sounds like a JIRA entry may be in > order. How do I go about doing that? I am not using ConfigSets as each Core > has its own unique Schema. > > Regards, > > Kelly > > -Original Message- > From: Erick Erickson > Sent: Thursday, May 31, 2018 8:50 PM > To: solr-user > Subject: Re: Replicate managed-schema in Solr Master/Slave Configuration > > On a quick glance at the code, I don't see anything requiring an xml > extension for the managed schema. I suppose it's possible that the hyphen is > messing things up. > > You should see a message like: > > "Adding config files to list: " .. on replication if you turn on > debug-level logging. > > At worst, you could change the _name_ of your schema by changing > solrconfig.xml like below. > > > true > managed-schema > > > But I'll emphasize that you should _not_ have to do this from a quick code > inspection. Have you changed your index since changing the managed schema? > Replication doesn't do anything unless the index on the master is changed > relative to the slave. > > And are you using configsets? > > If the schema has changed _and_ you see a replication happens (i.e. > new docs appear on the slaves) _and_ the managed-schema still isn't > replicated, that would merit a JIRA. > > Best, > Erick > > On Thu, May 31, 2018 at 3:35 PM, Kelly Rusk wrote: >> Hello all, >> I need to replicate the managed-schema in my Solr 6.6.2 Master/Slave >> environment and have added the necessary replication handlers. However, as >> the managed-schema does not have a file extension it doesn't seem to get >> picked up/replicated: >> schema.xml,managed-schema,stopwords.txt >> How can I replicate the managed-schema file if it has no file extension? >> Regards, >> Kelly >> >
RE: Replicate managed-schema in Solr Master/Slave Configuration
Thanks Erick, I have made changes on the Master/indexing that are replicated to the Slave and the managed-schema does not come over. Sounds like a JIRA entry may be in order. How do I go about doing that? I am not using ConfigSets as each Core has its own unique Schema. Regards, Kelly -Original Message- From: Erick Erickson Sent: Thursday, May 31, 2018 8:50 PM To: solr-user Subject: Re: Replicate managed-schema in Solr Master/Slave Configuration On a quick glance at the code, I don't see anything requiring an xml extension for the managed schema. I suppose it's possible that the hyphen is messing things up. You should see a message like: "Adding config files to list: " .. on replication if you turn on debug-level logging. At worst, you could change the _name_ of your schema by changing solrconfig.xml like below. true managed-schema But I'll emphasize that you should _not_ have to do this from a quick code inspection. Have you changed your index since changing the managed schema? Replication doesn't do anything unless the index on the master is changed relative to the slave. And are you using configsets? If the schema has changed _and_ you see a replication happens (i.e. new docs appear on the slaves) _and_ the managed-schema still isn't replicated, that would merit a JIRA. Best, Erick On Thu, May 31, 2018 at 3:35 PM, Kelly Rusk wrote: > Hello all, > I need to replicate the managed-schema in my Solr 6.6.2 Master/Slave > environment and have added the necessary replication handlers. However, as > the managed-schema does not have a file extension it doesn't seem to get > picked up/replicated: > schema.xml,managed-schema,stopwords.txt > How can I replicate the managed-schema file if it has no file extension? > Regards, > Kelly >
Re: Replicate managed-schema in Solr Master/Slave Configuration
On a quick glance at the code, I don't see anything requiring an xml extension for the managed schema. I suppose it's possible that the hyphen is messing things up. You should see a message like: "Adding config files to list: " .. on replication if you turn on debug-level logging. At worst, you could change the _name_ of your schema by changing solrconfig.xml like below. true managed-schema But I'll emphasize that you should _not_ have to do this from a quick code inspection. Have you changed your index since changing the managed schema? Replication doesn't do anything unless the index on the master is changed relative to the slave. And are you using configsets? If the schema has changed _and_ you see a replication happens (i.e. new docs appear on the slaves) _and_ the managed-schema still isn't replicated, that would merit a JIRA. Best, Erick On Thu, May 31, 2018 at 3:35 PM, Kelly Rusk wrote: > Hello all, > I need to replicate the managed-schema in my Solr 6.6.2 Master/Slave > environment and have added the necessary replication handlers. However, as > the managed-schema does not have a file extension it doesn't seem to get > picked up/replicated: > schema.xml,managed-schema,stopwords.txt > How can I replicate the managed-schema file if it has no file extension? > Regards, > Kelly >
Replicate managed-schema in Solr Master/Slave Configuration
Hello all, I need to replicate the managed-schema in my Solr 6.6.2 Master/Slave environment and have added the necessary replication handlers. However, as the managed-schema does not have a file extension it doesn't seem to get picked up/replicated: schema.xml,managed-schema,stopwords.txt How can I replicate the managed-schema file if it has no file extension? Regards, Kelly
REMINDER: Apache EU Roadshow 2018 in Berlin is less than 2 weeks away!
Hello Apache Supporters and Enthusiasts This is a reminder that our Apache EU Roadshow in Berlin is less than two weeks away and we need your help to spread the word. Please let your work colleagues, friends and anyone interested in any attending know about our Apache EU Roadshow event. We have a great schedule including tracks on Apache Tomcat, Apache Http Server, Microservices, Internet of Things (IoT) and Cloud Technologies. You can find more details at the link below: https://s.apache.org/0hnG Ticket prices will be going up on 8^th June 2018, so please make sure that you register soon if you want to beat the price increase. https://foss-backstage.de/tickets Remember that registering for the Apache EU Roadshow also gives you access to FOSS Backstage so you can attend any talks and workshops from both conferences. And don’t forget that our Apache Lounge will be open throughout the whole conference as a place to meet up, hack and relax. We look forward to seeing you in Berlin! Thanks Sharan Foga, VP Apache Community Development http://apachecon.com/ @apachecon PLEASE NOTE: You are receiving this message because you are subscribed to a user@ or dev@ list of one or more Apache Software Foundation projects.
Query logs when query reached the solr server
Hi, 1. Is there a way to enable query log when the query first reached the solr server? 2. In recent Solr versions, there is a NOW value in the query log. Is it correct to assume that this is the time when query arrived on that server? Thanks Nawab
Re: Pointing 3 Solr Servers to a 3-node Zookeeper Cluster
On 5/31/2018 10:30 AM, THADC wrote: > I have a three-node zookeeper cluster running on ports 2181, 2182, and 2183. > I also am creating three solr server nodes (running as solr cloud > instances). I want the three solr nodes (on ports 7574, 8983, and 8990) to > be in that zookeeper cluster. Since the three zookeeper nodes are in the > same cluster, can I simply point each solr node arbitrarily to one of the > three zookeeper nodes? For instance like: > > $./bin/./bin/solr start -c -p 7574 -z localhost:2181 > $./bin/./bin/solr start -c -p 8983 -z localhost:2181 > $./bin/./bin/solr start -c -p 8990 -z localhost:2181 No. Each Solr instance will need to be pointed at all the zookeeper servers. bin/solr start -p 7574 -z localhost:2181,localhost:2182,localhost:2183 bin/solr start -p 8983 -z localhost:2181,localhost:2182,localhost:2183 bin/solr start -p 8990 -z localhost:2181,localhost:2182,localhost:2183 If you don't do that, then a failure of the server in the -z parameter will mean that the Solr instance cannot talk to zookeeper at all, which is going to cause problems. Note that with all the services on the same server, you have no redundancy. If that server fails, everything's down. If you're really only going to have one server, then you might as well only have one Solr instance and one ZK instance. For production, I would also recommend using a chroot, which would change the -z parameter a little bit. https://lucene.apache.org/solr/guide/7_3/taking-solr-to-production.html#zookeeper-chroot Thanks, Shawn
RE: No solr.log in solr cloud 7.3
FYI -- having two zookeeper servers is actually LESS stable than only having one. You need at least three. Here are links to a couple of notes in the zookeeper documentation that discuss this requirement: We have 9 server in production and 3 (2+1) in dev So, this 2 are for the dev and it's on the dev the problem. We have restore a save -10days and it work. Thanks. -Message d'origine- De : Shawn Heisey [mailto:apa...@elyograg.org] Envoyé : jeudi 31 mai 2018 17:19 À : solr-user@lucene.apache.org Objet : Re: No solr.log in solr cloud 7.3 On 5/31/2018 7:04 AM, msaunier wrote: > wget > http://apache.mirrors.ovh.net/ftp.apache.org/dist/lucene/solr/6.6.1/so > lr-6.6.1.tgz > tar -xzf solr-*.tgz > /opt/solr-*/bin/install_solr_service.sh /opt/solr-*.tgz > /etc/init.d/solr stop rm -f solr-*.tgz So you did use the service installer. > 2. > cat ZK_HOST=srv-formation-solr.citya.local:2181,srv-formation-solr2.citya. > local:2181 > SOLR_PID_DIR=/var/solr > SOLR_HOME=/var/solr/data > LOG4J_PROPS=/var/solr/log4j.properties > SOLR_LOGS_DIR=/var/solr/logs > SOLR_PORT=8983 > SOLR_HOST=$(ifconfig | grep 'inet 192\.168\.' | sed -e > 's/^[^0-9]*\(192[^ ]*\).*/\1/') SOLR_JAVA_MEM="-Xms4096m -Xmx4096m > -Denable.runtime.lib=true" > EOF Unless you have also edited the bin/solr script and/or log4j.properties, all your logs should be in /var/solr/logs. If you have edited those files, then that might have changed the location, and we won't have any idea where you'll need to look. FYI -- having two zookeeper servers is actually LESS stable than only having one. You need at least three. Here are links to a couple of notes in the zookeeper documentation that discuss this requirement: http://zookeeper.apache.org/doc/r3.4.12/zookeeperStarted.html#sc_RunningReplicatedZooKeeper http://zookeeper.apache.org/doc/r3.4.12/zookeeperAdmin.html#sc_zkMulitServerSetup Thanks, Shawn
Re: Solr 7, exact phrase search, empty results for some records
The analysis page has one major thing to be aware of: It sees what would be in the field _after_ query parsing. I applaud your use of it, it's where lots of problems are found ;). Try adding =query in the two cases. Particularly look at the parsedquery_tostring in the response and compare. And I don't _think_ this is the issue since you're specifying phrases, but split-on-whitespace default has changed, see: https://lucidworks.com/2017/04/18/multi-word-synonyms-solr-adds-query-time-support/ Good luck, Erick On Thu, May 31, 2018 at 8:04 AM, damian.pawski wrote: > Hi, > > I have updated Solr from 5.4.1 to 7.2.1. > > I have updated the settings accordingly, but in some cases when I am > searching for an exact phrase surrounded by quotes I am getting 0 results. > > In 5.4.1 I have > > > > > ignoreCase="true" expand="true"/> > > words="stopwords.txt" /> > generateWordParts="1" generateNumberParts="1" catenateWords="1" > catenateNumbers="1" catenateAll="0" splitOnCaseChange="1" > preserveOriginal="1"/> > > > > > > > > > > words="stopwords.txt"/> > generateWordParts="1" generateNumberParts="1" catenateWords="0" > catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/> > > > > > > > > > > In 7.2.1 I have > > > > synonyms="synonyms.txt" > ignoreCase="true" expand="true"/> > > words="stopwords.txt" /> > > generateWordParts="1" > generateNumberParts="1" catenateWords="1" catenateNumbers="1" > catenateAll="0" splitOnCaseChange="1" preserveOriginal="1"/> > > > > > > > > > > > > words="stopwords.txt"/> > generateWordParts="1" generateNumberParts="1" catenateWords="0" > catenateNumbers="0" catenateAll="0" splitOnCaseChange="1"/> > > > > > > > > > > > I couldn't find any pattern explaining, why for some records searches with > quotes work fine but for the others, 0 results are returned (I have checked > and the records that are missing are imported, as I can find > them by the Id). > > Could you point me to correct direction in terms how can I investigate this? > > I have checked the results of the "..analysis..." pages on both instances of > Solr for the problematic records and in both cases I am getting the same > outcome. > > Thank you > Damian > > > > > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Pointing 3 Solr Servers to a 3-node Zookeeper Cluster
Hello, I have a three-node zookeeper cluster running on ports 2181, 2182, and 2183. I also am creating three solr server nodes (running as solr cloud instances). I want the three solr nodes (on ports 7574, 8983, and 8990) to be in that zookeeper cluster. Since the three zookeeper nodes are in the same cluster, can I simply point each solr node arbitrarily to one of the three zookeeper nodes? For instance like: $./bin/./bin/solr start -c -p 7574 -z localhost:2181 $./bin/./bin/solr start -c -p 8983 -z localhost:2181 $./bin/./bin/solr start -c -p 8990 -z localhost:2181 thanks for your response. -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: No solr.log in solr cloud 7.3
On 5/31/2018 7:04 AM, msaunier wrote: > wget > http://apache.mirrors.ovh.net/ftp.apache.org/dist/lucene/solr/6.6.1/solr-6.6.1.tgz > tar -xzf solr-*.tgz > /opt/solr-*/bin/install_solr_service.sh /opt/solr-*.tgz > /etc/init.d/solr stop > rm -f solr-*.tgz So you did use the service installer. > 2. > cat ZK_HOST=srv-formation-solr.citya.local:2181,srv-formation-solr2.citya.local:2181 > SOLR_PID_DIR=/var/solr > SOLR_HOME=/var/solr/data > LOG4J_PROPS=/var/solr/log4j.properties > SOLR_LOGS_DIR=/var/solr/logs > SOLR_PORT=8983 > SOLR_HOST=$(ifconfig | grep 'inet 192\.168\.' | sed -e 's/^[^0-9]*\(192[^ > ]*\).*/\1/') > SOLR_JAVA_MEM="-Xms4096m -Xmx4096m -Denable.runtime.lib=true" > EOF Unless you have also edited the bin/solr script and/or log4j.properties, all your logs should be in /var/solr/logs. If you have edited those files, then that might have changed the location, and we won't have any idea where you'll need to look. FYI -- having two zookeeper servers is actually LESS stable than only having one. You need at least three. Here are links to a couple of notes in the zookeeper documentation that discuss this requirement: http://zookeeper.apache.org/doc/r3.4.12/zookeeperStarted.html#sc_RunningReplicatedZooKeeper http://zookeeper.apache.org/doc/r3.4.12/zookeeperAdmin.html#sc_zkMulitServerSetup Thanks, Shawn
Re: SolrJ, CloudSolrClient and basic authentication
On 5/31/2018 8:03 AM, Dimitris Kardarakos wrote: > Following the feedback in the "Index protected zip" thread, I am > trying to add documents to the index using SolrJ API. > > The server is in SolrCloud mode with BasicAuthPlugin for authentication. > > I have not managed to figure out how to pass username/password to my > client. There are two ways to approach this. One approach is to build a custom HttpClient object that uses credentials by default, and then use that custom HttpClient object to build your CloudSolrClient. Exactly how to correctly build the HttpClient object will depend on exactly which HttpClient version you've included into your program. If you go with SolrJ dependency defaults, then the HttpClient version will depend on the SolrJ version. The other approach is the method described in the documentation, where credentials are added to each request object: https://lucene.apache.org/solr/guide/6_6/basic-authentication-plugin.html#BasicAuthenticationPlugin-UsingBasicAuthwithSolrJ There are several different kinds of request objects. A few examples: UpdateRequest, QueryRequest, CollectionAdminRequest. Thanks, Shawn
Solr 7, exact phrase search, empty results for some records
Hi, I have updated Solr from 5.4.1 to 7.2.1. I have updated the settings accordingly, but in some cases when I am searching for an exact phrase surrounded by quotes I am getting 0 results. In 5.4.1 I have In 7.2.1 I have I couldn't find any pattern explaining, why for some records searches with quotes work fine but for the others, 0 results are returned (I have checked and the records that are missing are imported, as I can find them by the Id). Could you point me to correct direction in terms how can I investigate this? I have checked the results of the "..analysis..." pages on both instances of Solr for the problematic records and in both cases I am getting the same outcome. Thank you Damian -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Solr query string compare to actual date and build a new string on this?
Hey, digging the web for a while now to find a solution due to a complex search string. what i've got: xml file with a date/description fields which are not part of the index: (start-date-time="2018-04-01T18:00:00.000+02:00" code-name="MD 28") (start-date-time="2018-04-07T15:00:00.000+02:00" code-name="MD 29") (start-date-time="2018-04-12T18:30:00.000+02:00" code-name="MD 30") what i want to have is a query string which checks the xml document against the actual date to query my solr search like this: select?q=*Goals*=roundTypeWithIndex%3A*insert actual/latest MD here*=json=true=0=50=true Is there any kind of possibility to do this by solr itself or i would really appriciate any kind of solution on this. Thanks Francois -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
SolrJ, CloudSolrClient and basic authentication
Hi. Following the feedback in the "Index protected zip" thread, I am trying to add documents to the index using SolrJ API. The server is in SolrCloud mode with BasicAuthPlugin for authentication. I have not managed to figure out how to pass username/password to my client. Thanks, -- Dimitris Kardarakos
search q via dynamic string depends on date
Hey, digging the web for a while now to find a solution due to a complex search string. what i've got: xml file with a date/description fields which are not part of the index: (start-date-time="2018-04-01T18:00:00.000+02:00" code-name="MD 28") (start-date-time="2018-04-07T15:00:00.000+02:00" code-name="MD 29") (start-date-time="2018-04-12T18:30:00.000+02:00" code-name="MD 30") what i want to have is a query string which checks the xml document against the actual date to query my solr search like this: select?q=*Goals*=roundTypeWithIndex%3A*insert actual/latest MD here*=json=true=0=50=true Is there any kind of possibility to do this by solr itself or i would really appriciate any kind of solution on this. Thanks Francois -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
RE: No solr.log in solr cloud 7.3
I start Solr with /etc/init.d/solr script:
#
#!/bin/sh
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
### BEGIN INIT INFO
# Provides: solr
# Required-Start:$remote_fs $syslog citya-zookeeper
# Required-Stop: $remote_fs $syslog citya-zookeeper
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Description: Controls Apache Solr as a Service
### END INIT INFO
# Example of a very simple *nix init script that delegates commands to the
bin/solr script
# Typical usage is to do:
#
# cp bin/init.d/solr /etc/init.d/solr
# chmod 755 /etc/init.d/solr
# chown root:root /etc/init.d/solr
# update-rc.d solr defaults
# update-rc.d solr enable
# Where you extracted the Solr distribution bundle
SOLR_INSTALL_DIR="/opt/solr"
if [ ! -d "$SOLR_INSTALL_DIR" ]; then
echo "$SOLR_INSTALL_DIR not found! Please check the SOLR_INSTALL_DIR setting
in your $0 script."
exit 1
fi
# Path to an include file that defines environment specific settings to
override default
# variables used by the bin/solr script. It's highly recommended to define this
script so
# that you can keep the Solr binary files separated from live files (pid, logs,
index data, etc)
# see bin/solr.in.sh for an example
SOLR_ENV="/etc/default/solr.in.sh"
if [ ! -f "$SOLR_ENV" ]; then
echo "$SOLR_ENV not found! Please check the SOLR_ENV setting in your $0
script."
exit 1
fi
# Specify the user to run Solr as; if not set, then Solr will run as root.
# Running Solr as root is not recommended for production environments
RUNAS="solr"
# verify the specified run as user exists
runas_uid="`id -u "$RUNAS"`"
if [ $? -ne 0 ]; then
echo "User $RUNAS not found! Please create the $RUNAS user before running
this script."
exit 1
fi
case "$1" in
start|stop|restart|status)
SOLR_CMD="$1"
;;
*)
echo "Usage: $0 {start|stop|restart|status}"
exit
esac
if [ -n "$RUNAS" ]; then
su -c "SOLR_INCLUDE=\"$SOLR_ENV\" \"$SOLR_INSTALL_DIR/bin/solr\" $SOLR_CMD" -
"$RUNAS"
else
SOLR_INCLUDE="$SOLR_ENV" "$SOLR_INSTALL_DIR/bin/solr" "$SOLR_CMD"
Fi
#
Where is the solr.in.sh file that you edited?
/etc/default/solr.in.sh
Solr Installation:
1.
cd /opt
wget
http://apache.mirrors.ovh.net/ftp.apache.org/dist/lucene/solr/6.6.1/solr-6.6.1.tgz
tar -xzf solr-*.tgz
/opt/solr-*/bin/install_solr_service.sh /opt/solr-*.tgz
/etc/init.d/solr stop
rm -f solr-*.tgz
2.
cat What procedure did you follow to install Solr?
> The procedure on the documentation to install SolR Cloud
You're going to have to be a lot more specific. The only documentation that I
consider to be relevant for installing Solr is NOT on the SolrCloud part of the
documentation. It's here:
https://lucene.apache.org/solr/guide/7_3/taking-solr-to-production.html#run-the-solr-installation-script
Have you used the service installer script? If so, what options did you give
it?
> How was it started?
> I think 1 weeks, but I only realized it yesterday. Before, no problems.
Not WHEN. I asked HOW you started it. What was the exact command that you
typed? Or was it started as part of system boot?
> Does you solr.in.sh have a reference to your log4j.properties file
> with the proper path?
Where is the solr.in.sh file that you edited? Whether that's the right one
will depend on the answer to the first question I
Update Solr Document
Hi solr-user, I am using Solr 7.2. I am newbie in solr , please forgive my nuisance. Lets say I have one solr collection(say Collection X) with 100 Documents(lets say 5 fields in one Document) and I have another collection(say Collection Y) with 1 lakh Documents (same 5 fields in one Document that we have in Collection X). Then, If I update only one field from one document from collection THEN Is "time required to update in collection X" == "time required to update in collection Y" ? question is: Is solr updates individual doc irrespective of collection size? if not how re-indexing works on Document update? Thanks, Rushikesh Garadade
Re: No solr.log in solr cloud 7.3
On 5/31/2018 1:49 AM, SAUNIER Maxence wrote: What procedure did you follow to install Solr? The procedure on the documentation to install SolR Cloud You're going to have to be a lot more specific. The only documentation that I consider to be relevant for installing Solr is NOT on the SolrCloud part of the documentation. It's here: https://lucene.apache.org/solr/guide/7_3/taking-solr-to-production.html#run-the-solr-installation-script Have you used the service installer script? If so, what options did you give it? How was it started? I think 1 weeks, but I only realized it yesterday. Before, no problems. Not WHEN. I asked HOW you started it. What was the exact command that you typed? Or was it started as part of system boot? Does you solr.in.sh have a reference to your log4j.properties file with the proper path? Where is the solr.in.sh file that you edited? Whether that's the right one will depend on the answer to the first question I asked -- exactly how did you install Solr? Have you edited any log4j.properties files? If so, where were they? Side note: Fault tolerant production zookeeper installs require at least three hosts. Your ZK_HOST setting only lists one. Thanks, Shawn
Re: Understanding SOLR Joins
On Tue, May 29, 2018 at 10:50 AM, Nancy Goyal <87.na...@gmail.com> wrote: > Hi, > > > I have implemented basic SOLR Joins between two collections. Currently in > my project implementation, we are getting data from multiple tables and > storing as single document in view and indexing that view. We got a > suggestion to implement the same with Joins but not sure if the same > functionalities can be achieved with JOINs or Block Join (Nested > documents)- > > *Data* > > There are multiple tables, one primary table having all the basic details > about product, the primary key is Product ID and then 7-8 other tables > having other details of product, it has Product ID column too but can have > multiple entries for single Product ID. > > *Can you please let me know if the below are possible-* > >1. Can we get data from multiple collections in the search results. The >results should contain only one record for a single product ID? > You can collection=depts,emps param to search across few collections (although it wasn't documented) Default merge behavior is to collapse recs with the same id to one, missing duplicated ids. >2. Can we search across multiple collections in a single query and then >club the results, so that final search results will have single result > for >each Product ID. > It sounds like grouping functionality, but I'm not sure it can do that with a few collections. Probably [subquery] is an option. >3. Can we perform join on more than 2 collections as we need to search >across 6-7 collections and then merge the data based on product ID. > For such advanced cases you may try to use Streaming Expressions, it can search and merge, but I've never do anything like this. >4. Can we query Parent and child in nested index at the same time. >Example- perform search on column1 from Parent and Column2 from Child > and >get the parent records with nested child in search results? > [subquery], [child] result transformers >5. If we can perform fielded search across multiple collections in the >single query,will the filters from different collections be returned in > a >single search response > This may probably done with [subquery]. > > The examples i got from internet have joins only between two collections > and searching only on single collection. > > Thanks & Regards, > Nancy Goyal > -- Sincerely yours Mikhail Khludnev
Impact of timeAllowed parameter
Hi, Wondering how would be the calling application informed that the search request has been impacted due to time-out vs it has completed normally? Is there something that is sent to the client as part of the response that time-out has been invoked? Thanks
Model type does not exist MultipleAdditiveTreesModel
Hi all,
I'm trying to upload the most simple model to solr 7.3.1 and i get an error:
the model:
{
"class" : "org.apache.solr.ltr.model.MultipleAdditiveTreesModel",
"name" : "my",
"features":[],
"params" : {
"trees" : [
{
"weight" : 1,
"root" : {
"value" : -10
}} ]}}
The error:
"error":{
"metadata":[
"error-class","org.apache.solr.common.SolrException",
"root-error-class","java.lang.IllegalArgumentException"],
"msg":"org.apache.solr.ltr.model.ModelException: Model type does not
exist org.apache.solr.ltr.model.MultipleAdditiveTreesModel",
"code":400}}
I inserted the configurations to solrconfig.xml like
and started solr using -Dsolr.ltr.enabled=true
please help me
Thanks you all ;)
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Solr Cloud 7.3.1 backups
Hi Rahul Solr is a secondary index. The system of record is a RDBMS. I'm currently looking at using AWS Elastic File System. Have you got any experience with this? I also thought about trying s3fs. When you say backup locally, what do you mean? Backup the files on disc without the associated Zookeeper config? Or something else? Thanks Greg On Thu, 31 May 2018 at 20:08, Rahul Singh wrote: > Greg, > > Is SolR your main system of record or is it a secondary index to a primary > data store? > > Depending on the answer to that question I would recommend different > options. > > If primary, then I would ask what is the underlying compute > infrastructure. Is it container, VM , or bare metal. > > There are some decent distributed shared file system services that could > be leveraged depending on the number of compute nodes. > > Shared file system is the best way to keep it consistent but it comes with > its draw backs. You can always backup locally and asynchronously sync to > shared FS too. > > -- > Rahul Singh > rahul.si...@anant.us > > Anant Corporation > On May 30, 2018, 5:16 PM -0400, Greg Roodt , wrote: > > Thanks for the confirmation Shawn. Distributed systems are hard, so this > > makes sense. > > > > I have a large, stable cluster (stable in terms of leadership and > > performance) with a single shard. The cluster scales up and down with > > additional PULL replicas over the day with the traffic curve. > > > > It's going to take a bit of coordination to get all nodes to mount a > shared > > volume when we take a backup and then unmount when done. > > > > Any idea what happens if a node joins or leaves during a backup? > > > > > > > > > > > > > > > > > > > > On Thu, 31 May 2018 at 06:14, Shawn Heisey wrote: > > > > > On 5/29/2018 3:01 PM, Greg Roodt wrote: > > > > What is the best way to perform a backup of a Solr Cloud cluster? Is > > > there > > > > a way to backup only the leader? From my tests with the collections > admin > > > > BACKUP command, all nodes in the cluster need to have access to a > shared > > > > filesystem. Surely that isn't necessary if you are backing up the > leader > > > or > > > > TLOG replica? > > > > > > If you have more than one Solr instance in your cloud, then all of > those > > > instances must have access to the same filesystem accessed from the > same > > > mount point. Together, they will write the entire collection to various > > > subdirectories in that location. > > > > > > I can't find any mention of whether backups are load balanced across > the > > > cloud, or if they always use leaders. I would assume the former. If > > > that's how it works, then you don't know which machine is going to do > > > the backup of a given shard. Even if the backup always uses leaders, > > > you can't always be sure of where a leader is. It can change from > > > moment to moment, especially if you're having stability problems with > > > your cloud. > > > > > > At restore time, there's a similar situation. You don't know which > > > machine(s) in the cloud are going to be actually loading index data > from > > > the backup location. So they all need to have access to the same data. > > > > > > Thanks, > > > Shawn > > > > > > >
Re: Solr Cloud 7.3.1 backups
Greg, Is SolR your main system of record or is it a secondary index to a primary data store? Depending on the answer to that question I would recommend different options. If primary, then I would ask what is the underlying compute infrastructure. Is it container, VM , or bare metal. There are some decent distributed shared file system services that could be leveraged depending on the number of compute nodes. Shared file system is the best way to keep it consistent but it comes with its draw backs. You can always backup locally and asynchronously sync to shared FS too. -- Rahul Singh rahul.si...@anant.us Anant Corporation On May 30, 2018, 5:16 PM -0400, Greg Roodt , wrote: > Thanks for the confirmation Shawn. Distributed systems are hard, so this > makes sense. > > I have a large, stable cluster (stable in terms of leadership and > performance) with a single shard. The cluster scales up and down with > additional PULL replicas over the day with the traffic curve. > > It's going to take a bit of coordination to get all nodes to mount a shared > volume when we take a backup and then unmount when done. > > Any idea what happens if a node joins or leaves during a backup? > > > > > > > > > > On Thu, 31 May 2018 at 06:14, Shawn Heisey wrote: > > > On 5/29/2018 3:01 PM, Greg Roodt wrote: > > > What is the best way to perform a backup of a Solr Cloud cluster? Is > > there > > > a way to backup only the leader? From my tests with the collections admin > > > BACKUP command, all nodes in the cluster need to have access to a shared > > > filesystem. Surely that isn't necessary if you are backing up the leader > > or > > > TLOG replica? > > > > If you have more than one Solr instance in your cloud, then all of those > > instances must have access to the same filesystem accessed from the same > > mount point. Together, they will write the entire collection to various > > subdirectories in that location. > > > > I can't find any mention of whether backups are load balanced across the > > cloud, or if they always use leaders. I would assume the former. If > > that's how it works, then you don't know which machine is going to do > > the backup of a given shard. Even if the backup always uses leaders, > > you can't always be sure of where a leader is. It can change from > > moment to moment, especially if you're having stability problems with > > your cloud. > > > > At restore time, there's a similar situation. You don't know which > > machine(s) in the cloud are going to be actually loading index data from > > the backup location. So they all need to have access to the same data. > > > > Thanks, > > Shawn > > > >
Re: Weird behavioural differences between pf in dismax and edismax
Hi, We personally use dismax as a more basic search endpoint so that users who are not aware for lucene syntax don't end up using special keywords or chars. which might affect their search queries. The switch between dismax and edismax is triggered by an advanced get param. I imagine there might be others who use it for similar purposes. Regards, Sam On Wed, May 30, 2018 at 7:29 PM, Alessandro Benedetti wrote: > Question in general for the community : > what is the dismax capable of doing that the edismax is not ? > Is it really necessary to keep both of them or the dismax could be > deprecated ? > > Cheers > > > > - > --- > Alessandro Benedetti > Search Consultant, R Software Engineer, Director > Sease Ltd. - www.sease.io > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html >
RE:No solr.log in solr cloud 7.3
What procedure did you follow to install Solr? The procedure on the documentation to install SolR Cloud How was it started? I think 1 weeks, but I only realized it yesterday. Before, no problems. What version of Solr? 7.3 What OS flavor are you running on? Debian Does you solr.in.sh have a reference to your log4j.properties file with the proper path? ZK_HOST=srv-formation-solr3.citya.local:2181 SOLR_PID_DIR=/var/solr SOLR_HOME=/var/solr/data LOG4J_PROPS=/var/solr/log4j.properties SOLR_LOGS_DIR=/var/solr/logs SOLR_PORT=8983 SOLR_HOST=192.168.37.109 SOLR_JAVA_MEM="-Xms4096m -Xmx4096m -Denable.runtime.lib=true -verbose:class" It's good. Thanks, Maxence, De : Leonard, Carl [cleon...@whisolutions.com] Envoyé : mercredi 30 mai 2018 22:29 À : solr-user@lucene.apache.org Objet : RE: No solr.log in solr cloud 7.3 Does you solr.in.sh have a reference to your log4j.properties file with the proper path? -Original Message- From: Shawn Heisey Sent: Wednesday, May 30, 2018 1:15 PM To: solr-user@lucene.apache.org Subject: Re: No solr.log in solr cloud 7.3 On 5/30/2018 8:40 AM, msaunier wrote: > Today, I don’t understand why, but I don’t have solr.log file. I have > just: > > drwxr-xr-x 1 solr solr 84 mai 30 16:19 archived > > -rw-r--r-- 1 solr solr 891352 mai 30 16:29 solr-8983-console.log > > -rw-r--r-- 1 solr solr 74068 mai 30 16:34 solr_gc.log.0.current What procedure did you follow to install Solr? How was it started? What version of Solr? What OS flavor are you running on? The answers to these questions will help determine where you should be looking. Thanks, Shawn
