Re: Solr Stop with -K
On Fri, Jul 29, 2016 at 06:46:23AM +, Srinivasa Meenavali wrote: > Can somebody explain advantage of Stop solr with "-k" option ? Please don't start a new thread by replying to a message in an existing thread. http://wikibin.org/articles/thread-hijacking.html Paul. -- Paul Hoffman Systems Librarian Fenway Libraries Online c/o Wentworth Institute of Technology 550 Huntington Ave. Boston, MA 02115 (617) 442-2384 (FLO main number)
Re: Regarding JSON indexing in SOLR 4.10
On Tue, Mar 29, 2016 at 11:30:06PM -0700, Aditya Desai wrote: > I am running SOLR 4.10 on port 8984 by changing the default port in > etc/jetty.xml. I am now trying to index all my JSON files to Solr running > on 8984. The following is the command > > curl 'http://localhost:8984/solr/update?commit=true' --data-binary *.json > -H 'Content-type:application/json' The wildcard is the problem; your shell is expanding --data-binary *.json to --data-binary foo.json bar.json baz.json and curl doesn't know how to download bar.json and baz.json. Try this instead: for file in *.json; do curl 'http://localhost:8984/solr/update?commit=true' --data-binary "$file" -H 'Content-type:application/json' done Paul. -- Paul Hoffman Systems Librarian Fenway Libraries Online c/o Wentworth Institute of Technology 550 Huntington Ave. Boston, MA 02115 (617) 442-2384 (FLO main number)
Re: Delete by query using JSON?
On Tue, Mar 22, 2016 at 10:25:06PM -0400, Jack Krupansky wrote: > See the correct syntax example here: > https://cwiki.apache.org/confluence/display/solr/Uploading+Data+with+Index+Handlers#UploadingDatawithIndexHandlers-SendingJSONUpdateCommands > > Your query is fine. Thanks; I thought the query was wrong, but the example you pointed me to clued me in to the real problem: I had neglected to specify Content-Type: application.json (d'oh!). Paul. -- Paul Hoffman Systems Librarian Fenway Libraries Online c/o Wentworth Institute of Technology 550 Huntington Ave. Boston, MA 02115 (617) 442-2384 (FLO main number)
Re: Delete by query using JSON?
On Tue, Mar 22, 2016 at 04:27:03PM -0700, Walter Underwood wrote: > “Why do you care?” might not be the best way to say it, but it is > essential to understand the difference between selection (filtering) > and ranking. > > As Solr params: > > * q is ranking and filtering > * fq is filtering only > * bq is ranking only Thanks, that is a very useful and concise synopsis. > When deleting documents, ordering does not matter, which is why we ask > why you care about the ordering. > > If the response is familiar to you, imagine how the questions sound to > people who have been working in search for twenty years. But even when > we are snippy, we still try to help. > > Many, many times, the question is wrong. The most common difficulty on > this list is an “XY problem”, where the poster has problem X and has > assumed solution Y, which is not the right solution. But they ask > about Y. So we will tell people that their approach is wrong, because > that is the most helpful thing we can do. Alex's response didn't seem snippy to me at all, and I agree wholeheartedly about the wrong-question problem -- in my case, not only was I asking the wrong question, but I shouldn't even have had to ask the (right) question at all! Thanks again, everyone. Paul. -- Paul Hoffman Systems Librarian Fenway Libraries Online c/o Wentworth Institute of Technology 550 Huntington Ave. Boston, MA 02115 (617) 442-2384 (FLO main number)
Delete by query using JSON?
I've been struggling to find the right syntax for deleting by query using JSON, where the query includes an fq parameter. I know how to delete *all* documents, but how would I delete only documents with field doctype = "cres"? I have tried the following along with a number of variations, all to no avail: $ curl -s -d @- 'http://localhost:8983/solr/blacklight-core/update?wt=json' <http://localhost:8983/solr/blacklight-core/select?q=&fq=doctype%3Acres&wt=json&fl=id' It seems like such a simple thing, but I haven't found any examples that use an fq. Could someone post an example? Thanks in advance, Paul. -- Paul Hoffman Systems Librarian Fenway Libraries Online c/o Wentworth Institute of Technology 550 Huntington Ave. Boston, MA 02115 (617) 442-2384 (FLO main number)
Re: Indexing using CSV
On Sun, Mar 20, 2016 at 06:11:32PM -0700, Jay Potharaju wrote:
> Hi,
> I am trying to index some data using csv files. The data contains
> description column, which can include quotes, comma, LF/CR & other special
> characters.
>
> I have it working but run into an issue with the following error
>
> line=5,can't read line: 5 values={NO LINES AVAILABLE}.
>
> What is the best way to debug this issue and secondly how do other people
> handle indexing data using csv data.
I would concentrate first on getting the CSV reader working verifiably,
which might be the hardest part -- CSV is not a file format, it's a
hodgepodge.
Paul.
--
Paul Hoffman
Systems Librarian
Fenway Libraries Online
c/o Wentworth Institute of Technology
550 Huntington Ave.
Boston, MA 02115
(617) 442-2384 (FLO main number)
Re: Solr won't start -- java.lang.ClassNotFoundException: org.eclipse.jetty.xml.XmlConfiguration
On Tue, Mar 15, 2016 at 07:58:21PM -0600, Shawn Heisey wrote: > On 3/15/2016 2:56 PM, Paul Hoffman wrote: > >> It sure looks like I started Solr from my blacklight project dir. > >> > >> Any ideas? Thanks, > >> > > You may need to get some help from the blacklight project. I've got > absolutely no idea what sort of integration they may have done with > Solr, what they may have changed, or how they've arranged the filesystem. > > Regarding the Jetty problem, in the directory where the "start.jar" that > you are running lives, there should be a lib directory, with various > jetty jars. The jetty-xml jar should be one of them. Here's a listing > of Jetty's lib directory from a Solr 4.9.1 install that I've got. I > have upgraded to a newer version of Jetty: > > root@bigindy5:/opt/solr4# ls -al lib > total 1496 > drwxr-xr-x 3 solr solr 4096 Aug 31 2015 . > drwxr-xr-x 13 solr solr 4096 Aug 31 2015 .. > drwxr-xr-x 2 solr solr 4096 Aug 31 2015 ext > -rw-r--r-- 1 solr solr 21162 Aug 31 2015 > jetty-continuation-8.1.14.v20131031.jar > -rw-r--r-- 1 solr solr 61908 Aug 31 2015 > jetty-deploy-8.1.14.v20131031.jar > -rw-r--r-- 1 solr solr 96122 Aug 31 2015 jetty-http-8.1.14.v20131031.jar > -rw-r--r-- 1 solr solr 104219 Aug 31 2015 jetty-io-8.1.14.v20131031.jar > -rw-r--r-- 1 solr solr 24770 Aug 31 2015 jetty-jmx-8.1.14.v20131031.jar > -rw-r--r-- 1 solr solr 89923 Aug 31 2015 > jetty-security-8.1.14.v20131031.jar > -rw-r--r-- 1 solr solr 357704 Aug 31 2015 > jetty-server-8.1.14.v20131031.jar > -rw-r--r-- 1 solr solr 101714 Aug 31 2015 > jetty-servlet-8.1.14.v20131031.jar > -rw-r--r-- 1 solr solr 287680 Aug 31 2015 jetty-util-8.1.14.v20131031.jar > -rw-r--r-- 1 solr solr 110096 Aug 31 2015 > jetty-webapp-8.1.14.v20131031.jar > -rw-r--r-- 1 solr solr 39065 Aug 31 2015 jetty-xml-8.1.14.v20131031.jar > -rw-r--r-- 1 solr solr 200387 Aug 31 2015 servlet-api-3.0.jar > > The Jetty included with blacklight may contain more jars than this. The > Solr jetty install is stripped down so it's very lean. > > Thanks, > Shawn > That's exactly what I have now -- I got them from the blacklight-jetty repo -- except the versions are slightly different (jetty-*-8.1.10.v20130312.jar instead of jetty-*-8.1.14.v20131031.jar). My assumption now is that I was running a significantly older version of Solr -- I was getting some deprecation warnings and an error that prevented loading my Blacklight core. However, using the new jetty jars and making some adjustments in my schema.xml has got the Solr end of things working again, so I'll take any further questions to the Blacklight list. Thanks again for your help, Paul. -- Paul Hoffman Systems Librarian Fenway Libraries Online c/o Wentworth Institute of Technology 550 Huntington Ave. Boston, MA 02115 (617) 442-2384 (FLO main number)
Re: Solr won't start -- java.lang.ClassNotFoundException: org.eclipse.jetty.xml.XmlConfiguration
On Tue, Mar 15, 2016 at 01:46:32PM -0600, Shawn Heisey wrote:
> On 3/15/2016 1:34 PM, Paul Hoffman wrote:
> > I've been running Solr successfully until this morning, when I stopped
> > it to pick up a change in my schema, and now it won't start up again.
> > I've whittled the problem down to this:
> >
> >
> > # cd /home/paul/proj/blacklight/jetty
> >
> > # java -jar start.jar -Djetty.port=8983 -Dsolr.solr.home=$PWD/solr
> > WARNING: System properties and/or JVM args set. Consider using --dry-run
> > or --exec
> > java.lang.ClassNotFoundException: org.eclipse.jetty.xml.XmlConfiguration
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> > at org.eclipse.jetty.start.Main.invokeMain(Main.java:440)
> > at org.eclipse.jetty.start.Main.start(Main.java:615)
> > at org.eclipse.jetty.start.Main.main(Main.java:96)
> > ClassNotFound: org.eclipse.jetty.xml.XmlConfiguration
>
> There are no Solr classes in that stacktrace. The class that can't be
> found is a Jetty class. I think the problem here is in Jetty, not
> Solr. It probably can't find a jar with a name like one of these:
>
> jetty-xml-8.1.14.v20131031.jar
> jetty-xml-9.2.13.v20150730.jar
>
> What version of Solr? I'm assuming it's not 5.x, since the command used
> to start those versions is very different, and Solr would probably not
> be located within a blacklight folder.
Thanks, Shawn. Which version indeed -- I have a mishmash of cruft lying
around from earlier attempts to get Solr and Blacklight running, so I
don't want to assume anything. I found the log file that shows me
stopping and starting Solr today:
# ls -ltr $(find $(locate log | egrep 'solr|jetty') -type f -mtime -1) | head
-n5
find: `/home/paul/proj/blacklight/jetty/logs/solr.log': No such file or
directory
-rw-rw-r-- 1 paul paul 2885083 Mar 15 11:38
/home/paul/proj/blacklight/jetty/logs/solr_log_20160315_1152
-rw-r--r-- 1 root root5088 Mar 15 11:49
/home/paul/proj/solr-5.3.1/server/logs/solr_log_20160315_1150
-rw-r--r-- 1 root root 26701 Mar 15 11:49
/home/paul/proj/solr-5.3.1/server/logs/solr_gc_log_20160315_1150
-rw-rw-r-- 1 paul paul5086 Mar 15 11:51
/home/paul/proj/solr-5.3.1/server/logs/solr_log_20160315_1546
-rw-rw-r-- 1 paul paul 23537 Mar 15 11:51
/home/paul/proj/solr-5.3.1/server/logs/solr_gc_log_20160315_1546
# LOGFILE=/home/paul/proj/blacklight/jetty/logs/solr_log_20160315_1152
# egrep -nw 'stopped|org.eclipse.jetty.server.Server;' $LOGFILE | tail
5852:INFO - 2016-01-08 16:16:32.222; org.eclipse.jetty.server.Server;
jetty-8.1.10.v20130312
6128:INFO - 2016-01-13 13:01:58.338; org.eclipse.jetty.server.Server;
jetty-8.1.10.v20130312
6281:INFO - 2016-01-14 08:41:03.025; org.eclipse.jetty.server.Server;
jetty-8.1.10.v20130312
7792:INFO - 2016-02-08 11:57:41.131; org.eclipse.jetty.server.Server;
jetty-8.1.10.v20130312
7957:INFO - 2016-02-08 12:01:48.361; org.eclipse.jetty.server.Server;
jetty-8.1.10.v20130312
8174:INFO - 2016-02-08 15:03:18.641; org.eclipse.jetty.server.Server;
jetty-8.1.10.v20130312
8773:INFO - 2016-02-10 12:05:25.639; org.eclipse.jetty.server.Server;
jetty-8.1.10.v20130312
12244:INFO - 2016-03-15 11:38:16.810; org.eclipse.jetty.server.Server;
Graceful shutdown SocketConnector@0.0.0.0:8983
12245:INFO - 2016-03-15 11:38:16.814; org.eclipse.jetty.server.Server;
Graceful shutdown
o.e.j.w.WebAppContext{/solr,file:/home/paul/proj/blacklight/jetty/solr-webapp/webapp/},/home/paul/proj/blacklight/jetty/webapps/solr.war
12262:INFO - 2016-03-15 11:38:18.473;
org.eclipse.jetty.server.handler.ContextHandler; stopped
o.e.j.w.WebAppContext{/solr,file:/home/paul/proj/blacklight/jetty/solr-webapp/webapp/},/home/paul/proj/blacklight/jetty/webapps/solr.war
It looks like the last time it was last restarted was on February 10
(line 8773). The log file doesn't show the Solr version directly, but
maybe the first lines will help:
# sed -n 8773,8795p $LOGFILE
INFO - 2016-02-10 12:05:25.639; org.eclipse.jetty.server.Server;
jetty-8.1.10.v20130312
INFO - 2016-02-10 12:05:25.703;
org.eclipse.jetty.deploy.providers.ScanningAppProvider; Deployment monitor
/home/paul/proj/blacklight/jetty/contexts at interval 0
INFO - 2016-02-10 12:05:25.714; org.eclipse.jetty.deploy.DeploymentManager;
Deployable added: /home/paul/proj/blacklight/jetty/
Solr won't start -- java.lang.ClassNotFoundException: org.eclipse.jetty.xml.XmlConfiguration
I've been running Solr successfully until this morning, when I stopped it to pick up a change in my schema, and now it won't start up again. I've whittled the problem down to this: # cd /home/paul/proj/blacklight/jetty # java -jar start.jar -Djetty.port=8983 -Dsolr.solr.home=$PWD/solr WARNING: System properties and/or JVM args set. Consider using --dry-run or --exec java.lang.ClassNotFoundException: org.eclipse.jetty.xml.XmlConfiguration at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.eclipse.jetty.start.Main.invokeMain(Main.java:440) at org.eclipse.jetty.start.Main.start(Main.java:615) at org.eclipse.jetty.start.Main.main(Main.java:96) ClassNotFound: org.eclipse.jetty.xml.XmlConfiguration Usage: java -jar start.jar [options] [properties] [configs] java -jar start.jar --help # for more information # readlink -e $(which java) /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java # uname -srvmpio Linux 3.16.0-57-generic #77~14.04.1-Ubuntu SMP Thu Dec 17 23:20:00 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux # env | fgrep JAVA [no output] I only have one JVM installed -- openjdk-8-jre-headless. Judging from the file timestamps within /usr/lib/jvm, the package hasn't been updated since last August at the latest; the server has only been up for 62 days. Just in case it matters, I was running Solr successfully under Blacklight's jetty wrapper, and the command line above is what it uses (or claims to use). Does anyone have any idea what might be causing this problem? Thanks in advance, Paul. -- Paul Hoffman Systems Librarian Fenway Libraries Online c/o Wentworth Institute of Technology 550 Huntington Ave. Boston, MA 02115 (617) 442-2384 (FLO main number)
Re: Solr server not starting
On Wed, Jan 06, 2016 at 05:11:06PM +0100, agonn Qurdina wrote: > Hi, > > I am using Solr server with Echoprint service > (https://github.com/echonest/echoprint-server). The first time I started > it everything worked perfectly. This is the way I started it: > > java -Dsolr.solr.home=/home/echoprint-server/solr/solr/solr/ > -Djava.awt.headless=true -Xmx2048m -Xms2048m -jar start.jar > > Then I stopped it and I cannot start it anymore as it gets stuck at the 3rd > row of execution: > > 2016-01-06 11:04:19.030::INFO: Logging to STDERR via > org.mortbay.log.StdErrLog > 2016-01-06 11:04:19.165::INFO: jetty-6.1.3 > 2016-01-06 11:04:19.231::INFO: Extract > jar:file:/home/echoprint-server/solr/solr/webapps/solr.war!/ to > /tmp/Jetty_0_0_0_0_8502_solr.war__solr__-rnc92a/webapp > > It does not continue to execute anymore. I check if it is running in the > processes list and it turns out it is NOT. Please help me to solve this > problem! > > Best regards, > > Agon This could be a permissions problem -- for example, perhaps you started it as root the first time and are now attempting to start it as some other user. Paul. -- Paul Hoffman Systems Librarian Fenway Libraries Online c/o Wentworth Institute of Technology 550 Huntington Ave. Boston, MA 02115 (617) 442-2384 (FLO main number)
