Upgrading tika
Hello community, I am using Tika to extract the text content from pdf files before indexing them. I used version 1.7 and it worked OK except it produced a lot warnings like "Font not found". Now I am trying to move to the newer version 1.19.1 and I have problems finding all necessary dependencies. First it gave me the exception that it cannot find the class "org/apache/cxf/jaxrs/ext/multipart/ContentDisposition". I have added jars "cxf-core-3.2.8.jar" and "cxf-rt-fromtend-jaxrs-2.6.3.jar" from the cxf project. Now it gives me an exception "java.lang.NoClassDefFoundError: javax/ws/rs/core/MultivaluedMap". Could anybody suggest me what files do I need to use the latest version of Tika and where to find them? Thank you. -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
All replicas created on the same node
Hi community,
I have solr 7.6 running on three nodes with about 400 collections with one
shard and 3 replicas per collection. I want replicas to be spread between
all 3 nodes so that for every collection I have one replica per collection
on each node.
I create collections via the SolrJ code.
for (String collectionName:){
create =
CollectionAdminRequest.createCollection(collectionName, source,
1, 3);
result = solrClient.request(create);
}
In solr 7.4 it worked fine, but in solr 7.6 created replicas are not spread
equally between nodes. In some collections all 3 replicas are created just
on one node, in some 2 replicas are created in one node and 1 in another
and some collections are created correctly: I replica per node.
Could anyone give me advice on why it happened and how to fix it?
Thank you.
Lev Tannen
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Why solr sends a request for a metrics every minute?
Jan,
After I suppress the metrics messages,
I found that there are other messages. They come also once a minute, but
only on the one out of 3 computers. Could you please explain me what do
these messages mean and why they are produced by only one computer?
Best wishes.
2019-02-07 20:18:37.089 INFO (commitScheduler-18-thread-1) [ ]
o.a.s.u.SolrIndexWriter Calling setCommitData with
IW:org.apache.solr.update.SolrIndexWriter@28a1b6e5
commitCommandVersion:1624842664242315264
2019-02-07 20:18:37.093 INFO (commitScheduler-16-thread-1) [ ]
o.a.s.u.DirectUpdateHandler2 start
commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
2019-02-07 20:18:37.093 INFO (qtp689401025-19) [c:.system s:shard1
r:core_node4 x:.system_shard1_replica_n3]
o.a.s.u.p.LogUpdateProcessorFactory [.system_shard1_replica_n3]
webapp=/solr path=/update
params={update.distrib=FROMLEADER&distrib.from=http://usahubslvcvw122.usa.doj.gov:8983/solr/.system_shard1_replica_n1/&wt=javabin&version=2}{add=[rrd|solr.collection.NJ-A-documents
(1624842664240218112)]} 0 2
2019-02-07 20:18:37.093 INFO (commitScheduler-16-thread-1) [ ]
o.a.s.u.SolrIndexWriter Calling setCommitData with
IW:org.apache.solr.update.SolrIndexWriter@4107d3ff commitCommandVersion:0
2019-02-07 20:18:37.093 INFO (qtp689401025-547) [c:.system s:shard1
r:core_node2 x:.system_shard1_replica_n1]
o.a.s.u.p.LogUpdateProcessorFactory [.system_shard1_replica_n1]
webapp=/solr path=/update
params={wt=javabin&version=2}{add=[rrd|solr.collection.NJ-A-documents
(1624842664240218112)]} 0 7
2019-02-07 20:18:37.096 INFO (qtp689401025-19) [c:.system s:shard1
r:core_node4 x:.system_shard1_replica_n3]
o.a.s.u.p.LogUpdateProcessorFactory [.system_shard1_replica_n3]
webapp=/solr path=/update
params={update.distrib=FROMLEADER&distrib.from=http://usahubslvcvw122.usa.doj.gov:8983/solr/.system_shard1_replica_n1/&wt=javabin&version=2}{add=[rrd|solr.collection.OHN-B-cases
(1624842664248606720)]} 0 0
2019-02-07 20:18:37.096 INFO (qtp689401025-576) [c:.system s:shard1
r:core_node2 x:.system_shard1_replica_n1]
o.a.s.u.p.LogUpdateProcessorFactory [.system_shard1_replica_n1]
webapp=/solr path=/update
params={wt=javabin&version=2}{add=[rrd|solr.collection.OHN-B-cases
(1624842664248606720)]} 0 2
2019-02-07 20:18:37.099 INFO (qtp689401025-19) [c:.system s:shard1
r:core_node4 x:.system_shard1_replica_n3]
o.a.s.u.p.LogUpdateProcessorFactory [.system_shard1_replica_n3]
webapp=/solr path=/update
params={update.distrib=FROMLEADER&distrib.from=http://usahubslvcvw122.usa.doj.gov:8983/solr/.system_shard1_replica_n1/&wt=javabin&version=2}{add=[rrd|solr.collection.NM-A-documents
(1624842664251752448)]} 0 0
2019-02-07 20:18:37.099 INFO (qtp689401025-580) [c:.system s:shard1
r:core_node2 x:.system_shard1_replica_n1]
o.a.s.u.p.LogUpdateProcessorFactory [.system_shard1_replica_n1]
webapp=/solr path=/update
params={wt=javabin&version=2}{add=[rrd|solr.collection.NM-A-documents
(1624842664251752448)]} 0 1
2019-02-07 20:18:37.101 INFO (qtp689401025-19) [c:.system s:shard1
r:core_node4 x:.system_shard1_replica_n3]
o.a.s.u.p.LogUpdateProcessorFactory [.system_shard1_replica_n3]
webapp=/solr path=/update
params={update.distrib=FROMLEADER&distrib.from=http://usahubslvcvw122.usa.doj.gov:8983/solr/.system_shard1_replica_n1/&wt=javabin&version=2}{add=[rrd|solr.collection.KYE-B-documents
(1624842664253849600)]} 0 0
2019-02-07 20:18:37.101 INFO (qtp689401025-21) [c:.system s:shard1
r:core_node2 x:.system_shard1_replica_n1]
o.a.s.u.p.LogUpdateProcessorFactory [.system_shard1_replica_n1]
webapp=/solr path=/update
params={wt=javabin&version=2}{add=[rrd|solr.collection.KYE-B-documents
(1624842664253849600)]} 0 1
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: Why solr sends a request for a metrics every minute?
Thank you Jan. Now, when I know what it is, I probably will not try to suppress the metrics itself, but instead will suppress the log message in log4j2.xml using an appropriate filter. This way I will have metrics in case I will figure out how to use it and will not clog the log. I hope this will cause performance degradation. -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Why solr sends a request for a metrics every minute?
Hello Solr community,
My solrcloud system consists of 3 machines, each running a zookeeper and a
solr server. It manages about 200 collections with 1 shard each.
When I run it, I see that every minutes samebody sends a request for some
metrics to my system. Because nobody can sent requests to my development
system I assume that it is solr sends these requests by itself to itself.
Could anybody explain me, what is going on and how can I controll such
requests?
Bellow is an example of log messages that are produced by these requests.
Regards
Lev Tannen
Message example:
2019-02-04 16:39:13.487 INFO (qtp817348612-16) [ ] o.a.s.s.HttpSolrCall
[admin] webapp=null path=/admin/metrics
params={wt=javabin&version=2&key=solr.core.OKN-A-documents.shard1.replica_n4:QUERY./select.requests&key=solr.core.CA5-
B-documents.shard1.replica_n4:UPDATE./update.requests&key=solr.core.LAM-B-cases.shard1.replica_n4:UPDATE./update.requests&key=solr.core.NV-A-documents.shard1.replica_n2:QUERY./select.requests&key=solr.core.FLN-B-documents.shard1.repl
ica_n1:INDEX.sizeInBytes&key=solr.core.GU-B-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.CO-B-documents.shard1.replica_n4:UPDATE./update.requests&key=solr.core.MD-A-documents.shard1.replica_n1:QUERY./select.requests&ke
y=solr.core.INS-B-documents.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.MD-A-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.INS-A-cases.shard1.replica_n4:QUERY./select.requests&key=solr.core.ILS-A-documents.shard1.
replica_n1:QUERY./select.requests&key=solr.core.VAW-B-cases.shard1.replica_n4:QUERY./select.requests&key=solr.core.ARW-A-cases.shard1.replica_n2:UPDATE./update.requests&key=solr.core.OKW-A-documents.shard1.replica_n1:QUERY./select.re
quests&key=solr.core.KS-A-cases.shard1.replica_n1:QUERY./select.requests&key=solr.core.NYN-B-cases.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.MA-B-cases.shard1.replica_n4:INDEX.sizeInBytes&key=solr.core.FLM-B-cases.shard1.repl
ica_n2:INDEX.sizeInBytes&key=solr.core.NYE-A-cases.shard1.replica_n1:QUERY./select.requests&key=solr.core.MN-A-documents.shard1.replica_n2:QUERY./select.requests&key=solr.core.NYS-B-cases.shard1.replica_n1:UPDATE./update.requests&key
=solr.core.MIE-B-documents.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.ILS-A-cases.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.CAN-B-cases.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.ARE-A-documents.shard1.replica_n
4:UPDATE./update.requests&key=solr.core.MOW-A-documents.shard1.replica_n5:QUERY./select.requests&key=solr.core.INS-B-cases.shard1.replica_n4:QUERY./select.requests&key=solr.core.ND-B-cases.shard1.replica_n1:UPDATE./update.requests&ke
y=solr.core.OHN-A-documents.shard1.replica_n1:UPDATE./update.requests&key=solr.core.UT-B-documents.shard1.replica_n2:UPDATE./update.requests&key=solr.core.TNE-B-cases.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.OHS-B-cases.shar
d1.replica_n2:UPDATE./update.requests&key=solr.core.KYW-B-documents.shard1.replica_n3:UPDATE./update.requests&key=solr.core.TXE-A-cases.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.ND-A-cases.shard1.replica_n4:UPDATE./update.req
uests&key=solr.core.NV-A-documents.shard1.replica_n2:UPDATE./update.requests&key=solr.core.WVN-A-documents.shard1.replica_n1:QUERY./select.requests&key=solr.core.UT-B-documents.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.TXN-B-
documents.shard1.replica_n1:UPDATE./update.requests&key=solr.core.CA4-B-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.HI-B-cases.shard1.replica_n2:QUERY./select.requests&key=solr.core.TNM-A-documents.shard1.replica_n2:U
PDATE./update.requests&key=solr.core.CA3-B-documents.shard1.replica_n4:UPDATE./update.requests&key=solr.core.NE-B-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.NCM-B-cases.shard1.replica_n2:UPDATE./update.requests&key=s
olr.core.CAS-B-documents.shard1.replica_n4:QUERY./select.requests&key=solr.core.GU-B-cases.shard1.replica_n4:UPDATE./update.requests&key=solr.core.CAC-A-cases.shard1.replica_n4:INDEX.sizeInBytes&key=solr.core.KS-A-documents.shard1.re
plica_n2:INDEX.sizeInBytes&key=solr.core.LAE-A-documents.shard1.replica_n2:UPDATE./update.requests&key=solr.core.CA7-B-documents.shard1.replica_n3:INDEX.sizeInBytes&key=solr.core.PAM-A-cases.shard1.replica_n4:QUERY./select.requests&k
ey=solr.core.HI-A-cases.shard1.replica_n2:QUERY./select.requests&key=solr.core.MIW-B-cases.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.LAE-B-documents.shard1.replica_n4:QUERY./select.requests&key=solr.core.CA10-B-documents.shar
d1.replica_n1:INDEX.sizeInBytes&key=solr.core.CAN-B-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.FLS-A-cases.shard1.replica_n2:QUERY./select.requests&key=solr.core.NCW-A-cases.shard1.replica_n4:QUERY./select.requests&k
ey=solr.core.VT-A-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.LAE-B-cases.shard1.replica_n4:INDEX.sizeInBytes&key=s
Re: HttpParser URI is too large
Thank you Eric, Actually I have figured this out, but there is a deeper problem: where do these messages come from? They are internal messages. If I suppress messages from "org.apache.solr.servlet.HttpSolrCall" I will suppress all query reporting and I do not want this. So the question is what is the meaning of these messages, why I am getting them and how to suppress them if I am really do not need them? Regards, Lev Tannen -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Re: HttpParser URI is too large
Thank you Jan. This solution worked. The warning message "URI is too large
>81920" disappeared. But this fix unleashed an another problem: The INFO
message that was suppressed by the previous error now is displayed in all
its length. And it is way too long because it lists all 100 collections. I
do not need this message in a log and I hope it can be prevented by setting
an appropriate logger level to DEBUG or to ERROR in solr4j2.xml, but I do
not know the name of this logger. Could you or somebody in the community to
figure this out from the following fragment of the log?
Regards
Lev Tannen
32610 2019-01-29 18:07:01.870 INFO (qtp817348612-19) [ ]
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
params={wt=javabin&version=2&key=solr.core.ME-B-cases.shard1.replica_n4:UPDATE./update.requests&key=s
olr.core.TXS-A-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.LAW-A-documents.shard1.replica_n4:UPDATE./update.requests&key=solr.core.CA1-A-documents.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.NYE
-A-documents.shard1.replica_n4:UPDATE./update.requests&key=solr.core.TXS-B-cases.shard1.replica_n4:QUERY./select.requests&key=solr.core.VI-B-documents.shard1.replica_n4:UPDATE./update.requests&key=solr.core.GU-A-docu
ments.shard1.replica_n2:QUERY./select.requests&key=solr.core.GAN-B-documents.shard1.replica_n4:QUERY./select.requests&key=solr.core.VT-B-documents.shard1.replica_n1:QUERY./select.requests&key=solr.core.CA7-B-document
s.shard1.replica_n5:INDEX.sizeInBytes&key=solr.core.MIW-B-cases.shard1.replica_n4:INDEX.sizeInBytes&key=solr.core.MSS-A-documents.shard1.replica_n2:UPDATE./update.requests&key=solr.core.CA2-A-documents.shard1.replica
_n4:UPDATE./update.requests&key=solr.core.NYS-A-cases.shard1.replica_n4:UPDATE./update.requests&key=solr.core.VAE-B-cases.shard1.replica_n4:INDEX.sizeInBytes&key=solr.core.NYS-B-documents.shard1.replica_n1:INDEX.size
InBytes&key=solr.core.NCE-B-cases.shard1.replica_n4:QUERY./select.requests&key=solr.core.FLN-B-cases.shard1.replica_n4:INDEX.sizeInBytes&key=solr.core.TNM-A-cases.shard1.replica_n1:UPDATE./update.requests&key=solr.co
re.OKW-A-cases.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.PR-B-cases.shard1.replica_n1:UPDATE./update.requests&key=solr.core.INS-B-documents.shard1.replica_n4:INDEX.sizeInBytes&key=solr.core.CA2-B-documents.sh
ard1.replica_n3:QUERY./select.requests&key=solr.core.GU-A-cases.shard1.replica_n1:QUERY./select.requests&key=solr.core.FLM-B-cases.shard1.replica_n4:UPDATE./update.requests&key=solr.core.ARE-B-documents.shard1.replic
a_n4:QUERY./select.requests&key=solr.core.NYN-B-documents.shard1.replica_n4:INDEX.sizeInBytes&key=solr.core.GAN-B-documents.shard1.replica_n4:UPDATE./update.requests&key=solr.core.GAN-A-cases.shard1.replica_n1:UPDATE
./update.requests&key=solr.core.CAN-B-cases.shard1.replica_n4:INDEX.sizeInBytes&key=solr.core.NCE-B-documents.shard1.replica_n4:UPDATE./update.requests&key=solr.core.WIE-B-cases.shard1.replica_n2:UPDATE./update.reque
sts&key=solr.core.VAE-B-cases.shard1.replica_n4:QUERY./select.requests&key=solr.core.CAS-A-documents.shard1.replica_n2:UPDATE./update.requests&key=solr.core.PR-A-cases.shard1.replica_n4:UPDATE./update.requests&key=so
lr.core.CAS-A-documents.shard1.replica_n2:INDEX.sizeInBytes&key=solr.core.ALM-B-documents.shard1.replica_n2:UPDATE./update.requests&key=solr.core.VAW-A-cases.shard1.replica_n4:QUERY./select.requests&key=solr.core.FLM
...
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
