Re: Solr - zoo with more than 1000 collections

2018-06-29 Thread Yago Riveiro
queue work stacks recovery operations due the low throughput to consume the queue. Regards. -- Yago Riveiro On 29 Jun 2018 17:38 +0100, Bertrand Mahé , wrote: > Hi, > > > > In order to store timeseries data and perform deletion easily, we create a > several collections pe

Re: Largest number of indexed documents used by Solr

2018-04-03 Thread Yago Riveiro
2G of ram Regards. -- Yago Riveiro On 4 Apr 2018 02:15 +0100, Abhi Basu <9000r...@gmail.com>, wrote: > We have tested Solr 4.10 with 200 million docs with avg doc size of 250 KB. > No issues with performance when using 3 shards / 2 replicas. > > > > On Tue, Apr 3, 2018 at

Re: Solr 6. 3 Can not talk to ZK Updates are disabled

2018-04-02 Thread Yago Riveiro
Hi murugesh, This error happen normally when you are in long GC pauses. Try to rise the heap memory. The only way to recover from this is restarting the affected node. Regard. -- Yago Riveiro On 2 Apr 2018 15:39 +0100, murugesh karmegam , wrote: > We noticed this issue in our solr clust

Re: Protect a collection to be deleted

2017-12-13 Thread Yago Riveiro
That can work, but the goal it’s to avoid human error (like the UI that enforce you to type de name of the collection on delete) independently of the access level. Regards -- /Yago Riveiro On 12 Dec 2017 20:24 +, Anshum Gupta , wrote: > You might want to explore Rule based authorizat

Re: Protect a collection to be deleted

2017-12-12 Thread Yago Riveiro
Thanks Shawn for address the question to Jira. Indeed I want to continue to insert data in the collection. I found that delete a collection by mistake using the API it’s to easy and prone to human error. Regards, -- Yago Riveiro On 12 Dec 2017 19:05 +, Shawn Heisey , wrote: > On 12

RE: Protect a collection to be deleted

2017-12-12 Thread Yago Riveiro
I don’t know if it’s possible but, if we can mark the collection like protected, we can avoid DELETE command to remove de collection. Maybe set the flag when CREATE command is executed? This is an interesting feature to avoid human errors, and relatively easy to implement. Regards -- Yago

Protect a collection to be deleted

2017-12-12 Thread Yago Riveiro
Hi, Is it possible in Solr protect a collection to be deleted through a property? Regards - Best regards /Yago -- Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Solr 6.5.1 process crash after jshort_disjoint_arraycopy error

2017-11-15 Thread Yago Riveiro
Nop, I never found a fix for this problem, sorry. Regards. -- Yago Riveiro On 15 Nov 2017 09:44 +, tothis , wrote: > Hi Yago, > > we are facing the same problem. Could you solve it somehow? > > thx > > > > > -- > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: How many collections in a solrcloud are too many, how to determine this?

2017-08-09 Thread Yago Riveiro
minutes. The tradeoff is not easy, depends of the number of machines, the volume of data, hardware and so on. -- /Yago Riveiro On 8 Aug 2017 20:27 +0100, Webster Homer , wrote: > Yes we do see replicas go into recovery. > > Most of our clouds are hosted in the google cloud. So flaky net

Re: IndexReaders cannot exceed 2 Billion

2017-08-07 Thread Yago Riveiro
You have the maximum number of docs in a single shard. If I'm not wrong, the only solution is split the index in more shards (if you are running solrcloud mode). -- /Yago Riveiro On 7 Aug 2017, 16:48 +0100, Wael Kader , wrote: > Hello, > > I faced an issue that is making me go

Re: Truncated chunk in CloudSolrStream

2017-05-25 Thread Yago Riveiro
Nop, this happened since 6.3.0 (when I started use the CloudSolrStream), now I’m using 6.5.1 code. Normally this happen with streams with more than 4M documents. Can be related with network? Is there any TTL in the CloudSolrStream at connection level? -- /Yago Riveiro On 25 May 2017 13:14

Truncated chunk in CloudSolrStream

2017-05-25 Thread Yago Riveiro
I have a process that uses the CloudSolrStream to run a streaming expression and I can see this exception frequently: Caused by: org.apache.http.TruncatedChunkException: Truncated chunk ( expected size: 32768; actual size: 1100) at org.apache.http.impl.io.ChunkedInputStream.read(ChunkedInp

Re: LukeRequestHandler not returning all fields in the index

2017-05-22 Thread Yago Riveiro
Ok ... then I have no way to know the full list of fields in my collection without doing a LukeRequest to all of the shards and do a merge in the end, isn't it? Streaming expressions doesn't allow * wildcard, the LukeRequest doesn't return all fields .. no way to pull all data from a collection in

LukeRequestHandler not returning all fields in the index

2017-05-22 Thread Yago Riveiro
I'm struggle with a situation that I think can be a bug The LukeRequestHandler is not returning all fields that exists in one collection with 12 shards on 12 nodes (1 shard on each node) Running this request "http://localhost:8983/solr/collection/admin/luke"; in each node the list of fields are t

Couldn't decorate docValues for field message in logs

2017-05-05 Thread Yago Riveiro
Hi, I have a field type in my schema configured as: The goal of this field type is allow fields to be faceted and displaying data is necessary. The field be searchable is not a requisite. While I'm indexing data I have this annoying warning in logs: Couldn't decorate docValues for field: [fie

Re: Export endpoint broken in solr 6.5.1?

2017-05-05 Thread Yago Riveiro
Joel, Thank for the advice, indeed the /export handler was referenced in the config. The streaming expression is working. - Best regards /Yago -- View this message in context: http://lucene.472066.n3.nabble.com/Export-endpoint-broken-in-solr-6-5-1-tp4333416p4333504.html Sent from the Sol

Re: Export endpoint broken in solr 6.5.1?

2017-05-04 Thread Yago Riveiro
Older build with that was upgraded from 6.3.0 to 6.5.1. The config used in 6.3.0 are the same used in 6.5.1 without changes. Should I update my configs? -- /Yago Riveiro On 4 May 2017, 21:45 +0100, Joel Bernstein , wrote: > Did this error come from a standard 6.5.1 build, or form a build t

Export endpoint broken in solr 6.5.1?

2017-05-04 Thread Yago Riveiro
I'm trying to run this streaming expression search(data,qt="/export",q="*:*",fl="id",sort="id asc") and I'm hitting this exception: 2017-05-04 17:24:05.156 ERROR (qtp1937348256-378) [c:data s:shard7 r:core_node38 x:data_shard7_replica1] o.a.s.c.s.i.s.ExceptionStream java.io.IOException: java.uti

Solr 6.5.1 process crash after jshort_disjoint_arraycopy error

2017-05-03 Thread Yago Riveiro
# A fatal error has been detected by the Java Runtime Environment: Hi, I'm running 6.5.1 using Java 8 build 1.8.0_131-b11 and solr's process crash with this log # SIGBUS (0x7) at pc=0x7fd2c87ea014, pid=4468, tid=0x7fd1f487e700 # # JRE version: Java(TM) SE Runtime Environment (8.0_131-b11

Aliases feature scales?

2017-04-19 Thread Yago Riveiro
Hi, Does Anyone know if there is any theoretical limit related to the number of aliases that a Solr cluster can handle? If I create like 10K aliases would I experiment any kind of bottleneck? Regards - Best regards /Yago -- View this message in context: http://lucene.472066.n3.nabble

Re: Solr 6.3.0, possible SYN flooding on port 8983. Sending cookies.

2017-03-04 Thread Yago Riveiro
I’m using guzzle 3 for HTTP (it’s old but it’s the only one that works in 5.3) and the documentation says that use persistent connection (but you know … is PHP, weird things happen). Maybe I need to dump data to disk an use Java to post it ... -- /Yago Riveiro On 4 Mar 2017 16:50 +

Re: Solr 6.3.0, possible SYN flooding on port 8983. Sending cookies.

2017-03-04 Thread Yago Riveiro
The weird thing is that the lsof command shows that connections are made between 2 solr instances and not from the origin of new income data ... -- /Yago Riveiro On 4 Mar 2017 10:32 +, Mikhail Khludnev , wrote: > I hardly can comment regarding PHP. But if you call curl as an exter

Re: Solr 6.3.0, possible SYN flooding on port 8983. Sending cookies.

2017-03-04 Thread Yago Riveiro
Hi Mikhail, I’m not using SSL, and the way I call Solr is through a php script that use Curl -- /Yago Riveiro On 4 Mar 2017 08:54 +, Mikhail Khludnev , wrote: > Hello, Yago. > It usually happens when client doesn't reuse http connections. How do you > call Solr? Is there SSL

Solr 6.3.0, possible SYN flooding on port 8983. Sending cookies.

2017-03-03 Thread Yago Riveiro
Hello, I have this log in my dmesg: possible SYN flooding on port 8983. Sending cookies. The Solr instance (6.3.0) has not accepting more http connections. I ran this: _lsof -nPi |grep \:8983 | wc -l_ and the number of connection to port 8983 is about 14K in CLOSE_WAIT ou ESTABLISHED state. An

Re: [Benchmark SOLR] JETTY VS TOMCAT

2017-01-27 Thread Yago Riveiro
Solr run tests with jetty. I ran in nasty bugs in solr in the past with tomcat. My advise it’s that speed is only one more metric, robustness and reliability matter too. -- /Yago Riveiro On 27 Jan 2017 15:38 +, William Bell , wrote: > Did you try: > > Set your accep

Re: Streams return default values for fields that doesn't exist in the document

2017-01-21 Thread Yago Riveiro
6.3.0 I will try again with 6.4.0 Thank Erick -- /Yago Riveiro On 21 Jan 2017, 21:23 +, Erick Erickson , wrote: > What version of Solr? See: https://issues.apache.org/jira/browse/SOLR-9166 > > Best, > Erick > > On Sat, Jan 21, 2017 at 1:08 PM, Yago Riveiro wrote: >

Streams return default values for fields that doesn't exist in the document

2017-01-21 Thread Yago Riveiro
I'm trying to use the streaming API to reindex data from one collection to another. I have a lot of dynamic fields on my documents and not every document has the same fields, therefore, to fetch the list if fields that exists in the collection, I need to run a luke query to fetch all of them. I r

Re: CloudSolrStream can't set the setZkClientTimeout and setZkConnectTimeout properties

2017-01-19 Thread Yago Riveiro
I can see some reconnects in my logs, the process of consuming the stream doesn't broke and continue as normal. The timeout is 10s but I can see in logs that after 6s the reconnect is triggered, I don't know if it's the default behaviour or the zk timeout it's not honoured. - Best regards

Re: Question about Lucene FieldCache

2017-01-09 Thread Yago Riveiro
Ok, then I need to configure to reduce the size of the cache. Thanks for the help Mikhail. -- /Yago Riveiro On 9 Jan 2017 17:01 +, Mikhail Khludnev , wrote: > This probably says why > https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/core/SolrConfi

CloudSolrStream can't set the setZkClientTimeout and setZkConnectTimeout properties

2017-01-09 Thread Yago Riveiro
Hi, Using the CloudSolrStream, is it possible define the setZkConnectTimeout and setZkClientTimeout of internal CloudSolrClient? The default negotiation timeout is set to 10 seconds. Regards, /Yago - Best regards /Yago -- View this message in context: http://lucene.472066.n3.nabble.com

Re: Help needed in breaking large index file into smaller ones

2017-01-09 Thread Yago Riveiro
You can try to reindex your data to another collection with more shards -- /Yago Riveiro On 9 Jan 2017 14:15 +, Narsimha Reddy CHALLA , wrote: > No, it does not work by splitting. First of all lucene index files are not > text files. There is a segment_NN file which will refer index

Re: Question about Lucene FieldCache

2017-01-09 Thread Yago Riveiro
The documentation says that the only caches configurable are: - filterCache - queryResultCache - documentCache - user defined caches There is no entry for fieldValueCache and in my case all of list in the documentation are disable ... -- /Yago Riveiro On 9 Jan 2017 13:20 +, Mikhail

Re: Question about Lucene FieldCache

2017-01-09 Thread Yago Riveiro
Thanks for re reply Mikhail, Do you know if the 1 value is configurable? My insert rate is so high (5000 docs/s) that the cache it's quite useless. In the case of the Lucene field cache, it's possible "clean" it in some way? Some cache is eating my memory heap. - Best regards /Yago -

Question about Lucene FieldCache

2017-01-09 Thread Yago Riveiro
Hi, After some reading into the documentation, supposedly the Lucene FieldCache is the only one that it's not possible to disable. Fetching the config for a collection through the REST API I found an entry like this: "query": { "useFilterForSortedQuery": true, "queryResultWindowSize": 1, "queryR

Re: CloudSolrStream client doesn't validate sort order

2017-01-07 Thread Yago Riveiro
Ok, good to know :) - Best regards /Yago -- View this message in context: http://lucene.472066.n3.nabble.com/CloudSolrStream-client-doesn-t-validate-sort-order-tp4312936p4312943.html Sent from the Solr - User mailing list archive at Nabble.com.

CloudSolrStream client doesn't validate sort order

2017-01-07 Thread Yago Riveiro
Hi, The CloudSolrStream client (Solr 6.3.0) assumes that the sort param always have the order. starting in line 326: String[] sorts = sort.split(","); StreamComparator[] comps = new StreamComparator[sorts.length]; for(int i=0; ihttp://lucene.472066.n3.nabble.com/CloudSolrStream-clien

Re: Boolean type supports docValues?

2017-01-03 Thread Yago Riveiro
Reading the actual documentation is not clear ... After test it, 6.3.0 indeed have docValues support for boolean type. Thanks Erick. -- /Yago Riveiro On 3 Jan 2017 10:39 +, Yago Riveiro , wrote: > Hi, > > The boolean type has support for DocValues? the documentation says

Boolean type supports docValues?

2017-01-03 Thread Yago Riveiro
Hi, The boolean type has support for DocValues? the documentation says that only StrField, UUIDField and Trie* numeric fields have support ( doc ) but I found this Jira issue SOLR-9187

Re: Cannot talk to ZooKeeper - Updates are disabled (Solr 6.3.0)

2016-12-29 Thread Yago Riveiro
if I’m lost the Zookeeper quorum? -- /Yago Riveiro On 29 Dec 2016 16:07 +, Susheel Kumar , wrote: > I believe this comes when Zookeeper quorum is not maintained. Do not see > any way around except bringing the quorum back? > > Thanks, > Susheel > > On Thu, Dec 29,

Cannot talk to ZooKeeper - Updates are disabled (Solr 6.3.0)

2016-12-29 Thread Yago Riveiro
There is any way to recover from a exception "org.apache.solr.common.SolrException: Cannot talk to ZooKeeper - Updates are disabled" without restart the affected node node? Regards, /Yago - Best regards /Yago -- View this message in context: http://lucene.472066.n3.nabble.com/Cannot-talk-

Re: Uncaught exception java.lang.StackOverflowError in 6.3.0

2016-12-27 Thread Yago Riveiro
auto magic management stuff. You have all the power and therefore all the work :p Following your advices I will try to review the topology of my collection and try to point the oversharded collections. -- /Yago Riveiro On 27 Dec 2016 21:54 +, Toke Eskildsen , wrote: > Yago Riveiro wrote:

Re: Uncaught exception java.lang.StackOverflowError in 6.3.0

2016-12-27 Thread Yago Riveiro
t manage a big cluster without custom tools. Solr continues to improve with each version, and I saw issues with a lot of nice stuff like SOLR-9735 and SOLR-9241 -- /Yago Riveiro On 26 Dec 2016 22:10 +, Toke Eskildsen , wrote: > Yago Riveiro wrtoe: > > My cluster holds more than 10B

Bad version writing to ZK in 6.3.0

2016-12-26 Thread Yago Riveiro
Lately I can read this warning in my logs some times: Bad version writing to ZK using compare-and-set, will force refresh cluster state: KeeperErrorCode = BadVersion for /collections/X/state.json Why this happen? it's normal? -- /Yago - Best regards -- View this message in context:

Re: Uncaught exception java.lang.StackOverflowError in 6.3.0

2016-12-26 Thread Yago Riveiro
My cluster holds more than 10B documents stored in 15T. The size of my collections is variable but I have collections with 800M documents distributed over the 12 nodes, the amount of documents per shard is ~66M and indeed the performance is good. I need the collections to isolate the data of my c

ClassCastException: BasicResultContext cannot be cast to SolrDocumentList

2016-12-20 Thread Yago Riveiro
I'm hitting this exception in 6.3.0, any ideas? null:java.lang.ClassCastException: org.apache.solr.response.BasicResultContext cannot be cast to org.apache.solr.common.SolrDocumentList at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:315) at o

Re: Getting Error - Session expired for /collections/sprod/state.json

2016-12-16 Thread Yago Riveiro
Do some gc profiling to get some information about. It's possible you have configure a small heap and you are running in gc stop the world issues. Normally zookeeper erros are bounded to gc and network latency issues -- /Yago Riveiro On 16 Dec 2016, 09:49 +, Piyush Kunal , wrote: &g

Re: Getting Error - Session expired for /collections/sprod/state.json

2016-12-16 Thread Yago Riveiro
I had some of this error in my logs too on 6.3.0 My cluster also index like 20K docs/sec I don't know why. -- /Yago Riveiro On 16 Dec 2016, 08:39 +, Piyush Kunal , wrote: > Anyone has noticed such issue before? > > On Thu, Dec 15, 2016 at 4:36 PM, Piyush Kunal wrote:

Re: Uncaught exception java.lang.StackOverflowError in 6.3.0

2016-12-15 Thread Yago Riveiro
nodes in 10 minutes. I need to review the value to something more conservative maybe. -- /Yago Riveiro On 15 Dec 2016, 16:43 +, Erick Erickson , wrote: > Hmmm, have you changed coreLoadThreads? We had a problem with this a > while back with loading lots and lots of cores, see: &

Uncaught exception java.lang.StackOverflowError in 6.3.0

2016-12-15 Thread Yago Riveiro
Hi, I'm getting this error in my log 12/15/2016, 9:28:18 AM ERROR true ExecutorUtilUncaught exception java.lang.StackOverflowError thrown by thread: coreZkRegister-1-thread-48-processing-n:XXX.XXX.XXX.XXX:8983_solr x:collection1_shard3_replica2 s:shard3 c:collection1-visitors r

Zookeeper connection lost in 5.5.3

2016-11-28 Thread Yago Riveiro
Hi, I upgraded my cluster to 5.5.3 and now I'm having a lot of this warnings. Unable to read /collections/collectionX/leader_initiated_recovery/shard9/core_node12 due to: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /collections/collectionX/l

Re: How to enable JMX to monitor Jetty

2016-11-28 Thread Yago Riveiro
Hi, Rallavagu, the jetty-jmx.xml file is the basic file of the github repository or something custom? I modified the file modules/http.mod and I can't see jetty stuff ... - Best regards -- View this message in context: http://lucene.472066.n3.nabble.com/How-to-enable-JMX-to-monitor-Jetty-

Re: Load core process changed between 5.5.3 and 5.3.1

2016-11-20 Thread Yago Riveiro
Indeed in 5.3.1 the CPU spikes to 80 of load and now the cluster is more stable, slower but more stable. Thanks. \-- /Yago Riveiro ![](https://link.nylas.com/open/m7fkqw0yim04itb62itnp7r9/local- a46120b4-69a7?r=c29sci11c2VyQGx1Y2VuZS5hcGFjaGUub3Jn) On Nov 20 2016, at 4:31 pm

Load core process changed between 5.5.3 and 5.3.1

2016-11-20 Thread Yago Riveiro
Hi, I'm trying to upgrade my cluster from Solr version 5.3.1. to 5.5.3 and I noticed that the core loading process in 5.5.3 is different from 5.3.1. The number of core loaded in parallel in 5.5.3 are about 5 or 6, when in 5.3.1 all cores were published as state "recovering" all together. This i

Re: Permission error using install_solr_service script.sh

2016-10-01 Thread Yago Riveiro
And yes, I executed the script as root using the /opt folder as install folder -- /Yago Riveiro On 1 Oct 2016, 15:18 +0100, Shawn Heisey , wrote: > On 9/29/2016 3:42 AM, Yago Riveiro wrote: > > I'm having troubles to run the install_solr_service in Centos 7.2. > > &g

Re: Permission error using install_solr_service script.sh

2016-10-01 Thread Yago Riveiro
I was running the install into a vm, I deleted it and did a new provisioning. Now it woks. The weird thing is that before delete the vm I ran a chmod over that files and the install finished without that errors. -- /Yago Riveiro On 1 Oct 2016, 15:18 +0100, Shawn Heisey , wrote: > On 9

Permission error using install_solr_service script.sh

2016-09-29 Thread Yago Riveiro
Hi, I'm having troubles to run the install_solr_service in Centos 7.2. I have this errors: -bash: /opt/usr/solr/bin/solr: Permission denied -bash: /opt/usr/solr/bin/solr: Permission denied the problematic line is the line 315 on install_solr_service.sh find "$SOLR_VAR_DIR" -type f -print0 | x

Re: Whether SolrCloud can support 2 TB data?

2016-09-24 Thread Yago Riveiro
clean index (voodoo magic thing can happen, thrust me), and it will drain a huge amount of hardware resources to do it without downtime. \-- /Yago Riveiro ![](https://link.nylas.com/open/m7fkqw0yim04itb62itnp7r9/local-277ee09e- 1aee?r=c29sci11c2VyQGx1Y2VuZS5hcGFjaGUub3Jn) On Sep 24 201

Re: Whether SolrCloud can support 2 TB data?

2016-09-23 Thread Yago Riveiro
will know) -- /Yago Riveiro On 23 Sep 2016, 23:44 +0100, Pushkar Raste , wrote: > Solr is RAM hungry. Make sure that you have enough RAM to have most if the > index of a core in the RAM itself. > > You should also consider using really good SSDs. > > That would be a good start.

Re: Heap memory usage is -1 in UI

2016-09-23 Thread Yago Riveiro
onsumption in services like zabbix. -- /Yago Riveiro On 23 Sep 2016, 01:08 +0100, Shawn Heisey , wrote: > On 9/22/2016 4:59 PM, Yago Riveiro wrote: > > The Heap Memory Usage in the UI it's always -1. There is some way to > > get the amount of heap that a core consumes? >

Heap memory usage is -1 in UI

2016-09-22 Thread Yago Riveiro
The Heap Memory Usage in the UI it's always -1. There is some way to get the amount of heap that a core consumes? - Best regards -- View this message in context: http://lucene.472066.n3.nabble.com/Heap-memory-usage-is-1-in-UI-tp4297601.html Sent from the Solr - User mailing list archive at

Re: Miserable Experience Using Solr. Again.

2016-09-13 Thread Yago Riveiro
f http connection in each node, the utilization of the jetty thread pool and stuff like that. Solr is a great tool, but it's hard, too hard to get in. \-- /Yago Riveiro ![](https://link.nylas.com/open/m7fkqw0yim04itb62itnp7r9/local- 89046b47-a272?r=c29sci11c2VyQGx1Y2VuZS5hcGFjaGUub

Unable to upgrade from 5.4 to 5.5.2

2016-08-12 Thread Yago Riveiro
e way how shard synchronization is done. Can this issue be related with my problem? \-- /Yago Riveiro ![](https://link.nylas.com/open/m7fkqw0yim04itb62itnp7r9/local-2eefb82c- 7bf1?r=c29sci11c2VyQGx1Y2VuZS5hcGFjaGUub3Jn)

Re: Idle timeout expired: 50000/50000 ms

2016-07-14 Thread Yago Riveiro
Recently I started to buffering docs and send them to Solr in blocks of 250 and 50 workers. But now I'm hitting this issue too with Solr 5.3.1 Googling a bit I found this https://bugs.eclipse.org/bugs/show_bug.cgi?id=435322 that was fixes in 9.2 version. There is another link

Re: Re: Can Solr 5.5 recognize the index result generated by SolrIndexer of old version Nutch ?

2016-06-01 Thread Yago Riveiro
like Erick said, you will need to do this process in several steps before reach 5.x \-- /Yago Riveiro ![](https://link.nylas.com/open/m7fkqw0yim04itb62itnp7r9/local-1f481cc8-d5e2) On Jun 1 2016, at 5:22 pm, Erick Erickson <erickerick...@gmail.com> wrote: > https://luc

Re: Can Solr 5.5 recognize the index result generated by SolrIndexer of old version Nutch ?

2016-06-01 Thread Yago Riveiro
You need to upgrade your index to version 4.10 using the IndexUpgrade tool. -- Yago Riveiro On 1 Jun 2016 10:53 +0100, t...@sina.com, wrote: > Hi, > We plan to upgrade the solr server to 5.5.0. And we have a customized crawler > based on Nutch 1.2 and Solr 1.4.1. > > So, the q

Re: Facet by truncated date

2016-03-31 Thread Yago Riveiro
ounts? \-- /Yago Riveiro ![](https://link.nylas.com/open/m7fkqw0yim04itb62itnp7r9/eae9e3a3308049849ef01 3655c85f3ba) On Mar 31 2016, at 10:26 am, Emir Arnautovic <emir.arnauto...@sematext.com> wrote: > Hi Yago, Not sure if I misunderstood the case, but assuming you have date

Re: Facet by truncated date

2016-03-31 Thread Yago Riveiro
. date:"2016-03-31T12:00:0Z" truncated_date_s:'2016-03-31' or truncated_date_i:20160331 (this should be more memory efficient) \-- /Yago Riveiro ![](https://link.nylas.com/open/m7fkqw0yim04itb62itnp7r9/4708d221e9a24b519bab6 3936013ce59) On Mar 31 2016, at 10:08

Re: Unable to create collection in 5.5

2016-03-28 Thread Yago Riveiro
uster with thousand of collections is like no-no. Sorry but without a way to rename old collection to collection_*, the enforce to do not allow hyphens is frustrating as a user. \-- /Yago Riveiro ![](https://link.nylas.com/open/m7fkqw0yim04itb62itnp7r9/ec42b9ccdd9a4285b0ff0 8cf203af0f

Re: Unable to create collection in 5.5

2016-03-28 Thread Yago Riveiro
This kind of stuff can't be released without a way to rename the current collections with hyphens (even for 6.0) \-- /Yago Riveiro ![](https://link.nylas.com/open/m7fkqw0yim04itb62itnp7r9/d6c3ba33ed5f4ac8af3b2 9c07e2c5e91) On Mar 28 2016, at 5:19 pm, Anshum Gupta

Unable to create collection in 5.5

2016-03-28 Thread Yago Riveiro
Hi, With solr 5.5 I can't create a collection with the name collection-16, and in 5.3.1 I can do it, Why? 4001org.apache.solr.common.SolrExceptionorg.apache.solr.common.SolrExceptionInvalid name: 'collection-16' Identifiers must consist entirely of periods, underscores and alphanumerics400

Re: Next Solr Release - 5.5.1 or 6.0 ?

2016-03-24 Thread Yago Riveiro
-index can take several months). Optimize one replica at a time doesn't work (All replicas are optimize at the same time) killing CPU an IO and as result the cluster. Conclusion, if I need to do it again to upgrade to a newer version of Solr I'm in literally in troubles ... \-

Re: java.lang.NullPointerException in json facet hll function

2016-03-22 Thread Yago Riveiro
ields, isn't it? \-- /Yago Riveiro > On Mar 22 2016, at 5:00 pm, Yonik Seeley <ysee...@gmail.com> wrote: > > Hmmm, looks like the "hll" value is missing for some reason. It's not clear why that would happen... are you running any custom code? > > -Yo

java.lang.NullPointerException in json facet hll function

2016-03-22 Thread Yago Riveiro
Solr version: 5.3.1 With this query: group: { type:terms, limit:-1, field:group, sort:{index:asc}, numBuckets:true, facet:{ col_1_unique_visitors:'hll(visitor__visitor_i

Re: JSON facets, count a long or an integer in cloud and non-cloud modes

2016-03-22 Thread Yago Riveiro
I have a felling that this is related with the number of nodes of the cluster. My dev runs in cloud mode but only has one node, production has 12, and the version is the same. \-- /Yago Riveiro > On Mar 22 2016, at 9:13 am, Markus Jelsma <markus.jel...@openindex.io&

Re: JSON facets, count a long or an integer in cloud and non-cloud modes

2016-03-22 Thread Yago Riveiro
I have the same problem with a custom response writer. In production works but in my dev doesn't and are the same version 5.3.1 -- Yago Riveiro On 22 Mar 2016 08:47 +, Markus Jelsma, wrote: > Hello, > > Using SolrJ i built a method that consumes output produced by JSON face

IllegalArgumentException: Seeking to negative position

2016-03-08 Thread Yago Riveiro
I saw this exception in my log. What can caused this? java.lang.IllegalArgumentException: Seeking to negative position: MMapIndexInput(path="/opt/solr/node/collections/2016_shard9_replica2/data/index/_0.fdx") at org.apache.lucene.store.ByteBufferIndexInput$SingleBufferImpl.seek(ByteBufferI

How can I monitor the jetty thread pool

2016-03-07 Thread Yago Riveiro
Hi, How can I monitor the jetty thread pool? I want to do a zabbix graph with this info but the JMX doesn't show any entry for this. - Best regards -- View this message in context: http://lucene.472066.n3.nabble.com/How-can-I-monitor-the-jetty-thread-pool-tp4262298.html Sent from the Solr

Re: Bulk delete of Solr documents

2016-02-08 Thread Yago Riveiro
Yes. You can delete using a query http://blog.dileno.com/archive/201106/delete-documents-from-solr-index-by- query/ \-- /Yago Riveiro > On Feb 8 2016, at 4:35 pm, Anil <anilk...@gmail.com> wrote: > > Hi , > > Can we delete solr documents from a

Solr Replication error

2016-01-24 Thread Yago Riveiro
I cached this in my logs. Any reason to this happen? My Solr version is 5.3.1. Index fetch failed :org.apache.solr.common.SolrException: Index fetch failed : at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:515) at org.apache.solr.handler.IndexFetcher.fe

Re: Scaling SolrCloud

2016-01-21 Thread Yago Riveiro
Is not a typo. I was wrong, for zookeeper 2 nodes still count as majority. It's not the desirable configuration but is tolerable. Thanks Erick. \-- /Yago Riveiro > On Jan 21 2016, at 4:15 am, Erick Erickson <erickerick...@gmail.com> wrote: > > bq: 3 are to ris

Re: Scaling SolrCloud

2016-01-20 Thread Yago Riveiro
d with replicas with 100G take an hour or more (normally when this happen, we disable indexing to release IO and CPU and not kill the node with a load of 50 or 60). In this department my advice is "keep it simple" in the end is an HTTP POST to a node of the cluster. \-- /Yago Ri

Re: Scaling SolrCloud

2016-01-19 Thread Yago Riveiro
ull-recover replicas that in some point are out of sync, happen a lot restarting nodes (this is annoying with replicas with 100G), don't underestimate this point. Free space can save your life. \-- /Yago Riveiro > On Jan 19 2016, at 11:26 pm, Shawn Heisey <apa...@elyograg.org> w

Re: Json facet api method stream

2015-12-22 Thread Yago Riveiro
ceived % Xferd  Average Speed   Time    Time     Time  Current                                  Dload  Upload   Total   Spent    Left  Speed 100 90.7M    0 90.7M    0     0  9276k      0 --:--:--  0:00:10 --:--:-- 22.6M real0m10.026s user0m0.038s sys 0m0.245s [yago@dev-1 ~]$ diff dum

Re: Json facet api method stream

2015-12-22 Thread Yago Riveiro
url with the unique users and the average response time (minutes is acceptable, but less as possible is desirable) —/Yago Riveiro On Tue, Dec 22, 2015 at 3:27 PM, Yonik Seeley wrote: > On Tue, Dec 22, 2015 at 6:06 AM, Yago Riveiro wrote: >> I’m surprised with the difference of spe

Re: Json facet query error "null:java.lang.IllegalArgumentException"

2015-12-22 Thread Yago Riveiro
I’m in 5.3.1. I’m waiting some time to upgrade to 5.4 to see if some nasty bug is reported. But after hitting this issue I think that I should upgrade ... —/Yago Riveiro On Tue, Dec 22, 2015 at 3:17 PM, Yonik Seeley wrote: > OK found the issue: > https://issues.apache.org/jira/

Json facet query error "null:java.lang.IllegalArgumentException"

2015-12-22 Thread Yago Riveiro
Hi, I'm hitting an error when a try to run a json facet query in a node that doesn't have any shard that belongs to collection. The same query using using the legacy facet method works. http://devel-16:8983/solr/collection-perf/query?rows=0&q=*:*&json.facet={label:{type:terms,field:url,limit:-1,s

Indexing using a collection alias

2015-12-22 Thread Yago Riveiro
Hi, It's possible index documents using the alias and not the collection name, if the alias only point to one collection? The Solr collection API doesn't allow rename a collection, so I wan't to know if with aliases I can achieve this functionality. All documentation that I googled use the alias

Re: Json facet api method stream

2015-12-22 Thread Yago Riveiro
Ok, I’m surprised with the difference of speed between DV and stream, the same query (aggregate 7M unique keys) with stream method takes 21s and with DV is about 3 minutes ...  —/Yago Riveiro On Tue, Dec 22, 2015 at 1:46 AM, Yonik Seeley wrote: > On Mon, Dec 21, 2015 at 6:56 PM, Y

Json facet api method stream

2015-12-21 Thread Yago Riveiro
Hi, The json facet API method "stream" uses the docvalues internally for do the aggregation on the fly? I wan't to know if using this method justifies have the docvalues configured in schema. - Best regards -- View this message in context: http://lucene.472066.n3.nabble.com/Json-facet-api

Re: Nested document query with wrong numFound value

2015-12-11 Thread Yago Riveiro
lways  31814269823181426982280 and 31814269823181426982281 as a children of   3181426982318142698228   Can this issue be related to SOLR-5211?. —/Yago Riveiro On Fri, Dec 11, 2015 at 8:46 PM, Mikhail Khludnev wrote: > On Fri, Dec 11, 2015 at 11:05 PM, Yago Riveiro > wrote: >> When do yo

Re: Nested document query with wrong numFound value

2015-12-11 Thread Yago Riveiro
2280" } ] } responseHeader:  { status: 0, QTime: 3, params:  { q: "id:31814269823181426982281", fl: "id", q.op: "AND" } }, response:  { numFound: 1, start: 0, maxScore: 9.919363, docs:  [ { id: "31814269823181426982281&q

Re: How Json facet API works with domains and facet functions?

2015-12-11 Thread Yago Riveiro
ry and filters. —/Yago Riveiro On Fri, Dec 11, 2015 at 5:17 PM, Yonik Seeley wrote: > If you search on the parents and want to match child documents, I > think you want {!child} and not {!parent} in your queries or filters. > fq={!child of=...}date_query_on_parents > fq=child_prop:X >

How Json facet API works with domains and facet functions?

2015-12-11 Thread Yago Riveiro
Hi, How the json facet api works with domains and facet functions? I try to google some info and I do not find nothing useful. How can do a query that find all parents that match a clause (a date) and calculate the avg price of all of children that have property X? Following yonik's blog exampl

Re: Schema API, change the defaultoperator

2015-12-11 Thread Yago Riveiro
t handlers works with POST http calls? On Fri, Dec 11, 2015 at 2:26 PM, Shawn Heisey wrote: > On 12/11/2015 4:23 AM, Yago Riveiro wrote: >> How can I change the defaultoperator parameter through the schema API? > The default operator and default field settings in the schema have been

Re: Nested document query with wrong numFound value

2015-12-11 Thread Yago Riveiro
}, expand: { time: 0 }, debug: { time: 3 } } }, rawquerystring: "id:318142698

Schema API, change the defaultoperator

2015-12-11 Thread Yago Riveiro
Hi, How can I change the defaultoperator parameter through the schema API? Thanks. - Best regards -- View this message in context: http://lucene.472066.n3.nabble.com/Schema-API-change-the-defaultoperator-tp4244857.html Sent from the Solr - User mailing list archive at Nabble.com.

Nested document query with wrong numFound value

2015-12-11 Thread Yago Riveiro
Hi, I'm playing with the nested documents feature and after run this query: http://localhost:8983/solr/ecommerce-15/query?q=id:3181426982318142698228* The documents has the IDs: - Parent : 3181426982318142698228 - Child_1 : 31814269823181426982280 - Child_2 : 31814269823181426982281 I have t

Re: Json facet api NullPointerException

2015-11-12 Thread Yago Riveiro
I found the bug … In my query I have  sort: index, And should be sort:{index:desc|asc} I think that the json parser should raise a “json parsing error” ... —/Yago Riveiro On Thu, Nov 12, 2015 at 4:44 PM, Yago Riveiro wrote: > Solr 5.3.1 > —/Yago Riveiro > On Th

Re: Json facet api NullPointerException

2015-11-12 Thread Yago Riveiro
Solr 5.3.1 —/Yago Riveiro On Thu, Nov 12, 2015 at 4:21 PM, Yonik Seeley wrote: > Thanks for the report Yago, > What version is this? > -Yonik > On Thu, Nov 12, 2015 at 10:53 AM, Yago Riveiro wrote: >> Hi, >> >> I'm hitting this NullPointerException using t

Json facet api NullPointerException

2015-11-12 Thread Yago Riveiro
Hi, I'm hitting this NullPointerException using the json facet API. Same query using Facet component is working. Json facet query: curl -s http://node1:8983/solr/metrics/query -d 'q=datetime:[2015-10-01T00:00:00Z TO 2015-10-04T23:59:59Z]&rows=0&json.facet={ urls: { type: terms,

  1   2   3   >