Re: CDCR Alias support?

2017-05-09 Thread Webster Homer
Still no answer to this. I've been investigating using the collections API for backup and restore. If CDCR supports collection aliases this would make things much smoother as we would restore to a new collection and then switch the alias to reference the new collection. On Tue, Jan 10, 2017 at 10:

Re: CDCR with SSL enabled

2017-05-01 Thread Xie, Sean
From the QUEUE action, the output is: 0 0 34741356 2 stopped On 5/2/17, 1:43 AM, "Xie, Sean" wrote: Does CDCR support SSL encrypted SolrCloud? I have two clusters started with SSL, and CDCR setup instruction is followed on source and target. However, from the solr.log, I’m n

Re: CDCR & firewall holes

2017-05-01 Thread Susheel Kumar
I believe you need to open a) ports from source cluster to target zookeepers (usually 2181 unless you change it) b) ports from source to target solr ports (usually 8983 unless you change it) Thanks, Susheel On Mon, May 1, 2017 at 2:17 PM, Oakley, Craig (NIH/NLM/NCBI) [C] < craig.oak...@nih.gov>

Re: CDCR support with solr version 4.10.3

2017-03-15 Thread Erick Erickson
The first version of CDCR is Solr 6.0. CDCR over HDFS is also not supported currently, see SOLR-9861. Best, Erick On Wed, Mar 15, 2017 at 6:37 AM, Ganesh Kumar J wrote: > Hi Solr Team, > > > We are using apache solr 4.10.3 version along with CDH > 5.4.8. We like to know that

Re: CDCR logging is Needlessly verbose, fills up the file system fast

2017-01-10 Thread Webster Homer
I know that we have never set the schedule parameter to 1 millisecond. We have specified either 100 or 1000. I wondered why it was writing so frequently. I suspect a bug somewhere However, we will have multiple collections using cdcr, and in some cases the source collection will have multiple targ

Re: CDCR logging is Needlessly verbose, fills up the file system fast

2017-01-09 Thread Shawn Heisey
On 12/22/2016 8:10 AM, Webster Homer wrote: > While testing CDCR I found that it is writing tons of log messages per > second. Example: > 2016-12-21 23:24:41.652 INFO (qtp110456297-13) [c:sial-catalog-material > s:shard1 r:core_node1 x:sial-catalog-material_shard1_replica1] > o.a.s.c.S.Request [si

Re: CDCR How to recover from Corrupted transaction log

2017-01-09 Thread Webster Homer
The root cause was the aggressive logging filling up the file system. Our admins have the logs on the same file system with the data, so when the filesystem got full it couldn't write to the transaction logs which corrupted them Thank you for the tips on recovery, I will forward them to our admins

Re: CDCR How to recover from Corrupted transaction log

2017-01-06 Thread Shawn Heisey
On 1/6/2017 10:09 AM, Webster Homer wrote: > This happened while testing and was not in a production system. So we > just deleted both collections and recreated them after fixing the root > cause. If this had been a production system that would not have been > acceptable. What is the best way to re

Re: CDCR How to recover from Corrupted transaction log

2017-01-06 Thread Erick Erickson
If you just deleted the tlog files on the source, you'd likely miss updates, right? I can think of two ways to get source and target back in sync: 1> go ahead and delete the tlogs, then re-index from some point guaranteed to have been propagated to the target _before_ the tlog went wonky. 2> rebu

Re: CDCR logging is Needlessly verbose, fills up the file system fast

2017-01-06 Thread Shawn Heisey
On 1/6/2017 8:21 AM, Webster Homer wrote: > I figured our problem with the filesystem, by default the root logger > is configured with the CONSOLE logger, which is NOT rotated and > eventually filled up the file system. That doesn't exonerate the CDCR > logging problem though. The thing writes a hu

Re: CDCR logging is Needlessly verbose, fills up the file system fast

2017-01-06 Thread Webster Homer
I figured our problem with the filesystem, by default the root logger is configured with the CONSOLE logger, which is NOT rotated and eventually filled up the file system. That doesn't exonerate the CDCR logging problem though. The thing writes a huge amount of junk to the logs, information that lo

Re: CDCR logging is Needlessly verbose, fills up the file system fast

2017-01-03 Thread Shawn Heisey
On 1/3/2017 1:12 PM, Webster Homer wrote: > We use the default log4j.properties file which rolls the log file to > solr.log.1, solr.log.2 ... which isn't really the problem. What is > also happening is that solr.log.1 gets renamed to > solr_log_20170103_1110 with a timestamp as the file name. How d

Re: CDCR logging is Needlessly verbose, fills up the file system fast

2017-01-03 Thread Webster Homer
So since the logging messages are not from an actual class, how do I suppress the CDCR INFO messages? I really do want normal INFO messages but they get swamped by CDCR. Even when CDCR issues a WARN message, it issues thousands of them. We use the default log4j.properties file which rolls the log

Re: CDCR logging is Needlessly verbose, fills up the file system fast

2017-01-03 Thread Alan Woodward
It’s org.apache.solr.core.SolrCore.Request - not an actual class. Alan Woodward www.flax.co.uk > On 3 Jan 2017, at 16:08, Webster Homer wrote: > > I am working on changing the log rotation, but looking at the message: > > 2016-12-21 23:24:41.653 INFO (qtp110456297-18) [c:sial-catalog-materia

Re: CDCR logging is Needlessly verbose, fills up the file system fast

2017-01-03 Thread Webster Homer
I am working on changing the log rotation, but looking at the message: 2016-12-21 23:24:41.653 INFO (qtp110456297-18) [c:sial-catalog-material s:shard1 r:core_node1 x:sial-catalog-material_shard1_replica1] o.a.s.c.S.Request [sial-catalog-material_shard1_replica1] webapp=/solr path=/cdcr params={

Re: CDCR logging is Needlessly verbose, fills up the file system fast

2016-12-29 Thread Erick Erickson
Seems like a bandaid would be to insure your Solr logs rotate appropriately quickly. That doesn't address the CDCR loging verbosity, but it might get you by. You can also change the logging at the class level by appropriately editing the log4j properties file. Again perhaps not the best solution

Re: CDCR logging is Needlessly verbose, fills up the file system fast

2016-12-29 Thread Webster Homer
The logs filled up the file system and caused CDCR to fail due to a corrupted Tlog file. On Thu, Dec 22, 2016 at 9:10 AM, Webster Homer wrote: > While testing CDCR I found that it is writing tons of log messages per > second. Example: > 2016-12-21 23:24:41.652 INFO (qtp110456297-13) [c:sial-cat

Re: CDCR: Help With Tlog Growth Issues

2016-12-02 Thread Renaud Delbru
Hi Shalin, when the buffer is enabled, tlogs are not removed anymore, even if they were replicated [1]: "When buffering updates, the updates log will store all the updates indefinitely. " Once you disable the buffer, all the old tlogs should be cleaned (the next time the tlog cleaning proces

Re: CDCR: Help With Tlog Growth Issues

2016-12-01 Thread Shalin Shekhar Mangar
Even if buffer is enabled, the old tlogs should be remove once the updates in those tlogs have been replicated to the target. So the real question is why they haven't been removed automatically? On Thu, Dec 1, 2016 at 9:13 PM, Renaud Delbru wrote: > Hi Thomas, > > Looks like the buffer is enabled

Re: CDCR: Help With Tlog Growth Issues

2016-12-01 Thread Renaud Delbru
Hi Thomas, Looks like the buffer is enabled on the update log, and even if the updates were replicated, they are not removed. What is the output of the command `cdcr?action=STATUS` on both cluster ? If you see in the response `enabled`, then the buffer is enabled. To disable it, you should

Re: CDCR (Solr6.x) does not start

2016-11-09 Thread neerajbhatt
Hi Uwe I am facing the same error as you , I am getting error 6538 ERROR (qtp110456297-20) [c:multi_dc_poc s:shard1 r:core_node3 x:multi_dc_poc_shard1_replica2] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Action LASTPROCESSEDVERSION sent to non-leader replica at org.ap

Re: CDCR (Solr6.x) does not start

2016-07-08 Thread Uwe Reh
Hi Renaud, thank you for your response. You asked for some further information: 1. Log messages at the source cluster: As mentioned in my addendum "CDCR (Solr6.x) does not start (logfile)". I changed the log level for all Handlers to TRACE and I got three Messages for each shard caused by "Ac

Re: CDCR (Solr6.x) does not start

2016-07-05 Thread Renaud Delbru
Hi Uwe, At first look, your configuration seems correct, see my comments below. On 28/06/16 15:36, Uwe Reh wrote: 9. Start CDCR http://SOURCE:s_port/solr/scoll/cdcr?action=start&wt=json {"responseHeader":{"status":0,"QTime":13},"status":["process","started","buffer","enabled"]} ! (not even a

Re: CDCR (Solr6.x) does not start (logfile)

2016-06-29 Thread Uwe Reh
Hi, trying to get more information, I restarted the SOURCE node and watched the log. For each shard i got following triple: WARN org.apache.solr.handler.CdcrRequestHandler - Action LASTPROCESSEDVERSION sent to non-leader replica @ scoll:shard1 ERROR org.apache.solr.handler.RequestHandlerBa

<    1   2