Hey Matthew,
Unfortunately, our shard leaders are across multiple nodes thus a single
EBS couldn't work. Did you manage to get around this issue yourself?
Regards,
Ash
On Tue, Aug 11, 2020 at 9:00 PM matthew sporleder
wrote:
> I can already tell you it is EFS that is slow. I had to switch to
I can already tell you it is EFS that is slow. I had to switch to an ebs disk
for backups on a different project because efs couldn't keep up.
> On Aug 10, 2020, at 9:43 PM, Ashwin Ramesh wrote:
>
> Hey Aroop, the general process for our backup is:
> - Connect all machines to an EFS drive
> We have 16 shards each approx 30GB - total is ~480GB. I'm also pretty sure
> it's a network issue. Very interesting that you can index 20x the data in
> 15 min!
Not index but backup an index in 15min.
>>> It would also help to ensure your overseer is on a node with a role that
> exempts it
Hi Aroop,
We have 16 shards each approx 30GB - total is ~480GB. I'm also pretty sure
it's a network issue. Very interesting that you can index 20x the data in
15 min!
>> It would also help to ensure your overseer is on a node with a role that
exempts it from any Solr index responsibilities.
How
Hi Ashwin
Thanks for sharing this detail.
Do you mind sharing how big are each of these indices ?
I am almost sure this is network capacity and constraints related per your aws
setup.
Yes if you can confirm that the backup is complete, or you just want the system
to move on discarding the
Hey Aroop, the general process for our backup is:
- Connect all machines to an EFS drive (AWS's NFS service)
- Call the collections API to backup into EFS
- ZIP the directory once the backup is completed
- Copy the ZIP into an s3 bucket
I'll probably have to see which part of the process is the
12 hours is extreme, we take backups of 10TB worth of indexes in 15 mins using
the collection backup api.
How are you taking the backup?
Do you actually see any backup progress or u are just seeing the task in the
overseer queue linger ?
I have seen restore tasks hanging in the queue forever
Hi everybody,
We are using solr 7.6 (SolrCloud). We notices that when the backup is
running, we cannot add any replicas to the collection. By the looks of it,
the job to add the replica is put into the Overseer queue, but it is not
being processed. Is this expected? And are there any workarounds?
