Our write and read are 50/50. We need to bulk load file system metadata into 
Atlas and  tag department by department, then search.

Feng

From: Jiaxin Ye <[email protected]>
Sent: Sunday, October 18, 2020 1:47 AM
To: [email protected]
Subject: Re: Multiple Atlas instance in on Hadoop cluster

Is your case read heavy or write heavy? I was able to change the code to make 
it read-only on multiple instances on the cluster. I haven’t tried to make 
write on multiple instances though. I did try to make the Kafka topic and 
consumer multi-partitioned to scale up the write with some assumptions of our 
Hive system.

Thanks,
Jiaxin

On Sat, Oct 17, 2020 at 07:36 XU Feng 
<[email protected]<mailto:[email protected]>> wrote:
Yes Dennis, we can scale HBase and Solr out, but I am sure if it’s good enough. 
The REST service and Janus graph still run on single active node.

Thanks,

Feng Xu

From: Dennis Suhari <[email protected]<mailto:[email protected]>>
Sent: Friday, October 16, 2020 6:08 AM
To: [email protected]<mailto:[email protected]>
Subject: Re: Multiple Atlas instance in on Hadoop cluster

Hi Feng,

but you can „scale out“ by extending HBASE cluster e.g. if you use this as 
storage.

Br,

Dennis
Von meinem iPhone gesendet

Am 14.10.2020 um 20:06 schrieb XU Feng 
<[email protected]<mailto:[email protected]>>:

I am new to Apache Atlas, I know Atlas HA cannot scale out and this is a big 
limitation for my use cases. I was wondering if I could configure multiple 
Atlas HA instances in one Hadoop cluster to work around.

Thank you!

Feng

Reply via email to