Hi all,
we are wondering how we can speed up the export via atlas export api, in
our test case we exported 1 600 000 objects (only part of our data) in 8h ,
it gives about 55.5 objects per second.
Some time ago in Subject Atlas import taking huge amount of time
Sreenivasulu Nallapati wrote: *"**We have around 10000 entities in the
exported zip file. The export is taking **around 2-3 mins." *roughly
calculating this also gives about 55.5 objects per second.
for export request we use such parameters
{"itemsToExport": [ { "typeName": "hive_db", "uniqueAttributes": {
"name":"database_name" } }, ], "options
{"fetchType":"CONNECTED","matchType": "startsWith"}}
For storage backend we use an external hbase cluster whose performance
should not be a issue and the CPU utilization on atlas instance during
export is about 30%.
I assume we could try with database level backups, but first we wanted to
give import/export api a chance. Do you know any ways to speed up the
process?
