Hi Dominik, Yes, the export/import job takes longer time if you have many entities and they are connected each other. We do take up backups at database level as import/export is not going to help.
--- Regards, Sreeni From: Dominik Choma <[email protected]> Reply to: "[email protected]" <[email protected]> Date: Thursday, 12 November 2020 at 2:31 PM To: "[email protected]" <[email protected]> Subject: rate of data export from atlas This email is from an external sender. Hi all, we are wondering how we can speed up the export via atlas export api, in our test case we exported 1 600 000 objects (only part of our data) in 8h , it gives about 55.5 objects per second. Some time ago in Subject Atlas import taking huge amount of time Sreenivasulu Nallapati wrote: "We have around 10000 entities in the exported zip file. The export is taking around 2-3 mins." roughly calculating this also gives about 55.5 objects per second. for export request we use such parameters {"itemsToExport": [ { "typeName": "hive_db", "uniqueAttributes": { "name":"database_name" } }, ], "options {"fetchType":"CONNECTED","matchType": "startsWith"}} For storage backend we use an external hbase cluster whose performance should not be a issue and the CPU utilization on atlas instance during export is about 30%. I assume we could try with database level backups, but first we wanted to give import/export api a chance. Do you know any ways to speed up the process?
