[ https://issues.apache.org/jira/browse/ATLAS-3396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16926904#comment-16926904 ]
Ashutosh Mestry commented on ATLAS-3396: ---------------------------------------- Master PC build: [https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/1399/] > ZipSource: Usage of In-memory Maps Limits the Size of Imports Performed > ----------------------------------------------------------------------- > > Key: ATLAS-3396 > URL: https://issues.apache.org/jira/browse/ATLAS-3396 > Project: Atlas > Issue Type: Bug > Components: atlas-core > Affects Versions: 0.8.4 > Reporter: Ashutosh Mestry > Assignee: Ashutosh Mestry > Priority: Major > Fix For: 0.8.4, trunk > > Attachments: > ATLAS-3396-ZipSource-Memory-usage-improvement-for-la-branch-0.8.patch, > ATLAS-3396-ZipSourceWithBackingDirectory-Implementat-master.patch > > > *Background* > _ZipSource_ is the container for working with Atlas-exported ZIP files. The > current implementation fetches the contents of the entire ZIP file in memory. > This greatly limits the size of ZIP files that can be imported. > *Solution Guidance* > New class _ZipSourceWIthBackingDirectory_, this will use a user-specified (or > default) temporary directory to preserve the contents of the ZIP file. This > will reduce the memory burden and large ZIP files can be imported. > Minor refactoring to _ImportService_ will be necessary to support the new > variation of _EntityImportStream._ > -- This message was sent by Atlassian Jira (v8.3.2#803003)