I am using LOAD CSV to update nodes of an existing label. The label contains 4M existing nodes. The CSV file contains 600k rows; some new, some updates.
My Centos 6.x box has 36G of ram. Using the 3.0.1 tuning guide, I have set pagecache to 24G and heap_initial_size/heap_max_size to 8G. Attached is my cql file. When executing this cql file neo4j consumes heap space until space is exhausted. The query plan (attached) appears to indicate there are no issues so I would expect period commits to re-use heap. I have adjusted pagecache and heap sizes up and down but every load attempts fails when heap is exhausted. Any ideas or thoughts? -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
USING PERIODIC COMMIT 100 LOAD CSV WITH HEADERS FROM 'file:/export/warehouse/DW/accounts_XX_08-23-2016_03-03.txt' as row FIELDTERMINATOR '\t' merge (x:XXaccounts {AccountsID:row.AccountsID}) on create set x+=row on match set x=row;
Compiler CYPHER 3.0 Planner COST Runtime INTERPRETED +---------------------------------+----------------+------+---------+-----------+----------------------------+ | Operator | Estimated Rows | Rows | DB Hits | Variables | Other | +---------------------------------+----------------+------+---------+-----------+----------------------------+ | +ProduceResults | 1 | 0 | 0 | | | | | +----------------+------+---------+-----------+----------------------------+ | +EmptyResult | | 0 | 0 | | | | | +----------------+------+---------+-----------+----------------------------+ | +Apply | 1 | 0 | 0 | row, x | | | |\ +----------------+------+---------+-----------+----------------------------+ | | +AntiConditionalApply | 1 | 0 | 0 | row, x | | | | |\ +----------------+------+---------+-----------+----------------------------+ | | | +SetNodePropertyFromMap | 1 | 0 | 0 | row, x | | | | | | +----------------+------+---------+-----------+----------------------------+ | | | +MergeCreateNode | 1 | 0 | 0 | x -- row | | | | | | +----------------+------+---------+-----------+----------------------------+ | | | +Argument | 1 | 0 | 0 | row | | | | | +----------------+------+---------+-----------+----------------------------+ | | +ConditionalApply | 1 | 0 | 0 | row, x | | | | |\ +----------------+------+---------+-----------+----------------------------+ | | | +SetNodePropertyFromMap | 1 | 0 | 0 | row, x | | | | | | +----------------+------+---------+-----------+----------------------------+ | | | +Argument | 1 | 0 | 0 | row -- x | | | | | +----------------+------+---------+-----------+----------------------------+ | | +Optional | 1 | 0 | 0 | x | | | | | +----------------+------+---------+-----------+----------------------------+ | | +NodeUniqueIndexSeek(Locking) | 1 | 0 | 0 | x | :XXaccounts(AccountsID) | | | +----------------+------+---------+-----------+----------------------------+ | +Limit | 0 | 0 | 0 | row | Literal(0) | | | +----------------+------+---------+-----------+----------------------------+ | +LoadCSV | 1 | 1 | 0 | row | | +---------------------------------+----------------+------+---------+-----------+----------------------------+ Total database accesses: 0