As the tx log is not forced to disk you might loose a few seconds of inserted data when it crashes
But the data on disk is still consistent Sent from mobile device Am 06.02.2014 um 20:06 schrieb Debajyoti Roy <[email protected]>: > ((GraphDatabaseAPI)neo4jGraph).tx().unforced().begin() is awesome but are > there any down sides to doing this? > > > > > On Tuesday, February 4, 2014 3:38:13 AM UTC-5, Michael Hunger wrote: >> >> No, it just indicates that the API might change in the future. >> >> Am 04.02.2014 um 09:16 schrieb Sotiris Beis <[email protected]>: >> >>> I don't have blocks of data, I measure the insertion time of 1,000 nodes >>> and their edges (which I call a block). I am doing that because I want to >>> simulate the creation of a graph by single element insertion. >>> >>> This >>> Transaction tx = ((GraphDatabaseAPI)neo4jGraph).tx().unforced().begin(); >>> does the job, but the tx() function says it's deprecated. Is this going to >>> be a problem? >>> >>> Sotiris >>> >>> >>> On 02/03/2014 06:41 PM, Michael Hunger wrote: >>>> Why do you do individual inserts when you have blocks of data? >>>> >>>> You can often aggregate events on the application level to be inserted as >>>> a bigger batch. >>>> see: http://maxdemarzi.com/2013/09/05/scaling-writes/ >>>> >>>> Otherwise you can also release the force-write-to-log constraint and use >>>> >>>> Transaction tx = ((GraphDatabaseAPI)neo4jGraph).tx().unforced().begin(); >>>> >>>> Instead of neo4jGraph.beginTx(); >>>> >>>> >>>> Am 03.02.2014 um 15:40 schrieb Sotiris Beis <[email protected]>: >>>> >>>>> In addition to my previous question is there any suggestion to improve >>>>> the performance of the following code. I use this to simulate the needed >>>>> time to create a graph if the graph is created by single insertions >>>>> (incrementally not batch). So I commit every transaction and I measure >>>>> the time that a block needs to be inserted. A block consists of a 1,000 >>>>> nodes and their edges. Here is the code: >>>>> >>>>> import java.io.BufferedReader; >>>>> import java.io.FileInputStream; >>>>> import java.io.IOException; >>>>> import java.io.InputStreamReader; >>>>> import java.util.ArrayList; >>>>> import java.util.List; >>>>> >>>>> import org.neo4j.graphdb.GraphDatabaseService; >>>>> import org.neo4j.graphdb.Node; >>>>> import org.neo4j.graphdb.RelationshipType; >>>>> import org.neo4j.graphdb.Transaction; >>>>> import org.neo4j.graphdb.factory.GraphDatabaseFactory; >>>>> import org.neo4j.graphdb.factory.GraphDatabaseSetting; >>>>> import org.neo4j.graphdb.index.Index; >>>>> >>>>> public class Neo4jSingleInsertion implements Insertion { >>>>> >>>>> public static String INSERTION_TIMES_OUTPUT_PATH = >>>>> "data/neo4j.insertion.times"; >>>>> >>>>> private static int count; >>>>> >>>>> private GraphDatabaseService neo4jGraph = null; >>>>> private Index<Node> nodeIndex = null; >>>>> >>>>> private static enum RelTypes implements RelationshipType { >>>>> SIMILAR >>>>> } >>>>> >>>>> public static void main(String args[]) { >>>>> Neo4jSingleInsertion test = new Neo4jSingleInsertion(); >>>>> test.startup("data/neo4j"); >>>>> test.createGraph("data/enronEdges.txt"); >>>>> test.shutdown(); >>>>> } >>>>> >>>>> public void startup(String neo4jDBDir) { >>>>> System.out.println("The Neo4j database is now starting . . . ."); >>>>> neo4jGraph = new >>>>> GraphDatabaseFactory().newEmbeddedDatabase(neo4jDBDir); >>>>> nodeIndex = neo4jGraph.index().forNodes("nodes"); >>>>> } >>>>> >>>>> public void shutdown() { >>>>> System.out.println("The Neo4j database is now shuting down . . . >>>>> ."); >>>>> if(neo4jGraph != null) { >>>>> neo4jGraph.shutdown(); >>>>> nodeIndex = null; >>>>> } >>>>> } >>>>> >>>>> public void createGraph(String datasetDir) { >>>>> count++; >>>>> System.out.println("Incrementally creating the Neo4j database . . >>>>> . ."); >>>>> List<Double> insertionTimes = new ArrayList<Double>(); >>>>> try { >>>>> BufferedReader reader = new >>>>> BufferedReader(new InputStreamReader(new FileInputStream(datasetDir))); >>>>> String line; >>>>> int nodesCounter = 0; >>>>> int lineCounter = 1; >>>>> Transaction tx = null; >>>>> long start = System.currentTimeMillis(); >>>>> long duration; >>>>> while((line = reader.readLine()) != null) { >>>>> if(lineCounter > 4) { >>>>> String[] parts = line.split("\t"); >>>>> >>>>> Node srcNode = nodeIndex.get("nodeId", >>>>> parts[0]).getSingle(); >>>>> if(srcNode == null) { >>>>> tx = neo4jGraph.beginTx(); >>>>> srcNode = neo4jGraph.createNode(); >>>>> srcNode.setProperty("nodeId", parts[0]); >>>>> nodeIndex.add(srcNode, "nodeId", parts[0]); >>>>> tx.success(); >>>>> tx.finish(); >>>>> nodesCounter++; >>>>> } >>>>> >>>>> if(nodesCounter == 1000) { >>>>> duration = System.currentTimeMillis() >>>>> - start; >>>>> insertionTimes.add((double) duration); >>>>> nodesCounter = 0; >>>>> start = System.currentTimeMillis(); >>>>> } >>>>> >>>>> Node dstNode = nodeIndex.get("nodeId", >>>>> parts[1]).getSingle(); >>>>> if(dstNode == null) { >>>>> tx = neo4jGraph.beginTx(); >>>>> dstNode = neo4jGraph.createNode(); >>>>> dstNode.setProperty("nodeId", parts[1]); >>>>> nodeIndex.add(dstNode, "nodeId", parts[1]); >>>>> tx.success(); >>>>> tx.finish(); >>>>> nodesCounter++; >>>>> } >>>>> >>>>> tx = neo4jGraph.beginTx(); >>>>> srcNode.createRelationshipTo(dstNode, >>>>> RelTypes.SIMILAR); >>>>> tx.success(); >>>>> tx.finish(); >>>>> >>>>> if(nodesCounter == 1000) { >>>>> duration = System.currentTimeMillis() >>>>> - start; >>>>> insertionTimes.add((double) duration); >>>>> nodesCounter = 0; >>>>> start = System.currentTimeMillis(); >>>>> } >>>>> } >>>>> lineCounter++; >>>>> } >>>>> duration = System.currentTimeMillis() - >>>>> start; >>>>> insertionTimes.add((double) duration); >>>>> reader.close(); >>>>> } >>>>> catch (IOException e) { >>>>> e.printStackTrace(); >>>>> } >>>>> Utils utils = new Utils(); >>>>> utils.writeTimes(insertionTimes, >>>>> Neo4jSingleInsertion.INSERTION_TIMES_OUTPUT_PATH+"."+count); >>>>> } >>>>> >>>>> } >>>>> >>>>> Thanks, >>>>> Sotiris >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google Groups >>>>> "Neo4j" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send an >>>>> email to [email protected]. >>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>> >>>> -- >>>> You received this message because you are subscribed to a topic in the >>>> Google Groups "Neo4j" group. >>>> To unsubscribe from this topic, visit >>>> https://groups.google.com/d/topic/neo4j/uZqRgCBc9lg/unsubscribe. >>>> To unsubscribe from this group and all its topics, send an email to >>>> [email protected]. >>>> For more options, visit https://groups.google.com/groups/opt_out. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Neo4j" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >>> For more options, visit https://groups.google.com/groups/opt_out. > > -- > You received this message because you are subscribed to the Google Groups > "Neo4j" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
