Re: [orientdb] ETL for professional import to OrientDB
This is my code (I didn't use actually Book and Author tables, I used my
domain specific tables, but here I illustrated problem with these common
names to make it easier to understand).
This works, but my question is could this be done using just ETL without
any coding.
MapInteger, Vertex bookMap = new HashMap();
String queryString = SELECT * FROM Books;
Statement statement = connection.createStatement();
ResultSet rs = statement.executeQuery(queryString);
while (rs.next()) {
Vertex bookVertex = graph.addVertex(class:Book);
Integer bookId = rs.getInt(1);
bookVertex.setProperty(...);
// fill vertex from result set
bookMap.put(bookId, bookVertex);
}
rs.close();
statement.close();
MapInteger, Vertex authorMap = new HashMap();
queryString = SELECT * FROM Authors;
statement = connection.createStatement();
rs = statement.executeQuery(queryString);
while (rs.next()) {
Vertex authorVertex = graph.addVertex(class:Author);
Integer authorId = rs.getInt(1);
authorVertex.setProperty(...);
// fill vertex from result set
authorMap.put(authorId, authorVertex);
}
rs.close();
statement.close();
graph.commit();
queryString = SELECT * FROM BookAuthor;
statement = connection.createStatement();
rs = statement.executeQuery(queryString);
while (rs.next()) {
Integer bookId = rs.getInt(2);
Integer authorId = rs.getInt(2);
Edge edge = graph.addEdge(null, bookMap.get(bookId),
authorMap.get(authorId), BookAuthor);
edge.setProperty();
}
rs.close();
statement.close();
graph.commit();
On Monday, 20 October 2014 20:51:22 UTC+2, Lvc@ wrote:
@Bojan, How did you import edges? Can you share the code? It's hard to
help without any information.
Lvc@
On 20 October 2014 13:31, 'Curtis Mosters' via OrientDB
orient-...@googlegroups.com javascript: wrote:
Is you code secret? I still don't understand the real issue you have,
sorry.
Am Montag, 20. Oktober 2014 15:26:27 UTC+2 schrieb Bojan Vukotić:
Well, that's exactly what I need (and what I already did). I thought
maybe that ETL has some API that we could use to make this easier (I used
'pure' OrientDB API to implement this)
On Monday, 20 October 2014 15:15:27 UTC+2, Curtis Mosters wrote:
I think I don't understand your issue:
First of all you import all Books. So you have all the data in there.
Now setting an index on the Book.ID.
After that the Authors are imported. They are matched with the ID of
the Book. I think in your case the Author matched with a Book is the same
like the Book.ID? Or do you have another file containing the relations of
the ID's?
Can you post here the first 10 lines of each file maybe? That would
help a low.
Am Montag, 20. Oktober 2014 14:55:10 UTC+2 schrieb Bojan Vukotić:
I took example from here http://www.orientechnologies.
com/docs/last/orientdb-etl.wiki/Import-from-DBMS.html
When can I find ETL/Java example?
On Monday, 20 October 2014 14:43:11 UTC+2, Curtis Mosters wrote:
Well I think it's way better to create a Java example. Then you
understand what is happening in the background. Otherwhise in my tests
the
ETL way had the same speed, but these tests are 3-4 month old. I will
redo
them soon. Did you take the example of ETL from OrientDB? Otherwhise
look
above for some examples. Or even post yours here?
Am Montag, 20. Oktober 2014 12:36:57 UTC+2 schrieb Bojan Vukotić:
I prefer do it with ETL, if it is possible, I would like to avoid
programming. If not, Java is also a good solution.
So, example how to do it in ETL? And regarding ETL, I was playing
with it, it imports vertices nicely, but when I want to import edges
(100
000 of them) it is extremely slow :( How to improve this?
On Monday, 20 October 2014 12:16:08 UTC+2, Curtis Mosters wrote:
Well you have several ways. Do you want to do it with JAVA oder the
ETL plugin?
In any case I think it should be
Vertices: Author, Book
Edge: WROTE
WDYT?
Am Montag, 20. Oktober 2014 10:51:03 UTC+2 schrieb Bojan Vukotić:
Hi guys!
The whole discussion here is how to create edges from one table to
another, but what to do if we have more complex cases where we have
connected 2 (or even more) tables? Example, n:n relation: book and
authors,
book can have one or more authors and author can work on one or more
books.
Tables:
Book {
book_id
book_name,
}
Author {
author_id,
author_name,
..
}
Author_on_Book {
ab_id,
book_id,
author_id,
description // describes
Re: [orientdb] OGlobalConfiguration.CACHE_LOCAL_ENABLED
Hi, By design instance of document database should be treated as lightweight connection to database storage which adds data unmarshaling features, so it means that open/close should be lightweight operation. But now it is not true, we are going to change it, which means that open/close operation will cost few ns instead of ms as it is now. On Mon, Oct 20, 2014 at 10:15 PM, odbuser odbus...@gmail.com wrote: At once we fix https://github.com/orientechnologies/orientdb/issues/2901 and https://github.com/orientechnologies/orientdb/issues/2900 and can use document db instance as jdbc connection and this problem will gone. What does this mean? On Monday, October 20, 2014 4:41:19 AM UTC-4, Andrey Lomakin wrote: Hi, It is very cheap. At once we fix https://github.com/orientechnologies/orientdb/issues/2901 and https://github.com/orientechnologies/orientdb/issues/2900 and can use document db instance as jdbc connection and this problem will gone. On Sun, Oct 19, 2014 at 7:07 AM, odbuser odbu...@gmail.com wrote: I worked around this by clearing the local cache before each use on the thread. If building the local cache is expensive, I don't like the workaround since the majority of times, I do not want the local cache enabled. orientGraph.getRawGraph().getLocalCache().invalidate(); On Friday, October 17, 2014 11:27:13 AM UTC-4, odbuser wrote: I spoke too soon. I am getting the OConcurrentModificationException. I have one thread that does some work and updates a record followed by another thread that gets the record and updates it. The second thread fails with OConcurrentModificationException b/c it's getting the record from the cache and it doesn't match the one in the database. How would I deal with this situation? On Thursday, October 16, 2014 8:22:16 PM UTC-4, odbuser wrote: By CME I assume you mean ConcurrentModificationException. I am not sure what you mean by single thread mode. I am using orientdb with multiple threads. I don't remember the impact of having the cache on but so far have not encountered any issues. I was just wondering why the option was removed as it wasn't explained in the issue. On Thursday, October 16, 2014 10:19:25 AM UTC-4, Andrey Lomakin wrote: Hi, No you always should use local cache but level2 cache was removed, if you do not use local cache you will get CME on single thread mode. But you probably have objections. Could you list them ? On Thu, Oct 16, 2014 at 5:03 PM, odbuser odbu...@gmail.com wrote: OGlobalConfiguration.CACHE_LOCAL_ENABLED (cache.local.enabled) was removed in 2.0-SNAPSHOT since 2.0-M2. Is there a replacement? -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-databa...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Best regards, Andrey Lomakin. Orient Technologies the Company behind OrientDB -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-databa...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Best regards, Andrey Lomakin. Orient Technologies the Company behind OrientDB -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Best regards, Andrey Lomakin. Orient Technologies the Company behind OrientDB -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] ETL for professional import to OrientDB
I tried to use ETL as described here
http://www.orientechnologies.com/docs/last/orientdb-etl.wiki/Import-from-DBMS.html
I used slightly modified approach (I don't know is it a correct one). I
tried to import BookAuthor as vertex first (went fast and with no
problems), than I tried to import Book as a vertex and to create out
relation from Book to BookAuthor. This lasted for ages (I have about 50 000
rows in Books table and 150 000 in BookAuthor)
{
config: {
log: error
},
extractor : {
jdbc: { driver: com.microsoft.sqlserver.jdbc.SQLServerDriver,
url: jdbc:sqlserver://localhost:1433;DatabaseName=SomeDB,
userName: sa,
fetchSize: 500,
userPassword: sa,
query: select * from BookAuthor }
},
transformers : [
{ vertex: { class: BookAuthor} }
],
loader : {
orientdb: {
dbURL: plocal:d:/orientdb/myorient,
dbAutoCreate: true,
dbType: graph,
batchCommit: 1000
}
}
}
{
config: {
log: error
},
extractor : {
jdbc: { driver: com.microsoft.sqlserver.jdbc.SQLServerDriver,
url: jdbc:sqlserver://localhost:1433;DatabaseName=SomeDB,
userName: sa,
fetchSize: 500,
userPassword: sa,
query: select * from Book }
},
transformers : [
{ vertex: { class: Book} },
{ edge: { class: Book_BookAuthor, direction : out,
joinFieldName: bookId,
lookup:BookAuthor.bookId, unresolvedLinkAction:CREATE}
}
],
loader : {
orientdb: {
dbURL: plocal:d:/orientdb/myorient,
dbAutoCreate: true,
dbType: graph,
batchCommit: 1000
}
}
}
On Monday, 20 October 2014 20:51:22 UTC+2, Lvc@ wrote:
@Bojan, How did you import edges? Can you share the code? It's hard to
help without any information.
Lvc@
On 20 October 2014 13:31, 'Curtis Mosters' via OrientDB
orient-...@googlegroups.com javascript: wrote:
Is you code secret? I still don't understand the real issue you have,
sorry.
Am Montag, 20. Oktober 2014 15:26:27 UTC+2 schrieb Bojan Vukotić:
Well, that's exactly what I need (and what I already did). I thought
maybe that ETL has some API that we could use to make this easier (I used
'pure' OrientDB API to implement this)
On Monday, 20 October 2014 15:15:27 UTC+2, Curtis Mosters wrote:
I think I don't understand your issue:
First of all you import all Books. So you have all the data in there.
Now setting an index on the Book.ID.
After that the Authors are imported. They are matched with the ID of
the Book. I think in your case the Author matched with a Book is the same
like the Book.ID? Or do you have another file containing the relations of
the ID's?
Can you post here the first 10 lines of each file maybe? That would
help a low.
Am Montag, 20. Oktober 2014 14:55:10 UTC+2 schrieb Bojan Vukotić:
I took example from here http://www.orientechnologies.
com/docs/last/orientdb-etl.wiki/Import-from-DBMS.html
When can I find ETL/Java example?
On Monday, 20 October 2014 14:43:11 UTC+2, Curtis Mosters wrote:
Well I think it's way better to create a Java example. Then you
understand what is happening in the background. Otherwhise in my tests
the
ETL way had the same speed, but these tests are 3-4 month old. I will
redo
them soon. Did you take the example of ETL from OrientDB? Otherwhise
look
above for some examples. Or even post yours here?
Am Montag, 20. Oktober 2014 12:36:57 UTC+2 schrieb Bojan Vukotić:
I prefer do it with ETL, if it is possible, I would like to avoid
programming. If not, Java is also a good solution.
So, example how to do it in ETL? And regarding ETL, I was playing
with it, it imports vertices nicely, but when I want to import edges
(100
000 of them) it is extremely slow :( How to improve this?
On Monday, 20 October 2014 12:16:08 UTC+2, Curtis Mosters wrote:
Well you have several ways. Do you want to do it with JAVA oder the
ETL plugin?
In any case I think it should be
Vertices: Author, Book
Edge: WROTE
WDYT?
Am Montag, 20. Oktober 2014 10:51:03 UTC+2 schrieb Bojan Vukotić:
Hi guys!
The whole discussion here is how to create edges from one table to
another, but what to do if we have more complex cases where we have
connected 2 (or even more) tables? Example, n:n relation: book and
authors,
book can have one or more authors and author can work on one or more
books.
Tables:
Book {
book_id
book_name,
}
Author {
author_id,
author_name,
..
}
Author_on_Book {
ab_id,
book_id,
author_id,
description // describes what this author did on this book
}
How to migrate this case? Should Author_on_Book be migrated as a
vertex or edge? How to write scripts in this case? (in real life we
could
have even more foreign keys in Author_on_Book table )
Re: [orientdb] Inserting a large file into OrientDB
Hey Andrey just sent you the code I am posting it again since it only got
sent privately it seems
public class Test {
static Sail sail;
public void testAll(File f) throws Exception {
KeyIndexableGraph og = new OrientGraphNoTx(plocal:server7/test, admin,
admin);
Sail sail = new GraphSail(og);
sail.initialize();
OGlobalConfiguration.SBTREE_MAX_KEY_SIZE.setValue(11000);
try {
SailLoader loader = new SailLoader(sail);
SailConnection sc = sail.getConnection();
try {
sc.begin();
loader.load(f);
sc.rollback();
} finally {
sc.close();
}
} finally {
sail.shutDown();
}
}
On Monday, 20 October 2014 11:02:47 UTC+2, Andrey Lomakin wrote:
HI,
Do you have test ?
I need to look how to reproduce this error.
On Fri, Oct 17, 2014 at 10:34 PM, NewUser sykam...@gmail.com
javascript: wrote:
Hi!
I have created a schemaless database in OrientDB in which I am trying to
load a large .nt file, I have not created any manual indexes and I am
trying to see how the system behaves with the automatic indexing. The
indexes were created upon starting the upload as from what I see in the
orientdb studio. However, the load was not successful and I encountered the
following error:
com.orientechnologies.common.directmemory.ODirectMemoryViolationException:
Negative offset was provided
at
com.orientechnologies.common.directmemory.ODirectMemoryPointer.rangeCheck(ODirectMemoryPointer.java:157)
at
com.orientechnologies.common.directmemory.ODirectMemoryPointer.get(ODirectMemoryPointer.java:57)
at
com.orientechnologies.orient.core.storage.impl.local.paginated.base.ODurablePage.moveData(ODurablePage.java:215)
at
com.orientechnologies.orient.core.index.sbtree.local.OSBTreeBucket.addEntry(OSBTreeBucket.java:286)
at
com.orientechnologies.orient.core.index.sbtree.local.OSBTree.splitNonRootBucket(OSBTree.java:1685)
at
com.orientechnologies.orient.core.index.sbtree.local.OSBTree.splitBucket(OSBTree.java:1622)
at
com.orientechnologies.orient.core.index.sbtree.local.OSBTree.put(OSBTree.java:606)
at
com.orientechnologies.orient.core.index.engine.OSBTreeIndexEngine.put(OSBTreeIndexEngine.java:317)
at
com.orientechnologies.orient.core.index.OIndexMultiValues.commitSnapshot(OIndexMultiValues.java:228)
at
com.orientechnologies.orient.core.index.OIndexAbstract.commit(OIndexAbstract.java:709)
at
com.orientechnologies.orient.core.tx.OTransactionOptimistic$CommitIndexesCallback.run(OTransactionOptimistic.java:104)
at
com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage.commit(OLocalPaginatedStorage.java:1080)
at
com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:464)
at
com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:148)
at
com.orientechnologies.orient.core.db.record.ODatabaseRecordTx.commit(ODatabaseRecordTx.java:142)
at
com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:511)
at
com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:503)
at
com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.commit(ONetworkProtocolBinary.java:1096)
at
com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary.executeRequest(ONetworkProtocolBinary.java:344)
at
com.orientechnologies.orient.server.network.protocol.binary.OBinaryNetworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:169)
at
com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:45)
Oct 17, 2014 9:17:46 PM com.tinkerpop.blueprints.oupls.sail.SailLoader
load
INFO: loaded 435000 statements in 353916ms
Oct 17, 2014 9:17:49 PM com.orientechnologies.common.log.OLogManager log
SEVERE: Error during context close for db remote:diufpc307/DBTest
com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.ONestedRollbackException:
Atomic operation was rolled back by internal component
at
com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.OAtomicOperationsManager.endAtomicOperation(OAtomicOperationsManager.java:73)
at
com.orientechnologies.orient.core.storage.impl.local.OStorageLocalAbstract.endStorageTx(OStorageLocalAbstract.java:123)
at
com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage.commit(OLocalPaginatedStorage.java:1082)
at
com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:464)
at
com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:148)
at
Re: [orientdb] SQL query performance with OR in the WHERE clause
Hi,
what version of OrientDB are you using? We had two issues related to OR
operators and index usage:
https://github.com/orientechnologies/orientdb/issues/2898
https://github.com/orientechnologies/orientdb/issues/2893
and we fixed them some days ago.
Could you try again on latest SNAPSHOT?
thanks
Luigi
2014-10-20 19:53 GMT+02:00 Kanak Agarwal agarwal.ka...@gmail.com:
Hello
My V class contains two properties prop1 and prop2 both of which are
indexed using non unique hash index.
The queries on prop1 alone (select from V where prop1 = 'xyz') or prop2
alone (select from V where prop1 = 'abc') use the index and perform well.
On the other hand, a query that uses OR in the WHERE clause (select from V
where prop1 = 'xyz' OR prop2 = abc') is very slow - it's not using index
and seems to be doing linear scan.
Any suggestions on how to optimize this query.
Here are the SQL explain results
- prop1 query (select from V where prop1 = 'xyz')
[{@type:d,limit:-1,compositeIndexUsed:1,involvedIndexes:[V.prop1],fullySortedByIndex:false,indexIsUsedInOrderBy:false,current:#9:2,evaluated:20,fetchingFromTargetElapsed:0,documentReads:20,documentAnalyzedCompatibleClass:20,recordReads:20,elapsed:0.216,resultType:collection,resultSize:20,@rid:#-1:-1}]
- prop2 query (select from V where prop2 = 'abc')
[{@type:d,limit:-1,compositeIndexUsed:1,involvedIndexes:[V.prop2],fullySortedByIndex:false,indexIsUsedInOrderBy:false,current:#9:20025,evaluated:20,fetchingFromTargetElapsed:0,documentReads:20,documentAnalyzedCompatibleClass:20,recordReads:20,elapsed:0.208,resultType:collection,resultSize:20,@rid:#-1:-1}]
- prop1 or prop2 query (select from V where prop1 = 'xyz' or prop2 = 'abc')
[{@type:d,current:#9:3,evaluated:4,fetchingFromTargetElapsed:249,documentReads:4,documentAnalyzedCompatibleClass:4,recordReads:4,elapsed:249.68301,resultType:collection,resultSize:40,@rid:#-1:-1}]
--
---
You received this message because you are subscribed to the Google Groups
OrientDB group.
To unsubscribe from this group and stop receiving emails from it, send an
email to orient-database+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
---
You received this message because you are subscribed to the Google Groups
OrientDB group.
To unsubscribe from this group and stop receiving emails from it, send an email
to orient-database+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] Re: Repairing security structures ?
I will. Thank you On Monday, 20 October 2014 08:08:07 UTC, Andrey Lomakin wrote: Hi Stefan, It is fixed in 2.0-SNAPSHOT could you try ? On Sat, Oct 18, 2014 at 11:42 PM, ste...@activitystream.com javascript: wrote: btw. this is 2.M2 and embedded server becomes unresponsive. Currently I have no option to repair this and the database becomes unusable/corrupt. Regards, -Stefan On Friday, October 17, 2014 8:03:54 PM UTC, ste...@activitystream.com wrote: Hi, Can someone please tell me what is happening the ODB server becomes unresponsive and this starts to show up: Oct 17, 2014 8:00:41 PM com.orientechnologies.common.log.OLogManager log WARNING: Repairing security structures... Regards, -Stefan -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-databa...@googlegroups.com javascript:. For more options, visit https://groups.google.com/d/optout. -- Best regards, Andrey Lomakin. Orient Technologies the Company behind OrientDB -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[orientdb] Multiple vertex objects referring to the same database record are not in sync
Hi all! I am in the process of debugging an application based on OrientDB (2.0M2). The problem I am facing are some OConcurrentModificationException. The issue appears with both the memory and the plocal storage engines (I did not try remote). It looks they are caused by the fact that multiple OrientVertex objects which refer to the same database record are (in general) not in sync. This also means that there is no guarantee in general that a value which is returned by a query is the most up to date. The following code replicates the issue. I agree it may look a bit awkward, but it is just the smallest example I could come up with from our code. OrientGraphFactory graphFactory = new OrientGraphFactory(DATABASE_URL); graphFactory.setAutoStartTx(false); OrientGraph graph = graphFactory.getTx(); graph.begin(); OrientVertex firstVertexHandle = graph.addVertex(null, PROPERTY_NAME, 0); graph.commit(); firstVertexHandle.setProperty(PROPERTY_NAME, 1); // Note that if any of the following 3 lines are commented, then the problem does not appear graph.begin(); graph.getVertices(someproperty, somevalue).iterator().hasNext(); graph.commit(); Object vertexId = firstVertexHandle.getId(); Vertex secondVertexHandle = graph.getVertex(vertexId); secondVertexHandle.setProperty(PROPERTY_NAME, 2); // prints 1 System.out.println(firstVertexHandle.getProperty(PROPERTY_NAME)); // prints 2 System.out.println(Value queried from the second vertex handle is + secondVertexHandle.getProperty(PROPERTY_NAME)); // The following throws an OConcurrentModificationException firstVertexHandle.setProperty(PROPERTY_NAME, 3); Is this a bug in OrientDB? What should the expected behavior with respect to multiple Vertex objects referring to the same database record? -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] Multiple vertex objects referring to the same database record are not in sync
Hi, Do you have local cache enabled ? It explicitly created to avoid such problems. On Tue, Oct 21, 2014 at 12:48 PM, Emanuele Milani milani.emanuele...@gmail.com wrote: Hi all! I am in the process of debugging an application based on OrientDB (2.0M2). The problem I am facing are some OConcurrentModificationException. The issue appears with both the memory and the plocal storage engines (I did not try remote). It looks they are caused by the fact that multiple OrientVertex objects which refer to the same database record are (in general) not in sync. This also means that there is no guarantee in general that a value which is returned by a query is the most up to date. The following code replicates the issue. I agree it may look a bit awkward, but it is just the smallest example I could come up with from our code. OrientGraphFactory graphFactory = new OrientGraphFactory(DATABASE_URL); graphFactory.setAutoStartTx(false); OrientGraph graph = graphFactory.getTx(); graph.begin(); OrientVertex firstVertexHandle = graph.addVertex(null, PROPERTY_NAME, 0); graph.commit(); firstVertexHandle.setProperty(PROPERTY_NAME, 1); // Note that if any of the following 3 lines are commented, then the problem does not appear graph.begin(); graph.getVertices(someproperty, somevalue).iterator().hasNext(); graph.commit(); Object vertexId = firstVertexHandle.getId(); Vertex secondVertexHandle = graph.getVertex(vertexId); secondVertexHandle.setProperty(PROPERTY_NAME, 2); // prints 1 System.out.println(firstVertexHandle.getProperty(PROPERTY_NAME)); // prints 2 System.out.println(Value queried from the second vertex handle is + secondVertexHandle.getProperty(PROPERTY_NAME)); // The following throws an OConcurrentModificationException firstVertexHandle.setProperty(PROPERTY_NAME, 3); Is this a bug in OrientDB? What should the expected behavior with respect to multiple Vertex objects referring to the same database record? -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Best regards, Andrey Lomakin. Orient Technologies the Company behind OrientDB -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [orientdb] Multiple vertex objects referring to the same database record are not in sync
Hi I got reason of your issue, could you try 2.0-SNAPSHOT. Problem should be fixed there. On Tue, Oct 21, 2014 at 1:59 PM, Andrey Lomakin lomakin.and...@gmail.com wrote: Hi, Do you have local cache enabled ? It explicitly created to avoid such problems. On Tue, Oct 21, 2014 at 12:48 PM, Emanuele Milani milani.emanuele...@gmail.com wrote: Hi all! I am in the process of debugging an application based on OrientDB (2.0M2). The problem I am facing are some OConcurrentModificationException. The issue appears with both the memory and the plocal storage engines (I did not try remote). It looks they are caused by the fact that multiple OrientVertex objects which refer to the same database record are (in general) not in sync. This also means that there is no guarantee in general that a value which is returned by a query is the most up to date. The following code replicates the issue. I agree it may look a bit awkward, but it is just the smallest example I could come up with from our code. OrientGraphFactory graphFactory = new OrientGraphFactory(DATABASE_URL); graphFactory.setAutoStartTx(false); OrientGraph graph = graphFactory.getTx(); graph.begin(); OrientVertex firstVertexHandle = graph.addVertex(null, PROPERTY_NAME, 0); graph.commit(); firstVertexHandle.setProperty(PROPERTY_NAME, 1); // Note that if any of the following 3 lines are commented, then the problem does not appear graph.begin(); graph.getVertices(someproperty, somevalue).iterator().hasNext(); graph.commit(); Object vertexId = firstVertexHandle.getId(); Vertex secondVertexHandle = graph.getVertex(vertexId); secondVertexHandle.setProperty(PROPERTY_NAME, 2); // prints 1 System.out.println(firstVertexHandle.getProperty(PROPERTY_NAME)); // prints 2 System.out.println(Value queried from the second vertex handle is + secondVertexHandle.getProperty(PROPERTY_NAME)); // The following throws an OConcurrentModificationException firstVertexHandle.setProperty(PROPERTY_NAME, 3); Is this a bug in OrientDB? What should the expected behavior with respect to multiple Vertex objects referring to the same database record? -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Best regards, Andrey Lomakin. Orient Technologies the Company behind OrientDB -- Best regards, Andrey Lomakin. Orient Technologies the Company behind OrientDB -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[orientdb] OCommandExecutorNotFoundException creating OrientDB database via jar file
We want to import data on a Linux system. So we have 2 Java files for the
import. One (*OrientDBMain.java*) is for creating an object that is doing
all the work. The other (*OrientDB.java*) contains all the database
functions like setting an index and so on. The second files starts with:
public class OrientDB {
private OrientGraphFactory factoryGraph;
private ODatabaseDocumentTx db;
private String csvPath;
@SuppressWarnings(resource)
public OrientDB(String dbPath) {
OGlobalConfiguration.STORAGE_KEEP_OPEN.setValue(true);
OGlobalConfiguration.ENVIRONMENT_CONCURRENT.setValue(false);
db = new ODatabaseDocumentTx(dbPath).create();
}
...
Running this in Eclipse on a Windows and MAC works well. No error and data
is correctly imported.
But now we wanted to run this on our Linux system. So we setup a Maven
project (m2e) and exported a Jar file with all the dependencies here:
project xmlns=http://maven.apache.org/POM/4.0.0; xmlns:xsi=
http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;
modelVersion4.0.0/modelVersion
groupIdmapegy/groupId
artifactIdorientdbcsv/artifactId
version0.0.2-SNAPSHOT/version
packagingjar/packaging
nameorientdbcsv/name
urlhttp://maven.apache.org/url
properties
project.build.sourceEncodingUTF-8/project.build.sourceEncoding
/properties
dependencies
dependency
groupIdjunit/groupId
artifactIdjunit/artifactId
version3.8.1/version
scopetest/scope
/dependency
dependency
groupIdcom.orientechnologies/groupId
artifactIdorientdb-graphdb/artifactId
version2.0-M1/version
/dependency
/dependencies
build
plugins
plugin
artifactIdmaven-assembly-plugin/artifactId
configuration
archive
manifest
mainClassmapegy.orientdbcsv.OrientDBMain/mainClass
/manifest
/archive
descriptorRefs
descriptorRefjar-with-dependencies/descriptorRef
/descriptorRefs
/configuration
executions
execution
idmake-assembly/id
phasepackage/phase
goals
goalsingle/goal
/goals
/execution
/executions
/plugin
plugin
groupIdorg.apache.maven.plugins/groupId
artifactIdmaven-compiler-plugin/artifactId
version2.3.2/version
configuration
source${jdk.version}/source
target${jdk.version}/target
/configuration
/plugin
/plugins
/build
/project
But if we run that Jar now we are getting some errors. They say that there
is some OCommandExecutorNotFoundException thrown. But how can that be if it
run great directly in Eclipse? We have 2 args {pathtoimportdata,
pathtodestinationdb}.
C:\eclipse-workspace\orientdbcsv\targetjava -jar orientdbcsv-0.0.2-
SNAPSHOT-jar
-with-dependencies.jar C:\data
C:\orientdb-community-2.0-M1\databases\test
Exception in thread main com.orientechnologies.orient.core.exception.
ODatabase
Exception: Cannot create database
at com.orientechnologies.orient.core.db.record.
ODatabaseRecordAbstract.c
reate(ODatabaseRecordAbstract.java:289)
at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract
.create(
ODatabaseWrapperAbstract.java:61)
at com.orientechnologies.orient.core.db.
ODatabaseRecordWrapperAbstract.c
reate(ODatabaseRecordWrapperAbstract.java:72)
at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract
.create(
ODatabaseWrapperAbstract.java:56)
at com.orientechnologies.orient.core.db.
ODatabaseRecordWrapperAbstract.c
reate(ODatabaseRecordWrapperAbstract.java:66)
at mapegy.orientdbcsv.OrientDB.init(OrientDB.java:36)
at mapegy.orientdbcsv.OrientDBMain.main(OrientDBMain.java:6)
Caused by: com.orientechnologies.orient.core.command.
OCommandExecutorNotFoundExc
eption: Cannot find a command executor for the command request: sql.
select count
(*) from ORole where name.type() not in [STRING] and name is not null
at com.orientechnologies.orient.core.sql.
OCommandExecutorSQLDelegate.par
se(OCommandExecutorSQLDelegate.java:48)
at com.orientechnologies.orient.core.sql.
OCommandExecutorSQLDelegate.par
se(OCommandExecutorSQLDelegate.java:33)
at com.orientechnologies.orient.core.storage.OStorageEmbedded.
command(OS
torageEmbedded.java:69)
at com.orientechnologies.orient.core.command.
OCommandRequestTextAbstract
.execute(OCommandRequestTextAbstract.java:59)
at com.orientechnologies.orient.core.metadata.schema.OClassImpl.
checkPer
Re: [orientdb] SQL query performance with OR in the WHERE clause
Hi
Thanks for the response. I am seeing the issue in the OrientDB 2.0-M2
community edition.
Is this the latest SNAPSHOT?
On Tuesday, October 21, 2014 2:23:04 AM UTC-5, Luigi Dell'Aquila wrote:
Hi,
what version of OrientDB are you using? We had two issues related to OR
operators and index usage:
https://github.com/orientechnologies/orientdb/issues/2898
https://github.com/orientechnologies/orientdb/issues/2893
and we fixed them some days ago.
Could you try again on latest SNAPSHOT?
thanks
Luigi
2014-10-20 19:53 GMT+02:00 Kanak Agarwal agarwa...@gmail.com
javascript::
Hello
My V class contains two properties prop1 and prop2 both of which are
indexed using non unique hash index.
The queries on prop1 alone (select from V where prop1 = 'xyz') or prop2
alone (select from V where prop1 = 'abc') use the index and perform well.
On the other hand, a query that uses OR in the WHERE clause (select from
V where prop1 = 'xyz' OR prop2 = abc') is very slow - it's not using index
and seems to be doing linear scan.
Any suggestions on how to optimize this query.
Here are the SQL explain results
- prop1 query (select from V where prop1 = 'xyz')
[{@type:d,limit:-1,compositeIndexUsed:1,involvedIndexes:[V.prop1],fullySortedByIndex:false,indexIsUsedInOrderBy:false,current:#9:2,evaluated:20,fetchingFromTargetElapsed:0,documentReads:20,documentAnalyzedCompatibleClass:20,recordReads:20,elapsed:0.216,resultType:collection,resultSize:20,@rid:#-1:-1}]
- prop2 query (select from V where prop2 = 'abc')
[{@type:d,limit:-1,compositeIndexUsed:1,involvedIndexes:[V.prop2],fullySortedByIndex:false,indexIsUsedInOrderBy:false,current:#9:20025,evaluated:20,fetchingFromTargetElapsed:0,documentReads:20,documentAnalyzedCompatibleClass:20,recordReads:20,elapsed:0.208,resultType:collection,resultSize:20,@rid:#-1:-1}]
- prop1 or prop2 query (select from V where prop1 = 'xyz' or prop2 =
'abc')
[{@type:d,current:#9:3,evaluated:4,fetchingFromTargetElapsed:249,documentReads:4,documentAnalyzedCompatibleClass:4,recordReads:4,elapsed:249.68301,resultType:collection,resultSize:40,@rid:#-1:-1}]
--
---
You received this message because you are subscribed to the Google Groups
OrientDB group.
To unsubscribe from this group and stop receiving emails from it, send an
email to orient-databa...@googlegroups.com javascript:.
For more options, visit https://groups.google.com/d/optout.
--
---
You received this message because you are subscribed to the Google Groups
OrientDB group.
To unsubscribe from this group and stop receiving emails from it, send an email
to orient-database+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[orientdb] Lucene in Neo4j has some misbehaviours in terms of reliable search querys - compared to OrientDB
I'm still in the evaluation of *Neo4j *vs. *OrientDB*. Most importantly I
need Lucene as full-text index engine. So I created on both databases the
same schema with the same data (300Mio lines). I'm also experienced with
querying different things in both systems. I used the Standard Analyzer on
both sides. The OrientDB test query results are all fine and really good in
terms of reliability and speed. The speed of Neo4j is also ok but the
results are kind of bad in most of the cases. So let's come to the
different issues I have with Neo4j Lucene indexing. I always give you an
example of how it would look in OrientDB and which result set you should be
getting out of the query.
So in these examples, there are Applns that have title(s). Titles are
indexed with Lucene in both databases. Applns also have an ID just to
demonstrate the ordering. At the end of each query I have some *questions*
about them. It would be great to get some *feedback *or even *answers *about
them.
Query #0: One word query with no order
Well this query is very simple. It shall be tested how the database behave
if there is just a simple word and nothing else. As you can see the Neo4j
result is way longer then the one from OrientDB. OrientDB is using TFIDF to
keep the results short and more reliable to the actual search. As you can
see as first result in OrientDB, there is title with SOLAR. That is totally
missing in Neo4j, too.
In Neo4j: *START n=node:titles('title:solar') RETURN n.title,n.ID LIMIT 10*
1. SOLAR RADIATION SHIELDING PARTICULATE AND SOLAR RADIATION SHIELDING
RESIN MATERIAL DISPERSED WITH ...38321319
2. Solar module for cooling solar cells on the underside of a solar
panel has air inlet and outlet openings ...12944121
3. Solar construction component for solar thermal assemblies, solar
thermal assembly, method for operating a solar...324146113
4. ...
In OrientDB: *SELECT title,ID FROM Appln WHERE title LUCENE solar LIMIT
10*
1. SOLAR24900187
2. Solar unit and solar apparatus1876343
3. Solar module with solar concentrator13496706
4. ...
Questions:
1. Why is Neo4j not using TFIDF or what do they use instead?
2. Is Neo4j able to use some ordering of the keyword match?
3. Is it possible to change TFIDF to somethign else in OrientDB?
Query #1: One word query with order by ID
Neo4j is ordering the ID's before using TFIDF. As known from Query#0 Neo4j
is not using TFIDF so it's basically just searching via first results of
the Lucene query. In OrientDB besides it's still searching by good TFIDF's
and then ordering.
In Neo4j:* START n=node:titles('title:solar') RETURN n.title,n.ID ORDER BY
n.ID ASC LIMIT 10*
1. Stackable flat-roof/floor frame for solar panels318
2. Method for producing contact for solar cells636
3. Solar cell and fabrication method thereof1217
4. ...
In OrientDB: *SELECT title,ID FROM Appln WHERE title LUCENE solar ORDER
BY ID ASC LIMIT 10*
1. Solar unit and solar apparatus 1876343
2. Solar module with solar concentrator13496706
3. SOLAR TRACKER FOR SOLAR COLLECTOR16543688
4. ...
Questions:
1. How would a search in OrientDB look like that should be ordered by
the ID and still matching the best TFIDF of them.
2. Is there a way in Neo4j to order the Lucene match before ordering by
the ID?
Query #2: One word with using a star search
Star search had no influence on the Neo4j results. OrientDB results changed
in a good way.
In Neo4j: *START n=node:titles('title:solar*') RETURN n.title,n.ID ORDER BY
n.ID ASC LIMIT 10*
1. Stackable flat-roof/floor frame for solar panels318
2. Method for producing contact for solar cells636
3. Solar cell and fabrication method thereof1217
4. ...
In OrientDB: *SELECT title,ID FROM Appln WHERE title LUCENE solar* ORDER
BY ID ASC LIMIT 10*
1. High performance solar methane generator8354701
2. All-plastic honeycomb solar water-heater8355379
3. Plate type solar energy heat collector plate core and its
manufacturing method8356173
4. ...
Questions:
1. Does Neo4j ignore star searches?
Query #3: Searching for 2 words devided by a space
The strange here is that you need to change 'title:solar panel' to that
query here. Otherwhise you just get errors. OrientDB seems good so far.
In Neo4j:* START n=node:titles(title=solar panel) RETURN n.title,n.ID
ORDER BY n.ID ASC LIMIT 10*
1. Returned 0 rows in 817 ms
In OrientDB: *SELECT title,ID FROM Appln WHERE title LUCENE solar panel
ORDER BY ID ASC LIMIT 10*
1. SOLAR PANEL1584567
2. SOLAR PANEL1616547
3. SOLAR PANEL2078382
4. SOLAR PANEL2078383
5. Solar panel2178466
6. ...
Questions:
1. Why does Neo4j need a special Query here to at least don't throw any
error?
2. Why is the query failing and not giving anything back? I know that
Neo4j
[orientdb] lucene spatial - import 1.7.4 db to 1.7.9 throws error
Here is an error that is thrown when I try to import the 1.7.4 db that has a spatial index to 1.7.9: - Index 'stores.latlong'...Error on database import happened just before line 0, column 1582755 com.orientechnologies.orient.core.index.OIndexException: Index type: SPATIAL is not supported. Types are [UNIQUE,FULLTEXT,FULLTEXT_HASH_INDEX,DICTIONARY_HASH_INDEX,DICTIONARY,UNIQUE_HASH_INDEX,NOTUNIQUE_HASH_INDEX,SPATIAL,NOTUNIQUE] This is a weird error, since it says that Spatial is not supported, bu then lists Spatial as an available index option. Any thoughts on how to fix this? -- --- You received this message because you are subscribed to the Google Groups OrientDB group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
