Hi Curtis,
getting your structure on MySQL:
- Person{id,name}
- Post{person_id,title}
You could configure 2 pipelines:
1. one with extraction of "select * from Person"
2. one with extraction of "select * from Post"
*Example for (1)*
{
"config": {
"verbose": true
},
"extractor" : {
"jdbc": { "driver": "com.mysql.jdbc.Driver",
"url": "jdbc:mysql://localhost/mysql",
"userName": "root",
"userPassword": "",
"query": "select * from Person" }
},
"transformers" : [
{ "vertex": { "class": "Person"} }
],
"loader" : {
"orientdb": {
"dbURL": "plocal:/temp/databases/orientdb",
"dbAutoCreate": true
}
}
}
*And this for (2):*
{
"config": {
"verbose": true
},
"extractor" : {
"jdbc": { "driver": "com.mysql.jdbc.Driver",
"url": "jdbc:mysql://localhost/mysql",
"userName": "root",
"userPassword": "",
"query": "select * from Post" }
},
"transformers" : [
{ "vertex": { "class": "Post"} },
* { "edge": { "class": "Person", "direction" : "in", *
* "joinFieldName": "person_id", "lookup":"Person.id",
"unresolvedLinkAction":"CREATE"} }* ],
"loader" : {
"orientdb": {
"dbURL": "plocal:/temp/databases/orientdb",
"dbAutoCreate": true
}
}
}
Lvc@
ᐧ
On 25 August 2014 15:23, 'Curtis Mosters' via OrientDB <
orient-database@googlegroups.com> wrote:
> Well there is an idea that might be easy to be implemented.
>
> So the idea is to *auto-create* an edge. Let's take an example:
>
> MySQL classes are:
>
> - Person{id,name}
> - Post{,person_id,title}
>
> Now we also create such classes in OrientDB. Now we tell OrientDB if there
> is INSERT INTO Person or Post -> create edge on "id" to "person_id". I
> don't but isn't that pretty simple for a database to do?
> Well I have to say all of that is possible easily done with Java API but I
> really want it with ETL because there is no real need for me to use the
> Java API.
>
> Am Montag, 25. August 2014 14:15:00 UTC+2 schrieb Curtis Mosters:
>
>> Could you maybe take some time to create such an example?
>>
>> The current "edge" is bit hard to understand:
>>
>> edge: {
>> class: "hasPost",
>> joinFieldName: "person_id",
>> lookup: "person.id",
>> unresolvedLinkAction: "CREATE",
>> if: "person_id is not null"
>> }
>>
>> How can I say here to Link from person on post? I just need a small
>> example and also the MySQL strcuture if possible. That would make
>> everything clear. Thanks Luca.
>>
>> Am Montag, 25. August 2014 13:23:55 UTC+2 schrieb Lvc@:
>>>
>>> Curtis,
>>> You could create 2 or more etl scripts, some that extract vertices and
>>> other that extract edges.
>>>
>>> Lvc@
>>>
>>> ᐧ
>>>
>>>
>>> On 25 August 2014 12:25, 'Curtis Mosters' via OrientDB <
>>> orient-...@googlegroups.com> wrote:
>>>
>>>> Hi Luca, well your example is just reading one table from MySQL. One
>>>> table with creating edges on it's entries. But in the usual cases that's
>>>> not the way how to import from a MySQL. Or maybe I just didn't understood
>>>> that example completely. But so far as I have seen it, you used one table
>>>> from MySQL that contained {id,...,friend_id}. You created profiles on that
>>>> id and then created edge on that id's. Is that right?
>>>>
>>>> But just once again I have those data as usually stored in another
>>>> table. So that cannot work for me.
>>>>
>>>> Am Donnerstag, 21. August 2014 15:56:05 UTC+2 schrieb Lvc@:
>>>>>
>>>>> ᐧ
>>>>> On 21 August 2014 14:45, 'Curtis Mosters' via OrientDB <
>>>>> orient-...@googlegroups.com> wrote:
>>>>>
>>>>>> This is a great example Luca, thanks first of all.
>>>>>>
>>>>>> But could you additionally add information about the MySQL schema and
>>>>>> do I understand it correctly that you get the classes Person,Friend and
>>>>>> an
>>>>>> edge. But where comes the friend data?
>>>>>>
>>>>>
>>>>>
>>>>> Friend in my case is an Edge class. (create class Friend extends E).
>>>>> It's useful to create ad-hoc edge classes instead that the generic "E":
>>>>>
>>>>> http://www.orientechnologies.com/docs/1.7.8/orientdb.wiki/Gr
>>>>> aph-Schema.html
>>>>>
>>>>> Lvc@
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> Am Donnerstag, 21. August 2014 13:25:47 UTC+2 schrieb Lvc@:
>>>>>>>
>>>>>>> Hi Curtis,
>>>>>>> ETL is very powerful and we're improving it everyday with users'
>>>>>>> feedback. We want to let it to be final for 2.0, so we can bundle with
>>>>>>> it.
>>>>>>> By the way Enterprise Edition
>>>>>>> <http://www.orientechnologies.com/orientdb-enterprise> will have a
>>>>>>> Web Interface to edit it, and in the future also to debug it.
>>>>>>>
>>>>>>> Unfortunately I don't understand in your example table1 & 2 with
>>>>>>> both name_id field. Look at this example about Friendship. I want to
>>>>>>> connect a social network like app.
>>>>>>>
>>>>>>> This is an example of ETL configuration. My comments begins with //
>>>>>>> but remove them from real config file:
>>>>>>>
>>>>>>> {
>>>>>>> config: {
>>>>>>> verbose: true
>>>>>>> },
>>>>>>> begin: [
>>>>>>> ],
>>>>>>> extractor: {
>>>>>>> "jdbc": { "driver": "com.mysql.jdbc.Driver",
>>>>>>> "url": "jdbc:mysql://localhost/mysocialnetwork",
>>>>>>> "userName": "root",
>>>>>>> "userPassword": "root",
>>>>>>> "query": "select * from profile"
>>>>>>> }
>>>>>>> },
>>>>>>> transformers : [
>>>>>>> {
>>>>>>> // THIS IS ONLY A LOG, USEFUL TO TRACE WHAT ARRIVES FROM MYSQL
>>>>>>> log: {
>>>>>>> prefix: "MySQL -> "
>>>>>>> }
>>>>>>> },
>>>>>>> {
>>>>>>> merge: {
>>>>>>> // LOOKUP BY ID: IF ALREADY PRESENT MERGE THE FIELDS BY
>>>>>>> OVERWRITING THE DIFFERENT ONES
>>>>>>> joinFieldName: "*id*",
>>>>>>> lookup: "Profile.*id*"
>>>>>>> }
>>>>>>> },
>>>>>>> {
>>>>>>> vertex: {
>>>>>>> // TRANSFORM IT IN A VERTEX
>>>>>>> class: "Profile"
>>>>>>> }
>>>>>>> },
>>>>>>> {
>>>>>>> edge: {
>>>>>>> // CONNECT THE FRIEND IN MYSQL COLUMN 'friend_id'
>>>>>>> class: "Friend", // Friend is the edges' class
>>>>>>> joinFieldName: "friend_id", //
>>>>>>> lookup: "Profile.id",
>>>>>>> unresolvedLinkAction: "CREATE", // IF DOESN'T EXIST YET,
>>>>>>> CREATE THE VERTEX EMPTY. WILL BE FURTHER UPDATED
>>>>>>> if: "friend_id is not null"
>>>>>>> }
>>>>>>> },
>>>>>>> {
>>>>>>> log: {
>>>>>>> prefix: "Vertex -> "
>>>>>>> }
>>>>>>> }
>>>>>>> ],
>>>>>>> loader : {
>>>>>>> orientdb: {
>>>>>>> dbURL: "remote:localhost/yourdb",
>>>>>>> dbUser: "admin",
>>>>>>> dbPassword: "admin",
>>>>>>> dbAutoCreate: true,
>>>>>>> tx: true,
>>>>>>> batchCommit: 1000,
>>>>>>> dbType: "graph",
>>>>>>> classes: [
>>>>>>> ],
>>>>>>> indexes: [
>>>>>>> ]
>>>>>>> }
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> Lvc@
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 21 August 2014 13:05, 'Curtis Mosters' via OrientDB <
>>>>>>> orient-...@googlegroups.com> wrote:
>>>>>>>
>>>>>>>> Hey so I have now some experiences with ETL. I think this is great
>>>>>>>> for import. That's why I want to improve the importing process now.
>>>>>>>>
>>>>>>>> 1. First of all is it possible to run several imports
>>>>>>>> parallel(not with several consoles) or if one is done start another
>>>>>>>> instantly?
>>>>>>>> 2. How do I create an edge (https://github.com/orientechn
>>>>>>>> ologies/orientdb-etl/wiki/Transformer#edge
>>>>>>>>
>>>>>>>> <https://github.com/orientechnologies/orientdb-etl/wiki/Transformer#edge>
>>>>>>>> is not helpful for me)
>>>>>>>>
>>>>>>>> Let's say I have 2 tables in MySQL:
>>>>>>>>
>>>>>>>> table1 with {*name_id*,*name*} and table2 with {*name_id*,*text_id*
>>>>>>>> ,*text*}
>>>>>>>>
>>>>>>>> Now I want to get them connected (name_id) in OrientDB.
>>>>>>>>
>>>>>>>> One way would now be to import both tables as Class. That's easy
>>>>>>>> and then create edges with the *name_id*. Everything is fine with
>>>>>>>> that, but it would be much easier to generate edges with ETL if that is
>>>>>>>> anyway possible. So I don't know what is more efficient. Someone else
>>>>>>>> also
>>>>>>>> said that Sails (https://github.com/vjsrinath/sails-orientdb)
>>>>>>>> might be a good choice.
>>>>>>>>
>>>>>>>> If ETL allows that I could imagine about the following way:
>>>>>>>>
>>>>>>>> - read line of table1 -> safe in memory
>>>>>>>> - look in table2 if name_id exists
>>>>>>>> - -> if yes, create vertex Tabel1 and Table2
>>>>>>>> - -> if no, create vertex Tabel1 and Table2 and create an
>>>>>>>> edge
>>>>>>>> - and so on
>>>>>>>>
>>>>>>>> Of course this is very crazy if you imagine 30 tables or so. Was
>>>>>>>> just an idea, just to check if that is already possible somehow.
>>>>>>>>
>>>>>>>> --
>>>>>>>>
>>>>>>>> ---
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "OrientDB" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>> send an email to orient-databa...@googlegroups.com.
>>>>>>>>
>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>
>>>>>> ---
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "OrientDB" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to orient-databa...@googlegroups.com.
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>
>>>>> --
>>>>
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "OrientDB" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to orient-databa...@googlegroups.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "OrientDB" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to orient-database+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
---
You received this message because you are subscribed to the Google Groups
"OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to orient-database+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
