Hi, Carlos. Nice analysis. It looks a good to me to start with the serialized approach. Personally, I tend to be too much ambitious, so I encourage to keep it simple :) Reading your analysis brings me lots of ideas (always time consuming ideas), like creating two Stores for each case with blahblahbla. So: *avoid my errors, keep it simple* ;) For sure, if later you have more time to spend, things can be improved, but no need to rush, I guess.
Just my oppinion. I know nothing :P Maybe others with knowledge of other Stores think the other way around. Regards, Alfonso Nishikawa 2018-06-26 3:57 GMT-01:00 carlos muñoz <[email protected]>: > Hi all > > As Alfonso mentioned the Ignite backend could implement the 3-union fields > either by using joined tables or storing serialized objects as binary. Here > some comments on those approaches. > > Using joined tables would definitely has an impact in performance due the > fact distributed joins [1] imply some subquery/aggregation overhead. > Moreover, the efficiency and results of such joins will depend on the > specific configuration used into the Ignite Cluster. On the other hand, > from a developing point of view, explicitly referencing persistent entities > within class mapping (JPA - like ) will introduce some complexity to the > project (fetching strategies, relations handling, etc.) and as far I > understand Apache Gore was not conceived in that way. > > Serializing objects on the other hand sticks to other backends > implementation as I noticed while analyzing Gora code. It seems to keep > code simple and IO operations efficient due the fact that embedded > persistent entities are stored as simple literals into the data stores. In > addition, Ignite has the data type BINARY [2] which suits the needs of such > fields. > > In a nutshell, I suggest taking the serialization approach and adapting > other backends implementation of 3-union fields in the Ignite backend. But > I am not really an expert on the subject so please feel free to give your > feedback. > > > [1] https://apacheignite-sql.readme.io/docs/distributed-joins > [2] https://apacheignite-sql.readme.io/docs/data-types#section-binary > > > 2018-06-06 5:05 GMT-05:00 Alfonso Nishikawa <[email protected]>: > > > Hi, Carlos. > > > > About the mapping files and the fields 'boss' and 'webpage', yes, they > > reference two other Persistent entities. You can see the schema of > Employee > > [1], were it has a boss (referencing itself) and a webpage (with the > > WebPage schema defined there). > > > > How to handle this depends on the actual backend. In HBase we serialize > > with avro each of the fields that are Persistent entities as shown at [2] > > (we get there from [3] > [4] > [5], but this is for HBase). > > On the other hand, if I am not worng, MongoDB Store has no problem with > > hierarchical data, so it just writes it as it is. > > > > In your case, it depends on the features of your backend and what > approach > > wold be better overall. I am seeing that Ignite has SQL + key/value > > properties... how interesting :) > > I know nothing about, but I would think about the performance costs of > > joined tables, the possibility of mapping each persistent into a table, > or > > the approach like HBase serializing (as a key/value store). Or maybe > Ignite > > has some functionality for complex data. > > Just thinking about it without knowing. > > > > If I left things unclear don't doubt to ask :) > > > > Regards, > > > > Alfonso > > > > > > [1] > > https://github.com/apache/gora/blob/apache-gora-0.8/ > > gora-core/src/examples/avro/employee.json > > [2] > > https://github.com/apache/gora/blob/apache-gora-0.8/ > > gora-hbase/src/main/java/org/apache/gora/hbase/util/ > > HBaseByteInterface.java#L233 > > [3] > > https://github.com/apache/gora/blob/apache-gora-0.8/ > > gora-hbase/src/main/java/org/apache/gora/hbase/store/ > HBaseStore.java#L240 > > [4] > > https://github.com/apache/gora/blob/apache-gora-0.8/ > > gora-hbase/src/main/java/org/apache/gora/hbase/store/ > HBaseStore.java#L262 > > [5] > > https://github.com/apache/gora/blob/apache-gora-0.8/ > > gora-hbase/src/main/java/org/apache/gora/hbase/store/ > HBaseStore.java#L331 > > > > > > 2018-06-06 4:38 GMT-01:00 carlos muñoz <[email protected]>: > > > > > Hi all, > > > > > > Currently I am working on the Ignite backend, I am using my own fork > [1]. > > > This is what I have done so far: > > > * Read mapping files. > > > * Read configuration files (Ignite parameters). > > > * Initialize Ignite DataStore through JDBC connection. > > > * Implement some trivial methods (#close(), #flush(), > > #getSchemaName()...). > > > * Set up an embedded Ignite instance within test packages. > > > > > > The code is broken because I created a DataStoreTestBase implementation > > for > > > Ignite and due the fact that most DataStore methods are not implemented > > yet > > > the tests are failing. But the mapping/configuration reading and > > connection > > > initialization are working fine. > > > > > > I noticed that the mapping files (gora-XX-mapping.xml) have two fields > > > 'boss' and 'webpage', which I assume are references to other objects > > (other > > > mapped classes). I was wondering how the backend should deal with them. > > Is > > > there some documentation regarding that?. > > > > > > Regards, > > > Carlos > > > > > > > > > [1] https://github.com/carlosrmng/gora/tree/GORA-535 > > > > > >
