Hi, we have a more or less small problem with DataFrame creation.
In our current project we have a more or less complex data model corresponding to documents and word positions in it. In the last month we refactor our architecture to use neo4j as persistence. Before that we used PostgreSQL and try to implement all old algorithms with spark. Currently we are not using a spark database connector but we will integrate it in the next few weeks. Now we have the problem, that neo4j in combination with spring data had a impact on our domain models such that we have the a reference of models the current object relates to. That ends in cyclic references and we don’t know how to resolve this issue. The problem is, that we already implement a database abstraction layer in a fashion that we don’t have to worry about how to get the data, we only have to specify which data we need. So we use reflection at a few locations. We have now a few approaches in mind: 1. Ask our domain for simple objects without cyclic references 2. Implement our own Encoders Are we missing something. We appreciate every hint so resolve this. Kind Regards, Pascal Stammer --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
