Hello, I’m working on the Emma integration, and I would need to write a generic DataQuanta to a temporary file, and then read it back later. What would be the best way to do this? It’s not trivial because I don’t know the concrete type of the DataQuanta, i.e., I’m just working with DataQuanta[A]. (I have a ClassTag for A.)
For example, the same functionality is achieved when Emma compiles to Flink by writing and reading the DataSet[A] in a binary format with the serializer that Flink has for A: https://github.com/emmalanguage/emma/blob/master/emma-flink/src/main/scala/org/emmalanguage/api/flink/FlinkOps.scala#L76-L92 At first, I thought that the ObjectFileSinks would help, but now I’m not sure. The ObjectFileSinks seem to be available only at the backend level. Best, Gábor
