Hmm, then what Stephan suggested might indeed be the best solution.
On Fri, Dec 5, 2014 at 5:43 PM, Paris Carbone <par...@kth.se> wrote: > @Alioscha It will be initially binary data (byte[]) from sensor streams > transformed into Hashmaps of numeric types and some additional metadata. > > Paris > >> On 05 Dec 2014, at 13:14, Paris Carbone <par...@kth.se> wrote: >> >> Great, thanks both for the recommendations! >> >> Paris >> >>> On 05 Dec 2014, at 12:56, Stephan Ewen <se...@apache.org> wrote: >>> >>> If you use Flink's LongValue, StringValue, DoubleValue, you can also use >>> Flink's MapValue. If you subclass it (without adding code) it is very >>> efficient. It stores type information in the subclass and that way handles >>> key and value types. >>> >>> On Fri, Dec 5, 2014 at 11:33 AM, Robert Metzger <rmetz...@apache.org> wrote: >>> >>>> As possible workarounds, you could >>>> a) Implement your own serialization by implementing the "Value" interface. >>>> b) Use the Hadoop >>>> >>>> http://hadoop.apache.org/docs/r2.3.0/api/org/apache/hadoop/io/MapWritable.html >>>> MapWritable Class. You have to use hadoop's LongWritable and IntWritable >>>> for the types of the map but Flink should be able to handle Writable's in >>>> POJOs. >>>> >>>> I would recommend option b). >>>> >>>> On Fri, Dec 5, 2014 at 10:22 AM, Stephan Ewen <se...@apache.org> wrote: >>>> >>>>> We are in the midst of replacing Avro in the serialization, so this >>>> should >>>>> spoon be fixed properly. >>>>> >>>>> Until then, you could try to re-package the collection. Something like an >>>>> array of map entries, or so. Would that feasible? >>>>> >>>>> Stephan >>>>> Am 04.12.2014 21:42 schrieb "Aljoscha Krettek" <aljos...@apache.org>: >>>>> >>>>>> I don't know any workaround. But maybe Avro should be avoided >>>>>> altogether for your requirements. >>>>>> >>>>>> What is the data that you want to move between operations? >>>>>> >>>>>> On Thu, Dec 4, 2014 at 7:13 PM, Paris Carbone <par...@kth.se> wrote: >>>>>>> Hello, >>>>>>> >>>>>>> It seems that Avro fails to serialise POJOs that contain non-String >>>> or >>>>>> stringable keys< >>>>>> >>>>> >>>> https://apache.googlesource.com/avro/+/40650540dcb8ca8a6b6235de5cdd36c0f6e2eb31/lang/java/avro/src/main/java/org/apache/avro/reflect/ReflectData.java#361 >>>>>>> . >>>>>>> eg. in the example here< >>>>>> >>>>> >>>> https://github.com/senorcarbone/incubator-flink/blob/72b6798b50396c962fc6cea20a2bcdd51eec06f4/flink-examples/flink-java-examples/src/main/java/org/apache/flink/examples/java/testing/LongMapKeyIssueExample.java >>>>>> >>>>>> I get a compiler exception caused by: >>>>>>> >>>>>>> org.apache.avro.AvroTypeException: Map key class not String: class >>>>>> java.lang.Long >>>>>>> >>>>>>> Is there any known workaround/recommendation for this except for >>>> using >>>>>> String keys? >>>>>>> I need this for a use case of low latency data-intensive streaming so >>>>>> String conversions should be avoided. >>>>>>> >>>>>>> Paris >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >> >
