Github user ilooner commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/262#discussion_r61682347
--- Diff:
library/src/main/java/org/apache/apex/malhar/lib/state/spillable/SpillableComplexComponent.java
---
@@ -105,6 +105,26 @@
Serde<V, byte[]> serdeValue);
/**
+ * This is a method for creating a {@link SpillableByteMultiset}. This
method
+ * auto-generates an identifier for the data structure.
+ * @param <T> The type of the elements.
+ * @param bucket The bucket that this {@link SpillableByteMultiset} will
be spilled too.
+ * @param serde The Serializer/Deserializer to use for data stored in
the {@link SpillableByteMultiset}.
+ * @return A {@link SpillableByteMultiset}.
+ */
+ <T> SpillableByteMultiset<T> newSpillableMultiset(long bucket, Serde<T,
byte[]> serde);
+
+ /**
+ * This is a method for creating a {@link SpillableByteMultiset}.
+ * @param <T> The type of the elements.
+ * @param identifier The identifier for this {@link
SpillableByteMultiset}.
+ * @param bucket The bucket that this {@link SpillableByteMultiset} will
be spilled too.
+ * @param serde The Serializer/Deserializer to use for data stored in
the {@link SpillableByteMultiset}.
+ * @return A {@link SpillableByteMultiset}.
+ */
+ <T> SpillableByteMultiset<T> newSpillableMultiset(byte[] identifier,
long bucket, Serde<T, byte[]> serde);
--- End diff --
@tweise The multiset just keeps track of each element and its count. That
could be represented as a map. Event time could be extracted from the tuple or
processing time could be used to distribute the data over multiple data files.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
