Github user amberarrow commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/273#discussion_r63195600
--- Diff:
library/src/main/java/com/datatorrent/lib/io/fs/AbstractSingleFileOutputOperator.java
---
@@ -39,10 +41,37 @@
@NotNull
protected String outputFileName;
+ /**
+ * partitionedFileName string format specifier
+ e.g. fileName_physicalPartionId -> %s_%d
+ */
+ private String partitionedFileNameformat = "%s_%d";
+
+ /**
+ * Derived name for file based on physicalPartitionId
+ */
+ private transient String partitionedFileName;
+
+ /**
+ * Physical partition id for the current partition.
+ */
+ private transient int physicalPartitionId;
+
+ /**
+ * Initializing current partition id, partitionedFileName etc.
{@inheritDoc}
+ */
+ @Override
+ public void setup(OperatorContext context)
--- End diff --
I'm wondering why we even need this class; can we not simply add a boolean
property to the base class called "appendPartitionId" ?
Also, the ticket description says "If SingleFileOutputOperator is
partitioned into **more than one** physical instances then it should append
partitionId to file name" (my emphasis), I don't see any check for that
condition before appending the partition id.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
