[
https://issues.apache.org/jira/browse/APEXMALHAR-2077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15285653#comment-15285653
]
ASF GitHub Bot commented on APEXMALHAR-2077:
--------------------------------------------
Github user amberarrow commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/273#discussion_r63443902
--- Diff:
library/src/main/java/com/datatorrent/lib/io/fs/AbstractSingleFileOutputOperator.java
---
@@ -39,10 +41,37 @@
@NotNull
protected String outputFileName;
+ /**
+ * partitionedFileName string format specifier
+ e.g. fileName_physicalPartionId -> %s_%d
+ */
+ private String partitionedFileNameformat = "%s_%d";
+
+ /**
+ * Derived name for file based on physicalPartitionId
+ */
+ private transient String partitionedFileName;
+
+ /**
+ * Physical partition id for the current partition.
+ */
+ private transient int physicalPartitionId;
+
+ /**
+ * Initializing current partition id, partitionedFileName etc.
{@inheritDoc}
+ */
+ @Override
+ public void setup(OperatorContext context)
--- End diff --
Yogi, if you can just implement the check if the format string is null or
empty, we can merge this. We can defer the issue of whether dynamically
changing the file name is supported for the next release. We have no guarantee
on the value of the operator id (not guaranteed to start at 0).
> SingleFileOutputOperator should append partitionId to file name
> ---------------------------------------------------------------
>
> Key: APEXMALHAR-2077
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2077
> Project: Apache Apex Malhar
> Issue Type: Improvement
> Reporter: Yogi Devendra
> Assignee: Yogi Devendra
> Priority: Minor
> Fix For: 3.4.0
>
>
> If SingleFileOutputOperator is partitioned into more than one physical
> instances then it should append partitionId to file name. So that, each
> partition can independently write to separate file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
