[
https://issues.apache.org/jira/browse/NIFI-1118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201678#comment-15201678
]
Joseph Witt commented on NIFI-1118:
-----------------------------------
Thanks [~markbean]. That does match my understanding so wanted to be sure.
I agree with you the current feature is not sufficiently implemented and has a
bug that should be fixed.
I do not share the view though that the best course of action for our user base
is to remove the feature at this time. That is because there are plenty of
cases for which it works perfectly well including those for which it was
originally created. Indeed that realization is why i am concerned.
I am also fully in support of the original intent of this ticket which is to
add features to improve the safety of processing given large line lengths and
the filtering of header lines.
It is clear though in adding those features other issues were discovered and
the approach taken to get there in my opinion leaves us too exposed for the
user base at this time.
I propose the following:
- Create a JIRA for 0.6.1 or next appropriate release to fix the defect in the
current feature and to document that this feature may be removed in the next
major release.
- Create a JIRA as this one was originally intended to specifically add the
features to limit line length and to filter header lines.
- Create a JIRA for 1.x that removes the line ending feature and accounts for
the necessary user migration/documentation guidance.
I remain apologetic for waffling as I realize it leaves you feeling like you
wasted your time. That was completely my mistake and I realize I created this
angst. I'm asking for your assistance in working through that mistake in a way
that will not hurt current legit uses of this processor and which allows all
the other great effort that has gone into the 0.6.0 release to continue.
> Enable SplitText processor to limit line length and filter header lines
> -----------------------------------------------------------------------
>
> Key: NIFI-1118
> URL: https://issues.apache.org/jira/browse/NIFI-1118
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Mark Bean
> Assignee: Joe Skora
> Fix For: 0.6.0
>
>
> Include the following functionality to the SplitText processor:
> 1) Maximum size limit of the split file(s)
> A new split file will be created if the next line to be added to the current
> split file exceeds a user-defined maximum file size
> 2) Header line marker
> User-defined character(s) can be used to identify the header line(s) of the
> data file rather than a predetermined number of lines
> These changes are additions, not a replacement of any property or behavior.
> In the case of header line marker, the existing property "Header Line Count"
> must be zero for the new property and behavior to be used.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
