[ https://issues.apache.org/jira/browse/BEAM-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143701#comment-16143701 ]
Etienne Chauchot commented on BEAM-2802: ---------------------------------------- - text files could have no new line character at all (one line text file) but only the custom delimiters - Adding a prior parse and replace (custom delimiter to \n for example) treatment will be counter productive on big files. For your first question: the delimiter could be n bytes containing no {{\n}} or, we have a use case in which the record delimiter is {{\002\n}} and the file contains also {{\n}} that must be inside the output record. For your second question: if the text file contains both {{\r\n}} and the delimiter, the output records should be single Strings that internally contain {{\r\n}}. > TextIO should allow specifying a custom delimiter > ------------------------------------------------- > > Key: BEAM-2802 > URL: https://issues.apache.org/jira/browse/BEAM-2802 > Project: Beam > Issue Type: New Feature > Components: sdk-java-extensions > Reporter: Etienne Chauchot > Assignee: Etienne Chauchot > Priority: Minor > > Currently TextIO use {{\r}} {{\n}} or {{\r\n}} or a mix of the two to split a > text file into PCollection elements. It might happen that a record is spread > across more than one line. In that case we should be able to specify a custom > record delimiter to be used in place of the default ones. -- This message was sent by Atlassian JIRA (v6.4.14#64029)