[
https://issues.apache.org/jira/browse/BEAM-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15204593#comment-15204593
]
Frances Perry commented on BEAM-138:
------------------------------------
TextIO is really about a specific file format -- it requires
newline-deliminated records. It'd be great to increase the number of things it
can read those from though. [~dhalp...@google.com] You probably know the status
of generalizing the file system?
> Extend TextIO to new protocols (and maybe rename to FileIO)
> -----------------------------------------------------------
>
> Key: BEAM-138
> URL: https://issues.apache.org/jira/browse/BEAM-138
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-extensions
> Reporter: Jean-Baptiste Onofré
> Assignee: Jean-Baptiste Onofré
>
> The current TextIO supports:
> - local file when using directly path like /path/to...
> - Google Service file using path like gs:...
> On the other hand, we have a contribution (from Tom) to support HDFS.
> For an user perspective, it would be easier to use an unique IO supporting
> different protocol:
> - file:
> - gs:
> - hdfs:
> - mvn:
> - ...
> It would also be convenient to be able to combine protocols and eventually
> use a different coder (for instance xml:hdfs:).
> In that case, maybe I would make sense to rename TextIO as generic FileIO.
> Thoughts ?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
