[
https://issues.apache.org/jira/browse/PARQUET-154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257952#comment-14257952
]
Manish Agarwal edited comment on PARQUET-154 at 12/24/14 6:27 AM:
------------------------------------------------------------------
for write : If the path is not Qualified and we pass path and file system
separately in a write call then there should not be any issue . it would open
a handle using the file System on the unqualified path .
once we do it , ParquetFileWriter(FileSystem fs, MessageType schema, Path
file) wil not be required to do FileSystem fs =
file.getFileSystem(configuration); as it will use the file system passed form
the user
was (Author: manish.agarwal):
for write : If the path is not Qualified and we pass path and file system
separately in a write call then there should not be any issue . it would open
a handle using the file System on the unqualified path .
once we do it , ParquetFileWriter(FileSystem fs, MessageType schema, Path
file) wil not be required to so FileSystem fs =
file.getFileSystem(configuration); as it will use the file system passed form
the user
> parquet Constructors not taking file System object
> --------------------------------------------------
>
> Key: PARQUET-154
> URL: https://issues.apache.org/jira/browse/PARQUET-154
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
> Affects Versions: parquet-mr_1.6.0
> Reporter: Manish Agarwal
>
> I am trying to create a file in parquet file format and in RC file format .
>
> No Parquet constructor accepts fileSystem object as an argument. This means
> that i will have to append the uri from the file system in front of the
> filepath object everytime i need to create a new file .
> In RC format the file system object is allowed to be passed in the
> constructor .
> The advantage of passing the file System object into the constructor is that
> i can specify my yarn instance file system pointer to be used while
> creating the file and its quite straight forward . For example RC file
> constructors uses the file System which we have passed .
> In parquet i see everywhere file System object being derived out of the
> Parameters or created a fresh .
> Is there some reason we avoided using the fileSystem object in constructor ?
> If we allow a file System object constructor as well , I would not have to
> worry about modifying my file name to contain the uri part .
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
