[
https://issues.apache.org/jira/browse/PARQUET-154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manish Agarwal updated PARQUET-154:
-----------------------------------
Description:
I am trying to create a file in parquet file format and in RC file format .
No Parquet constructor accepts fileSystem object as an argument. This means
that i will have to append the uri from the file system in front of the
filepath object everytime i need to create a new file .
In RC format the file system object is allowed to be passed in the constructor
.
The advantage of passing the file System object into the constructor is that i
can specify my yarn instance file system pointer to be used while creating
the file and its quite straight forward . For example RC file constructors
uses the file System which we have passed .
In parquet i see everywhere file System object being derived out of the
Parameters or created a fresh .
Is there some reason we avoided using the fileSystem object in constructor ?
If we allow a file System object constructor as well , I would not have to
worry about modifying my file name to contain the uri part .
was:
I am trying to create a file in parquet file format and in RC file format .
Np Parquet constructor accepts fileSystem object as an argument. This means
that i will have to append the uri from the file system in front of the
filepath object everytime i need to create a new file .
In RC format the file system object is allowed to be passed in the constructor
.
The advantage of passing the file System object into the constructor is that i
can specify my yarn instance file system pointer to be used while creating
the file and its quite straight forward . For example RC file constructors
uses the file System which we have passed .
In parquet i see everywhere file System object being derived out of the
Parameters or created a fresh .
Is there some reason we avoided using the fileSystem object in constructor ?
If we allow a file System object constructor as well , I would not have to
worry about modifying my file name to contain the uri part .
> parquet Constructors not taking file System object
> --------------------------------------------------
>
> Key: PARQUET-154
> URL: https://issues.apache.org/jira/browse/PARQUET-154
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
> Affects Versions: parquet-mr_1.6.0
> Reporter: Manish Agarwal
>
> I am trying to create a file in parquet file format and in RC file format .
>
> No Parquet constructor accepts fileSystem object as an argument. This means
> that i will have to append the uri from the file system in front of the
> filepath object everytime i need to create a new file .
> In RC format the file system object is allowed to be passed in the
> constructor .
> The advantage of passing the file System object into the constructor is that
> i can specify my yarn instance file system pointer to be used while
> creating the file and its quite straight forward . For example RC file
> constructors uses the file System which we have passed .
> In parquet i see everywhere file System object being derived out of the
> Parameters or created a fresh .
> Is there some reason we avoided using the fileSystem object in constructor ?
> If we allow a file System object constructor as well , I would not have to
> worry about modifying my file name to contain the uri part .
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
