[
https://issues.apache.org/jira/browse/PARQUET-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14267901#comment-14267901
]
Mariappan Asokan commented on PARQUET-134:
------------------------------------------
Created pull request https://github.com/apache/incubator-parquet-mr/pull/100
> Enhance ParquetWriter with file creation flag
> ---------------------------------------------
>
> Key: PARQUET-134
> URL: https://issues.apache.org/jira/browse/PARQUET-134
> Project: Parquet
> Issue Type: Improvement
> Reporter: Mariappan Asokan
>
> Currently, Parquet files can be created only with "create in exclusive mode"
> (like {{O_CREAT|O_EXCL}} in UNIX {{open}} system call.) This may be
> inconvenient in a situation where user knows that overwriting an existing
> file is okay. This is especially true for Parquet files created on a local
> disk. For example, a user might want to specify a named pipe (on UNIX) as
> the target, encrypt the bytes on the fly by reading from the named pipe in
> another process, and write to disk. The named pipe file has to exist first.
> I am thinking that we can have OVERWRITE and CREATE modes for now. If
> appending to Parquet files is supported in the future, we can add APPEND mode
> later. These mode flags can be defined as constants in
> {{ParquetWriter.java.}}
> There are 7 constructors in {{ParquetWriter}} class. I am thinking of adding
> one more that takes the mode flag (and all the arguments of the constructor
> with the most arguments today.) Also, a new constructor of
> {{ParquetFileWriter}} will take the mode flag as an argument and the
> following statement
> {code}
> this.out = fs.create(file, false);
> {code}
> will be modified so that for OVERWRITE mode, the second argument to
> {{create()}} will be set to {{true.}}
> I can submit a patch with the above changes and a test. Committers, please
> give your feedback with suggestions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
