steveloughran commented on a change in pull request #1792: HADOOP-16790: Add
Write Convenience Methods
URL: https://github.com/apache/hadoop/pull/1792#discussion_r363253470
##########
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileUtil.java
##########
@@ -1633,4 +1638,119 @@ public static boolean compareFs(FileSystem srcFs,
FileSystem destFs) {
// check for ports
return srcUri.getPort()==dstUri.getPort();
}
+
+ /**
+ * Writes bytes to a file. This utility method opens the file for writing,
+ * creating the file if it does not exist, or overwrites an existing file.
All
+ * bytes in the byte array are written to the file.
+ *
+ * @param fs the files system with which to create the file
+ * @param path the path to the file
+ * @param bytes the byte array with the bytes to write
+ *
+ * @return the file system
+ *
+ * @throws NullPointerException if any of the arguments are {@code null}
+ * @throws IOException if an I/O error occurs creating or writing to the file
+ */
+ public static FileSystem write(final FileSystem fs, final Path path,
+ final byte[] bytes) throws IOException {
+
+ Objects.requireNonNull(path);
+ Objects.requireNonNull(bytes);
+
+ try (FSDataOutputStream out = fs.create(path)) {
+ out.write(bytes);
+ }
+
+ return fs;
+ }
+
+ /**
+ * Write lines of text to a file. Each line is a char sequence and is written
+ * to the file in sequence with each line terminated by the platform's line
+ * separator, as defined by the system property {@code
+ * line.separator}. Characters are encoded into bytes using the specified
+ * charset. This utility method opens the file for writing, creating the file
+ * if it does not exist, or overwrites an existing file.
+ *
+ * @param fs the files system with which to create the file
+ * @param path the path to the file
+ * @param lines a Collection to iterate over the char sequences
+ * @param cs the charset to use for encoding
+ *
+ * @return the file system
+ *
+ * @throws NullPointerException if any of the arguments are {@code null}
+ * @throws IOException if an I/O error occurs creating or writing to the file
+ */
+ public static FileSystem write(final FileSystem fs, final Path path,
+ final Iterable<? extends CharSequence> lines, final Charset cs)
+ throws IOException {
+
+ Objects.requireNonNull(path);
+ Objects.requireNonNull(lines);
+ Objects.requireNonNull(cs);
+
+ CharsetEncoder encoder = cs.newEncoder();
+ try (FSDataOutputStream out = fs.create(path);
+ BufferedWriter writer =
+ new BufferedWriter(new OutputStreamWriter(out, encoder))) {
+ for (CharSequence line : lines) {
+ writer.append(line);
+ writer.newLine();
+ }
+ }
+ return fs;
+ }
+
+ /**
+ * Write a line of text to a file. Characters are encoded into bytes using
the
+ * specified charset. This utility method opens the file for writing,
creating
+ * the file if it does not exist, or overwrites an existing file.
+ *
+ * @param fs the files system with which to create the file
+ * @param path the path to the file
+ * @param charseq the char sequence to write to the file
+ * @param cs the charset to use for encoding
+ *
+ * @return the file system
+ *
+ * @throws NullPointerException if any of the arguments are {@code null}
+ * @throws IOException if an I/O error occurs creating or writing to the file
+ */
+ public static FileSystem write(final FileSystem fs, final Path path,
Review comment:
add in overwrite options. We've been dealing with 404 caching in S3A, which
relies on createFile(overwrite = false). Unless you make the default, it must
be something callers can use.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
