hadoop-yetus commented on a change in pull request #1003: HADOOP-16384: prune resilience. URL: https://github.com/apache/hadoop/pull/1003#discussion_r297437262
########## File path: hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md ########## @@ -1100,6 +1100,51 @@ property should be configured, and the name of that table should be different incurring AWS charges. +### How to dump the table to a CSV file + +There's an unstable, unsupported command to list the contents of a table +to a CSV, or more specifically a TSV file, on the local system + +``` +hadoop org.apache.hadoop.fs.s3a.s3guard.DumpS3GuardTable s3a://bucket-x/ out.csv +``` +This generates a file which can then be viewed on the command line or editor: + +``` +"path" "type" "is_auth_dir" "deleted" "is_empty_dir" "len" "updated" "updated_s" "last_modified" "last_modified_s" "etag" "version" +"s3a://bucket-x/FileSystemContractBaseTest" "file" "false" "true" "UNKNOWN" 0 1561484415455 "Tue Jun 25 18:40:15 BST 2019" 1561483826881 "Tue Jun 25 18:30:26 BST 2019" "" "" +"s3a://bucket-x/Users" "file" "false" "true" "UNKNOWN" 0 1561484415455 "Tue Jun 25 18:40:15 BST 2019" 1561484376835 "Tue Jun 25 18:39:36 BST 2019" "" "" +"s3a://bucket-x/dest-6f578c72-eb40-4767-a89d-66a6a5b89578" "file" "false" "true" "UNKNOWN" 0 1561484415455 "Tue Jun 25 18:40:15 BST 2019" 1561483757615 "Tue Jun 25 18:29:17 BST 2019" "" "" +"s3a://bucket-x/file.txt" "file" "false" "true" "UNKNOWN" 0 1561484415455 "Tue Jun 25 18:40:15 BST 2019" 1561484382603 "Tue Jun 25 18:39:42 BST 2019" "" "" +"s3a://bucket-x/fork-0001" "file" "false" "true" "UNKNOWN" 0 1561484415455 "Tue Jun 25 18:40:15 BST 2019" 1561484378086 "Tue Jun 25 18:39:38 BST 2019" "" "" +"s3a://bucket-x/fork-0002" "file" "false" "true" "UNKNOWN" 0 1561484415455 "Tue Jun 25 18:40:15 BST 2019" 1561484380177 "Tue Jun 25 18:39:40 BST 2019" "" "" +"s3a://bucket-x/fork-0003" "file" "false" "true" "UNKNOWN" 0 1561484415455 "Tue Jun 25 18:40:15 BST 2019" 1561484379690 "Tue Jun 25 18:39:39 BST 2019" "" "" +``` + +This is unstable: the output format may change without warning. +To understand the meaning of the fields, consult the documentation. +They are, currently: + +| field | meaning | source | +|-------|---------| -------| +| `path` | path of an entry | filestatus | +| `type` | type | filestatus | +| `is_auth_dir` | directory entry authoritative status | metadata | +| `deleted` | tombstone marker | metadata | +| `is_empty_dir` | does the entry represent an empty directory | metadata | Review comment: whitespace:end of line ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org