subject:"\[jira\] \[Updated\] \(HDFS\-13744\) OIV tool should better handle control characters present in file or directory names"

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

2018-09-12 Thread Sean Mackrory (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-13744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Mackrory updated HDFS-13744:
-
   Resolution: Fixed
Fix Version/s: 3.2.0
   Status: Resolved  (was: Patch Available)

> OIV tool should better handle control characters present in file or directory 
> names
> ---
>
> Key: HDFS-13744
> URL: https://issues.apache.org/jira/browse/HDFS-13744
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: hdfs, tools
>Affects Versions: 2.6.5, 2.9.1, 2.8.4, 2.7.6, 3.0.3
>Reporter: Zsolt Venczel
>Assignee: Zsolt Venczel
>Priority: Critical
> Fix For: 3.2.0
>
> Attachments: HDFS-13744.01.patch, HDFS-13744.02.patch, 
> HDFS-13744.03.patch
>
>
> In certain cases when control characters or white space is present in file or 
> directory names OIV tool processors can export data in a misleading format.
> In the below examples we have EXAMPLE_NAME as a file and a directory name 
> where the directory has a line feed character at the end (the actual 
> production case has multiple line feeds and multiple spaces)
>  * Delimited processor case:
>  ** misleading example:
> {code:java}
> /user/data/EXAMPLE_NAME
> ,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
> /user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
> 10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
> {code}
>  * 
>  ** expected example as suggested by 
> [https://tools.ietf.org/html/rfc4180#section-2]:
> {code:java}
> "/user/data/EXAMPLE_NAME%x0A",0,2017-04-24 04:34,1969-12-31 
> 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
> "/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
> 10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
> {code}
>  * XML processor case:
>  ** misleading example:
> {code:java}
> 479867791DIRECTORYEXAMPLE_NAME
> 1493033668294user:group:0775
> 113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
> {code}
>  * 
>  ** expected example as specified in 
> [https://www.w3.org/TR/REC-xml/#sec-line-ends]:
> {code:java}
> 479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775
> 113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
> {code}
>  * JSON:
>  The OIV Web Processor behaves correctly and produces the following:
> {code:java}
> {
>   "FileStatuses": {
> "FileStatus": [
>   {
> "fileId": 113632535,
> "accessTime": 1494954320141,
> "replication": 3,
> "owner": "user",
> "length": 520,
> "permission": "674",
> "blockSize": 134217728,
> "modificationTime": 1472205657504,
> "type": "FILE",
> "group": "group",
> "childrenNum": 0,
> "pathSuffix": "EXAMPLE_NAME"
>   },
>   {
> "fileId": 479867791,
> "accessTime": 0,
> "replication": 0,
> "owner": "user",
> "length": 0,
> "permission": "775",
> "blockSize": 0,
> "modificationTime": 1493033668294,
> "type": "DIRECTORY",
> "group": "group",
> "childrenNum": 0,
> "pathSuffix": "EXAMPLE_NAME\n"
>   }
> ]
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

2018-09-04 Thread Sean Mackrory (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-13744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Mackrory updated HDFS-13744:
-
Attachment: HDFS-13744.03.patch

> OIV tool should better handle control characters present in file or directory 
> names
> ---
>
> Key: HDFS-13744
> URL: https://issues.apache.org/jira/browse/HDFS-13744
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: hdfs, tools
>Affects Versions: 2.6.5, 2.9.1, 2.8.4, 2.7.6, 3.0.3
>Reporter: Zsolt Venczel
>Assignee: Zsolt Venczel
>Priority: Critical
> Attachments: HDFS-13744.01.patch, HDFS-13744.02.patch, 
> HDFS-13744.03.patch
>
>
> In certain cases when control characters or white space is present in file or 
> directory names OIV tool processors can export data in a misleading format.
> In the below examples we have EXAMPLE_NAME as a file and a directory name 
> where the directory has a line feed character at the end (the actual 
> production case has multiple line feeds and multiple spaces)
>  * Delimited processor case:
>  ** misleading example:
> {code:java}
> /user/data/EXAMPLE_NAME
> ,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
> /user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
> 10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
> {code}
>  * 
>  ** expected example as suggested by 
> [https://tools.ietf.org/html/rfc4180#section-2]:
> {code:java}
> "/user/data/EXAMPLE_NAME%x0A",0,2017-04-24 04:34,1969-12-31 
> 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
> "/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
> 10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
> {code}
>  * XML processor case:
>  ** misleading example:
> {code:java}
> 479867791DIRECTORYEXAMPLE_NAME
> 1493033668294user:group:0775
> 113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
> {code}
>  * 
>  ** expected example as specified in 
> [https://www.w3.org/TR/REC-xml/#sec-line-ends]:
> {code:java}
> 479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775
> 113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
> {code}
>  * JSON:
>  The OIV Web Processor behaves correctly and produces the following:
> {code:java}
> {
>   "FileStatuses": {
> "FileStatus": [
>   {
> "fileId": 113632535,
> "accessTime": 1494954320141,
> "replication": 3,
> "owner": "user",
> "length": 520,
> "permission": "674",
> "blockSize": 134217728,
> "modificationTime": 1472205657504,
> "type": "FILE",
> "group": "group",
> "childrenNum": 0,
> "pathSuffix": "EXAMPLE_NAME"
>   },
>   {
> "fileId": 479867791,
> "accessTime": 0,
> "replication": 0,
> "owner": "user",
> "length": 0,
> "permission": "775",
> "blockSize": 0,
> "modificationTime": 1493033668294,
> "type": "DIRECTORY",
> "group": "group",
> "childrenNum": 0,
> "pathSuffix": "EXAMPLE_NAME\n"
>   }
> ]
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

2018-08-30 Thread Zsolt Venczel (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-13744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zsolt Venczel updated HDFS-13744:
-
Attachment: HDFS-13744.02.patch

> OIV tool should better handle control characters present in file or directory 
> names
> ---
>
> Key: HDFS-13744
> URL: https://issues.apache.org/jira/browse/HDFS-13744
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: hdfs, tools
>Affects Versions: 2.6.5, 2.9.1, 2.8.4, 2.7.6, 3.0.3
>Reporter: Zsolt Venczel
>Assignee: Zsolt Venczel
>Priority: Critical
> Attachments: HDFS-13744.01.patch, HDFS-13744.02.patch
>
>
> In certain cases when control characters or white space is present in file or 
> directory names OIV tool processors can export data in a misleading format.
> In the below examples we have EXAMPLE_NAME as a file and a directory name 
> where the directory has a line feed character at the end (the actual 
> production case has multiple line feeds and multiple spaces)
>  * Delimited processor case:
>  ** misleading example:
> {code:java}
> /user/data/EXAMPLE_NAME
> ,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
> /user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
> 10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
> {code}
>  * 
>  ** expected example as suggested by 
> [https://tools.ietf.org/html/rfc4180#section-2]:
> {code:java}
> "/user/data/EXAMPLE_NAME%x0A",0,2017-04-24 04:34,1969-12-31 
> 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
> "/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
> 10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
> {code}
>  * XML processor case:
>  ** misleading example:
> {code:java}
> 479867791DIRECTORYEXAMPLE_NAME
> 1493033668294user:group:0775
> 113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
> {code}
>  * 
>  ** expected example as specified in 
> [https://www.w3.org/TR/REC-xml/#sec-line-ends]:
> {code:java}
> 479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775
> 113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
> {code}
>  * JSON:
>  The OIV Web Processor behaves correctly and produces the following:
> {code:java}
> {
>   "FileStatuses": {
> "FileStatus": [
>   {
> "fileId": 113632535,
> "accessTime": 1494954320141,
> "replication": 3,
> "owner": "user",
> "length": 520,
> "permission": "674",
> "blockSize": 134217728,
> "modificationTime": 1472205657504,
> "type": "FILE",
> "group": "group",
> "childrenNum": 0,
> "pathSuffix": "EXAMPLE_NAME"
>   },
>   {
> "fileId": 479867791,
> "accessTime": 0,
> "replication": 0,
> "owner": "user",
> "length": 0,
> "permission": "775",
> "blockSize": 0,
> "modificationTime": 1493033668294,
> "type": "DIRECTORY",
> "group": "group",
> "childrenNum": 0,
> "pathSuffix": "EXAMPLE_NAME\n"
>   }
> ]
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

2018-08-16 Thread Zsolt Venczel (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-13744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zsolt Venczel updated HDFS-13744:
-
Status: Patch Available  (was: In Progress)

> OIV tool should better handle control characters present in file or directory 
> names
> ---
>
> Key: HDFS-13744
> URL: https://issues.apache.org/jira/browse/HDFS-13744
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: hdfs, tools
>Affects Versions: 3.0.3, 2.7.6, 2.8.4, 2.9.1, 2.6.5
>Reporter: Zsolt Venczel
>Assignee: Zsolt Venczel
>Priority: Critical
> Attachments: HDFS-13744.01.patch
>
>
> In certain cases when control characters or white space is present in file or 
> directory names OIV tool processors can export data in a misleading format.
> In the below examples we have EXAMPLE_NAME as a file and a directory name 
> where the directory has a line feed character at the end (the actual 
> production case has multiple line feeds and multiple spaces)
>  * Delimited processor case:
>  ** misleading example:
> {code:java}
> /user/data/EXAMPLE_NAME
> ,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
> /user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
> 10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
> {code}
>  * 
>  ** expected example as suggested by 
> [https://tools.ietf.org/html/rfc4180#section-2]:
> {code:java}
> "/user/data/EXAMPLE_NAME%x0A",0,2017-04-24 04:34,1969-12-31 
> 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
> "/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
> 10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
> {code}
>  * XML processor case:
>  ** misleading example:
> {code:java}
> 479867791DIRECTORYEXAMPLE_NAME
> 1493033668294user:group:0775
> 113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
> {code}
>  * 
>  ** expected example as specified in 
> [https://www.w3.org/TR/REC-xml/#sec-line-ends]:
> {code:java}
> 479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775
> 113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
> {code}
>  * JSON:
>  The OIV Web Processor behaves correctly and produces the following:
> {code:java}
> {
>   "FileStatuses": {
> "FileStatus": [
>   {
> "fileId": 113632535,
> "accessTime": 1494954320141,
> "replication": 3,
> "owner": "user",
> "length": 520,
> "permission": "674",
> "blockSize": 134217728,
> "modificationTime": 1472205657504,
> "type": "FILE",
> "group": "group",
> "childrenNum": 0,
> "pathSuffix": "EXAMPLE_NAME"
>   },
>   {
> "fileId": 479867791,
> "accessTime": 0,
> "replication": 0,
> "owner": "user",
> "length": 0,
> "permission": "775",
> "blockSize": 0,
> "modificationTime": 1493033668294,
> "type": "DIRECTORY",
> "group": "group",
> "childrenNum": 0,
> "pathSuffix": "EXAMPLE_NAME\n"
>   }
> ]
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

2018-08-16 Thread Zsolt Venczel (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-13744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zsolt Venczel updated HDFS-13744:
-
Description: 
In certain cases when control characters or white space is present in file or 
directory names OIV tool processors can export data in a misleading format.

In the below examples we have EXAMPLE_NAME as a file and a directory name where 
the directory has a line feed character at the end (the actual production case 
has multiple line feeds and multiple spaces)
 * Delimited processor case:
 ** misleading example:
{code:java}
/user/data/EXAMPLE_NAME
,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
/user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}

 * 
 ** expected example as suggested by 
[https://tools.ietf.org/html/rfc4180#section-2]:
{code:java}
"/user/data/EXAMPLE_NAME%x0A",0,2017-04-24 04:34,1969-12-31 
16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
"/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}

 * XML processor case:
 ** misleading example:
{code:java}
479867791DIRECTORYEXAMPLE_NAME
1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}

 * 
 ** expected example as specified in 
[https://www.w3.org/TR/REC-xml/#sec-line-ends]:
{code:java}
479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}

 * JSON:
 The OIV Web Processor behaves correctly and produces the following:
{code:java}
{
  "FileStatuses": {
"FileStatus": [
  {
"fileId": 113632535,
"accessTime": 1494954320141,
"replication": 3,
"owner": "user",
"length": 520,
"permission": "674",
"blockSize": 134217728,
"modificationTime": 1472205657504,
"type": "FILE",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME"
  },
  {
"fileId": 479867791,
"accessTime": 0,
"replication": 0,
"owner": "user",
"length": 0,
"permission": "775",
"blockSize": 0,
"modificationTime": 1493033668294,
"type": "DIRECTORY",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME\n"
  }
]
  }
}
{code}

  was:
In certain cases when control characters or white space is present in file or 
directory names OIV tool processors can export data in a misleading format.

In the below examples we have EXAMPLE_NAME as a file and a directory name where 
the directory has a line feed character at the end (the actual production case 
has multiple line feeds and multiple spaces)
 * CSV processor case:
 ** misleading example:
{code:java}
/user/data/EXAMPLE_NAME
,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
/user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}

 ** expected example as suggested by 
[https://tools.ietf.org/html/rfc4180#section-2]:
{code:java}
"/user/data/EXAMPLE_NAME%x0A",0,2017-04-24 04:34,1969-12-31 
16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
"/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}

 * XML processor case:
 ** misleading example:
{code:java}
479867791DIRECTORYEXAMPLE_NAME
1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}

 ** expected example as specified in 
[https://www.w3.org/TR/REC-xml/#sec-line-ends]:
{code:java}
479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}

 * JSON:
 The OIV Web Processor behaves correctly and produces the following:
{code:java}
{
  "FileStatuses": {
"FileStatus": [
  {
"fileId": 113632535,
"accessTime": 1494954320141,
"replication": 3,
"owner": "user",
"length": 520,
"permission": "674",
"blockSize": 134217728,
"modificationTime": 1472205657504,
"type": "FILE",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME"
  },
  {
"fileId": 479867791,
"accessTime": 0,
"replication": 0,
"owner": "user",
"length": 0,
"permission": "775",
"blockSize": 0,
"modificationTime": 1493033668294,
"type": "DIRECTORY",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME\n"
  }
]
  }
}
{code}


> OIV tool should better handle control characters present in file or directory 
> names
> ---
>
> Key: HDFS-13744

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

2018-08-16 Thread Zsolt Venczel (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-13744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zsolt Venczel updated HDFS-13744:
-
Attachment: HDFS-13744.01.patch

> OIV tool should better handle control characters present in file or directory 
> names
> ---
>
> Key: HDFS-13744
> URL: https://issues.apache.org/jira/browse/HDFS-13744
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: hdfs, tools
>Affects Versions: 2.6.5, 2.9.1, 2.8.4, 2.7.6, 3.0.3
>Reporter: Zsolt Venczel
>Assignee: Zsolt Venczel
>Priority: Critical
> Attachments: HDFS-13744.01.patch
>
>
> In certain cases when control characters or white space is present in file or 
> directory names OIV tool processors can export data in a misleading format.
> In the below examples we have EXAMPLE_NAME as a file and a directory name 
> where the directory has a line feed character at the end (the actual 
> production case has multiple line feeds and multiple spaces)
>  * CSV processor case:
>  ** misleading example:
> {code:java}
> /user/data/EXAMPLE_NAME
> ,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
> /user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
> 10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
> {code}
>  ** expected example as suggested by 
> [https://tools.ietf.org/html/rfc4180#section-2]:
> {code:java}
> "/user/data/EXAMPLE_NAME%x0A",0,2017-04-24 04:34,1969-12-31 
> 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
> "/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
> 10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
> {code}
>  * XML processor case:
>  ** misleading example:
> {code:java}
> 479867791DIRECTORYEXAMPLE_NAME
> 1493033668294user:group:0775
> 113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
> {code}
>  ** expected example as specified in 
> [https://www.w3.org/TR/REC-xml/#sec-line-ends]:
> {code:java}
> 479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775
> 113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
> {code}
>  * JSON:
>  The OIV Web Processor behaves correctly and produces the following:
> {code:java}
> {
>   "FileStatuses": {
> "FileStatus": [
>   {
> "fileId": 113632535,
> "accessTime": 1494954320141,
> "replication": 3,
> "owner": "user",
> "length": 520,
> "permission": "674",
> "blockSize": 134217728,
> "modificationTime": 1472205657504,
> "type": "FILE",
> "group": "group",
> "childrenNum": 0,
> "pathSuffix": "EXAMPLE_NAME"
>   },
>   {
> "fileId": 479867791,
> "accessTime": 0,
> "replication": 0,
> "owner": "user",
> "length": 0,
> "permission": "775",
> "blockSize": 0,
> "modificationTime": 1493033668294,
> "type": "DIRECTORY",
> "group": "group",
> "childrenNum": 0,
> "pathSuffix": "EXAMPLE_NAME\n"
>   }
> ]
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

2018-08-16 Thread Zsolt Venczel (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-13744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zsolt Venczel updated HDFS-13744:
-
Description: 
In certain cases when control characters or white space is present in file or 
directory names OIV tool processors can export data in a misleading format.

In the below examples we have EXAMPLE_NAME as a file and a directory name where 
the directory has a line feed character at the end (the actual production case 
has multiple line feeds and multiple spaces)
 * CSV processor case:
 ** misleading example:
{code:java}
/user/data/EXAMPLE_NAME
,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
/user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}

 ** expected example as suggested by 
[https://tools.ietf.org/html/rfc4180#section-2]:
{code:java}
"/user/data/EXAMPLE_NAME%x0A",0,2017-04-24 04:34,1969-12-31 
16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
"/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}

 * XML processor case:
 ** misleading example:
{code:java}
479867791DIRECTORYEXAMPLE_NAME
1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}

 ** expected example as specified in 
[https://www.w3.org/TR/REC-xml/#sec-line-ends]:
{code:java}
479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}

 * JSON:
 The OIV Web Processor behaves correctly and produces the following:
{code:java}
{
  "FileStatuses": {
"FileStatus": [
  {
"fileId": 113632535,
"accessTime": 1494954320141,
"replication": 3,
"owner": "user",
"length": 520,
"permission": "674",
"blockSize": 134217728,
"modificationTime": 1472205657504,
"type": "FILE",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME"
  },
  {
"fileId": 479867791,
"accessTime": 0,
"replication": 0,
"owner": "user",
"length": 0,
"permission": "775",
"blockSize": 0,
"modificationTime": 1493033668294,
"type": "DIRECTORY",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME\n"
  }
]
  }
}
{code}

  was:
In certain cases when control characters or white space is present in file or 
directory names OIV tool processors can export data in a misleading format.

In the below examples we have EXAMPLE_NAME as a file and a directory name where 
the directory has a line feed character at the end (the actual production case 
has multiple line feeds and multiple spaces)
 * CSV processor case:
 ** misleading example:
{code:java}
/user/data/EXAMPLE_NAME
,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
/user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}
 ** expected example as suggested by 
[https://tools.ietf.org/html/rfc4180#section-2]:
{code:java}
"/user/data/EXAMPLE_NAME%x0D",0,2017-04-24 04:34,1969-12-31 
16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
"/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}

 * XML processor case:
 ** misleading example:
{code:java}
479867791DIRECTORYEXAMPLE_NAME
1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}
 ** expected example as specified in 
[https://www.w3.org/TR/REC-xml/#sec-line-ends]:
{code:java}
479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}

 * JSON:
 The OIV Web Processor behaves correctly and produces the following:
{code:java}
{
  "FileStatuses": {
"FileStatus": [
  {
"fileId": 113632535,
"accessTime": 1494954320141,
"replication": 3,
"owner": "user",
"length": 520,
"permission": "674",
"blockSize": 134217728,
"modificationTime": 1472205657504,
"type": "FILE",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME"
  },
  {
"fileId": 479867791,
"accessTime": 0,
"replication": 0,
"owner": "user",
"length": 0,
"permission": "775",
"blockSize": 0,
"modificationTime": 1493033668294,
"type": "DIRECTORY",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME\n"
  }
]
  }
}
{code}


> OIV tool should better handle control characters present in file or directory 
> names
> ---
>
> Key: HDFS-13744
>

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

2018-07-18 Thread Zsolt Venczel (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-13744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zsolt Venczel updated HDFS-13744:
-
Description: 
In certain cases when control characters or white space is present in file or 
directory names OIV tool processors can export data in a misleading format.

In the below examples we have EXAMPLE_NAME as a file and a directory name where 
the directory has a line feed character at the end (the actual production case 
has multiple line feeds and multiple spaces)
 * CSV processor case:
 ** misleading example:
{code:java}
/user/data/EXAMPLE_NAME
,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
/user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}
 ** expected example as suggested by 
[https://tools.ietf.org/html/rfc4180#section-2]:
{code:java}
"/user/data/EXAMPLE_NAME%x0D",0,2017-04-24 04:34,1969-12-31 
16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
"/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}

 * XML processor case:
 ** misleading example:
{code:java}
479867791DIRECTORYEXAMPLE_NAME
1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}
 ** expected example as specified in 
[https://www.w3.org/TR/REC-xml/#sec-line-ends]:
{code:java}
479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}

 * JSON:
 The OIV Web Processor behaves correctly and produces the following:
{code:java}
{
  "FileStatuses": {
"FileStatus": [
  {
"fileId": 113632535,
"accessTime": 1494954320141,
"replication": 3,
"owner": "user",
"length": 520,
"permission": "674",
"blockSize": 134217728,
"modificationTime": 1472205657504,
"type": "FILE",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME"
  },
  {
"fileId": 479867791,
"accessTime": 0,
"replication": 0,
"owner": "user",
"length": 0,
"permission": "775",
"blockSize": 0,
"modificationTime": 1493033668294,
"type": "DIRECTORY",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME\n"
  }
]
  }
}
{code}

  was:
In certain cases when control characters or white space is present in file or 
directory names OIV tool processors can export data in a misleading format.

In the below examples we have EXAMPLE_NAME as a file and a directory name where 
the directory has a line feed character at the end (the actual production case 
has multiple line feeds and multiple spaces)
 * CSV processor case:
 ** misleading example:
{code:java}
/user/data/EXAMPLE_NAME
,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
/user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}
 ** expected example as suggested by 
[https://tools.ietf.org/html/rfc4180#section-2:]
{code:java}
"/user/data/EXAMPLE_NAME%x0D",0,2017-04-24 04:34,1969-12-31 
16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
"/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}

 * XML processor case:
 ** misleading example:
{code:java}
479867791DIRECTORYEXAMPLE_NAME
1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}
 ** expected example as specified in 
[https://www.w3.org/TR/REC-xml/#sec-line-ends:]
{code:java}
479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}

 * JSON:
 The OIV Web Processor behaves correctly and produces the following:
{code:java}
{
  "FileStatuses": {
"FileStatus": [
  {
"fileId": 113632535,
"accessTime": 1494954320141,
"replication": 3,
"owner": "user",
"length": 520,
"permission": "674",
"blockSize": 134217728,
"modificationTime": 1472205657504,
"type": "FILE",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME"
  },
  {
"fileId": 479867791,
"accessTime": 0,
"replication": 0,
"owner": "user",
"length": 0,
"permission": "775",
"blockSize": 0,
"modificationTime": 1493033668294,
"type": "DIRECTORY",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME\n"
  }
]
  }
}
{code}


> OIV tool should better handle control characters present in file or directory 
> names
> ---
>
> Key: HDFS-13744
>

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

2018-07-18 Thread Zsolt Venczel (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-13744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zsolt Venczel updated HDFS-13744:
-
Description: 
In certain cases when control characters or white space is present in file or 
directory names OIV tool processors can export data in a misleading format.

In the below examples we have EXAMPLE_NAME as a file and a directory name where 
the directory has a line feed character at the end (the actual production case 
has multiple line feeds and multiple spaces)
 * CSV processor case:
 ** misleading example:
{code:java}
/user/data/EXAMPLE_NAME
,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
/user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}
 ** expected example as suggested by 
[https://tools.ietf.org/html/rfc4180#section-2:]
{code:java}
"/user/data/EXAMPLE_NAME%x0D",0,2017-04-24 04:34,1969-12-31 
16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
"/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}

 * XML processor case:
 ** misleading example:
{code:java}
479867791DIRECTORYEXAMPLE_NAME
1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}
 ** expected example as specified in 
[https://www.w3.org/TR/REC-xml/#sec-line-ends:]
{code:java}
479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}

 * JSON:
 The OIV Web Processor behaves correctly and produces the following:
{code:java}
{
  "FileStatuses": {
"FileStatus": [
  {
"fileId": 113632535,
"accessTime": 1494954320141,
"replication": 3,
"owner": "user",
"length": 520,
"permission": "674",
"blockSize": 134217728,
"modificationTime": 1472205657504,
"type": "FILE",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME"
  },
  {
"fileId": 479867791,
"accessTime": 0,
"replication": 0,
"owner": "user",
"length": 0,
"permission": "775",
"blockSize": 0,
"modificationTime": 1493033668294,
"type": "DIRECTORY",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME\n"
  }
]
  }
}
{code}

  was:
In certain cases when control characters or white space is present in file or 
directory names OIV tool processors can export data in a misleading format.

In the below examples we have EXAMPLE_NAME as a file and a directory name where 
the directory has a line feed character at the end (the actual production case 
has multiple line feeds and multiple spaces)
 * CSV processor case:
 ** misleading example:
{code:java}
/user/data/EXAMPLE_NAME
,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
/user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}
 ** expected example as suggested by 
[https://tools.ietf.org/html/rfc4180#section-2:]
{code:java}
"/user/data/EXAMPLE_NAME%x0D",0,2017-04-24 04:34,1969-12-31 
16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
"/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}

 * XML processor case:
 ** misleading example:
{code:java}
479867791DIRECTORYEXAMPLE_NAME
1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}
 ** expected example as specified in 
[https://www.w3.org/TR/REC-xml/#sec-line-ends:]
{code:java}
479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775

479867791DIRECTORYEXAMPLE_NAME1493033668294user:group:0775
{code}

 * JSON:
 The OIV Web Processor behaves correctly and produces the following:
{code:java}
{
  "FileStatuses": {
"FileStatus": [
  {
"fileId": 113632535,
"accessTime": 1494954320141,
"replication": 3,
"owner": "user",
"length": 520,
"permission": "674",
"blockSize": 134217728,
"modificationTime": 1472205657504,
"type": "FILE",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME"
  },
  {
"fileId": 479867791,
"accessTime": 0,
"replication": 0,
"owner": "user",
"length": 0,
"permission": "775",
"blockSize": 0,
"modificationTime": 1493033668294,
"type": "DIRECTORY",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME\n"
  }
]
  }
}
{code}


> OIV tool should better handle control characters present in file or directory 
> names
> ---
>
> Key: HDFS-13744
> URL:

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

2018-07-18 Thread Zsolt Venczel (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-13744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zsolt Venczel updated HDFS-13744:
-
Description: 
In certain cases when control characters or white space is present in file or 
directory names OIV tool processors can export data in a misleading format.

In the below examples we have EXAMPLE_NAME as a file and a directory name where 
the directory has a line feed character at the end (the actual production case 
has multiple line feeds and multiple spaces)
 * CSV processor case:
 ** misleading example:
{code:java}
/user/data/EXAMPLE_NAME
,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
/user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}
 ** expected example as suggested by 
[https://tools.ietf.org/html/rfc4180#section-2:]
{code:java}
"/user/data/EXAMPLE_NAME%x0D",0,2017-04-24 04:34,1969-12-31 
16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
"/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}

 * XML processor case:
 ** misleading example:
{code:java}
479867791DIRECTORYEXAMPLE_NAME
1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}
 ** expected example as specified in 
[https://www.w3.org/TR/REC-xml/#sec-line-ends:]
{code:java}
479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775

479867791DIRECTORYEXAMPLE_NAME1493033668294user:group:0775
{code}

 * JSON:
 The OIV Web Processor behaves correctly and produces the following:
{code:java}
{
  "FileStatuses": {
"FileStatus": [
  {
"fileId": 113632535,
"accessTime": 1494954320141,
"replication": 3,
"owner": "user",
"length": 520,
"permission": "674",
"blockSize": 134217728,
"modificationTime": 1472205657504,
"type": "FILE",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME"
  },
  {
"fileId": 479867791,
"accessTime": 0,
"replication": 0,
"owner": "user",
"length": 0,
"permission": "775",
"blockSize": 0,
"modificationTime": 1493033668294,
"type": "DIRECTORY",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME\n"
  }
]
  }
}
{code}

  was:
In certain cases when control characters or white space is present in file or 
directory names OIV tool processors can export data in a misleading format.

In the below examples we have EXAMPLE_NAME as a file and a directory name where 
the directory has a line feed character at the end (the actual production case 
has multiple line feeds and multiple spaces)
 * CSV processor case:
 ** misleading example:
{code:java}
/user/data/EXAMPLE_NAME
,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
/user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}
 ** expected example as suggested by 
[https://tools.ietf.org/html/rfc4180#section-2:]
{code:java}
"/user/data/EXAMPLE_NAME%x0D",0,2017-04-24 04:34,1969-12-31 
16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
"/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}

 * XML processor case:
 ** misleading example:
{code:java}
479867791DIRECTORYEXAMPLE_NAME
1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}
 ** expected example as specified in 
[https://www.w3.org/TR/REC-xml/#sec-line-ends:]
{code:java}
479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775

479867791DIRECTORYEXAMPLE_NAME
1493033668294user:group:0775
{code}

 * JSON:
 The OIV Web Processor behaves correctly and produces the following:
{code:java}
{
  "FileStatuses": {
"FileStatus": [
  {
"fileId": 113632535,
"accessTime": 1494954320141,
"replication": 3,
"owner": "user",
"length": 520,
"permission": "674",
"blockSize": 134217728,
"modificationTime": 1472205657504,
"type": "FILE",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME"
  },
  {
"fileId": 479867791,
"accessTime": 0,
"replication": 0,
"owner": "user",
"length": 0,
"permission": "775",
"blockSize": 0,
"modificationTime": 1493033668294,
"type": "DIRECTORY",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME\n"
  }
]
  }
}
{code}


> OIV tool should better handle control characters present in file or directory 
> names
> ---
>
> Key: HDFS-13744
> URL:

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

2018-07-18 Thread Zsolt Venczel (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-13744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zsolt Venczel updated HDFS-13744:
-
Description: 
In certain cases when control characters or white space is present in file or 
directory names OIV tool processors can export data in a misleading format.

In the below examples we have EXAMPLE_NAME as a file and a directory name where 
the directory has a line feed character at the end (the actual production case 
has multiple line feeds and multiple spaces)
 * CSV processor case:
 ** misleading example:
{code:java}
/user/data/EXAMPLE_NAME
,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
/user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}
 ** expected example as suggested by 
[https://tools.ietf.org/html/rfc4180#section-2:]
{code:java}
"/user/data/EXAMPLE_NAME%x0D",0,2017-04-24 04:34,1969-12-31 
16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
"/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}

 * XML processor case:
 ** misleading example:
{code:java}
479867791DIRECTORYEXAMPLE_NAME
1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}
 ** expected example as specified in 
[https://www.w3.org/TR/REC-xml/#sec-line-ends:]
{code:java}
479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775

479867791DIRECTORYEXAMPLE_NAME
1493033668294user:group:0775
{code}

 * JSON:
 The OIV Web Processor behaves correctly and produces the following:
{code:java}
{
  "FileStatuses": {
"FileStatus": [
  {
"fileId": 113632535,
"accessTime": 1494954320141,
"replication": 3,
"owner": "user",
"length": 520,
"permission": "674",
"blockSize": 134217728,
"modificationTime": 1472205657504,
"type": "FILE",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME"
  },
  {
"fileId": 479867791,
"accessTime": 0,
"replication": 0,
"owner": "user",
"length": 0,
"permission": "775",
"blockSize": 0,
"modificationTime": 1493033668294,
"type": "DIRECTORY",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME\n"
  }
]
  }
}
{code}

  was:
In certain cases when control characters or white space is present in file or 
directory names OIV tool processors can export data in a misleading format.

In the below examples we have EXAMPLE_NAME as a file and a directory name where 
the directory has a line feed character at the end (the actual production case 
has multiple line feeds and multiple spaces)
 * CSV processor case:
 ** misleading example:
{code:java}
/user/data/EXAMPLE_NAME
,0,2017-04-24 04:34,1969-12-31 16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
/user/data/EXAMPLE_NAME,2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}

 ** expected example as suggested by 
[https://tools.ietf.org/html/rfc4180#section-2:]
{code:java}
"/user/data/EXAMPLE_NAME%x0D",0,2017-04-24 04:34,1969-12-31 
16:00,0,0,0,-1,-1,drwxrwxr-x+,user,group
"/user/data/EXAMPLE_NAME",2016-08-26 03:00,2017-05-16 
10:05,134217728,1,520,0,0,-rw-rwxr--+,user,group
{code}

 * XML processor case:
 ** misleading example:
{code:java}
479867791DIRECTORYEXAMPLE_NAME
1493033668294user:group:0775

113632535FILEEXAMPLE_NAME314722056575041494954320141134217728user:group:0674
{code}

 ** expected example as specified in 
[https://www.w3.org/TR/REC-xml/#sec-line-ends:]
{code:java}
479867791DIRECTORYEXAMPLE_NAME#xA1493033668294user:group:0775

479867791DIRECTORYEXAMPLE_NAME
1493033668294user:group:0775
{code}

 * JSON:
 The OIV Web Processor behaves correctly and produces the following:
{code:java}
{
  "FileStatuses": {
"FileStatus": [
  {
"fileId": 113632535,
"accessTime": 1494954320141,
"replication": 3,
"owner": "user",
"length": 520,
"permission": "674",
"blockSize": 134217728,
"modificationTime": 1472205657504,
"type": "FILE",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME"
  },
  {
"fileId": 479867791,
"accessTime": 0,
"replication": 0,
"owner": "user",
"length": 0,
"permission": "775",
"blockSize": 0,
"modificationTime": 1493033668294,
"type": "DIRECTORY",
"group": "group",
"childrenNum": 0,
"pathSuffix": "EXAMPLE_NAME\n"
  }
]
  }
}
{code}


> OIV tool should better handle control characters present in file or directory 
> names
> ---
>
> Key: HDFS-13744
> URL:

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

[jira] [Updated] (HDFS-13744) OIV tool should better handle control characters present in file or directory names

11 matches

Site Navigation

Mail list logo

Footer information