YUBI LEE created ZEPPELIN-5594:
----------------------------------
Summary: HDFS file id should be read as "long", not "int".
Key: ZEPPELIN-5594
URL: https://issues.apache.org/jira/browse/ZEPPELIN-5594
Project: Zeppelin
Issue Type: Bug
Components: Interpreters
Affects Versions: 0.10.0
Reporter: YUBI LEE
{code:java}
ERROR [2021-11-29 14:46:51,519]
({FIFOScheduler-org.apache.zeppelin.file.FileInterpreter1871691756-Worker-1}
HDFSFileInterpreter.java[listAll]:249) - listall: listDir /user/*******
com.google.gson.JsonSyntaxException: java.lang.NumberFormatException: Expected
an int but was 4947954640 at line 1 column 585 path
$.FileStatuses.FileStatus[2].fileId {code}
I found this error when using file(hdfs) interpreter.
Some paths are okay.
I met this exception when trying to access paths with high file id (above
integer).
I think that "blockSize" & "fileId" should be declared as "long"
Refer to HdfsFileStatus.java,
{code:java}
class Builder {
// Changing default values will affect cases where values are not
// specified. Be careful!
private long length = 0L;
private boolean isdir = false;
private int replication = 0;
private long blocksize = 0L;
private long mtime = 0L;
private long atime = 0L;
private FsPermission permission = null;
private EnumSet<Flags> flags = EnumSet.noneOf(Flags.class);
private String owner = null;
private String group = null;
private byte[] symlink = null;
private byte[] path = EMPTY_NAME;
private long fileId = -1L;
private int childrenNum = 0;
private FileEncryptionInfo feInfo = null;
private byte storagePolicy =
HdfsConstants.BLOCK_STORAGE_POLICY_ID_UNSPECIFIED;
private ErasureCodingPolicy ecPolicy = null;
private LocatedBlocks locations = null;
...
{code}
* HDFSFileInterpreter.java:
[https://github.com/apache/zeppelin/blob/85ed8e2e51e1ea10df38d4710216343efe218d60/file/src/main/java/org/apache/zeppelin/file/HDFSFileInterpreter.java#L87]
* HdfsFileStatus.java:
[https://github.com/apache/hadoop/blob/branch-3.1.2/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsFileStatus.java#L62-L83]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
