[jira] Commented: (HDFS-717) Proposal for exceptions thrown by FileContext and Abstract File System
[ https://issues.apache.org/jira/browse/HDFS-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12802966#action_12802966 ] Sanjay Radia commented on HDFS-717: --- The proposal is to declare the precise set of exception (Going forward these APIs will throw more specific exceptions). This is generally good practice and you are right that Avro needs this info. So we should declare the precise exceptions not just in FileContext but also in the protocol definition. Proposal for exceptions thrown by FileContext and Abstract File System -- Key: HDFS-717 URL: https://issues.apache.org/jira/browse/HDFS-717 Project: Hadoop HDFS Issue Type: Improvement Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Fix For: 0.22.0 Currently the APIs in FileContext throw only IOException. Going forward these APIs will throw more specific exceptions. This jira proposes following hierarchy of exceptions to be thrown by FileContext and AFS (Abstract File System) classes. InterruptedException (java.lang.InterruptedException) IOException /* Following exceptions extend IOException */ FileNotFoundException FileAlreadyExistsException DirectoryNotEmptyException NotDirectoryException AccessDeniedException IsDirectoryException InvalidPathNameException FileSystemException /* Following exceptions extend FileSystemException */ FileSystemNotReadyException ReadOnlyFileSystemException QuotaExceededException OutOfSpaceException RemoteException (java.rmi.RemoteException) Most of the IOExceptions above are caused by invalid user input, while FileSystemException is thrown when FS is in such a state that the requested operation cannot proceed. Please note that the proposed RemoteException is from standard java rmi package, which also extends IOException. HDFS throws many exceptions which are not in the above list. The DFSClient will unwrap the exceptions thrown by HDFS, and any exception not in the above list will be thrown as IOException or FileSystemException. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-717) Proposal for exceptions thrown by FileContext and Abstract File System
[ https://issues.apache.org/jira/browse/HDFS-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12801040#action_12801040 ] Sanjay Radia commented on HDFS-717: --- +1 on the proposal. Proposal for exceptions thrown by FileContext and Abstract File System -- Key: HDFS-717 URL: https://issues.apache.org/jira/browse/HDFS-717 Project: Hadoop HDFS Issue Type: Improvement Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Fix For: 0.22.0 Currently the APIs in FileContext throw only IOException. Going forward these APIs will throw more specific exceptions. This jira proposes following hierarchy of exceptions to be thrown by FileContext and AFS (Abstract File System) classes. InterruptedException (java.lang.InterruptedException) IOException /* Following exceptions extend IOException */ FileNotFoundException FileAlreadyExistsException DirectoryNotEmptyException NotDirectoryException AccessDeniedException IsDirectoryException InvalidPathNameException FileSystemException /* Following exceptions extend FileSystemException */ FileSystemNotReadyException ReadOnlyFileSystemException QuotaExceededException OutOfSpaceException RemoteException (java.rmi.RemoteException) Most of the IOExceptions above are caused by invalid user input, while FileSystemException is thrown when FS is in such a state that the requested operation cannot proceed. Please note that the proposed RemoteException is from standard java rmi package, which also extends IOException. HDFS throws many exceptions which are not in the above list. The DFSClient will unwrap the exceptions thrown by HDFS, and any exception not in the above list will be thrown as IOException or FileSystemException. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-717) Proposal for exceptions thrown by FileContext and Abstract File System
[
https://issues.apache.org/jira/browse/HDFS-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12796790#action_12796790
]
Suresh Srinivas commented on HDFS-717:
--
The following are the layers between the application and the Service
Implementation (such as NameNode).
Application = Client library = =RPC client= = Network = = RPC server =
= Service Impl
Key goals:
# InterruptedExceptions in the client library should not be ignored. This will
help in clean application shutdown. InterruptedException on the server side
should not be ignored; see below.
# Applications must be able to differentiate between RPC layer exception from
the exceptions in the Service Impl. Applications can choose to retry a request
based on different categories of exceptions received.
# Exceptions declared in the API should be propagated end to end over RPC from
the server to the application. All undeclared exceptions from the Service Impl
including InterruptedException should be handled by the RPC layer.
# Changes needed in applications to move to FileContext from FileSystem should
be minimal.
Proposal:
Exceptions will be organized as shown below.
# IOException
#* exceptions as declared in the RPC API - note the detailed method exception
will be declared even though they are a subclass of IOException
#* RPCException - exceptions in the rpc layer
#** RPCClientException - exception encountered in RPC client
#** RPCServerException - exception encountered in RPC server
#** UnexpectedServerException - unexpected exception from the Service Impl to
RPC handlers.
# RunTimeException
#* HadoopIllegalArgumentException - sublcass of IllegalArgumentException
indicates illegal or inappropriate argument.
#* HadoopInterruptedException - subclass of RunTimeException thrown on
encountering InterruptedException.
#* UnsupportedOperationException - thrown to indicate the requested operation
is not supported.
Rationale:
# declared exception should be subclass of IOException as before - no changes
here.
# group the rpc exceptions categorized by client side and server side.
# use runtime exception for InterrruptedException - simplifies migration to
FileContext. Subclass of IOException not used as applications might have catch
and ignore code.
# HadoopIllegalArgumentException instead of the java IllegalArgumentException -
helps differentiate exception in Hadoop implementation from exception thrown
from java libraries. Applications can choose to catch IllegalArgumentException.
# unsupported operation is indicated by unchecked UnsupportedOperationException
- subclass of IOException not used as applications might have catch and ignore
code. Using RunTimeException since applications cannot recover from this
condition.
Implementation details:
InterruptedException handling:
# Client side changes
#* Client library (both API interface and RPC client) and InputStream and
OutputStream returned by FileContext throw unchecked HadoopInterruptedException
on InterruptedException.
# Server changes:
#* InterruptedException is currently ignored in the Service Impl layer. With
this change the Service Impl will throw the exception. Methods in protocol
classes such as ClientProtocol will specify InterruptedException in throws
clause.
#* On InterruptedException, RPC handlers close the socket connection to the
client. Client handles this failure same as loss of connection.
RPC layer changes
# RPC layer marshalls HadoopInterruptedException,
HadoopIllegalArgumentException, UnsupportedException from Service Impl all the
way to the client.
# RPC layer throws RPCClientException, RPCServerException and
UnexpectedServerException.
FileContext and AbstractFileSystem and protocol changes:
# Methods in FileContext declare IOException and the relevant subclasses of
IOExceptions. This helps document the specific exceptions thrown and in
marshalling the exception from the server to application over RPC.
RPCExceptions are not declared as thrown in FileContext and AbstractFileSystem,
as some implementation might not use RPC layer (Local file system).
example:
{noformat}
public FSDataInputStream open(Path path) throws IOException,
FileNotFoundException, AccessDeniedException;
{noformat}
# Protocol methods (such as ClientProtocol) will the throw exceptions similar
to FileContext, along with InterruptedException.
Finally the FileContext will throw the following exceptions. The exception
hierarchy is flattened. The semantics remains as defined in the earlier
comments.
# IOException
#* ServerNotReadyException (NameNode safemode etc)
#* OutOfSpaceException for write operations
#* AccessControlException
#* InvalidPathNameException
#* FileNotFoundException
#* FileAlreadyExistsException
#* DirectoryNotEmptyException
#* NotDirectoryException
#* DirectoryNotAllowedException
Proposal for exceptions thrown by FileContext and Abstract File System
[jira] Commented: (HDFS-717) Proposal for exceptions thrown by FileContext and Abstract File System
[ https://issues.apache.org/jira/browse/HDFS-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12773593#action_12773593 ] Sanjay Radia commented on HDFS-717: --- I disagree here. We should unwrap only the ones that are declared in the method signature. RPC/network exceptions should be thrown as RemoteException I also considered RemoteException as a container for passing exceptions through RPC. If the name-node throws an IOException the DFSClient should unwrap it and throw the original directly rather than wrapped into the RemoteException. At this stage I don't have a strong opinion on what exception is used as the container across the wire. I also need to look into how avro is passing exceptions across the wire; will do so shortly. The RPC layer can be smarter about passing only declared exceptions and converting others into the remote exception in the RPC layer - I recall doing something like that in previous lifetime. If the RemoteException is the wrapper/container then yes client side should unwrap it, but only unwrap what is declared since these are the only classes available on the client side. The client side does not have all the Server-side exception classes. So if the server side throws a internal FooException that extends IOException the client side can only throw the IOException not the FooException. Proposal for exceptions thrown by FileContext and Abstract File System -- Key: HDFS-717 URL: https://issues.apache.org/jira/browse/HDFS-717 Project: Hadoop HDFS Issue Type: Improvement Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Fix For: 0.22.0 Currently the APIs in FileContext throw only IOException. Going forward these APIs will throw more specific exceptions. This jira proposes following hierarchy of exceptions to be thrown by FileContext and AFS (Abstract File System) classes. InterruptedException (java.lang.InterruptedException) IOException /* Following exceptions extend IOException */ FileNotFoundException FileAlreadyExistsException DirectoryNotEmptyException NotDirectoryException AccessDeniedException IsDirectoryException InvalidPathNameException FileSystemException /* Following exceptions extend FileSystemException */ FileSystemNotReadyException ReadOnlyFileSystemException QuotaExceededException OutOfSpaceException RemoteException (java.rmi.RemoteException) Most of the IOExceptions above are caused by invalid user input, while FileSystemException is thrown when FS is in such a state that the requested operation cannot proceed. Please note that the proposed RemoteException is from standard java rmi package, which also extends IOException. HDFS throws many exceptions which are not in the above list. The DFSClient will unwrap the exceptions thrown by HDFS, and any exception not in the above list will be thrown as IOException or FileSystemException. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-717) Proposal for exceptions thrown by FileContext and Abstract File System
[ https://issues.apache.org/jira/browse/HDFS-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12773613#action_12773613 ] Sanjay Radia commented on HDFS-717: --- Nicolas I propose to have the following hierarchy: ... +1 to use the name SystemException instead of FileSystemException as this can be reused in other client-server protocols. On the FileSystemOperationException - I guess you are trying to distinguish from the SystemException grouping and other non-file-system exceptions. So with your proposal will the method signature be: * open(...) throws IOException, FileSystemOperationException, SystemException, RemoteException, InterruptedIOException OR * open(...) throws FileSystemOperationException, SystemException, RemoteException, InterruptedIOException (i.e Are there any IOException thrown besides FileSystemOperationException, SystemException, RemoteException, InterruptedIOException?). This will prevents us from using any existing IOExceptions like FileNotFoundException. Proposal for exceptions thrown by FileContext and Abstract File System -- Key: HDFS-717 URL: https://issues.apache.org/jira/browse/HDFS-717 Project: Hadoop HDFS Issue Type: Improvement Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Fix For: 0.22.0 Currently the APIs in FileContext throw only IOException. Going forward these APIs will throw more specific exceptions. This jira proposes following hierarchy of exceptions to be thrown by FileContext and AFS (Abstract File System) classes. InterruptedException (java.lang.InterruptedException) IOException /* Following exceptions extend IOException */ FileNotFoundException FileAlreadyExistsException DirectoryNotEmptyException NotDirectoryException AccessDeniedException IsDirectoryException InvalidPathNameException FileSystemException /* Following exceptions extend FileSystemException */ FileSystemNotReadyException ReadOnlyFileSystemException QuotaExceededException OutOfSpaceException RemoteException (java.rmi.RemoteException) Most of the IOExceptions above are caused by invalid user input, while FileSystemException is thrown when FS is in such a state that the requested operation cannot proceed. Please note that the proposed RemoteException is from standard java rmi package, which also extends IOException. HDFS throws many exceptions which are not in the above list. The DFSClient will unwrap the exceptions thrown by HDFS, and any exception not in the above list will be thrown as IOException or FileSystemException. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-717) Proposal for exceptions thrown by FileContext and Abstract File System
[ https://issues.apache.org/jira/browse/HDFS-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12773620#action_12773620 ] Sanjay Radia commented on HDFS-717: --- UnsupportedOperationException: This one is tricky. See my comment in the rename jira: https://issues.apache.org/jira/browse/HADOOP-6240?focusedCommentId=12756632page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12756632 Basically the argument is that folks routinely write applications that are to be deployed on HDFS, but these apps are often tested on the local file system. Unfortunately, the unsupported exception will be thrown when testing the app. Further wrt to setReplication, I think our API needs improvement: the setReplication operation needs to be more abstract like want redundancy or not want redundancy. No sure what the API should look like. A filesystem that uses raid can give you reliability by parity rather than replication. I don't think it is a good idea to have setReplication throw UnsupportedOpsException. Perhaps we should have methods like IsFooOperationSupported(). I am raising issues rather then offering answers. Proposal for exceptions thrown by FileContext and Abstract File System -- Key: HDFS-717 URL: https://issues.apache.org/jira/browse/HDFS-717 Project: Hadoop HDFS Issue Type: Improvement Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Fix For: 0.22.0 Currently the APIs in FileContext throw only IOException. Going forward these APIs will throw more specific exceptions. This jira proposes following hierarchy of exceptions to be thrown by FileContext and AFS (Abstract File System) classes. InterruptedException (java.lang.InterruptedException) IOException /* Following exceptions extend IOException */ FileNotFoundException FileAlreadyExistsException DirectoryNotEmptyException NotDirectoryException AccessDeniedException IsDirectoryException InvalidPathNameException FileSystemException /* Following exceptions extend FileSystemException */ FileSystemNotReadyException ReadOnlyFileSystemException QuotaExceededException OutOfSpaceException RemoteException (java.rmi.RemoteException) Most of the IOExceptions above are caused by invalid user input, while FileSystemException is thrown when FS is in such a state that the requested operation cannot proceed. Please note that the proposed RemoteException is from standard java rmi package, which also extends IOException. HDFS throws many exceptions which are not in the above list. The DFSClient will unwrap the exceptions thrown by HDFS, and any exception not in the above list will be thrown as IOException or FileSystemException. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-717) Proposal for exceptions thrown by FileContext and Abstract File System
[ https://issues.apache.org/jira/browse/HDFS-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12773633#action_12773633 ] Jitendra Nath Pandey commented on HDFS-717: --- BTW this jira should have been a hadoop-common and not hadoop-hdfs jira. Agreed. The corresponding jira in hadoop-common is HADOOP-6361 Proposal for exceptions thrown by FileContext and Abstract File System -- Key: HDFS-717 URL: https://issues.apache.org/jira/browse/HDFS-717 Project: Hadoop HDFS Issue Type: Improvement Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Fix For: 0.22.0 Currently the APIs in FileContext throw only IOException. Going forward these APIs will throw more specific exceptions. This jira proposes following hierarchy of exceptions to be thrown by FileContext and AFS (Abstract File System) classes. InterruptedException (java.lang.InterruptedException) IOException /* Following exceptions extend IOException */ FileNotFoundException FileAlreadyExistsException DirectoryNotEmptyException NotDirectoryException AccessDeniedException IsDirectoryException InvalidPathNameException FileSystemException /* Following exceptions extend FileSystemException */ FileSystemNotReadyException ReadOnlyFileSystemException QuotaExceededException OutOfSpaceException RemoteException (java.rmi.RemoteException) Most of the IOExceptions above are caused by invalid user input, while FileSystemException is thrown when FS is in such a state that the requested operation cannot proceed. Please note that the proposed RemoteException is from standard java rmi package, which also extends IOException. HDFS throws many exceptions which are not in the above list. The DFSClient will unwrap the exceptions thrown by HDFS, and any exception not in the above list will be thrown as IOException or FileSystemException. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-717) Proposal for exceptions thrown by FileContext and Abstract File System
[
https://issues.apache.org/jira/browse/HDFS-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12773676#action_12773676
]
Tsz Wo (Nicholas), SZE commented on HDFS-717:
-
On the FileSystemOperationException ...
I am suggesting
* open(...) throws SystemException, RemoteException, InterruptedIOException,
AccessControlException, InvalidPathNameException, FileNotFoundException,
IsDirectoryException
SystemException, RemoteException and InterruptedIOException should be thrown by
all methods. Then, the specific FileSystemOperationException subclasses are
listed out explicitly. Other subclasses of IOException should never be thrown.
... This will prevents us from using any existing IOExceptions like
FileNotFoundException.
I actually suggest create a new o.a.h.fs.FileNotFoundException, which extends
FileSystemOperationException and prevent using java.io.FileNotFoundException
because java.io.FileNotFoundException is overloaded with different meaning.
The following is quoted from the [java.io.FileNotFoundException
API|http://java.sun.com/javase/6/docs/api/].
{quote}
It will also be thrown by these constructors if the file does exist but for
some reason is inaccessible, for example when an attempt is made to open a
read-only file for writing.
{quote}
Proposal for exceptions thrown by FileContext and Abstract File System
--
Key: HDFS-717
URL: https://issues.apache.org/jira/browse/HDFS-717
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Jitendra Nath Pandey
Assignee: Jitendra Nath Pandey
Fix For: 0.22.0
Currently the APIs in FileContext throw only IOException. Going forward these
APIs will throw more specific exceptions.
This jira proposes following hierarchy of exceptions to be thrown by
FileContext and AFS (Abstract File System) classes.
InterruptedException (java.lang.InterruptedException)
IOException
/* Following exceptions extend IOException */
FileNotFoundException
FileAlreadyExistsException
DirectoryNotEmptyException
NotDirectoryException
AccessDeniedException
IsDirectoryException
InvalidPathNameException
FileSystemException
/* Following exceptions extend
FileSystemException */
FileSystemNotReadyException
ReadOnlyFileSystemException
QuotaExceededException
OutOfSpaceException
RemoteException (java.rmi.RemoteException)
Most of the IOExceptions above are caused by invalid user input, while
FileSystemException is thrown when FS is in such a state that the requested
operation cannot proceed.
Please note that the proposed RemoteException is from standard java rmi
package, which also extends IOException.
HDFS throws many exceptions which are not in the above list. The DFSClient
will unwrap the exceptions thrown by HDFS, and any exception not in the above
list will be thrown as IOException or FileSystemException.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-717) Proposal for exceptions thrown by FileContext and Abstract File System
[ https://issues.apache.org/jira/browse/HDFS-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12773308#action_12773308 ] Sanjay Radia commented on HDFS-717: --- Can you put forth the use case why we have two different sub-types of exceptions: IOException and FileSystemException? I can see a case for exceptions that are system problems, that is, not caused by an invalid parameter supplied by the caller. For example, the file system is full, or it is in safe mode. Generally the caller cannot do much about this and hence it is worth grouping such exceptions. Today your application may run fine and tomorrow the very same application may fail because of a system problem. This is not bug in the application. So FileSystemExceptions are not application errors but system errors.(It is worth considering making these runtime exception and not checked exceptions; but this is orthogonal to the grouping question.) What else should fall into this group? Lack of quotas falls into the same category since the admin sets the quotas and the user does has not much control over this. Should access-control exceptions should be in the same group? I think not because this exception is mostly, but not always, caused by an application error. Proposal for exceptions thrown by FileContext and Abstract File System -- Key: HDFS-717 URL: https://issues.apache.org/jira/browse/HDFS-717 Project: Hadoop HDFS Issue Type: Improvement Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Fix For: 0.22.0 Currently the APIs in FileContext throw only IOException. Going forward these APIs will throw more specific exceptions. This jira proposes following hierarchy of exceptions to be thrown by FileContext and AFS (Abstract File System) classes. InterruptedException (java.lang.InterruptedException) IOException /* Following exceptions extend IOException */ FileNotFoundException FileAlreadyExistsException DirectoryNotEmptyException NotDirectoryException AccessDeniedException IsDirectoryException InvalidPathNameException FileSystemException /* Following exceptions extend FileSystemException */ FileSystemNotReadyException ReadOnlyFileSystemException QuotaExceededException OutOfSpaceException RemoteException (java.rmi.RemoteException) Most of the IOExceptions above are caused by invalid user input, while FileSystemException is thrown when FS is in such a state that the requested operation cannot proceed. Please note that the proposed RemoteException is from standard java rmi package, which also extends IOException. HDFS throws many exceptions which are not in the above list. The DFSClient will unwrap the exceptions thrown by HDFS, and any exception not in the above list will be thrown as IOException or FileSystemException. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-717) Proposal for exceptions thrown by FileContext and Abstract File System
[ https://issues.apache.org/jira/browse/HDFS-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12773360#action_12773360 ] Tsz Wo (Nicholas), SZE commented on HDFS-717: - - The term FileSystemException is not clear whether it is a system problem or a file system operation problem. A better name may be SystemException. It also allow other systems like JobTracker to use it. - It may be better to group FileNotFoundException, FileAlreadyExistsException, etc. How about extending FileSystemOperationException? - I propose to have the following hierarchy: -* java.io.IOException -** *java.rmi.RemoteException* --- network/ipc related exception -** *java.io.InterruptedIOException* --- the client thread is interrupted when it is waiting for a blocking IO (e.g. waiting for server reply). -** *SystemException* --- capture system problems, usually, server side problems. -*** UnsupportedOperationException --- e.g. calling setReplication(..) on LocalFileSystem - ReadOnlyFileSystemException --- e.g. calling create(..) on HftpFileSystem -*** SystemNotReadyException --- e.g. NameNode/JobTracker is starting up -*** QuotaExceededException -*** OutOfSpaceException -*** UncaughtRuntimeException --- server side uncaught RuntimeException -** *FileSystemOperationException* -*** AccessControlException -*** InvalidPathNameException -*** FileNotFoundException -*** FileAlreadyExistsException -*** DirectoryNotEmptyException -*** NotDirectoryException -*** IsDirectoryException Proposal for exceptions thrown by FileContext and Abstract File System -- Key: HDFS-717 URL: https://issues.apache.org/jira/browse/HDFS-717 Project: Hadoop HDFS Issue Type: Improvement Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Fix For: 0.22.0 Currently the APIs in FileContext throw only IOException. Going forward these APIs will throw more specific exceptions. This jira proposes following hierarchy of exceptions to be thrown by FileContext and AFS (Abstract File System) classes. InterruptedException (java.lang.InterruptedException) IOException /* Following exceptions extend IOException */ FileNotFoundException FileAlreadyExistsException DirectoryNotEmptyException NotDirectoryException AccessDeniedException IsDirectoryException InvalidPathNameException FileSystemException /* Following exceptions extend FileSystemException */ FileSystemNotReadyException ReadOnlyFileSystemException QuotaExceededException OutOfSpaceException RemoteException (java.rmi.RemoteException) Most of the IOExceptions above are caused by invalid user input, while FileSystemException is thrown when FS is in such a state that the requested operation cannot proceed. Please note that the proposed RemoteException is from standard java rmi package, which also extends IOException. HDFS throws many exceptions which are not in the above list. The DFSClient will unwrap the exceptions thrown by HDFS, and any exception not in the above list will be thrown as IOException or FileSystemException. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-717) Proposal for exceptions thrown by FileContext and Abstract File System
[ https://issues.apache.org/jira/browse/HDFS-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12773353#action_12773353 ] Tsz Wo (Nicholas), SZE commented on HDFS-717: - Throw java.io.InterruptedIOException instead of InterruptedException. We thought of InterruptedException to capture interrupts to any thread and not just threads busy with IO. Do you mean that a server thread is interrupted by some reason and then the server side InterruptedException is forwarded to the client? This sounds like a system problem. I was considering the case that the a client is waiting for a blocking IO (e.g. waiting for server reply) and the client thread is interrupted. In such case, InterruptedIOException should be used. Proposal for exceptions thrown by FileContext and Abstract File System -- Key: HDFS-717 URL: https://issues.apache.org/jira/browse/HDFS-717 Project: Hadoop HDFS Issue Type: Improvement Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Fix For: 0.22.0 Currently the APIs in FileContext throw only IOException. Going forward these APIs will throw more specific exceptions. This jira proposes following hierarchy of exceptions to be thrown by FileContext and AFS (Abstract File System) classes. InterruptedException (java.lang.InterruptedException) IOException /* Following exceptions extend IOException */ FileNotFoundException FileAlreadyExistsException DirectoryNotEmptyException NotDirectoryException AccessDeniedException IsDirectoryException InvalidPathNameException FileSystemException /* Following exceptions extend FileSystemException */ FileSystemNotReadyException ReadOnlyFileSystemException QuotaExceededException OutOfSpaceException RemoteException (java.rmi.RemoteException) Most of the IOExceptions above are caused by invalid user input, while FileSystemException is thrown when FS is in such a state that the requested operation cannot proceed. Please note that the proposed RemoteException is from standard java rmi package, which also extends IOException. HDFS throws many exceptions which are not in the above list. The DFSClient will unwrap the exceptions thrown by HDFS, and any exception not in the above list will be thrown as IOException or FileSystemException. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-717) Proposal for exceptions thrown by FileContext and Abstract File System
[ https://issues.apache.org/jira/browse/HDFS-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12772717#action_12772717 ] Tsz Wo (Nicholas), SZE commented on HDFS-717: - Sound good. Some comments: - Throw java.io.InterruptedIOException instead of InterruptedException. - Add UnsupportedOperationException as a subclass of FileSystemException. - Is FileSystemException a subclass of IOException? Proposal for exceptions thrown by FileContext and Abstract File System -- Key: HDFS-717 URL: https://issues.apache.org/jira/browse/HDFS-717 Project: Hadoop HDFS Issue Type: Improvement Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Fix For: 0.22.0 Currently the APIs in FileContext throw only IOException. Going forward these APIs will throw more specific exceptions. This jira proposes following hierarchy of exceptions to be thrown by FileContext and AFS (Abstract File System) classes. InterruptedException (java.lang.InterruptedException) IOException /* Following exceptions extend IOException */ FileNotFoundException FileAlreadyExistsException DirectoryNotEmptyException NotDirectoryException AccessDeniedException IsDirectoryException InvalidPathNameException FileSystemException /* Following exceptions extend FileSystemException */ FileSystemNotReadyException ReadOnlyFileSystemException QuotaExceededException OutOfSpaceException RemoteException (java.rmi.RemoteException) Most of the IOExceptions above are caused by invalid user input, while FileSystemException is thrown when FS is in such a state that the requested operation cannot proceed. Please note that the proposed RemoteException is from standard java rmi package, which also extends IOException. HDFS throws many exceptions which are not in the above list. The DFSClient will unwrap the exceptions thrown by HDFS, and any exception not in the above list will be thrown as IOException or FileSystemException. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-717) Proposal for exceptions thrown by FileContext and Abstract File System
[ https://issues.apache.org/jira/browse/HDFS-717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12772795#action_12772795 ] Jitendra Nath Pandey commented on HDFS-717: --- Is FileSystemException a subclass of IOException? Yes FileSystemException is a subclass of IOException. Add UnsupportedOperationException as a subclass of FileSystemException. I agree to add this one. Throw java.io.InterruptedIOException instead of InterruptedException. We thought of InterruptedException to capture interrupts to any thread and not just threads busy with IO. why we have two different sub-types of exceptions? FileSystemException is thrown when FS is in such a state that the requested operation cannot proceed for example if filesystem is in safemode, we can throw FileSystemNotReadyException which is a subtype of FileSystemException. The other IOExceptions are thrown when there are inconsistencies in the user inputs and operation cannot succees for example creating a file that already exists. If it is java.nio.file.FileSystemException then it will appear only in java 7. FileSystemException is inspired by java.nio.file.FileSystemException but we will implement it because it is not in java 6. What is IsDirectoryException? Sounds more like a method name. IsDirectoryException is inspired by Unix error code EISDIR, which means that expected input was a file but a directory is being passed. We can rename it to DirectoryNotAllowedException. Should QuotaExceededException be a subclass of IOException, same as AccessDeniedException? All the exceptions except Interrupted exceptions are subclasses of IOException. Do you plan to unwrap ALL RemoteExceptions in DFSClient? Yes all these exceptions will be unwrapped. But that implementation is not part of this jira. Proposal for exceptions thrown by FileContext and Abstract File System -- Key: HDFS-717 URL: https://issues.apache.org/jira/browse/HDFS-717 Project: Hadoop HDFS Issue Type: Improvement Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Fix For: 0.22.0 Currently the APIs in FileContext throw only IOException. Going forward these APIs will throw more specific exceptions. This jira proposes following hierarchy of exceptions to be thrown by FileContext and AFS (Abstract File System) classes. InterruptedException (java.lang.InterruptedException) IOException /* Following exceptions extend IOException */ FileNotFoundException FileAlreadyExistsException DirectoryNotEmptyException NotDirectoryException AccessDeniedException IsDirectoryException InvalidPathNameException FileSystemException /* Following exceptions extend FileSystemException */ FileSystemNotReadyException ReadOnlyFileSystemException QuotaExceededException OutOfSpaceException RemoteException (java.rmi.RemoteException) Most of the IOExceptions above are caused by invalid user input, while FileSystemException is thrown when FS is in such a state that the requested operation cannot proceed. Please note that the proposed RemoteException is from standard java rmi package, which also extends IOException. HDFS throws many exceptions which are not in the above list. The DFSClient will unwrap the exceptions thrown by HDFS, and any exception not in the above list will be thrown as IOException or FileSystemException. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
