[jira] [Comment Edited] (HDFS-14082) RBF: Add option to fail operations when a subcluster is unavailable
[
https://issues.apache.org/jira/browse/HDFS-14082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598066#comment-17598066
]
Íñigo Goiri edited comment on HDFS-14082 at 8/30/22 8:50 PM:
-
[~smarthan], I would be ok making the default false.
For your solution, we would need to surface this to the client though.
was (Author: elgoiri):
[~smarthan], I would be ok making the default false.
> RBF: Add option to fail operations when a subcluster is unavailable
> ---
>
> Key: HDFS-14082
> URL: https://issues.apache.org/jira/browse/HDFS-14082
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Íñigo Goiri
>Assignee: Íñigo Goiri
>Priority: Major
> Fix For: HDFS-13891
>
> Attachments: HDFS-14082-HDFS-13891.002.patch,
> HDFS-14082-HDFS-13891.003.patch, HDFS-14082.000.patch, HDFS-14082.001.patch
>
>
> When a subcluster is unavailable, we succeed operations like
> {{getListing()}}. We should add an option to fail the operation if one of the
> subclusters is unavailable.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14082) RBF: Add option to fail operations when a subcluster is unavailable
[
https://issues.apache.org/jira/browse/HDFS-14082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598066#comment-17598066
]
Íñigo Goiri edited comment on HDFS-14082 at 8/30/22 8:49 PM:
-
[~smarthan], I would be ok making the default false.
was (Author: elgoiri):
[~smarthan], partial listing also happens in the regular namenode and the
DFSClient knows how to handle this.
I would be ok making the default false though.
> RBF: Add option to fail operations when a subcluster is unavailable
> ---
>
> Key: HDFS-14082
> URL: https://issues.apache.org/jira/browse/HDFS-14082
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Íñigo Goiri
>Assignee: Íñigo Goiri
>Priority: Major
> Fix For: HDFS-13891
>
> Attachments: HDFS-14082-HDFS-13891.002.patch,
> HDFS-14082-HDFS-13891.003.patch, HDFS-14082.000.patch, HDFS-14082.001.patch
>
>
> When a subcluster is unavailable, we succeed operations like
> {{getListing()}}. We should add an option to fail the operation if one of the
> subclusters is unavailable.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14082) RBF: Add option to fail operations when a subcluster is unavailable
[
https://issues.apache.org/jira/browse/HDFS-14082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692596#comment-16692596
]
Yiqun Lin edited comment on HDFS-14082 at 11/20/18 7:28 AM:
hI [~elgoiri], would you mind addressing my previous comments? I suppose you
have missed that. Here I think we only take care for that if the whole files
are returned or not.
{quote}I mean we make following change and not use specific number.
// Test the behavior when everything is fine
+ FileSystem fs = getRouterFileSystem();
+ FileStatus[] files = fs.listStatus(new Path("/"));
int originalCount= files.length
// simplify the assertion logic
assertNotEqual("...", originalCount , files.length);
Catching one place I am missing before:
fail("I should throw an exception");
Can we reword this to "listStatus call should throw an exception"?
{quote}
Others look good to me..
was (Author: linyiqun):
hI [~elgoiri], would you mind addressing my previous comments? I suppose you
have missed that.
{quote}I mean we make following change and not use specific number.
// Test the behavior when everything is fine
+ FileSystem fs = getRouterFileSystem();
+ FileStatus[] files = fs.listStatus(new Path("/"));
int originalCount= files.length
// simplify the assertion logic
assertNotEqual("...", originalCount , files.length);
Catching one place I am missing before:
fail("I should throw an exception");
Can we reword this to "listStatus call should throw an exception"?
{quote}
Others look good to me..
> RBF: Add option to fail operations when a subcluster is unavailable
> ---
>
> Key: HDFS-14082
> URL: https://issues.apache.org/jira/browse/HDFS-14082
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Íñigo Goiri
>Assignee: Íñigo Goiri
>Priority: Major
> Attachments: HDFS-14082-HDFS-13891.002.patch, HDFS-14082.000.patch,
> HDFS-14082.001.patch
>
>
> When a subcluster is unavailable, we succeed operations like
> {{getListing()}}. We should add an option to fail the operation if one of the
> subclusters is unavailable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14082) RBF: Add option to fail operations when a subcluster is unavailable
[
https://issues.apache.org/jira/browse/HDFS-14082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16690191#comment-16690191
]
Yiqun Lin edited comment on HDFS-14082 at 11/17/18 2:04 AM:
{quote}Then, I would keep it backwards compatible setting it to true.
{quote}
The default value should be false i think. We are allowed to get partial list
when someone subcluster is unavailable by default.
{quote}What do you propose here? We definetely can make 6 a variable but not
sure about 4/5.
{quote}
I mean we make following change and not use specific number.
{code:java}
// Test the behavior when everything is fine
+FileSystem fs = getRouterFileSystem();
+FileStatus[] files = fs.listStatus(new Path("/"));
int originalCount= files.length
// simplify the assertion logic
assertNotEqual("...", originalCount , files.length);
{code}
Catching one place I am missing before:
{noformat}
fail("I should throw an exception");
{noformat}
Can we reword this to "{{listStatus call should throw an exception"}}?
BTW, this JIRA is targeted for trunk? As i see the patch is for trunk.
was (Author: linyiqun):
{quote}Then, I would keep it backwards compatible setting it to true.
{quote}
The default value should be false i think. We are allowed to get partial list
when someone subcluster is unavailable by default.
{quote}What do you propose here? We definetely can make 6 a variable but not
sure about 4/5.
{quote}
I mean we make following change and not use specific number.
{code:java}
// Test the behavior when everything is fine
+FileSystem fs = getRouterFileSystem();
+FileStatus[] files = fs.listStatus(new Path("/"));
int originalCount= files.length
// simplify the assertion logic
assertNotEqual("...", originalCount , files.length);
{code}
:
BTW, this JIRA is targeted for trunk? As i see the patch is for trunk.
> RBF: Add option to fail operations when a subcluster is unavailable
> ---
>
> Key: HDFS-14082
> URL: https://issues.apache.org/jira/browse/HDFS-14082
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Íñigo Goiri
>Assignee: Íñigo Goiri
>Priority: Major
> Attachments: HDFS-14082.000.patch, HDFS-14082.001.patch
>
>
> When a subcluster is unavailable, we succeed operations like
> {{getListing()}}. We should add an option to fail the operation if one of the
> subclusters is unavailable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14082) RBF: Add option to fail operations when a subcluster is unavailable
[
https://issues.apache.org/jira/browse/HDFS-14082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16690191#comment-16690191
]
Yiqun Lin edited comment on HDFS-14082 at 11/17/18 1:50 AM:
{quote}Then, I would keep it backwards compatible setting it to true.
{quote}
The default value should be false i think. We are allowed to get partial list
when someone subcluster is unavailable by default.
{quote}What do you propose here? We definetely can make 6 a variable but not
sure about 4/5.
{quote}
I mean we make following change and not use specific number.
{code:java}
// Test the behavior when everything is fine
+FileSystem fs = getRouterFileSystem();
+FileStatus[] files = fs.listStatus(new Path("/"));
int originalCount= files.length
// simplify the assertion logic
assertNotEqual("...", originalCount , files.length);
{code}
:
BTW, this JIRA is targeted for trunk? As i see the patch is for trunk.
was (Author: linyiqun):
{quote}Then, I would keep it backwards compatible setting it to true.
{quote}
The default value should be false i think. We are allowed to get partial list
when someone subcluster is unavailable by default.
{quote}What do you propose here? We definetely can make 6 a variable but not
sure about 4/5.
{quote}
I mean we make following change and not use specific number.
{code:java}
// Test the behavior when everything is fine
+FileSystem fs = getRouterFileSystem();
+FileStatus[] files = fs.listStatus(new Path("/"));
int originalCount= files.length
// simplify the assertion logic
assertNotEqual("...", originalCount , files.length);
{code}
:
> RBF: Add option to fail operations when a subcluster is unavailable
> ---
>
> Key: HDFS-14082
> URL: https://issues.apache.org/jira/browse/HDFS-14082
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Íñigo Goiri
>Assignee: Íñigo Goiri
>Priority: Major
> Attachments: HDFS-14082.000.patch, HDFS-14082.001.patch
>
>
> When a subcluster is unavailable, we succeed operations like
> {{getListing()}}. We should add an option to fail the operation if one of the
> subclusters is unavailable.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
