[jira] [Commented] (HDFS-15572) RBF: Quota updating every 1 min once, if I setquota 50 on mount path, in a minute i am able to write the files morethan quota in mount path.
[
https://issues.apache.org/jira/browse/HDFS-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17214037#comment-17214037
]
Takanobu Asanuma commented on HDFS-15572:
-
If reducing {{dfs.federation.router.cache.ttl}} or
{{dfs.federation.router.quota-cache.update.interval}}, router would update the
cache near real-time with the current implementation. But it will cause
frequency read operations to state store if it is too short.
> RBF: Quota updating every 1 min once, if I setquota 50 on mount path, in a
> minute i am able to write the files morethan quota in mount path.
> -
>
> Key: HDFS-15572
> URL: https://issues.apache.org/jira/browse/HDFS-15572
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Affects Versions: 3.1.1
>Reporter: Harshakiran Reddy
>Assignee: Hemanth Boyina
>Priority: Major
>
> IN Router State store quota updating every 1 min once only, if i set quota 50
> on mount path am able to write more that quota in the mount path here quota
> will not work out.
> {noformat}
> 1. Create a destinations dir in Namespaces
> 2. Create a mount path with multiple destinations
> 3. Setquota 50 on mount path
> 4. write a files morethan 50 + in the mount path in a minute
> {noformat}
> *Excepted Result:-*
> Here after setquota mount path should not allow morethan that at any cost.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15572) RBF: Quota updating every 1 min once, if I setquota 50 on mount path, in a minute i am able to write the files morethan quota in mount path.
[
https://issues.apache.org/jira/browse/HDFS-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17212885#comment-17212885
]
Hemanth Boyina commented on HDFS-15572:
---
can we have quota real time updated along with existing periodic invoke
[~tasanuma] [~elgoiri] ?
> RBF: Quota updating every 1 min once, if I setquota 50 on mount path, in a
> minute i am able to write the files morethan quota in mount path.
> -
>
> Key: HDFS-15572
> URL: https://issues.apache.org/jira/browse/HDFS-15572
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Affects Versions: 3.1.1
>Reporter: Harshakiran Reddy
>Assignee: Hemanth Boyina
>Priority: Major
>
> IN Router State store quota updating every 1 min once only, if i set quota 50
> on mount path am able to write more that quota in the mount path here quota
> will not work out.
> {noformat}
> 1. Create a destinations dir in Namespaces
> 2. Create a mount path with multiple destinations
> 3. Setquota 50 on mount path
> 4. write a files morethan 50 + in the mount path in a minute
> {noformat}
> *Excepted Result:-*
> Here after setquota mount path should not allow morethan that at any cost.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15572) RBF: Quota updating every 1 min once, if I setquota 50 on mount path, in a minute i am able to write the files morethan quota in mount path.
[
https://issues.apache.org/jira/browse/HDFS-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194623#comment-17194623
]
Hemanth Boyina commented on HDFS-15572:
---
i am not completely sure about consistency for mount table operations , but the
router quota usage has some consistency issues as the router will not have the
real time quota updated in it , as routes relies on RouterQuotaUpdateService
which periodically queries the quota usage from subclusters
> RBF: Quota updating every 1 min once, if I setquota 50 on mount path, in a
> minute i am able to write the files morethan quota in mount path.
> -
>
> Key: HDFS-15572
> URL: https://issues.apache.org/jira/browse/HDFS-15572
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Affects Versions: 3.1.1
>Reporter: Harshakiran Reddy
>Assignee: Hemanth Boyina
>Priority: Major
>
> IN Router State store quota updating every 1 min once only, if i set quota 50
> on mount path am able to write more that quota in the mount path here quota
> will not work out.
> {noformat}
> 1. Create a destinations dir in Namespaces
> 2. Create a mount path with multiple destinations
> 3. Setquota 50 on mount path
> 4. write a files morethan 50 + in the mount path in a minute
> {noformat}
> *Excepted Result:-*
> Here after setquota mount path should not allow morethan that at any cost.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15572) RBF: Quota updating every 1 min once, if I setquota 50 on mount path, in a minute i am able to write the files morethan quota in mount path.
[
https://issues.apache.org/jira/browse/HDFS-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194407#comment-17194407
]
Takanobu Asanuma commented on HDFS-15572:
-
If I understand correctly, RBF doesn't guarantee strong consistency for mount
table operations.
> RBF: Quota updating every 1 min once, if I setquota 50 on mount path, in a
> minute i am able to write the files morethan quota in mount path.
> -
>
> Key: HDFS-15572
> URL: https://issues.apache.org/jira/browse/HDFS-15572
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Affects Versions: 3.1.1
>Reporter: Harshakiran Reddy
>Assignee: Hemanth Boyina
>Priority: Major
>
> IN Router State store quota updating every 1 min once only, if i set quota 50
> on mount path am able to write more that quota in the mount path here quota
> will not work out.
> {noformat}
> 1. Create a destinations dir in Namespaces
> 2. Create a mount path with multiple destinations
> 3. Setquota 50 on mount path
> 4. write a files morethan 50 + in the mount path in a minute
> {noformat}
> *Excepted Result:-*
> Here after setquota mount path should not allow morethan that at any cost.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15572) RBF: Quota updating every 1 min once, if I setquota 50 on mount path, in a minute i am able to write the files morethan quota in mount path.
[
https://issues.apache.org/jira/browse/HDFS-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17194053#comment-17194053
]
Hemanth Boyina commented on HDFS-15572:
---
thanks [~Harsha1206] for reporting the issue
RouterQuotaUpdateService#periodicInvoke by default updates the quota for every
1min , Because of this yes there could be chance of creating more number of
files than quota set value as router quota store will not be having updated
quota value from name services
> RBF: Quota updating every 1 min once, if I setquota 50 on mount path, in a
> minute i am able to write the files morethan quota in mount path.
> -
>
> Key: HDFS-15572
> URL: https://issues.apache.org/jira/browse/HDFS-15572
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Affects Versions: 3.1.1
>Reporter: Harshakiran Reddy
>Assignee: Hemanth Boyina
>Priority: Major
>
> IN Router State store quota updating every 1 min once only, if i set quota 50
> on mount path am able to write more that quota in the mount path here quota
> will not work out.
> {noformat}
> 1. Create a destinations dir in Namespaces
> 2. Create a mount path with multiple destinations
> 3. Setquota 50 on mount path
> 4. write a files morethan 50 + in the mount path in a minute
> {noformat}
> *Excepted Result:-*
> Here after setquota mount path should not allow morethan that at any cost.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
