[GitHub] [hadoop-ozone] timmylicheng edited a comment on issue #720: HDDS-3185 Construct a standalone ratis server for SCM.

2020-03-25 Thread GitBox
timmylicheng edited a comment on issue #720: HDDS-3185 Construct a standalone ratis server for SCM. URL: https://github.com/apache/hadoop-ozone/pull/720#issuecomment-604254744 @elek Hey Marton, Sorry this patch was meant to merge into HDDS-2823 as a dev branch for SCM HA. So far

[GitHub] [hadoop-ozone] timmylicheng commented on issue #720: HDDS-3185 Construct a standalone ratis server for SCM.

2020-03-25 Thread GitBox
timmylicheng commented on issue #720: HDDS-3185 Construct a standalone ratis server for SCM. URL: https://github.com/apache/hadoop-ozone/pull/720#issuecomment-604254744 @elek Hey Marton, So far the doc work is ongoing in parallel with prototyping. The doc still needs improvements and

[jira] [Commented] (HDDS-3001) NFS support for Ozone

2020-03-25 Thread Prashant Pogde (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067384#comment-17067384 ] Prashant Pogde commented on HDDS-3001: -- Attaching the design document. Please take a

[jira] [Updated] (HDDS-3001) NFS support for Ozone

2020-03-25 Thread Prashant Pogde (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Pogde updated HDDS-3001: - Attachment: NFS Support for Ozone.pdf > NFS support for Ozone > - > >

[jira] [Comment Edited] (HDDS-3254) Datanode memory increase so much

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067371#comment-17067371 ] runzhiwang edited comment on HDDS-3254 at 3/26/20, 6:06 AM: [~

[jira] [Comment Edited] (HDDS-3254) Datanode memory increase so much

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067371#comment-17067371 ] runzhiwang edited comment on HDDS-3254 at 3/26/20, 6:06 AM: [~

[jira] [Commented] (HDDS-3254) Datanode memory increase so much

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067371#comment-17067371 ] runzhiwang commented on HDDS-3254: -- [~shashikant] I have stop send request to s3gateway f

[jira] [Commented] (HDDS-3254) Datanode memory increase so much

2020-03-25 Thread Shashikant Banerjee (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067369#comment-17067369 ] Shashikant Banerjee commented on HDDS-3254: --- [~yjxxtd], after killing the datano

[GitHub] [hadoop-ozone] bshashikant commented on a change in pull request #709: HDDS-3244. Improve write efficiency by opening RocksDB only once

2020-03-25 Thread GitBox
bshashikant commented on a change in pull request #709: HDDS-3244. Improve write efficiency by opening RocksDB only once URL: https://github.com/apache/hadoop-ozone/pull/709#discussion_r398305798 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/cont

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #709: HDDS-3244. Improve write efficiency by opening RocksDB only once

2020-03-25 Thread GitBox
runzhiwang commented on a change in pull request #709: HDDS-3244. Improve write efficiency by opening RocksDB only once URL: https://github.com/apache/hadoop-ozone/pull/709#discussion_r398310403 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/conta

[GitHub] [hadoop-ozone] bshashikant commented on a change in pull request #709: HDDS-3244. Improve write efficiency by opening RocksDB only once

2020-03-25 Thread GitBox
bshashikant commented on a change in pull request #709: HDDS-3244. Improve write efficiency by opening RocksDB only once URL: https://github.com/apache/hadoop-ozone/pull/709#discussion_r398305798 ## File path: hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/cont

[jira] [Updated] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3240: - Attachment: screenshot-2.png > Improve write efficiency by creating container in parallel. > -

[jira] [Updated] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3240: - Description: Now follower cannot create container until leader finish creating container. But follower an

[jira] [Updated] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3240: - Description: Now follower cannot create container until leader finish creating container. But follower an

[jira] [Commented] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067325#comment-17067325 ] runzhiwang commented on HDDS-3240: -- I'm working on it > Improve write efficiency by crea

[jira] [Updated] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3240: - Description: Now follower cannot create container until leader finish creating container. But follower an

[jira] [Updated] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3240: - Description: Now follower cannot create container until leader finish creating container. But follower an

[jira] [Updated] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3240: - Description: Now follower cannot create container until leader finish creating container. But follower an

[jira] [Reopened] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang reopened HDDS-3240: -- > Improve write efficiency by creating container in parallel. >

[jira] [Issue Comment Deleted] (HDDS-3240) Improve write efficiency by creating container in parallel.

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3240: - Comment: was deleted (was: duplicated with https://jira.apache.org/jira/browse/HDDS-3244) > Improve write

[jira] [Updated] (HDDS-3283) Improve write efficiency by improving RocksDB writeBatch

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3283: - Description: >From the image, you can find the whole write cost 140ms, but RocksDB >[writeBatch|https://g

[jira] [Commented] (HDDS-3283) Improve write efficiency by improving RocksDB writeBatch

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067287#comment-17067287 ] runzhiwang commented on HDDS-3283: -- I'm working on it. > Improve write efficiency by imp

[jira] [Updated] (HDDS-3283) Improve write efficiency by improving RocksDB writeBatch

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3283: - Description: >From the image, you can find the whole write cost 140ms, but RocksDB >writeBatch of two OM

[jira] [Updated] (HDDS-3283) Improve write efficiency by improving RocksDB writeBatch

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang updated HDDS-3283: - Attachment: screenshot-1.png > Improve write efficiency by improving RocksDB writeBatch >

[jira] [Assigned] (HDDS-3283) Improve write efficiency by improving RocksDB writeBatch

2020-03-25 Thread runzhiwang (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] runzhiwang reassigned HDDS-3283: Assignee: runzhiwang > Improve write efficiency by improving RocksDB writeBatch > -

[jira] [Created] (HDDS-3283) Improve write efficiency by improving RocksDB writeBatch

2020-03-25 Thread runzhiwang (Jira)
runzhiwang created HDDS-3283: Summary: Improve write efficiency by improving RocksDB writeBatch Key: HDDS-3283 URL: https://issues.apache.org/jira/browse/HDDS-3283 Project: Hadoop Distributed Data Store

[jira] [Updated] (HDDS-3164) Add Recon endpoint to serve missing containers and its metadata.

2020-03-25 Thread Vivek Ratnavel Subramanian (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vivek Ratnavel Subramanian updated HDDS-3164: - Description: Add a Recon API endpoint to serve missing containers informat

[GitHub] [hadoop-ozone] xiaoyuyao opened a new pull request #724: HDDS-3282. ozone.http.filter.initializers can't be set properly for S…

2020-03-25 Thread GitBox
xiaoyuyao opened a new pull request #724: HDDS-3282. ozone.http.filter.initializers can't be set properly for S… URL: https://github.com/apache/hadoop-ozone/pull/724 ## What changes were proposed in this pull request? Change to use the Initializer class from hadoop-common while keeping t

[jira] [Updated] (HDDS-3282) ozone.http.filter.initializers can't be set properly for SPNEGO auth

2020-03-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3282: - Labels: pull-request-available (was: ) > ozone.http.filter.initializers can't be set properly for

[jira] [Commented] (HDDS-3282) ozone.http.filter.initializers can't be set properly for SPNEGO auth

2020-03-25 Thread Xiaoyu Yao (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067254#comment-17067254 ] Xiaoyu Yao commented on HDDS-3282: -- {code:java} SCM start failed with exception java.lang

[jira] [Created] (HDDS-3282) ozone.http.filter.initializers can't be set properly for SPNEGO auth

2020-03-25 Thread Xiaoyu Yao (Jira)
Xiaoyu Yao created HDDS-3282: Summary: ozone.http.filter.initializers can't be set properly for SPNEGO auth Key: HDDS-3282 URL: https://issues.apache.org/jira/browse/HDDS-3282 Project: Hadoop Distributed

[jira] [Updated] (HDDS-3281) Add timeouts to all robot tests

2020-03-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3281: - Labels: pull-request-available (was: ) > Add timeouts to all robot tests > --

[GitHub] [hadoop-ozone] hanishakoneru opened a new pull request #723: HDDS-3281. Add timeouts to all robot tests

2020-03-25 Thread GitBox
hanishakoneru opened a new pull request #723: HDDS-3281. Add timeouts to all robot tests URL: https://github.com/apache/hadoop-ozone/pull/723 ## What changes were proposed in this pull request? We have seen in some CI runs that the acceptance test suit is getting cancelled as it run

[jira] [Created] (HDDS-3281) Add timeouts to all robot tests

2020-03-25 Thread Hanisha Koneru (Jira)
Hanisha Koneru created HDDS-3281: Summary: Add timeouts to all robot tests Key: HDDS-3281 URL: https://issues.apache.org/jira/browse/HDDS-3281 Project: Hadoop Distributed Data Store Issue Typ

[jira] [Updated] (HDDS-3273) OM HA: getconf must return all OMs

2020-03-25 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Wagle updated HDDS-3273: -- Summary: OM HA: getconf must return all OMs (was: getconf must return all OMs) > OM HA: getcon

[jira] [Assigned] (HDDS-2976) Recon throws error while trying to get snapshot in secure environment

2020-03-25 Thread Siddharth Wagle (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Wagle reassigned HDDS-2976: - Assignee: Siddharth Wagle (was: Shweta) > Recon throws error while trying to get snapsho

[GitHub] [hadoop-ozone] swagle commented on issue #718: HDDS-3224. Enforce volume and bucket name rule at create time.

2020-03-25 Thread GitBox
swagle commented on issue #718: HDDS-3224. Enforce volume and bucket name rule at create time. URL: https://github.com/apache/hadoop-ozone/pull/718#issuecomment-604141630 > > > I have a few comments. > > > > > > Thanks, Bharat for your comments. I will make those changes requeste

[GitHub] [hadoop-ozone] bharatviswa504 commented on issue #718: HDDS-3224. Enforce volume and bucket name rule at create time.

2020-03-25 Thread GitBox
bharatviswa504 commented on issue #718: HDDS-3224. Enforce volume and bucket name rule at create time. URL: https://github.com/apache/hadoop-ozone/pull/718#issuecomment-604138297 > > I have a few comments. > > Thanks, Bharat for your comments. I will make those changes requested but

[jira] [Created] (HDDS-3280) Ozone BaseHTTPServer should honor ozone.security.enabled config

2020-03-25 Thread Vivek Ratnavel Subramanian (Jira)
Vivek Ratnavel Subramanian created HDDS-3280: Summary: Ozone BaseHTTPServer should honor ozone.security.enabled config Key: HDDS-3280 URL: https://issues.apache.org/jira/browse/HDDS-3280 P

[GitHub] [hadoop-ozone] fapifta commented on a change in pull request #710: HDDS-3173. Provide better default JVM options

2020-03-25 Thread GitBox
fapifta commented on a change in pull request #710: HDDS-3173. Provide better default JVM options URL: https://github.com/apache/hadoop-ozone/pull/710#discussion_r398210290 ## File path: hadoop-ozone/dist/src/shell/hdds/hadoop-functions.sh ## @@ -1541,6 +1541,21 @@ functio

[GitHub] [hadoop-ozone] fapifta commented on issue #710: HDDS-3173. Provide better default JVM options

2020-03-25 Thread GitBox
fapifta commented on issue #710: HDDS-3173. Provide better default JVM options URL: https://github.com/apache/hadoop-ozone/pull/710#issuecomment-604118733 Thank you for addressing my concern, sounds reasonable, and I can accept the current solution. I agree that most of the users wil

[jira] [Updated] (HDDS-3164) Add Recon endpoint to serve missing containers and its metadata.

2020-03-25 Thread Aravindan Vijayan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravindan Vijayan updated HDDS-3164: Resolution: Fixed Status: Resolved (was: Patch Available) Thank you for the contrib

[GitHub] [hadoop-ozone] avijayanhwx commented on issue #714: HDDS-3164. Add Recon endpoint to serve missing containers and its metadata.

2020-03-25 Thread GitBox
avijayanhwx commented on issue #714: HDDS-3164. Add Recon endpoint to serve missing containers and its metadata. URL: https://github.com/apache/hadoop-ozone/pull/714#issuecomment-604104783 Thank you for the contribution @vivekratnavel. -

[GitHub] [hadoop-ozone] avijayanhwx merged pull request #714: HDDS-3164. Add Recon endpoint to serve missing containers and its metadata.

2020-03-25 Thread GitBox
avijayanhwx merged pull request #714: HDDS-3164. Add Recon endpoint to serve missing containers and its metadata. URL: https://github.com/apache/hadoop-ozone/pull/714 This is an automated message from the Apache Git Service.

[GitHub] [hadoop-ozone] arp7 commented on issue #643: HDDS-2339. Add OzoneManager to MiniOzoneChaosCluster

2020-03-25 Thread GitBox
arp7 commented on issue #643: HDDS-2339. Add OzoneManager to MiniOzoneChaosCluster URL: https://github.com/apache/hadoop-ozone/pull/643#issuecomment-604061845 Folks - any further comments on this? If not can we get this in? T

[GitHub] [hadoop-ozone] vivekratnavel commented on issue #714: HDDS-3164. Add Recon endpoint to serve missing containers and its metadata.

2020-03-25 Thread GitBox
vivekratnavel commented on issue #714: HDDS-3164. Add Recon endpoint to serve missing containers and its metadata. URL: https://github.com/apache/hadoop-ozone/pull/714#issuecomment-604043200 Acceptance test failure is not related to the change. It fails due to no space available on disk

[GitHub] [hadoop-ozone] smengcl edited a comment on issue #721: HDDS-3279. Rebase OFS branch

2020-03-25 Thread GitBox
smengcl edited a comment on issue #721: HDDS-3279. Rebase OFS branch URL: https://github.com/apache/hadoop-ozone/pull/721#issuecomment-603999618 Test failure assessment: - acceptance failure is from smoketest `ozonesecure-mr-mapreduce`. I recall witnessing the same failure before but not

[GitHub] [hadoop-ozone] smengcl merged pull request #703: HDDS-2945. Implement ofs://: Add robot tests for mkdir.

2020-03-25 Thread GitBox
smengcl merged pull request #703: HDDS-2945. Implement ofs://: Add robot tests for mkdir. URL: https://github.com/apache/hadoop-ozone/pull/703 This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [hadoop-ozone] sodonnel commented on issue #719: HDDS-3270. Allow safemode listeners to be notified when some precheck rules pass

2020-03-25 Thread GitBox
sodonnel commented on issue #719: HDDS-3270. Allow safemode listeners to be notified when some precheck rules pass URL: https://github.com/apache/hadoop-ozone/pull/719#issuecomment-604029510 Getting intermitted unit test failure on: ``` [ERROR] Errors: [ERROR] TestDeadNodeHa

[GitHub] [hadoop-ozone] smengcl edited a comment on issue #721: HDDS-3279. Rebase OFS branch

2020-03-25 Thread GitBox
smengcl edited a comment on issue #721: HDDS-3279. Rebase OFS branch URL: https://github.com/apache/hadoop-ozone/pull/721#issuecomment-603999618 Test failure assessment: - acceptance failure is from smoketest `ozonesecure-mr-mapreduce`. I recall witnessing the same failure before but not

[GitHub] [hadoop-ozone] smengcl commented on issue #721: HDDS-3279. Rebase OFS branch

2020-03-25 Thread GitBox
smengcl commented on issue #721: HDDS-3279. Rebase OFS branch URL: https://github.com/apache/hadoop-ozone/pull/721#issuecomment-603999618 Test failure assessment: - acceptance failure is from smoketest `ozonesecure-mr-mapreduce`. I recall witnessing the same failure before but not doing

[GitHub] [hadoop-ozone] smengcl commented on issue #721: HDDS-3279. Rebase OFS branch

2020-03-25 Thread GitBox
smengcl commented on issue #721: HDDS-3279. Rebase OFS branch URL: https://github.com/apache/hadoop-ozone/pull/721#issuecomment-603995561 > You actually don't need to submit a PR. Simply git pull --rebase, and then git push -f to force push the branch. But good to see additional check after

[GitHub] [hadoop-ozone] cxorm commented on a change in pull request #399: HDDS-2424. Add the recover-trash command server side handling.

2020-03-25 Thread GitBox
cxorm commented on a change in pull request #399: HDDS-2424. Add the recover-trash command server side handling. URL: https://github.com/apache/hadoop-ozone/pull/399#discussion_r397964460 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key

[GitHub] [hadoop-ozone] jojochuang commented on issue #721: HDDS-3279. Rebase OFS branch

2020-03-25 Thread GitBox
jojochuang commented on issue #721: HDDS-3279. Rebase OFS branch URL: https://github.com/apache/hadoop-ozone/pull/721#issuecomment-603983510 You actually don't need to submit a PR. Simply git pull --rebase, and then git push -f to force push the branch. But good to see additional check afte

[GitHub] [hadoop-ozone] elek commented on issue #720: HDDS-3185 Construct a standalone ratis server for SCM.

2020-03-25 Thread GitBox
elek commented on issue #720: HDDS-3185 Construct a standalone ratis server for SCM. URL: https://github.com/apache/hadoop-ozone/pull/720#issuecomment-603977290 Thanks to work on this @timmylicheng I am confused a little bit because this was the last message on the mailing list:

[GitHub] [hadoop-ozone] avijayanhwx commented on issue #722: HDDS-3074. Make the configuration of container scrub consistent.

2020-03-25 Thread GitBox
avijayanhwx commented on issue #722: HDDS-3074. Make the configuration of container scrub consistent. URL: https://github.com/apache/hadoop-ozone/pull/722#issuecomment-603974479 LGTM +1 pending CI. This is an automated messag

[jira] [Commented] (HDDS-3241) Invalid container reported to SCM should be deleted

2020-03-25 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066888#comment-17066888 ] Marton Elek commented on HDDS-3241: --- bq. The issue described by you is a case where the

[GitHub] [hadoop-ozone] elek commented on issue #702: HDDS-3241. Invalid container reported to SCM should be deleted.

2020-03-25 Thread GitBox
elek commented on issue #702: HDDS-3241. Invalid container reported to SCM should be deleted. URL: https://github.com/apache/hadoop-ozone/pull/702#issuecomment-603972493 Note: see the discussion in the jira: https://issues.apache.org/jira/browse/HDDS-3241 -

[jira] [Updated] (HDDS-3074) Make the configuration of container scrub consistent

2020-03-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3074: - Labels: pull-request-available (was: ) > Make the configuration of container scrub consistent > -

[GitHub] [hadoop-ozone] cku328 opened a new pull request #722: HDDS-3074. Make the configuration of container scrub consistent.

2020-03-25 Thread GitBox
cku328 opened a new pull request #722: HDDS-3074. Make the configuration of container scrub consistent. URL: https://github.com/apache/hadoop-ozone/pull/722 ## What changes were proposed in this pull request? When we set `hdds.container.scrub.enabled` (in ozone-site.xml) to true, co

[GitHub] [hadoop-ozone] smengcl commented on a change in pull request #696: HDDS-3056. Allow users to list volumes they have access to, and optionally allow all users to list all volumes

2020-03-25 Thread GitBox
smengcl commented on a change in pull request #696: HDDS-3056. Allow users to list volumes they have access to, and optionally allow all users to list all volumes URL: https://github.com/apache/hadoop-ozone/pull/696#discussion_r398019230 ## File path: hadoop-hdds/common/src/main/re

[GitHub] [hadoop-ozone] xiaoyuyao commented on issue #665: HDDS-3160. Disable index and filter block cache for RocksDB.

2020-03-25 Thread GitBox
xiaoyuyao commented on issue #665: HDDS-3160. Disable index and filter block cache for RocksDB. URL: https://github.com/apache/hadoop-ozone/pull/665#issuecomment-603963226 Thanks @elek for the additional info. That helps. I'm just curious on the bottom-right, why rocks DB suddenly had a s

[jira] [Updated] (HDDS-3088) maxRetries value is too large while trying to reconnect to SCM server

2020-03-25 Thread Mukul Kumar Singh (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDDS-3088: Priority: Major (was: Blocker) > maxRetries value is too large while trying to reconnect to

[jira] [Updated] (HDDS-3279) Rebase OFS branch

2020-03-25 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-3279: - Status: Patch Available (was: Open) > Rebase OFS branch > - > > Key: HDDS

[jira] [Updated] (HDDS-3279) Rebase OFS branch

2020-03-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDDS-3279: - Labels: pull-request-available (was: ) > Rebase OFS branch > - > >

[GitHub] [hadoop-ozone] smengcl opened a new pull request #721: HDDS-3279. Rebase OFS branch

2020-03-25 Thread GitBox
smengcl opened a new pull request #721: HDDS-3279. Rebase OFS branch URL: https://github.com/apache/hadoop-ozone/pull/721 ## What changes were proposed in this pull request? Merge commits on master branch to OFS dev branch: `git merge master` Also need to manually apply a coupl

[jira] [Resolved] (HDDS-3243) Recon should not have the ability to send Create/Close Container commands to Datanode.

2020-03-25 Thread Aravindan Vijayan (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aravindan Vijayan resolved HDDS-3243. - Resolution: Fixed PR merged. > Recon should not have the ability to send Create/Close Con

[GitHub] [hadoop-ozone] avijayanhwx merged pull request #712: HDDS-3243. Recon should not have the ability to send Create/Close Con…

2020-03-25 Thread GitBox
avijayanhwx merged pull request #712: HDDS-3243. Recon should not have the ability to send Create/Close Con… URL: https://github.com/apache/hadoop-ozone/pull/712 This is an automated message from the Apache Git Service. To r

[GitHub] [hadoop-ozone] avijayanhwx commented on issue #712: HDDS-3243. Recon should not have the ability to send Create/Close Con…

2020-03-25 Thread GitBox
avijayanhwx commented on issue #712: HDDS-3243. Recon should not have the ability to send Create/Close Con… URL: https://github.com/apache/hadoop-ozone/pull/712#issuecomment-603937582 Thanks for the review @vivekratnavel & @bharatviswa504. --

[jira] [Updated] (HDDS-3279) Rebase OFS branch

2020-03-25 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-3279: - Description: Merge commits on master branch to OFS dev branch: {{git merge master}} Also need to manually

[jira] [Updated] (HDDS-3279) Rebase OFS branch

2020-03-25 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-3279: - Description: Need to manually apply a couple of changes in master branch to OFS classes as well: - HDDS-3

[jira] [Updated] (HDDS-3279) Rebase OFS branch

2020-03-25 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-3279: - Summary: Rebase OFS branch (was: Rebase OFS dev branch: adapt changes from master branch commits) > Reba

[jira] [Assigned] (HDDS-3178) Add unit tests for OMGetDelegationToken Request and Response

2020-03-25 Thread YiSheng Lien (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YiSheng Lien reassigned HDDS-3178: -- Assignee: YiSheng Lien > Add unit tests for OMGetDelegationToken Request and Response > ---

[jira] [Resolved] (HDDS-3278) Acceptance test ozonesecure-mr-mapreduce failing

2020-03-25 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stephen O'Donnell resolved HDDS-3278. - Resolution: Not A Problem > Acceptance test ozonesecure-mr-mapreduce failing > ---

[jira] [Commented] (HDDS-3278) Acceptance test ozonesecure-mr-mapreduce failing

2020-03-25 Thread Stephen O'Donnell (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066784#comment-17066784 ] Stephen O'Donnell commented on HDDS-3278: - The reason for this failure, is that I

[GitHub] [hadoop-ozone] cxorm commented on a change in pull request #399: HDDS-2424. Add the recover-trash command server side handling.

2020-03-25 Thread GitBox
cxorm commented on a change in pull request #399: HDDS-2424. Add the recover-trash command server side handling. URL: https://github.com/apache/hadoop-ozone/pull/399#discussion_r397964460 ## File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/response/key

[jira] [Updated] (HDDS-3279) Rebase OFS dev branch: adapt changes from master branch commits

2020-03-25 Thread Siyao Meng (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siyao Meng updated HDDS-3279: - Description: So far need to apply [changes|https://github.com/apache/hadoop-ozone/commit/bb9933c96941a57b5

[jira] [Created] (HDDS-3279) Rebase OFS dev branch: adapt changes from master branch commits

2020-03-25 Thread Siyao Meng (Jira)
Siyao Meng created HDDS-3279: Summary: Rebase OFS dev branch: adapt changes from master branch commits Key: HDDS-3279 URL: https://issues.apache.org/jira/browse/HDDS-3279 Project: Hadoop Distributed Data

[jira] [Updated] (HDDS-3266) Intermittent TestSecureOzoneRpcClient failure due to DEADLINE_EXCEEDED

2020-03-25 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek updated HDDS-3266: -- Priority: Blocker (was: Major) > Intermittent TestSecureOzoneRpcClient failure due to DEADLINE_EXCEEDED

[jira] [Updated] (HDDS-3266) Intermittent TestSecureOzoneRpcClient failure due to DEADLINE_EXCEEDED

2020-03-25 Thread Marton Elek (Jira)
[ https://issues.apache.org/jira/browse/HDDS-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Elek updated HDDS-3266: -- Attachment: org.apache.hadoop.ozone.freon.TestRandomKeyGenerator-output.txt > Intermittent TestSecureOzo

[GitHub] [hadoop-ozone] avijayanhwx commented on a change in pull request #715: HDDS-3253. Initialize Recon metrics for prometheus at /prom endpoint

2020-03-25 Thread GitBox
avijayanhwx commented on a change in pull request #715: HDDS-3253. Initialize Recon metrics for prometheus at /prom endpoint URL: https://github.com/apache/hadoop-ozone/pull/715#discussion_r397943155 ## File path: hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/Recon

[jira] [Created] (HDDS-3278) Acceptance test ozonesecure-mr-mapreduce failing

2020-03-25 Thread Stephen O'Donnell (Jira)
Stephen O'Donnell created HDDS-3278: --- Summary: Acceptance test ozonesecure-mr-mapreduce failing Key: HDDS-3278 URL: https://issues.apache.org/jira/browse/HDDS-3278 Project: Hadoop Distributed Data St

[jira] [Created] (HDDS-3277) Datanodes do not close pipeline when pipeline directory is deleted.

2020-03-25 Thread Mukul Kumar Singh (Jira)
Mukul Kumar Singh created HDDS-3277: --- Summary: Datanodes do not close pipeline when pipeline directory is deleted. Key: HDDS-3277 URL: https://issues.apache.org/jira/browse/HDDS-3277 Project: Hadoop

[GitHub] [hadoop-ozone] elek commented on a change in pull request #687: HDDS-2184. Rename ozone scmcli to ozone admin

2020-03-25 Thread GitBox
elek commented on a change in pull request #687: HDDS-2184. Rename ozone scmcli to ozone admin URL: https://github.com/apache/hadoop-ozone/pull/687#discussion_r397878003 ## File path: hadoop-hdds/docs/content/tools/SCMCLI.md ## @@ -1,27 +0,0 @@ Review comment: I

[GitHub] [hadoop-ozone] elek commented on a change in pull request #687: HDDS-2184. Rename ozone scmcli to ozone admin

2020-03-25 Thread GitBox
elek commented on a change in pull request #687: HDDS-2184. Rename ozone scmcli to ozone admin URL: https://github.com/apache/hadoop-ozone/pull/687#discussion_r397875948 ## File path: hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/SafeModeCommands.java ##

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #668: HDDS-3139 Pipeline placement should max out pipeline usage

2020-03-25 Thread GitBox
timmylicheng commented on a change in pull request #668: HDDS-3139 Pipeline placement should max out pipeline usage URL: https://github.com/apache/hadoop-ozone/pull/668#discussion_r397858137 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/Pipe

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #668: HDDS-3139 Pipeline placement should max out pipeline usage

2020-03-25 Thread GitBox
timmylicheng commented on a change in pull request #668: HDDS-3139 Pipeline placement should max out pipeline usage URL: https://github.com/apache/hadoop-ozone/pull/668#discussion_r397857851 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/Pipe

[GitHub] [hadoop-ozone] timmylicheng commented on issue #668: HDDS-3139 Pipeline placement should max out pipeline usage

2020-03-25 Thread GitBox
timmylicheng commented on issue #668: HDDS-3139 Pipeline placement should max out pipeline usage URL: https://github.com/apache/hadoop-ozone/pull/668#issuecomment-603840741 Before pipeline max: 13, pipeline actual: 12 Pipeline count on node: 1 Pipeline count on node: 5 Pipeline

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #678: HDDS-3179 Pipeline placement based on Topology does not have fallback

2020-03-25 Thread GitBox
timmylicheng commented on a change in pull request #678: HDDS-3179 Pipeline placement based on Topology does not have fallback URL: https://github.com/apache/hadoop-ozone/pull/678#discussion_r397830892 ## File path: hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/pi

[GitHub] [hadoop-ozone] elek commented on a change in pull request #687: HDDS-2184. Rename ozone scmcli to ozone admin

2020-03-25 Thread GitBox
elek commented on a change in pull request #687: HDDS-2184. Rename ozone scmcli to ozone admin URL: https://github.com/apache/hadoop-ozone/pull/687#discussion_r397824173 ## File path: hadoop-hdds/tools/src/main/java/org/apache/hadoop/hdds/scm/cli/container/WithScmClient.java #

[GitHub] [hadoop-ozone] elek commented on issue #578: HDDS-3053. Decrease the number of the chunk writer threads

2020-03-25 Thread GitBox
elek commented on issue #578: HDDS-3053. Decrease the number of the chunk writer threads URL: https://github.com/apache/hadoop-ozone/pull/578#issuecomment-603816449 > Can we make it a data driven decision? I looked at the linked document. I didn't understand the test methodology. E.g. when

[GitHub] [hadoop-ozone] elek commented on issue #578: HDDS-3053. Decrease the number of the chunk writer threads

2020-03-25 Thread GitBox
elek commented on issue #578: HDDS-3053. Decrease the number of the chunk writer threads URL: https://github.com/apache/hadoop-ozone/pull/578#issuecomment-603814439 > So, reducing this will make request also processing slow Good point, and I think they should be separated anyway. Bu

[GitHub] [hadoop-ozone] elek edited a comment on issue #665: HDDS-3160. Disable index and filter block cache for RocksDB.

2020-03-25 Thread GitBox
elek edited a comment on issue #665: HDDS-3160. Disable index and filter block cache for RocksDB. URL: https://github.com/apache/hadoop-ozone/pull/665#issuecomment-603812461 > I mean do we see better BLOCK_CACHE_DATA_HIT @xiaoyuyao Good questions, I checked it and saved the screensh

[GitHub] [hadoop-ozone] elek commented on issue #665: HDDS-3160. Disable index and filter block cache for RocksDB.

2020-03-25 Thread GitBox
elek commented on issue #665: HDDS-3160. Disable index and filter block cache for RocksDB. URL: https://github.com/apache/hadoop-ozone/pull/665#issuecomment-603812461 > I mean do we see better BLOCK_CACHE_DATA_HIT Yes: ![image](https://user-images.githubusercontent.com/17054

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #678: HDDS-3179 Pipeline placement based on Topology does not have fallback

2020-03-25 Thread GitBox
timmylicheng commented on a change in pull request #678: HDDS-3179 Pipeline placement based on Topology does not have fallback URL: https://github.com/apache/hadoop-ozone/pull/678#discussion_r397795926 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pi

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #678: HDDS-3179 Pipeline placement based on Topology does not have fallback

2020-03-25 Thread GitBox
timmylicheng commented on a change in pull request #678: HDDS-3179 Pipeline placement based on Topology does not have fallback URL: https://github.com/apache/hadoop-ozone/pull/678#discussion_r397795809 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pi

[GitHub] [hadoop-ozone] timmylicheng commented on a change in pull request #678: HDDS-3179 Pipeline placement based on Topology does not have fallback

2020-03-25 Thread GitBox
timmylicheng commented on a change in pull request #678: HDDS-3179 Pipeline placement based on Topology does not have fallback URL: https://github.com/apache/hadoop-ozone/pull/678#discussion_r397795529 ## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pi

[GitHub] [hadoop-ozone] runzhiwang commented on a change in pull request #710: HDDS-3173. Provide better default JVM options

2020-03-25 Thread GitBox
runzhiwang commented on a change in pull request #710: HDDS-3173. Provide better default JVM options URL: https://github.com/apache/hadoop-ozone/pull/710#discussion_r397793264 ## File path: hadoop-ozone/dist/src/shell/hdds/hadoop-functions.sh ## @@ -1541,6 +1541,21 @@ func

[GitHub] [hadoop-ozone] elek commented on a change in pull request #710: HDDS-3173. Provide better default JVM options

2020-03-25 Thread GitBox
elek commented on a change in pull request #710: HDDS-3173. Provide better default JVM options URL: https://github.com/apache/hadoop-ozone/pull/710#discussion_r397785874 ## File path: hadoop-ozone/dist/src/shell/hdds/hadoop-functions.sh ## @@ -1541,6 +1541,21 @@ function h

[GitHub] [hadoop-ozone] elek commented on a change in pull request #710: HDDS-3173. Provide better default JVM options

2020-03-25 Thread GitBox
elek commented on a change in pull request #710: HDDS-3173. Provide better default JVM options URL: https://github.com/apache/hadoop-ozone/pull/710#discussion_r397784329 ## File path: hadoop-ozone/dist/src/test/shell/gc_opts.bats ## @@ -0,0 +1,41 @@ +#!/usr/bin/env bash +#

  1   2   >