[jira] [Comment Edited] (HDFS-7240) Object store in HDFS
[
https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16251930#comment-16251930
]
Daniel Takamori edited comment on HDFS-7240 at 11/14/17 6:47 PM:
-
*+This message is being proxy posted by an Infra admin because it contains
banned strings triggering our spam filter.
+*
Before we send out the [Vote] thread for ozone, I propose that we do two
community
meetings. This allows us to address any questions issues over a high bandwidth
medium.
Since many contributors/committers of ozone are spread across the world, the
first meeting is friendly toward Europe/Asia time Zone. I propose the following
time for the meeting.
||Location || Local Time || Time zone || UTC Offset
|Seattle / San Jose | Thursday, November 16, 2017 at 1:00:00 am (night) | PST |
UTC-8 hours
|London| Thursday, November 16, 2017 at 9:00:00 am | GMT | UTC
|Budapest/Brussels|Thursday, November 16, 2017 at 10:00:00 am | CET | UTC+1 hour
|Bangalore | Thursday, November 16, 2017 at 2:30:00 pm |IST | UTC+5:30 hours
|Shanghai | Thursday, November 16, 2017 at 5:00:00 pm | CST | UTC+8 hours
I propose that we have a follow-up meeting targetting Americas time zone, and I
propose *Friday, November 17, 2017, at 4:00 pm PST* for that meeting.
Here is the meeting info:
{noformat}
Topic: Ozone Merge meeting
Time: Nov 16, 2017 1:00 AM Pacific Time (US and Canada)
Join from PC, Mac, Linux, iOS or Android:
https://hortonworks.zoom.us/j/5451676776
Or join by phone:
+1 646 558 8656 (US Toll) or +1 669 900 6833 (US Toll)
+1 877 369 0926 (US Toll Free)
+1 877 853 5247 (US Toll Free)
Meeting ID: 545 167 6776
International numbers available:
https://hortonworks.zoom.us/zoomconference?m=rYZYSAOVLYtFE6wkwIrjJeqO3CP_I6ij
was (Author: pono):
*+This message is being proxy posted by an Infra admin because it contains
banned strings triggering our spam filter.
+*
Before we send out the [Vote] thread for ozone, I propose that we do two
community
meetings. This allows us to address any questions issues over a high bandwidth
medium.
Since many contributors/committers of ozone are spread across the world, the
first meeting is friendly toward Europe/Asia time Zone. I propose the following
time for the meeting.
||Location || Local Time || Time zone || UTC Offset
|Seattle / San Jose | Thursday, November 16, 2017 at 1:00:00 am (night) | PST |
UTC-8 hours
|London| Thursday, November 16, 2017 at 9:00:00 am | GMT | UTC
|Budapest/Brussels|Thursday, November 16, 2017 at 10:00:00 am | CET | UTC+1 hour
|Bangalore | Thursday, November 16, 2017 at 2:30:00 pm |IST | UTC+5:30 hours
|Shanghai | Thursday, November 16, 2017 at 5:00:00 pm | CST | UTC+8 hours
I propose that we have a follow-up meeting targetting Americas time zone, and I
propose *Friday, November 17, 2017, at 4:00 pm PST* for that meeting.
Here is the meeting info:
{noformat}
Topic: Ozone Merge meeting
Time: Nov 16, 2017 1:00 AM Pacific Time (US and Canada)
Join from PC, Mac, Linux, iOS or Android:
https://hortonworks.zoom.us/j/5451676776
Or join by phone:
+1 646 558 8656 (US Toll) or +1 669 900 6833 (US Toll)
+1 877 369 0926 (US Toll Free)
+1 877 853 5247 (US Toll Free)
Meeting ID: 545 167 6776
International numbers available:
https://hortonworks.zoom.us/zoomconference?m=rYZYSAOVLYtFE6wkwIrjJeqO3CP_I6ij
> Object store in HDFS
>
>
> Key: HDFS-7240
> URL: https://issues.apache.org/jira/browse/HDFS-7240
> Project: Hadoop HDFS
> Issue Type: New Feature
>Reporter: Jitendra Nath Pandey
>Assignee: Jitendra Nath Pandey
> Attachments: HDFS Scalability and Ozone.pdf, HDFS-7240.001.patch,
> HDFS-7240.002.patch, HDFS-7240.003.patch, HDFS-7240.003.patch,
> HDFS-7240.004.patch, HDFS-7240.005.patch, HDFS-7240.006.patch,
> MeetingMinutes.pdf, Ozone-architecture-v1.pdf, Ozonedesignupdate.pdf,
> ozone_user_v0.pdf
>
>
> This jira proposes to add object store capabilities into HDFS.
> As part of the federation work (HDFS-1052) we separated block storage as a
> generic storage layer. Using the Block Pool abstraction, new kinds of
> namespaces can be built on top of the storage layer i.e. datanodes.
> In this jira I will explore building an object store using the datanode
> storage, but independent of namespace metadata.
> I will soon update with a detailed design document.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-7240) Object store in HDFS
[
https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16251930#comment-16251930
]
Daniel Takamori commented on HDFS-7240:
---
*+This message is being proxy posted by an Infra admin because it contains
banned strings triggering our spam filter.
+*
Before we send out the [Vote] thread for ozone, I propose that we do two
community
meetings. This allows us to address any questions issues over a high bandwidth
medium.
Since many contributors/committers of ozone are spread across the world, the
first meeting is friendly toward Europe/Asia time Zone. I propose the following
time for the meeting.
||Location || Local Time || Time zone || UTC Offset
|Seattle / San Jose | Thursday, November 16, 2017 at 1:00:00 am (night) | PST |
UTC-8 hours
|London| Thursday, November 16, 2017 at 9:00:00 am | GMT | UTC
|Budapest/Brussels|Thursday, November 16, 2017 at 10:00:00 am | CET | UTC+1 hour
|Bangalore | Thursday, November 16, 2017 at 2:30:00 pm |IST | UTC+5:30 hours
|Shanghai | Thursday, November 16, 2017 at 5:00:00 pm | CST | UTC+8 hours
I propose that we have a follow-up meeting targetting Americas time zone, and I
propose *Friday, November 17, 2017, at 4:00 pm PST* for that meeting.
Here is the meeting info:
{noformat}
Topic: Ozone Merge meeting
Time: Nov 16, 2017 1:00 AM Pacific Time (US and Canada)
Join from PC, Mac, Linux, iOS or Android:
https://hortonworks.zoom.us/j/5451676776
Or join by phone:
+1 646 558 8656 (US Toll) or +1 669 900 6833 (US Toll)
+1 877 369 0926 (US Toll Free)
+1 877 853 5247 (US Toll Free)
Meeting ID: 545 167 6776
International numbers available:
https://hortonworks.zoom.us/zoomconference?m=rYZYSAOVLYtFE6wkwIrjJeqO3CP_I6ij
> Object store in HDFS
>
>
> Key: HDFS-7240
> URL: https://issues.apache.org/jira/browse/HDFS-7240
> Project: Hadoop HDFS
> Issue Type: New Feature
>Reporter: Jitendra Nath Pandey
>Assignee: Jitendra Nath Pandey
> Attachments: HDFS Scalability and Ozone.pdf, HDFS-7240.001.patch,
> HDFS-7240.002.patch, HDFS-7240.003.patch, HDFS-7240.003.patch,
> HDFS-7240.004.patch, HDFS-7240.005.patch, HDFS-7240.006.patch,
> MeetingMinutes.pdf, Ozone-architecture-v1.pdf, Ozonedesignupdate.pdf,
> ozone_user_v0.pdf
>
>
> This jira proposes to add object store capabilities into HDFS.
> As part of the federation work (HDFS-1052) we separated block storage as a
> generic storage layer. Using the Block Pool abstraction, new kinds of
> namespaces can be built on top of the storage layer i.e. datanodes.
> In this jira I will explore building an object store using the datanode
> storage, but independent of namespace metadata.
> I will soon update with a detailed design document.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-9650) Problem is logging of "Redundant addStoredBlock request received"
[ https://issues.apache.org/jira/browse/HDFS-9650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Takamori updated HDFS-9650: -- Summary: Problem is logging of "Redundant addStoredBlock request received" (was: Need to create issue in HDFS-jira) > Problem is logging of "Redundant addStoredBlock request received" > - > > Key: HDFS-9650 > URL: https://issues.apache.org/jira/browse/HDFS-9650 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Frode Halvorsen > > Description; > Hadoop 2.7.1. 2 namenodes in HA. 14 Datanodes. > Enough CPU,disk and RAM. > Just discovered that some datanodes must have been corrupted somehow. > When restarting a 'defect' ( works without failure except when restarting) > the active namenode suddenly is logging a lot of : "Redundant addStoredBlock > request received" > and finally the failover-controller takes the namenode down, fails over to > other node. This node also starts logging the same, and as soon as the fisrt > node is bac online, the failover-controller again kill the active node, and > does failover. > This node now was started after the datanode, and doesn't log "Redundant > addStoredBlock request received" anymore, and restart of the second name-node > works fine. > If I again restarts the datanode- the process repeats itself. > Problem is logging of "Redundant addStoredBlock request received" and why > does it happen ? > The failover-controller acts the same way as it did on 2.5/6 when we had a > lot of 'block does not belong to any replica'-messages. Namenode is too busy > to respond to heartbeats, and is taken down... > To resolv this, I have to take down the datanode, delete all data from it, > and start it up. Then cluster will reproduce the missing blocks, and the > failing datanode is working fine again... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Moved] (HDFS-9650) Need to create issue in HDFS-jira
[ https://issues.apache.org/jira/browse/HDFS-9650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Takamori moved INFRA-11079 to HDFS-9650: --- INFRA-Members: (was: [infrastructure-team]) Component/s: (was: JIRA) Workflow: no-reopen-closed, patch-avail (was: INFRA Workflow) Key: HDFS-9650 (was: INFRA-11079) Project: Hadoop HDFS (was: Infrastructure) > Need to create issue in HDFS-jira > - > > Key: HDFS-9650 > URL: https://issues.apache.org/jira/browse/HDFS-9650 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Frode Halvorsen > > Description; > Hadoop 2.7.1. 2 namenodes in HA. 14 Datanodes. > Enough CPU,disk and RAM. > Just discovered that some datanodes must have been corrupted somehow. > When restarting a 'defect' ( works without failure except when restarting) > the active namenode suddenly is logging a lot of : "Redundant addStoredBlock > request received" > and finally the failover-controller takes the namenode down, fails over to > other node. This node also starts logging the same, and as soon as the fisrt > node is bac online, the failover-controller again kill the active node, and > does failover. > This node now was started after the datanode, and doesn't log "Redundant > addStoredBlock request received" anymore, and restart of the second name-node > works fine. > If I again restarts the datanode- the process repeats itself. > Problem is logging of "Redundant addStoredBlock request received" and why > does it happen ? > The failover-controller acts the same way as it did on 2.5/6 when we had a > lot of 'block does not belong to any replica'-messages. Namenode is too busy > to respond to heartbeats, and is taken down... > To resolv this, I have to take down the datanode, delete all data from it, > and start it up. Then cluster will reproduce the missing blocks, and the > failing datanode is working fine again... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
