Re: [Proposal]New storage project: HBlock

[Sheng Wu]

Hi

Personally, and basically, I am feeling the team has misunderstood
the meaning of incubator and the requirements of building the community.
Same as the last time discussion, I still think they will be in a big
pressure as they have to deal with the basic feature development, community
build and following ASF incubator requirements at the same time if they are
accepted into the incubator. And at the same time, the team lacks the
experiences of open source community in or out of ASF.
I am not sure whether this is good for the project. Seem like a little
hurry to join the incubator.
More Comments inline.

Willing to listen to what other IPMCs think.

 于2020年3月10日周二 上午10:21写道：

> Hi, All,
>
> We are China Telecom Corporation Limited Cloud Computing Branch
> Corporation.
> We hope to contribute one of our projects named 'HBlock' to Apache.
> Here is the proposal of HBlock project, please feel free to let me know
> what
> the concerns and suggestions from you. Thank you so much.
>
> HBlock Proposal
>
> 1.Abstract
> The HBlock project will be an enterprise distributed block storage.
>
> 2.Proposal
> HBlock provides a distributed block storage with the following features:
> 2.1.User-space iSCSI target: HBlock will implement an iSCSI target that is
> RFC-7143 (https://tools.ietf.org/html/rfc7143) compliant written in pure
> Java designed to run on top of any mainstream Operating System, including
> Windows and Linux, as a user-space process.
> 2.2.Enterprise level features: HBlock will implement comprehensive
> enterprise level features, such as
> Asymmetric Logical Unit Access (ALUA, Information technology -SCSI Primary
> Commands - 4 (SPC-4), https://www.t10.org/cgi-bin/ac.pl?t=f&f=spc4r37.pdf),
>
> Persistent Reservations (PR, Information technology -SCSI Primary Commands
> -
> 4 (SPC-4), https://www.t10.org/cgi-bin/ac.pl?t=f&f=spc4r37.pdf),
> VMware vSphere Storage APIs - Array Integration(VAAI,
>
> https://www.vmware.com/techpapers/2012/vmware-vsphere-storage-apis-array-int
> egration-10337.html
> 
> ),
> Offloaded Data Transfer(ODX,
>
> https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-se
> rver-2012-R2-and-2012/hh831628(v=ws.11)
> ),
> so that it will support
> session-level fail-over,
> Oracle Real Application Cluster(Oracle RAC,
> https://www.oracle.com/database/technologies/rac.html) ,
> Cluster File System (CFS), VMware cluster and Windows cluster.
> 2.3.Low latency: HBlock will implement in-memory distributed cache to
> reduce
> write latency and improve Input / Output Operations Per Second (IOPS), and
> it will leverage storage-class memory to archive even higher durability
> without IOPS loss.
> 2.4.Smart Compaction and Garbage Collection(GC): HBlock will convert all
> the
> write operations into sequential append operations to improve the random
> write performance, and it will choose the best timing to compact and
> collect
> the garbage per Logic Unit (LU). Comparting to Solid State Drives (SSD's)
> internal Garbage Collection, such a global GC will reduce the need of SSD's
> internal GC, which indirectly make SSD have more usable space, and have
> even
> better GC strategy due to close to application. In essence, flash writes
> data in block (32MB) order. In order to realize random write, SSD disk will
> reserve a part of space for GC in the disk. Therefore, the more random
> write
> and delete, the more space needs to be reserved. HDFS based writes are
> sequential for SSD, so the space reserved in SSD is small. In short, as
> long
> as there is a GC, there must be reserved space, either in the HBlock layer
> or in the controller layer inside the SSD. Because HBlock is closer to LU,
> it can be more efficient GC. For example, a LU dedicated to video
> monitoring
> data basically writes video data in sequence, and starts writing again when
> the disk is full. This LU does not need any GC at all. If you do GC in the
> SSD layer, SSD will see the data of various LUs, and unnecessary movement
> will be made to the LU dedicated for video monitoring.
> 2.5.Hadoop Distributed File System (HDFS)-based: HBlock leverages HDFS a as
> persistent layer to avoid reinventing wheels. The iSCSI target will run on
> the client side of HDFS and directly read or write data from or to Data
> Nodes.
> 2.6.Easy to deploy: HBlock will provide easy-to-use utilities to make the
> installation process extremely easy. Since HBlock does not rely on any
> Operating System, deployment is easy unlike other storage systems that rely
> on in-kernel iSCSI module, such as Linux-IO (LIO), or SCST.
>

I noticed there are a lot of `will`s here in the Proposal section as the
project core features.
Are these language issues or all these features not available today?
Which parts have been implemented

Re: [Proposal]New storage project: HBlock

[Rich Bowen]

On 3/9/20 10:45 PM, Sheng Wu wrote:

Hi

Personally, and basically, I am feeling the team has misunderstood
the meaning of incubator and the requirements of building the community.
Same as the last time discussion, I still think they will be in a big
pressure as they have to deal with the basic feature development, community
build and following ASF incubator requirements at the same time if they are
accepted into the incubator. And at the same time, the team lacks the
experiences of open source community in or out of ASF.


I find this remark confusing. Surely this is what the incubator is *for* 
- to learn about open source community at the ASF.


A strong community and "basic feature development" are not requirements 
for entering the Incubator. Rather, the incubator is the place for 
community building (among other things).



I noticed there are a lot of `will`s here in the Proposal section as the
project core features.
Are these language issues or all these features not available today?
Which parts have been implemented?



There is no requirement that a project be a completed product when it 
comes to the ASF. Indeed, as our friend Stefano Mazzocchi observed, all 
those years ago, coming in with a completed product makes it a lot 
harder to build a strong community, because there's nothing for them to do.


I readily admit that I've been away from the Incubator for some time, 
but surely we don't require projects to have a robust feature set and 
vibrant community as a requirement of entry. That would seem completely 
contrary to the entire point of coming here.


--
Rich Bowen - rbo...@rcbowen.com
http://rcbowen.com/
@rbowen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [Proposal]New storage project: HBlock

[Ted Dunning]

Three things are very clear to me:

1) having an open source iSCSI implementation from a mature and experienced
storage stream is a very cool thing, especially if it can be targeted to
non HDFS storage relatively easily. Building such a thing requires very
high levels of experience and expertise that have generally been lacking in
the open source world.

2) this team is very naive about the negative impacts that Apache processes
will have on their development speed and will need lots of mentoring. Given
their release schedule, I think that there are symmetrical risks, first
that the team will be tempted to JFDI when getting features out the door
rather than communicate and share designs and second that if they build a
proper community overcoming language, timezone and large internal team
dynamics that the internal political costs will severe due to slower
development.

3) this team is very enthusiastic about making open source work and that
might be enough to allow them to succeed in spite of the difficulties.

The path to success here is, in my opinion, to require strong and engaged
mentorship and make it very clear before they come in that Apache may not
be a good fit due to the pressures they face to delivery on a schedule. If
incubation with a high risk of exit back to a non-Apache form is acceptable
to the project team, then it should be fine for Apache.



On Mon, Mar 9, 2020 at 7:45 PM Sheng Wu  wrote:

> Hi
>
> Personally, and basically, I am feeling the team has misunderstood
> the meaning of incubator and the requirements of building the community.
> Same as the last time discussion, I still think they will be in a big
> pressure as they have to deal with the basic feature development, community
> build and following ASF incubator requirements at the same time if they are
> accepted into the incubator. And at the same time, the team lacks the
> experiences of open source community in or out of ASF.
> I am not sure whether this is good for the project. Seem like a little
> hurry to join the incubator.
> More Comments inline.
>
> Willing to listen to what other IPMCs think.
>
>  于2020年3月10日周二 上午10:21写道：
>
> > Hi, All,
> >
> > We are China Telecom Corporation Limited Cloud Computing Branch
> > Corporation.
> > We hope to contribute one of our projects named 'HBlock' to Apache.
> > Here is the proposal of HBlock project, please feel free to let me know
> > what
> > the concerns and suggestions from you. Thank you so much.
> >
> > HBlock Proposal
> >
> > 1.Abstract
> > The HBlock project will be an enterprise distributed block storage.
> >
> > 2.Proposal
> > HBlock provides a distributed block storage with the following features:
> > 2.1.User-space iSCSI target: HBlock will implement an iSCSI target that
> is
> > RFC-7143 (https://tools.ietf.org/html/rfc7143) compliant written in pure
> > Java designed to run on top of any mainstream Operating System, including
> > Windows and Linux, as a user-space process.
> > 2.2.Enterprise level features: HBlock will implement comprehensive
> > enterprise level features, such as
> > Asymmetric Logical Unit Access (ALUA, Information technology -SCSI
> Primary
> > Commands - 4 (SPC-4),
> https://www.t10.org/cgi-bin/ac.pl?t=f&f=spc4r37.pdf),
> >
> > Persistent Reservations (PR, Information technology -SCSI Primary
> Commands
> > -
> > 4 (SPC-4), https://www.t10.org/cgi-bin/ac.pl?t=f&f=spc4r37.pdf),
> > VMware vSphere Storage APIs - Array Integration(VAAI,
> >
> >
> https://www.vmware.com/techpapers/2012/vmware-vsphere-storage-apis-array-int
> > egration-10337.html
> > <
> https://www.vmware.com/techpapers/2012/vmware-vsphere-storage-apis-array-integration-10337.html
> >
> > ),
> > Offloaded Data Transfer(ODX,
> >
> >
> https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-se
> > rver-2012-R2-and-2012/hh831628(v=ws.11)
> > <
> https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/hh831628(v=ws.11)
> >),
> > so that it will support
> > session-level fail-over,
> > Oracle Real Application Cluster(Oracle RAC,
> > https://www.oracle.com/database/technologies/rac.html) ,
> > Cluster File System (CFS), VMware cluster and Windows cluster.
> > 2.3.Low latency: HBlock will implement in-memory distributed cache to
> > reduce
> > write latency and improve Input / Output Operations Per Second (IOPS),
> and
> > it will leverage storage-class memory to archive even higher durability
> > without IOPS loss.
> > 2.4.Smart Compaction and Garbage Collection(GC): HBlock will convert all
> > the
> > write operations into sequential append operations to improve the random
> > write performance, and it will choose the best timing to compact and
> > collect
> > the garbage per Logic Unit (LU). Comparting to Solid State Drives (SSD's)
> > internal Garbage Collection, such a global GC will reduce the need of
> SSD's
> > internal GC, which indirectly make SSD have more usable space, and have
> > even
> > better GC strategy due to c

Re: [Proposal]New storage project: HBlock

[Kevin A. McGrail]

I have committed to champion and I think the points you make are good,
Ted.   Do you have the bandwidth to be a mentor?

I will work with them to set expectations about the process.  I have also
asked for them to do some community building now, too.
--
Kevin A. McGrail
Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171


On Wed, Mar 25, 2020 at 12:00 PM Ted Dunning  wrote:

> Three things are very clear to me:
>
> 1) having an open source iSCSI implementation from a mature and experienced
> storage stream is a very cool thing, especially if it can be targeted to
> non HDFS storage relatively easily. Building such a thing requires very
> high levels of experience and expertise that have generally been lacking in
> the open source world.
>
> 2) this team is very naive about the negative impacts that Apache processes
> will have on their development speed and will need lots of mentoring. Given
> their release schedule, I think that there are symmetrical risks, first
> that the team will be tempted to JFDI when getting features out the door
> rather than communicate and share designs and second that if they build a
> proper community overcoming language, timezone and large internal team
> dynamics that the internal political costs will severe due to slower
> development.
>
> 3) this team is very enthusiastic about making open source work and that
> might be enough to allow them to succeed in spite of the difficulties.
>
> The path to success here is, in my opinion, to require strong and engaged
> mentorship and make it very clear before they come in that Apache may not
> be a good fit due to the pressures they face to delivery on a schedule. If
> incubation with a high risk of exit back to a non-Apache form is acceptable
> to the project team, then it should be fine for Apache.
>
>
>
> On Mon, Mar 9, 2020 at 7:45 PM Sheng Wu  wrote:
>
> > Hi
> >
> > Personally, and basically, I am feeling the team has misunderstood
> > the meaning of incubator and the requirements of building the community.
> > Same as the last time discussion, I still think they will be in a big
> > pressure as they have to deal with the basic feature development,
> community
> > build and following ASF incubator requirements at the same time if they
> are
> > accepted into the incubator. And at the same time, the team lacks the
> > experiences of open source community in or out of ASF.
> > I am not sure whether this is good for the project. Seem like a little
> > hurry to join the incubator.
> > More Comments inline.
> >
> > Willing to listen to what other IPMCs think.
> >
> >  于2020年3月10日周二 上午10:21写道：
> >
> > > Hi, All,
> > >
> > > We are China Telecom Corporation Limited Cloud Computing Branch
> > > Corporation.
> > > We hope to contribute one of our projects named 'HBlock' to Apache.
> > > Here is the proposal of HBlock project, please feel free to let me know
> > > what
> > > the concerns and suggestions from you. Thank you so much.
> > >
> > > HBlock Proposal
> > >
> > > 1.Abstract
> > > The HBlock project will be an enterprise distributed block storage.
> > >
> > > 2.Proposal
> > > HBlock provides a distributed block storage with the following
> features:
> > > 2.1.User-space iSCSI target: HBlock will implement an iSCSI target that
> > is
> > > RFC-7143 (https://tools.ietf.org/html/rfc7143) compliant written in
> pure
> > > Java designed to run on top of any mainstream Operating System,
> including
> > > Windows and Linux, as a user-space process.
> > > 2.2.Enterprise level features: HBlock will implement comprehensive
> > > enterprise level features, such as
> > > Asymmetric Logical Unit Access (ALUA, Information technology -SCSI
> > Primary
> > > Commands - 4 (SPC-4),
> > https://www.t10.org/cgi-bin/ac.pl?t=f&f=spc4r37.pdf),
> > >
> > > Persistent Reservations (PR, Information technology -SCSI Primary
> > Commands
> > > -
> > > 4 (SPC-4), https://www.t10.org/cgi-bin/ac.pl?t=f&f=spc4r37.pdf),
> > > VMware vSphere Storage APIs - Array Integration(VAAI,
> > >
> > >
> >
> https://www.vmware.com/techpapers/2012/vmware-vsphere-storage-apis-array-int
> > > egration-10337.html
> > > <
> >
> https://www.vmware.com/techpapers/2012/vmware-vsphere-storage-apis-array-integration-10337.html
> > >
> > > ),
> > > Offloaded Data Transfer(ODX,
> > >
> > >
> >
> https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-se
> > > rver-2012-R2-and-2012/hh831628(v=ws.11)
> > > <
> >
> https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/hh831628(v=ws.11)
> > >),
> > > so that it will support
> > > session-level fail-over,
> > > Oracle Real Application Cluster(Oracle RAC,
> > > https://www.oracle.com/database/technologies/rac.html) ,
> > > Cluster File System (CFS), VMware cluster and Windows cluster.
> > > 2.3.Low latency: HBlock will implement in-memory distributed cache to
> > > reduce
> > > write latency and improve

Re: [Proposal]New storage project: HBlock

[Ted Dunning]

On Wed, Mar 25, 2020 at 1:56 PM Kevin A. McGrail 
wrote:

> I have committed to champion and I think the points you make are good,
> Ted.   Do you have the bandwidth to be a mentor?
>

 I don't have the time.

I am interested in the project, but just can't afford the time and effort
for mentoring a project like this that will need a lot of help and
education. This would be even harder because of timezones. Most of my
other-timezone meetings are with Europe (therefore early in my day). Adding
Asia meetings and calls (necessary for education, I think) would mean
burning both ends of the candle.

Re: [Proposal]New storage project: HBlock

[Kevin A. McGrail]

On 3/25/2020 5:39 PM, Ted Dunning wrote:
> On Wed, Mar 25, 2020 at 1:56 PM Kevin A. McGrail 
> wrote:
>
>> I have committed to champion and I think the points you make are good,
>> Ted.   Do you have the bandwidth to be a mentor?
>>
>  I don't have the time.
>
> I am interested in the project, but just can't afford the time and effort
> for mentoring a project like this that will need a lot of help and
> education. This would be even harder because of timezones. Most of my
> other-timezone meetings are with Europe (therefore early in my day). Adding
> Asia meetings and calls (necessary for education, I think) would mean
> burning both ends of the candle.

I understand but can't think of a better mentor for the project.  They
are looking for more mentors, btw, and I think HBlock presents a very
cool solution.  Anyone else willing to mentor them?

Regards,

KAM

-- 
Kevin A. McGrail
kmcgr...@apache.org

Member, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: [Proposal]New storage project: HBlock

[Justin Mclean]

Hi,

Just checking in on progress here and where this proposal is at.

Thanks,
Justin

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Re: [Proposal]New storage project: HBlock

[zhangguochen]

Hi Justin,

I am sorry for the delay feedback and thank you for your interest.
We have spent the time preparing more and working on HBlock. We would like
to enter the incubator but still need 2-more mentors to step forward.
Any volunteers to mentor HBlock and then we'd like a vote on our proposal to
start the journey to become Apache HBlock.

Best wishes.
Guochen.

-邮件原件-
发件人:
general-return-73210-zhangguochen=chinatelecom...@incubator.apache.org
 代
表 Justin Mclean
发送时间: 2020年5月24日 17:00
收件人: general@incubator.apache.org
主题: Re: [Proposal]New storage project: HBlock

Hi,

Just checking in on progress here and where this proposal is at.

Thanks,
Justin

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org



-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org