Re: [DISCUSS] Proposal for Ignite Extensions as a separate Bahir module or Incubator project

[Saikat Maitra]

Hi,

I looked into the way Apache Bahir manages their extensions for Spark and
Flink and it looks like they are much independent in terms of managing
their releases. They also have separate git repos for apache bahir and
apache bahir-flink.

Releases :
https://bahir.apache.org/downloads/spark/
https://bahir.apache.org/downloads/flink/

Repos :
https://github.com/apache/bahir
https://github.com/apache/bahir-flink


I am thinking if we are following the similar pattern we can create a
separate git repo under the Org apache / ignite-extentions or apache /
bahir-ignite.

If most of our integrations are data streaming connectors that we are most
interested to migrate to separate repository then joining Apache Bahir
project and managing independent release cycle will benefit us as it will
help foster cross community engagement and support. The purpose of Bahir is
also to host such extensions as ours.

I was reading this news article and it resonated similar ideas that we have
specific to managing release cycles
https://thenewstack.io/apache-bahir-gives-spark-extensions-new-home/

Please review and share your feedback.

Warm Regards,
Saikat













On Wed, Oct 23, 2019 at 4:29 PM Denis Magda  wrote:

> Folks,
>
> How about considering the option Dmitriy named as "0. placing integration
> in a separate module within space of Apache Ignite"?
>
> Nothing prevents us from following concepts of Bahir project in the sense
> that we'll be creating and managing separate repositories for Ignite
> extensions/modules but those will be governed by the Ignite community and
> all the contributors to the extensions will be becoming Ignite committers
> and PMC members. The more I think about this approach the more I like it.
> Any thoughts?
>
> -
> Denis
>
>
> On Wed, Oct 23, 2019 at 12:42 PM Dmitriy Pavlov 
> wrote:
>
> > Hi, Saikat, Alexey,
> >
> > Actually we have 3 ways to solve it.
> > 0. placing integration in a separate module within space of Apache Ignite
> > 1. Apache Bahir
> > 2. Apache Incubator
> >
> > I'm not sure if option 2 is the best one since it is more about building
> a
> > new community around Ignite Extensions, it may be tricky.
> >
> > But 0 and 1 seem to be perfectly OK.
> >
> > And I like option 1 most since it is very natural to move Ignite-Kafka,
> > Ignite-Camel to a separate project specially intended for integration.
> >
> > So if we stay with option 1 I would be glad to help. Count on my support
> > within the migration to Apache Bahir. Inter-project interaction and
> > integration are usually welcomed in the ASF.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > ср, 23 окт. 2019 г. в 09:31, Alexey Zinoviev :
> >
> > > Also, dear Saikat Maitra, could you please describe how you see the
> > > release cycles in Bahir Ignite Extensions and how it be related to
> Ignite
> > > release, 2.9, 3.0 for example.
> > >
> > > Thank you for your energy
> > >
> > > ср, 23 окт. 2019 г., 8:10 Alexey Zinoviev :
> > >
> > >> Please, give me permissions too, I'd glad to help with this modules
> > >> migration and support part of them in future, but also we need not
> only
> > >> contributor but a few Committer permissions to merge In repository in
> > other
> > >> side it could be very long proccess.
> > >>
> > >> Could you ask Bahir Community about that?
> > >>
> > >> ср, 23 окт. 2019 г., 2:31 Saikat Maitra :
> > >>
> > >>> Hi,
> > >>>
> > >>> I discussed with Apache Bahir community and they are interested to
> have
> > >>> Apache Ignite extensions as part of Apache Bahir project.
> > >>>
> > >>> I have also requested for contributor access in Jira for Apache Bahir
> > >>> project so that I can create issues and assign to myself. I can help
> > with
> > >>> code reviews as well.
> > >>>
> > >>> Also my thoughts on releases specific to dependencies for Apache
> Ignite
> > >>> is
> > >>> to do a fast follow up release for modules based on latest Apache
> > Ignite
> > >>> stable release.
> > >>>
> > >>> Here is the email thread for reference
> > >>> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > >>>
> > >>> I wanted to connect and get feedback on the proposal and if we are ok
> > to
> > >>> move the following Apache Ignite Extensions
> > >>>
> > >>>
> > >>>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >>>
> > >>> Regards,
> > >>> Saikat
> > >>>
> > >>>
> > >>> On Fri, Oct 18, 2019 at 9:44 PM Saikat Maitra <
> saikat.mai...@gmail.com
> > >
> > >>> wrote:
> > >>>
> > >>> > Hello,
> > >>> >
> > >>> > We wanted to discuss on a proposal to move and support the Apache
> > >>> Ignite
> > >>> > integrations as separate Ignite Extensions as discussed here
> > >>> >
> > >>>
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Pub-Sub-Streamer-Implementation-td43944.html
> > >>> > .
> > >>> >
> > >>> > The reason we wanted to move our Apache Ignite integration as
> > separate
> > >>> > Extensions is this 

Re: [DISCUSS] PMC Chair rotation time

[Nikita Ivanov]

+1 Alexey Goncharuk
--
Nikita Ivanov

> On Oct 21, 2019, at 10:21 PM, Denis Magda  wrote:
> 
> Igniters,
> 
> It’s been almost 3 years since my election as the PMC Chair and I’d like
> the community to give other PMC members an opportunity to serve in this
> role. It’s healthy to rotate the role more frequently and we’re already
> due. Though the chair doesn’t have formal power, he/she can bring a fresh
> perspective and help to navigate the community via trails not considered
> before.
> 
> Please propose candidates selecting among active PMC members:
> https://ignite.apache.org/community/resources.html#people
> 
> 
> Denis
> 
> 
>> On Monday, December 5, 2016, Dmitriy Setrakyan 
>> wrote:
>> 
>> I haven't forgotten. Just got back from a business trip. Will start a vote
>> this week.
>> 
>> On Wed, Nov 23, 2016 at 5:18 PM, Dmitriy Setrakyan 
>> wrote:
>> 
>>> Cos, I will start the vote soon. A bit over occupied with travel and
>>> holidays at this moment.
>>> 
>>> On Mon, Nov 21, 2016 at 12:33 PM, Konstantin Boudnik 
>>> wrote:
>>> 
 Ok,
 
 This was a great discussion, thanks for the deep insight, Roman!
 
 looks like this thread has stewed for a while and it is time to [VOTE].
 The
 following candidates were put forward during the [DISCUSS] (in the order
 of
 their appearance on the thread):
 
  Dmitriy Setrakyan
  Vladimir Ozerov
  Konstantin Boudnik
  Valentin Kulichenko
  Denis Magda
  Branko Čibej
 
 Considering the number of the candidates I propose to use
 First-past-the-post
 voting, with a single-round/single winner rules, where the candidate
 collecting the most votes (not the majority) wins. [1]
 
 Dmitriy, would you do the honors and start the formal [VOTE]?
 
 
 [1] https://en.wikipedia.org/wiki/First-past-the-post_voting
 
 Thanks,
  Cos
 
 On Thu, Nov 10, 2016 at 08:19PM, Roman Shaposhnik wrote:
> On Wed, Nov 2, 2016 at 11:32 AM, Dmitriy Setrakyan
>  wrote:
>> While all the candidates suggested so far look good, I would propose
 Denis
>> Magda as the next PMC chair. His participation in the project does
 not only
>> have to do with the code, but also with Ignite project overall,
 including
>> website, documentation, new tickets, design, etc.
> 
> Sorry to come to this thread rather late, but I wanted to offer a
 somewhat
> different perspective that hopefully can help reframe this discussion
> a little bit.
> Note, that what I'm about to say is coming from a non-coding (on
 Ignite) member
> of the PMC. In fact, I stayed on the PMC after graduation more with my
 ASF
> member hat on, rather than as somebody who wanted to directly
>> contribute
> to where the technology was going. My concern during your incubation
 days
> was (and still is!) how well are you guys doing on the community
 development
> front.
> 
> If you recall, the diversity requirement (the fact that more than 80%
 of project
> members work for the same company) came up during our graduation.
>> While
> it wasn't a blocking requirement it still was a very valid concern.
 Monocultures
> are really nasty for open source communities.
> 
> I think it would be fair to say that Ignite community hasn't really
> improved much
> diversity-wise since its graduation. And I honestly feel that this is,
> to a larger
> extent, because the focus is now missing. It was well understood that
 in order
> to graduate the community had to do all it can to demonstrate a
> positive trajectory
> in that direction. And we did. But we kind of dropped the ball on it
 since.
> 
> With that in mind, I offer my view on what a change in the Chair could
> offer: a renewed
> focus on community growth and development. If you agree with this
 premise than
> somebody like Branko or Cos who could act as a forcing function to
 constantly
> remind us about what else can we do to grow and develop this community
 would
> be an ideal choice for the Chair (after all Branko's "tough love" was
> probably the
> most useful part of Ignite's experience in the Incubator).
> 
> Now, of course, strictly speaking you don't have to be a Chair to do
> that. But lets
> face it -- that helps. And besides, a project's Chair doesn't really
> have any special
> powers when it comes to technological consensus, but anything that has
 to do
> with external community the VP title that the chair gets can help
>> quite
 a bit.
> 
> Long story short: if Branko or Cos (as former mentors) would like to
> commit to 6 more
> months of the same hands-on mentorship focused on community
>> development
 -- they
> would have my wholehearted support.
> 
> Thanks,
> Roman (with my PMC member 

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra]

Hi Denis,

Thank you for your response. Yes, I have created the following discussion
thread for Ignite extensions migration.

http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td44064.html

Regards,
Saikat

On Tue, Oct 22, 2019 at 8:17 PM Denis Magda  wrote:

> Saikat, thanks for checking with Bahir!
>
> However, let’s run this idea through the rest of the community before
> getting down to the implementation. As an alternate option, we might adopt
> their approach for Ignite extensions but keep those within Ignite community
> in separate repositories. Would you mind starting a separate discussion
> with Ignite dev?
>
> Denis
>
> On Tuesday, October 22, 2019, Saikat Maitra 
> wrote:
>
> > Hello Emmanouil,
> >
> > As discussed earlier, I discussed with Apache Bahir community and they
> are
> > interested to have Apache Ignite extensions as part of Apache Bahir
> > project.
> >
> > Can you please share Pub/Sub streamer implementation details with Apache
> > Bahir community and request for feedback.
> >
> > https://bahir.apache.org/
> >
> > I have also requested for contributor access in Jira for Apache Bahir
> > project so that I can create issues and assign to myself.
> >
> > I can help with code reviews as well.
> >
> > Here is the email thread for reference
> > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >
> > Regards,
> > Saikat
> >
> >
> >
> >
> >
> >
> >
> > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra 
> > wrote:
> >
> > > Hello,
> > >
> > > As discussed I have created following discussion threads on our
> migration
> > > proposals for Apache Ignite extensions:
> > >
> > >
> > > http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-
> > Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-
> > project-td29829.html
> > >
> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > >
> > > I will share update as I hear more information.
> > >
> > > Regards,
> > > Saikat
> > >
> > >
> > >
> > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra  >
> > > wrote:
> > >
> > >> Hello Denis,
> > >>
> > >> Sure, sounds good. I will create a separate discussion thread to get
> > >> community feedback on whether we
> > >> should join the Bahir or kick-off "Ignite Extensions" project.
> > >>
> > >> Regards,
> > >> Saikat
> > >>
> > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda 
> wrote:
> > >>
> > >>> Folks,
> > >>>
> > >>> The concept of Apache Bahir is precisely what we need! Saikat, thanks
> > for
> > >>> doing the research. Why I believe the * idea* fits us well:
> > >>>
> > >>>- All integrations can be stored in separate Github repositories
> and
> > >>>have their dev lifecycles. We've not obliged to couple all the
> > >>> integrations
> > >>>in a single repo. For instance, Spark can be located in a
> dedicated
> > >>> repo
> > >>>while streaming integrations might be in a single one. It's up to
> > us.
> > >>>- All the repositories and integrations belong to ASF. We're not
> > >>> dumping
> > >>>them on Github but rather keep supporting and developing in
> > >>> accordance with
> > >>>ASF vision and practices.
> > >>>- There is a way to reward contributors via committership and PMC
> > >>>membership. You won't get this if a project is just one of the
> > >>> millions of
> > >>>Github projects.
> > >>>
> > >>> Saikat, as Ignite PMC member, could you please kick-off a separate
> > >>> dev-list
> > >>> discussion to involve more community members and referring to this
> > >>> thread?
> > >>> I think the primary question the community needs to answer whether we
> > >>> should join the Bahir or kick-off "Ignite Extensions" project?
> > >>>
> > >>> -
> > >>> Denis
> > >>>
> > >>>
> > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> > zaleslaw@gmail.com>
> > >>> wrote:
> > >>>
> > >>> > Maybe we could move all our Streaming Integrations there, but what
> is
> > >>> about
> > >>> > maintaining and committer permissions to the new repositories?
> > >>> > I see the list of the committers and PMC members there
> > >>> > https://bahir.apache.org/community-members/
> > >>> >
> > >>> > Could somebody from Ignite community be added to this list as a
> > >>> > PMC/committer to maintain Ignite integrations?
> > >>> > If yes, I'd happy to join this project and collaborate with these
> > guys
> > >>> > together
> > >>> > If no, and we should start from the zero level with external PRs -
> > >>> hmmm,
> > >>> > it's better to have our own external repository with ApacheBahirr
> > >>> ideology.
> > >>> >
> > >>> > Also, I agree, that all connectors could be moved there (in
> > >>> ApacheBahirr
> > >>> > like repository), but not all modules from the page
> > >>> >
> > >>> >
> > >>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > 36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >>> > because
> > >>> > they 

Re: [DISCUSS] PMC Chair rotation time

[Denis Magda]

Folks,

Any other candidates or opinions before I start the vote?

-
Denis


On Tue, Oct 22, 2019 at 2:33 AM Ilya Kasnacheev 
wrote:

> Hello!
>
> +1 to all three :)
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 22 окт. 2019 г. в 09:24, Ivan Pavlukhin :
>
> > IMHO, today Dmitriy Pavlov is the best candidate for this role. He
> > invested a lot into community health and productivity (TC bot and much
> > more).
> >
> > вт, 22 окт. 2019 г. в 09:15, Vyacheslav Daradur :
> > >
> > > Igniters, I'd suggest 2 candidates who perfectly fit this role, in my
> > opinion:
> > >
> > > Nickolay Izhikov - one of the most active community member with the
> > > significant contributions, Ignite's evangelist and tech-speaker.
> > >
> > > Alexey Goncharuk - Ignite's veteran, one of the Ignite's code-base
> > > expert with the project's architecture vision.
> > >
> > > I hope they would like to be PMC Chair!)
> > >
> > >
> > >
> > >
> > > On Tue, Oct 22, 2019 at 8:21 AM Denis Magda  wrote:
> > > >
> > > > Igniters,
> > > >
> > > > It’s been almost 3 years since my election as the PMC Chair and I’d
> > like
> > > > the community to give other PMC members an opportunity to serve in
> this
> > > > role. It’s healthy to rotate the role more frequently and we’re
> already
> > > > due. Though the chair doesn’t have formal power, he/she can bring a
> > fresh
> > > > perspective and help to navigate the community via trails not
> > considered
> > > > before.
> > > >
> > > > Please propose candidates selecting among active PMC members:
> > > > https://ignite.apache.org/community/resources.html#people
> > > >
> > > >
> > > > Denis
> > > >
> > > >
> > > > On Monday, December 5, 2016, Dmitriy Setrakyan <
> dsetrak...@apache.org>
> > > > wrote:
> > > >
> > > > > I haven't forgotten. Just got back from a business trip. Will start
> > a vote
> > > > > this week.
> > > > >
> > > > > On Wed, Nov 23, 2016 at 5:18 PM, Dmitriy Setrakyan <
> > dsetrak...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Cos, I will start the vote soon. A bit over occupied with travel
> > and
> > > > > > holidays at this moment.
> > > > > >
> > > > > > On Mon, Nov 21, 2016 at 12:33 PM, Konstantin Boudnik <
> > c...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > >> Ok,
> > > > > >>
> > > > > >> This was a great discussion, thanks for the deep insight, Roman!
> > > > > >>
> > > > > >> looks like this thread has stewed for a while and it is time to
> > [VOTE].
> > > > > >> The
> > > > > >> following candidates were put forward during the [DISCUSS] (in
> > the order
> > > > > >> of
> > > > > >> their appearance on the thread):
> > > > > >>
> > > > > >>   Dmitriy Setrakyan
> > > > > >>   Vladimir Ozerov
> > > > > >>   Konstantin Boudnik
> > > > > >>   Valentin Kulichenko
> > > > > >>   Denis Magda
> > > > > >>   Branko Čibej
> > > > > >>
> > > > > >> Considering the number of the candidates I propose to use
> > > > > >> First-past-the-post
> > > > > >> voting, with a single-round/single winner rules, where the
> > candidate
> > > > > >> collecting the most votes (not the majority) wins. [1]
> > > > > >>
> > > > > >> Dmitriy, would you do the honors and start the formal [VOTE]?
> > > > > >>
> > > > > >>
> > > > > >> [1] https://en.wikipedia.org/wiki/First-past-the-post_voting
> > > > > >>
> > > > > >> Thanks,
> > > > > >>   Cos
> > > > > >>
> > > > > >> On Thu, Nov 10, 2016 at 08:19PM, Roman Shaposhnik wrote:
> > > > > >> > On Wed, Nov 2, 2016 at 11:32 AM, Dmitriy Setrakyan
> > > > > >> >  wrote:
> > > > > >> > > While all the candidates suggested so far look good, I would
> > propose
> > > > > >> Denis
> > > > > >> > > Magda as the next PMC chair. His participation in the
> project
> > does
> > > > > >> not only
> > > > > >> > > have to do with the code, but also with Ignite project
> > overall,
> > > > > >> including
> > > > > >> > > website, documentation, new tickets, design, etc.
> > > > > >> >
> > > > > >> > Sorry to come to this thread rather late, but I wanted to
> offer
> > a
> > > > > >> somewhat
> > > > > >> > different perspective that hopefully can help reframe this
> > discussion
> > > > > >> > a little bit.
> > > > > >> > Note, that what I'm about to say is coming from a non-coding
> (on
> > > > > >> Ignite) member
> > > > > >> > of the PMC. In fact, I stayed on the PMC after graduation more
> > with my
> > > > > >> ASF
> > > > > >> > member hat on, rather than as somebody who wanted to directly
> > > > > contribute
> > > > > >> > to where the technology was going. My concern during your
> > incubation
> > > > > >> days
> > > > > >> > was (and still is!) how well are you guys doing on the
> community
> > > > > >> development
> > > > > >> > front.
> > > > > >> >
> > > > > >> > If you recall, the diversity requirement (the fact that more
> > than 80%
> > > > > >> of project
> > > > > >> > members work for the same company) came up during our
> > graduation.
> > > > > While
> > > > > >> > it wasn't a blocking requirement it still was a very valid
> > 

Re: [DISCUSS] Proposal for Ignite Extensions as a separate Bahir module or Incubator project

[Denis Magda]

Folks,

How about considering the option Dmitriy named as "0. placing integration
in a separate module within space of Apache Ignite"?

Nothing prevents us from following concepts of Bahir project in the sense
that we'll be creating and managing separate repositories for Ignite
extensions/modules but those will be governed by the Ignite community and
all the contributors to the extensions will be becoming Ignite committers
and PMC members. The more I think about this approach the more I like it.
Any thoughts?

-
Denis


On Wed, Oct 23, 2019 at 12:42 PM Dmitriy Pavlov  wrote:

> Hi, Saikat, Alexey,
>
> Actually we have 3 ways to solve it.
> 0. placing integration in a separate module within space of Apache Ignite
> 1. Apache Bahir
> 2. Apache Incubator
>
> I'm not sure if option 2 is the best one since it is more about building a
> new community around Ignite Extensions, it may be tricky.
>
> But 0 and 1 seem to be perfectly OK.
>
> And I like option 1 most since it is very natural to move Ignite-Kafka,
> Ignite-Camel to a separate project specially intended for integration.
>
> So if we stay with option 1 I would be glad to help. Count on my support
> within the migration to Apache Bahir. Inter-project interaction and
> integration are usually welcomed in the ASF.
>
> Sincerely,
> Dmitriy Pavlov
>
> ср, 23 окт. 2019 г. в 09:31, Alexey Zinoviev :
>
> > Also, dear Saikat Maitra, could you please describe how you see the
> > release cycles in Bahir Ignite Extensions and how it be related to Ignite
> > release, 2.9, 3.0 for example.
> >
> > Thank you for your energy
> >
> > ср, 23 окт. 2019 г., 8:10 Alexey Zinoviev :
> >
> >> Please, give me permissions too, I'd glad to help with this modules
> >> migration and support part of them in future, but also we need not only
> >> contributor but a few Committer permissions to merge In repository in
> other
> >> side it could be very long proccess.
> >>
> >> Could you ask Bahir Community about that?
> >>
> >> ср, 23 окт. 2019 г., 2:31 Saikat Maitra :
> >>
> >>> Hi,
> >>>
> >>> I discussed with Apache Bahir community and they are interested to have
> >>> Apache Ignite extensions as part of Apache Bahir project.
> >>>
> >>> I have also requested for contributor access in Jira for Apache Bahir
> >>> project so that I can create issues and assign to myself. I can help
> with
> >>> code reviews as well.
> >>>
> >>> Also my thoughts on releases specific to dependencies for Apache Ignite
> >>> is
> >>> to do a fast follow up release for modules based on latest Apache
> Ignite
> >>> stable release.
> >>>
> >>> Here is the email thread for reference
> >>> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >>>
> >>> I wanted to connect and get feedback on the proposal and if we are ok
> to
> >>> move the following Apache Ignite Extensions
> >>>
> >>>
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >>>
> >>> Regards,
> >>> Saikat
> >>>
> >>>
> >>> On Fri, Oct 18, 2019 at 9:44 PM Saikat Maitra  >
> >>> wrote:
> >>>
> >>> > Hello,
> >>> >
> >>> > We wanted to discuss on a proposal to move and support the Apache
> >>> Ignite
> >>> > integrations as separate Ignite Extensions as discussed here
> >>> >
> >>>
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Pub-Sub-Streamer-Implementation-td43944.html
> >>> > .
> >>> >
> >>> > The reason we wanted to move our Apache Ignite integration as
> separate
> >>> > Extensions is this will help us to manage and maintain separate
> >>> lifecycle
> >>> > for Apache Ignite integrations.
> >>> >
> >>> > All the integrations will continue to be part of ASF and we will
> keep
> >>> > supporting and developing in accordance with ASF vision and
> practices.
> >>> >
> >>> > We are considering following two choices for moving to Apache Ignite
> >>> > Extensions:
> >>> >
> >>> > 1. Reach out to Apache Bahir community and propose to make Ignite
> >>> > Extensions a separate module as part of Apache Bahir project.
> >>> >
> >>> > https://bahir.apache.org/
> >>> >
> >>> >
> >>> >
> >>>
> https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces96
> >>> >
> >>> >
> >>> > 2. Reach out to Apache Incubator community and request for a new
> >>> project
> >>> > for Ignite Extensions.
> >>> >
> >>> > Please review and share feedback on our proposal.
> >>> >
> >>> > Warm Regards,
> >>> > Saikat
> >>> >
> >>>
> >>
>

Re: [DISCUSS] Proposal for Ignite Extensions as a separate Bahir module or Incubator project

[Emmanouil Gkatziouras]

Hi all,

I come from the Pub/Sub integration [1] thread.
I had a check on Bahir and making an implementation there would be nice.
My concern is if Bahir's description limits the choices on a connector
implementation.
'Apache Bahir provides extensions to multiple distributed analytic
platforms, extending their reach with a diversity of streaming connectors
and SQL data sources.'
For example in the future I would like to propose a feature for cache
invalidation based on the streamer implementation. Supposing this is ok
with the Ignite community is it ok under the scope of Bahir?

[1]
http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Pub-Sub-Streamer-Implementation-td43944.html

Kind regards,
Emmanouil

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Wed, 23 Oct 2019 at 20:42, Dmitriy Pavlov  wrote:

> Hi, Saikat, Alexey,
>
> Actually we have 3 ways to solve it.
> 0. placing integration in a separate module within space of Apache Ignite
> 1. Apache Bahir
> 2. Apache Incubator
>
> I'm not sure if option 2 is the best one since it is more about building a
> new community around Ignite Extensions, it may be tricky.
>
> But 0 and 1 seem to be perfectly OK.
>
> And I like option 1 most since it is very natural to move Ignite-Kafka,
> Ignite-Camel to a separate project specially intended for integration.
>
> So if we stay with option 1 I would be glad to help. Count on my support
> within the migration to Apache Bahir. Inter-project interaction and
> integration are usually welcomed in the ASF.
>
> Sincerely,
> Dmitriy Pavlov
>
> ср, 23 окт. 2019 г. в 09:31, Alexey Zinoviev :
>
> > Also, dear Saikat Maitra, could you please describe how you see the
> > release cycles in Bahir Ignite Extensions and how it be related to Ignite
> > release, 2.9, 3.0 for example.
> >
> > Thank you for your energy
> >
> > ср, 23 окт. 2019 г., 8:10 Alexey Zinoviev :
> >
> >> Please, give me permissions too, I'd glad to help with this modules
> >> migration and support part of them in future, but also we need not only
> >> contributor but a few Committer permissions to merge In repository in
> other
> >> side it could be very long proccess.
> >>
> >> Could you ask Bahir Community about that?
> >>
> >> ср, 23 окт. 2019 г., 2:31 Saikat Maitra :
> >>
> >>> Hi,
> >>>
> >>> I discussed with Apache Bahir community and they are interested to have
> >>> Apache Ignite extensions as part of Apache Bahir project.
> >>>
> >>> I have also requested for contributor access in Jira for Apache Bahir
> >>> project so that I can create issues and assign to myself. I can help
> with
> >>> code reviews as well.
> >>>
> >>> Also my thoughts on releases specific to dependencies for Apache Ignite
> >>> is
> >>> to do a fast follow up release for modules based on latest Apache
> Ignite
> >>> stable release.
> >>>
> >>> Here is the email thread for reference
> >>> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >>>
> >>> I wanted to connect and get feedback on the proposal and if we are ok
> to
> >>> move the following Apache Ignite Extensions
> >>>
> >>>
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >>>
> >>> Regards,
> >>> Saikat
> >>>
> >>>
> >>> On Fri, Oct 18, 2019 at 9:44 PM Saikat Maitra  >
> >>> wrote:
> >>>
> >>> > Hello,
> >>> >
> >>> > We wanted to discuss on a proposal to move and support the Apache
> >>> Ignite
> >>> > integrations as separate Ignite Extensions as discussed here
> >>> >
> >>>
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Pub-Sub-Streamer-Implementation-td43944.html
> >>> > .
> >>> >
> >>> > The reason we wanted to move our Apache Ignite integration as
> separate
> >>> > Extensions is this will help us to manage and maintain separate
> >>> lifecycle
> >>> > for Apache Ignite integrations.
> >>> >
> >>> > All the integrations will continue to be part of ASF and we will
> keep
> >>> > supporting and developing in accordance with ASF vision and
> practices.
> >>> >
> >>> > We are considering following two choices for moving to Apache Ignite
> >>> > Extensions:
> >>> >
> >>> > 1. Reach out to Apache Bahir community and propose to make Ignite
> >>> > Extensions a separate module as part of Apache Bahir project.
> >>> >
> >>> > https://bahir.apache.org/
> >>> >
> >>> >
> >>> >
> >>>
> https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces96
> >>> >
> >>> >
> >>> > 2. Reach out to Apache Incubator community and request for a new
> >>> project
> >>> > for Ignite Extensions.
> >>> >
> >>> > Please review and share feedback on our proposal.
> >>> >
> >>> > Warm Regards,
> >>> > Saikat
> >>> >
> >>>
> >>
>

Re: [DISCUSS] Proposal for Ignite Extensions as a separate Bahir module or Incubator project

[Dmitriy Pavlov]

Hi, Saikat, Alexey,

Actually we have 3 ways to solve it.
0. placing integration in a separate module within space of Apache Ignite
1. Apache Bahir
2. Apache Incubator

I'm not sure if option 2 is the best one since it is more about building a
new community around Ignite Extensions, it may be tricky.

But 0 and 1 seem to be perfectly OK.

And I like option 1 most since it is very natural to move Ignite-Kafka,
Ignite-Camel to a separate project specially intended for integration.

So if we stay with option 1 I would be glad to help. Count on my support
within the migration to Apache Bahir. Inter-project interaction and
integration are usually welcomed in the ASF.

Sincerely,
Dmitriy Pavlov

ср, 23 окт. 2019 г. в 09:31, Alexey Zinoviev :

> Also, dear Saikat Maitra, could you please describe how you see the
> release cycles in Bahir Ignite Extensions and how it be related to Ignite
> release, 2.9, 3.0 for example.
>
> Thank you for your energy
>
> ср, 23 окт. 2019 г., 8:10 Alexey Zinoviev :
>
>> Please, give me permissions too, I'd glad to help with this modules
>> migration and support part of them in future, but also we need not only
>> contributor but a few Committer permissions to merge In repository in other
>> side it could be very long proccess.
>>
>> Could you ask Bahir Community about that?
>>
>> ср, 23 окт. 2019 г., 2:31 Saikat Maitra :
>>
>>> Hi,
>>>
>>> I discussed with Apache Bahir community and they are interested to have
>>> Apache Ignite extensions as part of Apache Bahir project.
>>>
>>> I have also requested for contributor access in Jira for Apache Bahir
>>> project so that I can create issues and assign to myself. I can help with
>>> code reviews as well.
>>>
>>> Also my thoughts on releases specific to dependencies for Apache Ignite
>>> is
>>> to do a fast follow up release for modules based on latest Apache Ignite
>>> stable release.
>>>
>>> Here is the email thread for reference
>>> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>>>
>>> I wanted to connect and get feedback on the proposal and if we are ok to
>>> move the following Apache Ignite Extensions
>>>
>>>
>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>>>
>>> Regards,
>>> Saikat
>>>
>>>
>>> On Fri, Oct 18, 2019 at 9:44 PM Saikat Maitra 
>>> wrote:
>>>
>>> > Hello,
>>> >
>>> > We wanted to discuss on a proposal to move and support the Apache
>>> Ignite
>>> > integrations as separate Ignite Extensions as discussed here
>>> >
>>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Pub-Sub-Streamer-Implementation-td43944.html
>>> > .
>>> >
>>> > The reason we wanted to move our Apache Ignite integration as  separate
>>> > Extensions is this will help us to manage and maintain separate
>>> lifecycle
>>> > for Apache Ignite integrations.
>>> >
>>> > All the integrations will continue to be part of ASF and we will  keep
>>> > supporting and developing in accordance with ASF vision and practices.
>>> >
>>> > We are considering following two choices for moving to Apache Ignite
>>> > Extensions:
>>> >
>>> > 1. Reach out to Apache Bahir community and propose to make Ignite
>>> > Extensions a separate module as part of Apache Bahir project.
>>> >
>>> > https://bahir.apache.org/
>>> >
>>> >
>>> >
>>> https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces96
>>> >
>>> >
>>> > 2. Reach out to Apache Incubator community and request for a new
>>> project
>>> > for Ignite Extensions.
>>> >
>>> > Please review and share feedback on our proposal.
>>> >
>>> > Warm Regards,
>>> > Saikat
>>> >
>>>
>>

Re: Ignite-Spark integration meeting

[Nikolay Izhikov]

The only issue with the Spark integration tests - they should be runned under 
JDK8.
That's all.

I wrote about it recently [1]

[1] 
http://apache-ignite-developers.2346864.n4.nabble.com/Spark-examples-on-TC-td43591.html

В Ср, 23/10/2019 в 17:46 +0200, Alexey Zinoviev пишет:
> Good point, now they are not in the Best state, we have a few ideas how to
> fix, will discuss it later here in this thread.
> 
> If you have any objection write it, but my position - creation special
> Spark Visa on TC, like for ML, maybe two for different Spark versions, and
> all commits to Spark will require this Visa.
> 
> 
> ср, 23 окт. 2019 г., 16:25 Ivan Pavlukhin :
> 
> > Actually, I meant running tests on TeamCity.
> > 
> > ср, 23 окт. 2019 г. в 15:36, Alexey Zinoviev :
> > > 
> > > What kind of testing do you mean?  JUnit, integration via example or
> > > something else?
> > > 
> > > ср, 23 окт. 2019 г., 14:16 Ivan Pavlukhin :
> > > 
> > > > Correction.
> > > > 
> > > > How are you going to TEST integrations with different Spark versions?
> > > > 
> > > > ср, 23 окт. 2019 г. в 15:16, Ivan Pavlukhin :
> > > > > 
> > > > > Alexey,
> > > > > 
> > > > > Thank you for sharing the summary!
> > > > > 
> > > > > Small question. How are you going to integrations with different
> > 
> > Spark
> > > > versions?
> > > > > 
> > > > > чт, 17 окт. 2019 г. в 22:30, Denis Magda :
> > > > > > 
> > > > > > > 
> > > > > > >  1. We are going to release Apache Ignite 2.8 with limited
> > 
> > support of
> > > > > > >Spark 2.4.4 (known issues are listed here
> > > > > > >https://issues.apache.org/jira/browse/IGNITE-12054)
> > > > > > 
> > > > > > 
> > > > > > What does limited support mean? Is it about regressions (something
> > 
> > that
> > > > > > worked before but will fail) or capabilities unsupported even for
> > 
> > the
> > > > > > presently released version of the integration?
> > > > > > 
> > > > > > -
> > > > > > Denis
> > > > > > 
> > > > > > 
> > > > > > On Thu, Oct 17, 2019 at 7:55 AM Alexey Zinoviev <
> > > > 
> > > > zaleslaw@gmail.com>
> > > > > > wrote:
> > > > > > 
> > > > > > > The meeting notes and results of discussion are next:
> > > > > > > 
> > > > > > >1. We are going to release Apache Ignite 2.8 with limited
> > 
> > support
> > > > of
> > > > > > >Spark 2.4.4 (known issues are listed here
> > > > > > >https://issues.apache.org/jira/browse/IGNITE-12054)
> > > > > > >2. The AI 2.8 will support both, Spark 2.3 and Spark 2.4.4
> > 
> > with
> > > > known
> > > > > > >fixed bugs
> > > > > > >3. We will create the developer-profiles spark_2_3 and
> > 
> > spark_2_4
> > > > > > >separately from current scala profile
> > > > > > >4. We are ready to move the Ignite-Spark integration to the
> > > > 
> > > > separate
> > > > > > >module after release 2.8 with support of most actual Spark
> > > > 
> > > > versions and
> > > > > > >with independent release cycle
> > > > > > >5. We will work together on the next release roadmap with
> > 
> > Nikolay
> > > > and
> > > > > > >all interested igniters to improve the current solution (add
> > 
> > new
> > > > > > > features
> > > > > > >as thin client support, working with caches via binary object
> > 
> > and
> > > > so on)
> > > > > > >6. Nikolay is ready to help with the review of tickets as an
> > > > 
> > > > author of
> > > > > > >this component
> > > > > > > 
> > > > > > > Thanks
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > пн, 14 окт. 2019 г. в 14:36, Alexey Zinoviev <
> > 
> > zaleslaw@gmail.com
> > > > > :
> > > > > > > 
> > > > > > > > Yes, let's use the Russian for short discussion, the meeting
> > 
> > notes
> > > > will
> > > > > > > be
> > > > > > > > presented in English
> > > > > > > > 
> > > > > > > > пн, 14 окт. 2019 г. в 14:08, Nikolay Izhikov <
> > 
> > nizhi...@apache.org
> > > > > :
> > > > > > > > 
> > > > > > > > > Hello, Alexey.
> > > > > > > > > 
> > > > > > > > > I'm ready to take part in the meeting.
> > > > > > > > > It will be in Russian, isn't it?
> > > > > > > > > 
> > > > > > > > > В Пн, 14/10/2019 в 14:00 +0300, Alexey Zinoviev пишет:
> > > > > > > > > > Hi, Igniters, and especially Nikolay Izhikov
> > > > > > > > > > 
> > > > > > > > > > I suggest to arrange the meeting via ASF-slack in
> > > > > > > > > 
> > > > > > > > > #ignite-spark-integration
> > > > > > > > > > channel at 17 October, in 16 MSK Time.
> > > > > > > > > > The language of the meeting: Russian
> > > > > > > > > > 
> > > > > > > > > > *The topics to discuss:*
> > > > > > > > > > 
> > > > > > > > > >- Review of next tickets
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > >1. [Spark][Bug] IgniteSpark integration forget to close
> > 
> > the
> > > > > > > > > >   IgniteContext and stops the client node in case if
> > 
> > error
> > > > during
> > > > > > > > > >   PairFunction logic
> > > > > > > > > >   
> > > > 

Re: Ignite-Spark integration meeting

[Alexey Zinoviev]

Good point, now they are not in the Best state, we have a few ideas how to
fix, will discuss it later here in this thread.

If you have any objection write it, but my position - creation special
Spark Visa on TC, like for ML, maybe two for different Spark versions, and
all commits to Spark will require this Visa.


ср, 23 окт. 2019 г., 16:25 Ivan Pavlukhin :

> Actually, I meant running tests on TeamCity.
>
> ср, 23 окт. 2019 г. в 15:36, Alexey Zinoviev :
> >
> > What kind of testing do you mean?  JUnit, integration via example or
> > something else?
> >
> > ср, 23 окт. 2019 г., 14:16 Ivan Pavlukhin :
> >
> > > Correction.
> > >
> > > How are you going to TEST integrations with different Spark versions?
> > >
> > > ср, 23 окт. 2019 г. в 15:16, Ivan Pavlukhin :
> > > >
> > > > Alexey,
> > > >
> > > > Thank you for sharing the summary!
> > > >
> > > > Small question. How are you going to integrations with different
> Spark
> > > versions?
> > > >
> > > > чт, 17 окт. 2019 г. в 22:30, Denis Magda :
> > > > >
> > > > > >
> > > > > >  1. We are going to release Apache Ignite 2.8 with limited
> support of
> > > > > >Spark 2.4.4 (known issues are listed here
> > > > > >https://issues.apache.org/jira/browse/IGNITE-12054)
> > > > >
> > > > >
> > > > > What does limited support mean? Is it about regressions (something
> that
> > > > > worked before but will fail) or capabilities unsupported even for
> the
> > > > > presently released version of the integration?
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Thu, Oct 17, 2019 at 7:55 AM Alexey Zinoviev <
> > > zaleslaw@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > The meeting notes and results of discussion are next:
> > > > > >
> > > > > >1. We are going to release Apache Ignite 2.8 with limited
> support
> > > of
> > > > > >Spark 2.4.4 (known issues are listed here
> > > > > >https://issues.apache.org/jira/browse/IGNITE-12054)
> > > > > >2. The AI 2.8 will support both, Spark 2.3 and Spark 2.4.4
> with
> > > known
> > > > > >fixed bugs
> > > > > >3. We will create the developer-profiles spark_2_3 and
> spark_2_4
> > > > > >separately from current scala profile
> > > > > >4. We are ready to move the Ignite-Spark integration to the
> > > separate
> > > > > >module after release 2.8 with support of most actual Spark
> > > versions and
> > > > > >with independent release cycle
> > > > > >5. We will work together on the next release roadmap with
> Nikolay
> > > and
> > > > > >all interested igniters to improve the current solution (add
> new
> > > > > > features
> > > > > >as thin client support, working with caches via binary object
> and
> > > so on)
> > > > > >6. Nikolay is ready to help with the review of tickets as an
> > > author of
> > > > > >this component
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > > >
> > > > > >
> > > > > > пн, 14 окт. 2019 г. в 14:36, Alexey Zinoviev <
> zaleslaw@gmail.com
> > > >:
> > > > > >
> > > > > > > Yes, let's use the Russian for short discussion, the meeting
> notes
> > > will
> > > > > > be
> > > > > > > presented in English
> > > > > > >
> > > > > > > пн, 14 окт. 2019 г. в 14:08, Nikolay Izhikov <
> nizhi...@apache.org
> > > >:
> > > > > > >
> > > > > > >> Hello, Alexey.
> > > > > > >>
> > > > > > >> I'm ready to take part in the meeting.
> > > > > > >> It will be in Russian, isn't it?
> > > > > > >>
> > > > > > >> В Пн, 14/10/2019 в 14:00 +0300, Alexey Zinoviev пишет:
> > > > > > >> > Hi, Igniters, and especially Nikolay Izhikov
> > > > > > >> >
> > > > > > >> > I suggest to arrange the meeting via ASF-slack in
> > > > > > >> #ignite-spark-integration
> > > > > > >> > channel at 17 October, in 16 MSK Time.
> > > > > > >> > The language of the meeting: Russian
> > > > > > >> >
> > > > > > >> > *The topics to discuss:*
> > > > > > >> >
> > > > > > >> >- Review of next tickets
> > > > > > >> >
> > > > > > >> >
> > > > > > >> >1. [Spark][Bug] IgniteSpark integration forget to close
> the
> > > > > > >> >   IgniteContext and stops the client node in case if
> error
> > > during
> > > > > > >> >   PairFunction logic
> > > > > > >> >   
> > > > > > >> >   2. [Spark] Add initial support of Spark 2.4
> > > > > > >> >   
> > > > > > >> >   3. [Spark] Add joins to Spark Dataframe examples
> > > > > > >> >   
> > > > > > >> >
> > > > > > >> >
> > > > > > >> >- Known issues of migration to 2.4 version
> > > > > > >> >- Organization of multiple version support for Apache
> Spark
> > > > > > >> >
> > > > > > >> > P.S. meeting notes will be shared with the community
> > > > > > >> > P.P.S Nikolay, if you have any objections about the time or
> > > topics,
> > > > > > >> please
> > > > > > >> > let me know
> > > > > > >>
> > > > > > >
> > > > 

[jira] [Created] (IGNITE-12326) JDBC thin returns java.util.Date instead of java.sql.Date if input values, putted to cache via cache API are LocalDate or java.sql.Date.

[Alexander Lapin (Jira)]

Alexander Lapin created IGNITE-12326:


 Summary: JDBC thin returns java.util.Date instead of java.sql.Date 
if input values, putted to cache via cache API are  LocalDate or java.sql.Date.
 Key: IGNITE-12326
 URL: https://issues.apache.org/jira/browse/IGNITE-12326
 Project: Ignite
  Issue Type: Bug
Reporter: Alexander Lapin
 Fix For: 2.8


JDBC thin returns java.util.Date instead of java.sql.Date if input values, 
putted to cache via cache API are LocalDate or java.sql.Date.

Reproducers:
org.apache.ignite.jdbc.thin.JdbcThinCacheToJdbcDataTypesCoverageTest#testSqlDateDataType
org.apache.ignite.jdbc.thin.JdbcThinCacheToJdbcDataTypesCoverageTest#testLocalDateDataType



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Thin clients: WithExpiryPolicy

[Pavel Tupitsyn]

Ok, no objections

On Wed, Oct 23, 2019 at 4:36 PM Alexandr Shapkin  wrote:

> Pavel,
>
> I think that in some cases milliseconds could make sense,
> so I would like to keep the current precision.
>
> With this approach we will avoid unnecessary code changes as well as
> keeping API in consistence with the thick client method and
> cacheConfiguration settings.
>
>
> From: Pavel Tupitsyn
> Sent: Saturday, October 19, 2019 9:20 PM
> To: dev
> Subject: Re: Thin clients: WithExpiryPolicy
>
> Alexandr,
>
> Sounds good to me.
>
> Remaining question is - how do we serialize the expiry policy?
> Thick client passes this to JNI as 3 int64 values (duration in
> milliseconds).
> But I don't think we need millisecond precision for expiration, so we could
> pass seconds as float values (3*4 bytes).
>
> Thoughts?
>
> On Fri, Oct 18, 2019 at 8:21 PM Alexandr Shapkin 
> wrote:
>
> > Pavel,
> >
> > Thanks for sharing your thoughts.
> >
> > Right now I see that 2 reserved bytes come with every cache request:
> > cacheId and flags.
> > The first one is obvious, but the second one is used only for java thin
> > client with enabled KeepBinary mode.
> >
> > I think we could use the flags byte and write an expiration policy flag
> > when required.
> > Whenever the server sees that there is a request with expiration flag, we
> > deserialize a policy and apply it to the request.
> >
> > From: Pavel Tupitsyn
> > Sent: Friday, October 18, 2019 7:05 PM
> > To: dev
> > Subject: Re: Thin clients: WithExpiryPolicy
> >
> > Stateless approach looks a lot better to me.
> >
> > We have a choice:
> > * Keep expiry policy on server and send an ID with every request (like a
> > query cursor ID - 8 bytes)
> > * Send full expiry policy with every expiry-enabled request (24 bytes -
> or
> > maybe less? We should think about the format)
> >
> > Stateful approach will bring a lot of complexity if we consider Affinity
> > Awareness [1] mechanism (and also automatic reconnect).
> > We would have to keep ExpiryPolicyId for every server and choose the
> right
> > one based on the affinity for every operation.
> > This can easily negate any performance gain from saving 16 bytes.
> >
> > And there is always CacheConfiguration.ExpiryPolicyFactory, which allows
> us
> > to set up default expiry policy.
> >
> > > things could get worse if we decided to add a few more WithSomething*
> > methods
> > I don't think this is a good argument - we should decide on case by case
> > basis.
> > Anyway, other With* methods don't have any parameters, so they carry
> only 1
> > bit of information.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients
> >
> >
> > On Fri, Oct 18, 2019 at 5:14 PM Alexandr Shapkin 
> > wrote:
> >
> > > Igniters,
> > >
> > > I would like to add WithExpirePolicy support to thin clients. [1]
> > > For a thick client, we can obtain a reference to a cache wrapper
> instance
> > > and use cache API through it. At the same time, the thin client
> protocol
> > is
> > > stateless, we do not hold a reference to a cache but rather a cache
> name
> > > identifier is used for a server to create an appropriate cache
> instance.
> > >
> > > We could extend the protocol as we did with WithKeepBinaryMethod:
> > > every time we need to call some API on a cache with expiration, a
> > > serialized ExpiryPolicy (additional 3*8 bytes) would be sent. This
> > approach
> > > works well, but things could get worse if we decided to add a few more
> > > WithSomething* methods.
> > >
> > > Initially, I was thinking about introducing some state context to a
> > > protocol, similar to a QueryCursor API. For instance, we can save an
> > expire
> > > policy configuration for the first call and use some hash value based
> on
> > an
> > > ExpiryPolicy for further calls, just as we do for cache names. I.e.
> > > newCacheId = [cacheId, new AdditionalValues(expiryPolicyId,
> binaryModeId,
> > > )] But this approach complicates logic and leads to additional
> memory
> > > consumption.
> > >
> > > I think it's ok for now to use the first approach with ExpiryPolicy
> > > serialization.
> > > But any ideas are welcome.
> > >
> > > [1] - https://issues.apache.org/jira/browse/IGNITE-9033
> > >
> > >
> >
> >
>
>

Re: Ignite-Spark integration meeting

[Ivan Pavlukhin]

Actually, I meant running tests on TeamCity.

ср, 23 окт. 2019 г. в 15:36, Alexey Zinoviev :
>
> What kind of testing do you mean?  JUnit, integration via example or
> something else?
>
> ср, 23 окт. 2019 г., 14:16 Ivan Pavlukhin :
>
> > Correction.
> >
> > How are you going to TEST integrations with different Spark versions?
> >
> > ср, 23 окт. 2019 г. в 15:16, Ivan Pavlukhin :
> > >
> > > Alexey,
> > >
> > > Thank you for sharing the summary!
> > >
> > > Small question. How are you going to integrations with different Spark
> > versions?
> > >
> > > чт, 17 окт. 2019 г. в 22:30, Denis Magda :
> > > >
> > > > >
> > > > >  1. We are going to release Apache Ignite 2.8 with limited support of
> > > > >Spark 2.4.4 (known issues are listed here
> > > > >https://issues.apache.org/jira/browse/IGNITE-12054)
> > > >
> > > >
> > > > What does limited support mean? Is it about regressions (something that
> > > > worked before but will fail) or capabilities unsupported even for the
> > > > presently released version of the integration?
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Thu, Oct 17, 2019 at 7:55 AM Alexey Zinoviev <
> > zaleslaw@gmail.com>
> > > > wrote:
> > > >
> > > > > The meeting notes and results of discussion are next:
> > > > >
> > > > >1. We are going to release Apache Ignite 2.8 with limited support
> > of
> > > > >Spark 2.4.4 (known issues are listed here
> > > > >https://issues.apache.org/jira/browse/IGNITE-12054)
> > > > >2. The AI 2.8 will support both, Spark 2.3 and Spark 2.4.4 with
> > known
> > > > >fixed bugs
> > > > >3. We will create the developer-profiles spark_2_3 and spark_2_4
> > > > >separately from current scala profile
> > > > >4. We are ready to move the Ignite-Spark integration to the
> > separate
> > > > >module after release 2.8 with support of most actual Spark
> > versions and
> > > > >with independent release cycle
> > > > >5. We will work together on the next release roadmap with Nikolay
> > and
> > > > >all interested igniters to improve the current solution (add new
> > > > > features
> > > > >as thin client support, working with caches via binary object and
> > so on)
> > > > >6. Nikolay is ready to help with the review of tickets as an
> > author of
> > > > >this component
> > > > >
> > > > > Thanks
> > > > >
> > > > >
> > > > >
> > > > > пн, 14 окт. 2019 г. в 14:36, Alexey Zinoviev  > >:
> > > > >
> > > > > > Yes, let's use the Russian for short discussion, the meeting notes
> > will
> > > > > be
> > > > > > presented in English
> > > > > >
> > > > > > пн, 14 окт. 2019 г. в 14:08, Nikolay Izhikov  > >:
> > > > > >
> > > > > >> Hello, Alexey.
> > > > > >>
> > > > > >> I'm ready to take part in the meeting.
> > > > > >> It will be in Russian, isn't it?
> > > > > >>
> > > > > >> В Пн, 14/10/2019 в 14:00 +0300, Alexey Zinoviev пишет:
> > > > > >> > Hi, Igniters, and especially Nikolay Izhikov
> > > > > >> >
> > > > > >> > I suggest to arrange the meeting via ASF-slack in
> > > > > >> #ignite-spark-integration
> > > > > >> > channel at 17 October, in 16 MSK Time.
> > > > > >> > The language of the meeting: Russian
> > > > > >> >
> > > > > >> > *The topics to discuss:*
> > > > > >> >
> > > > > >> >- Review of next tickets
> > > > > >> >
> > > > > >> >
> > > > > >> >1. [Spark][Bug] IgniteSpark integration forget to close the
> > > > > >> >   IgniteContext and stops the client node in case if error
> > during
> > > > > >> >   PairFunction logic
> > > > > >> >   
> > > > > >> >   2. [Spark] Add initial support of Spark 2.4
> > > > > >> >   
> > > > > >> >   3. [Spark] Add joins to Spark Dataframe examples
> > > > > >> >   
> > > > > >> >
> > > > > >> >
> > > > > >> >- Known issues of migration to 2.4 version
> > > > > >> >- Organization of multiple version support for Apache Spark
> > > > > >> >
> > > > > >> > P.S. meeting notes will be shared with the community
> > > > > >> > P.P.S Nikolay, if you have any objections about the time or
> > topics,
> > > > > >> please
> > > > > >> > let me know
> > > > > >>
> > > > > >
> > > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >



-- 
Best regards,
Ivan Pavlukhin

RE: Thin clients: WithExpiryPolicy

[Alexandr Shapkin]

Pavel,

I think that in some cases milliseconds could make sense, 
so I would like to keep the current precision. 

With this approach we will avoid unnecessary code changes as well as
keeping API in consistence with the thick client method and cacheConfiguration 
settings.


From: Pavel Tupitsyn
Sent: Saturday, October 19, 2019 9:20 PM
To: dev
Subject: Re: Thin clients: WithExpiryPolicy

Alexandr,

Sounds good to me.

Remaining question is - how do we serialize the expiry policy?
Thick client passes this to JNI as 3 int64 values (duration in
milliseconds).
But I don't think we need millisecond precision for expiration, so we could
pass seconds as float values (3*4 bytes).

Thoughts?

On Fri, Oct 18, 2019 at 8:21 PM Alexandr Shapkin  wrote:

> Pavel,
>
> Thanks for sharing your thoughts.
>
> Right now I see that 2 reserved bytes come with every cache request:
> cacheId and flags.
> The first one is obvious, but the second one is used only for java thin
> client with enabled KeepBinary mode.
>
> I think we could use the flags byte and write an expiration policy flag
> when required.
> Whenever the server sees that there is a request with expiration flag, we
> deserialize a policy and apply it to the request.
>
> From: Pavel Tupitsyn
> Sent: Friday, October 18, 2019 7:05 PM
> To: dev
> Subject: Re: Thin clients: WithExpiryPolicy
>
> Stateless approach looks a lot better to me.
>
> We have a choice:
> * Keep expiry policy on server and send an ID with every request (like a
> query cursor ID - 8 bytes)
> * Send full expiry policy with every expiry-enabled request (24 bytes - or
> maybe less? We should think about the format)
>
> Stateful approach will bring a lot of complexity if we consider Affinity
> Awareness [1] mechanism (and also automatic reconnect).
> We would have to keep ExpiryPolicyId for every server and choose the right
> one based on the affinity for every operation.
> This can easily negate any performance gain from saving 16 bytes.
>
> And there is always CacheConfiguration.ExpiryPolicyFactory, which allows us
> to set up default expiry policy.
>
> > things could get worse if we decided to add a few more WithSomething*
> methods
> I don't think this is a good argument - we should decide on case by case
> basis.
> Anyway, other With* methods don't have any parameters, so they carry only 1
> bit of information.
>
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients
>
>
> On Fri, Oct 18, 2019 at 5:14 PM Alexandr Shapkin 
> wrote:
>
> > Igniters,
> >
> > I would like to add WithExpirePolicy support to thin clients. [1]
> > For a thick client, we can obtain a reference to a cache wrapper instance
> > and use cache API through it. At the same time, the thin client protocol
> is
> > stateless, we do not hold a reference to a cache but rather a cache name
> > identifier is used for a server to create an appropriate cache instance.
> >
> > We could extend the protocol as we did with WithKeepBinaryMethod:
> > every time we need to call some API on a cache with expiration, a
> > serialized ExpiryPolicy (additional 3*8 bytes) would be sent. This
> approach
> > works well, but things could get worse if we decided to add a few more
> > WithSomething* methods.
> >
> > Initially, I was thinking about introducing some state context to a
> > protocol, similar to a QueryCursor API. For instance, we can save an
> expire
> > policy configuration for the first call and use some hash value based on
> an
> > ExpiryPolicy for further calls, just as we do for cache names. I.e.
> > newCacheId = [cacheId, new AdditionalValues(expiryPolicyId, binaryModeId,
> > )] But this approach complicates logic and leads to additional memory
> > consumption.
> >
> > I think it's ok for now to use the first approach with ExpiryPolicy
> > serialization.
> > But any ideas are welcome.
> >
> > [1] - https://issues.apache.org/jira/browse/IGNITE-9033
> >
> >
>
>

[jira] [Created] (IGNITE-12325) GridCacheMapEntry reservation mechanism is broken with enabled cache store

[Pavel Kovalenko (Jira)]

Pavel Kovalenko created IGNITE-12325:


 Summary: GridCacheMapEntry reservation mechanism is broken with 
enabled cache store
 Key: IGNITE-12325
 URL: https://issues.apache.org/jira/browse/IGNITE-12325
 Project: Ignite
  Issue Type: Bug
  Components: cache
Affects Versions: 2.8
Reporter: Pavel Kovalenko
 Fix For: 2.8


Entry deferred deletion was disabled after 
https://issues.apache.org/jira/browse/IGNITE-11704 in transactional caches. 
However, if cache store is enabled there is a race with cache entry reservation 
after transactional remove and clear reservation after cache load:

{noformat}
java.lang.AssertionError: GridDhtCacheEntry [rdrs=ReaderId[] [ReaderId 
[nodeId=96c87c98-2524-4f9e-8a2f-6cfceda5, msgId=22663371, txFut=null], 
ReaderId [nodeId=68130805-0dc8-4ef4-abf7-7e7cde86, msgId=22663375, 
txFut=null], ReaderId [nodeId=b4a8abce-8d0e-4459-b93a-a734ad64, 
msgId=22663370, txFut=null]], part=8, super=GridDistributedCacheEntry 
[super=GridCacheMapEntry [key=KeyCacheObjectImpl [part=8, val=8, 
hasValBytes=true], val=null, ver=GridCacheVersion [topVer=0, order=0, 
nodeOrder=0], hash=8, extras=null, flags=2]]]
at 
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.clearReserveForLoad(GridCacheMapEntry.java:3616)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.clearReservationsIfNeeded(GridCacheAdapter.java:2429)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.access$400(GridCacheAdapter.java:179)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter$18.call(GridCacheAdapter.java:2309)
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter$18.call(GridCacheAdapter.java:2217)
at 
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6963)
at 
org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:967)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:844)

{noformat}

The issue can be resolved with enabled deferred delete if cache store is 
enabled.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Ignite-Spark integration meeting

[Alexey Zinoviev]

What kind of testing do you mean?  JUnit, integration via example or
something else?

ср, 23 окт. 2019 г., 14:16 Ivan Pavlukhin :

> Correction.
>
> How are you going to TEST integrations with different Spark versions?
>
> ср, 23 окт. 2019 г. в 15:16, Ivan Pavlukhin :
> >
> > Alexey,
> >
> > Thank you for sharing the summary!
> >
> > Small question. How are you going to integrations with different Spark
> versions?
> >
> > чт, 17 окт. 2019 г. в 22:30, Denis Magda :
> > >
> > > >
> > > >  1. We are going to release Apache Ignite 2.8 with limited support of
> > > >Spark 2.4.4 (known issues are listed here
> > > >https://issues.apache.org/jira/browse/IGNITE-12054)
> > >
> > >
> > > What does limited support mean? Is it about regressions (something that
> > > worked before but will fail) or capabilities unsupported even for the
> > > presently released version of the integration?
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Thu, Oct 17, 2019 at 7:55 AM Alexey Zinoviev <
> zaleslaw@gmail.com>
> > > wrote:
> > >
> > > > The meeting notes and results of discussion are next:
> > > >
> > > >1. We are going to release Apache Ignite 2.8 with limited support
> of
> > > >Spark 2.4.4 (known issues are listed here
> > > >https://issues.apache.org/jira/browse/IGNITE-12054)
> > > >2. The AI 2.8 will support both, Spark 2.3 and Spark 2.4.4 with
> known
> > > >fixed bugs
> > > >3. We will create the developer-profiles spark_2_3 and spark_2_4
> > > >separately from current scala profile
> > > >4. We are ready to move the Ignite-Spark integration to the
> separate
> > > >module after release 2.8 with support of most actual Spark
> versions and
> > > >with independent release cycle
> > > >5. We will work together on the next release roadmap with Nikolay
> and
> > > >all interested igniters to improve the current solution (add new
> > > > features
> > > >as thin client support, working with caches via binary object and
> so on)
> > > >6. Nikolay is ready to help with the review of tickets as an
> author of
> > > >this component
> > > >
> > > > Thanks
> > > >
> > > >
> > > >
> > > > пн, 14 окт. 2019 г. в 14:36, Alexey Zinoviev  >:
> > > >
> > > > > Yes, let's use the Russian for short discussion, the meeting notes
> will
> > > > be
> > > > > presented in English
> > > > >
> > > > > пн, 14 окт. 2019 г. в 14:08, Nikolay Izhikov  >:
> > > > >
> > > > >> Hello, Alexey.
> > > > >>
> > > > >> I'm ready to take part in the meeting.
> > > > >> It will be in Russian, isn't it?
> > > > >>
> > > > >> В Пн, 14/10/2019 в 14:00 +0300, Alexey Zinoviev пишет:
> > > > >> > Hi, Igniters, and especially Nikolay Izhikov
> > > > >> >
> > > > >> > I suggest to arrange the meeting via ASF-slack in
> > > > >> #ignite-spark-integration
> > > > >> > channel at 17 October, in 16 MSK Time.
> > > > >> > The language of the meeting: Russian
> > > > >> >
> > > > >> > *The topics to discuss:*
> > > > >> >
> > > > >> >- Review of next tickets
> > > > >> >
> > > > >> >
> > > > >> >1. [Spark][Bug] IgniteSpark integration forget to close the
> > > > >> >   IgniteContext and stops the client node in case if error
> during
> > > > >> >   PairFunction logic
> > > > >> >   
> > > > >> >   2. [Spark] Add initial support of Spark 2.4
> > > > >> >   
> > > > >> >   3. [Spark] Add joins to Spark Dataframe examples
> > > > >> >   
> > > > >> >
> > > > >> >
> > > > >> >- Known issues of migration to 2.4 version
> > > > >> >- Organization of multiple version support for Apache Spark
> > > > >> >
> > > > >> > P.S. meeting notes will be shared with the community
> > > > >> > P.P.S Nikolay, if you have any objections about the time or
> topics,
> > > > >> please
> > > > >> > let me know
> > > > >>
> > > > >
> > > >
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>

Re: Ignite-Spark integration meeting

[Ivan Pavlukhin]

Correction.

How are you going to TEST integrations with different Spark versions?

ср, 23 окт. 2019 г. в 15:16, Ivan Pavlukhin :
>
> Alexey,
>
> Thank you for sharing the summary!
>
> Small question. How are you going to integrations with different Spark 
> versions?
>
> чт, 17 окт. 2019 г. в 22:30, Denis Magda :
> >
> > >
> > >  1. We are going to release Apache Ignite 2.8 with limited support of
> > >Spark 2.4.4 (known issues are listed here
> > >https://issues.apache.org/jira/browse/IGNITE-12054)
> >
> >
> > What does limited support mean? Is it about regressions (something that
> > worked before but will fail) or capabilities unsupported even for the
> > presently released version of the integration?
> >
> > -
> > Denis
> >
> >
> > On Thu, Oct 17, 2019 at 7:55 AM Alexey Zinoviev 
> > wrote:
> >
> > > The meeting notes and results of discussion are next:
> > >
> > >1. We are going to release Apache Ignite 2.8 with limited support of
> > >Spark 2.4.4 (known issues are listed here
> > >https://issues.apache.org/jira/browse/IGNITE-12054)
> > >2. The AI 2.8 will support both, Spark 2.3 and Spark 2.4.4 with known
> > >fixed bugs
> > >3. We will create the developer-profiles spark_2_3 and spark_2_4
> > >separately from current scala profile
> > >4. We are ready to move the Ignite-Spark integration to the separate
> > >module after release 2.8 with support of most actual Spark versions and
> > >with independent release cycle
> > >5. We will work together on the next release roadmap with Nikolay and
> > >all interested igniters to improve the current solution (add new
> > > features
> > >as thin client support, working with caches via binary object and so 
> > > on)
> > >6. Nikolay is ready to help with the review of tickets as an author of
> > >this component
> > >
> > > Thanks
> > >
> > >
> > >
> > > пн, 14 окт. 2019 г. в 14:36, Alexey Zinoviev :
> > >
> > > > Yes, let's use the Russian for short discussion, the meeting notes will
> > > be
> > > > presented in English
> > > >
> > > > пн, 14 окт. 2019 г. в 14:08, Nikolay Izhikov :
> > > >
> > > >> Hello, Alexey.
> > > >>
> > > >> I'm ready to take part in the meeting.
> > > >> It will be in Russian, isn't it?
> > > >>
> > > >> В Пн, 14/10/2019 в 14:00 +0300, Alexey Zinoviev пишет:
> > > >> > Hi, Igniters, and especially Nikolay Izhikov
> > > >> >
> > > >> > I suggest to arrange the meeting via ASF-slack in
> > > >> #ignite-spark-integration
> > > >> > channel at 17 October, in 16 MSK Time.
> > > >> > The language of the meeting: Russian
> > > >> >
> > > >> > *The topics to discuss:*
> > > >> >
> > > >> >- Review of next tickets
> > > >> >
> > > >> >
> > > >> >1. [Spark][Bug] IgniteSpark integration forget to close the
> > > >> >   IgniteContext and stops the client node in case if error during
> > > >> >   PairFunction logic
> > > >> >   
> > > >> >   2. [Spark] Add initial support of Spark 2.4
> > > >> >   
> > > >> >   3. [Spark] Add joins to Spark Dataframe examples
> > > >> >   
> > > >> >
> > > >> >
> > > >> >- Known issues of migration to 2.4 version
> > > >> >- Organization of multiple version support for Apache Spark
> > > >> >
> > > >> > P.S. meeting notes will be shared with the community
> > > >> > P.P.S Nikolay, if you have any objections about the time or topics,
> > > >> please
> > > >> > let me know
> > > >>
> > > >
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin



-- 
Best regards,
Ivan Pavlukhin

Re: Ignite-Spark integration meeting

[Ivan Pavlukhin]

Alexey,

Thank you for sharing the summary!

Small question. How are you going to integrations with different Spark versions?

чт, 17 окт. 2019 г. в 22:30, Denis Magda :
>
> >
> >  1. We are going to release Apache Ignite 2.8 with limited support of
> >Spark 2.4.4 (known issues are listed here
> >https://issues.apache.org/jira/browse/IGNITE-12054)
>
>
> What does limited support mean? Is it about regressions (something that
> worked before but will fail) or capabilities unsupported even for the
> presently released version of the integration?
>
> -
> Denis
>
>
> On Thu, Oct 17, 2019 at 7:55 AM Alexey Zinoviev 
> wrote:
>
> > The meeting notes and results of discussion are next:
> >
> >1. We are going to release Apache Ignite 2.8 with limited support of
> >Spark 2.4.4 (known issues are listed here
> >https://issues.apache.org/jira/browse/IGNITE-12054)
> >2. The AI 2.8 will support both, Spark 2.3 and Spark 2.4.4 with known
> >fixed bugs
> >3. We will create the developer-profiles spark_2_3 and spark_2_4
> >separately from current scala profile
> >4. We are ready to move the Ignite-Spark integration to the separate
> >module after release 2.8 with support of most actual Spark versions and
> >with independent release cycle
> >5. We will work together on the next release roadmap with Nikolay and
> >all interested igniters to improve the current solution (add new
> > features
> >as thin client support, working with caches via binary object and so on)
> >6. Nikolay is ready to help with the review of tickets as an author of
> >this component
> >
> > Thanks
> >
> >
> >
> > пн, 14 окт. 2019 г. в 14:36, Alexey Zinoviev :
> >
> > > Yes, let's use the Russian for short discussion, the meeting notes will
> > be
> > > presented in English
> > >
> > > пн, 14 окт. 2019 г. в 14:08, Nikolay Izhikov :
> > >
> > >> Hello, Alexey.
> > >>
> > >> I'm ready to take part in the meeting.
> > >> It will be in Russian, isn't it?
> > >>
> > >> В Пн, 14/10/2019 в 14:00 +0300, Alexey Zinoviev пишет:
> > >> > Hi, Igniters, and especially Nikolay Izhikov
> > >> >
> > >> > I suggest to arrange the meeting via ASF-slack in
> > >> #ignite-spark-integration
> > >> > channel at 17 October, in 16 MSK Time.
> > >> > The language of the meeting: Russian
> > >> >
> > >> > *The topics to discuss:*
> > >> >
> > >> >- Review of next tickets
> > >> >
> > >> >
> > >> >1. [Spark][Bug] IgniteSpark integration forget to close the
> > >> >   IgniteContext and stops the client node in case if error during
> > >> >   PairFunction logic
> > >> >   
> > >> >   2. [Spark] Add initial support of Spark 2.4
> > >> >   
> > >> >   3. [Spark] Add joins to Spark Dataframe examples
> > >> >   
> > >> >
> > >> >
> > >> >- Known issues of migration to 2.4 version
> > >> >- Organization of multiple version support for Apache Spark
> > >> >
> > >> > P.S. meeting notes will be shared with the community
> > >> > P.P.S Nikolay, if you have any objections about the time or topics,
> > >> please
> > >> > let me know
> > >>
> > >
> >



-- 
Best regards,
Ivan Pavlukhin

Re: [DISCUSS] Proposal for Ignite Extensions as a separate Bahir module or Incubator project

[Alexey Zinoviev]

Also, dear Saikat Maitra, could you please describe how you see the release
cycles in Bahir Ignite Extensions and how it be related to Ignite release,
2.9, 3.0 for example.

Thank you for your energy

ср, 23 окт. 2019 г., 8:10 Alexey Zinoviev :

> Please, give me permissions too, I'd glad to help with this modules
> migration and support part of them in future, but also we need not only
> contributor but a few Committer permissions to merge In repository in other
> side it could be very long proccess.
>
> Could you ask Bahir Community about that?
>
> ср, 23 окт. 2019 г., 2:31 Saikat Maitra :
>
>> Hi,
>>
>> I discussed with Apache Bahir community and they are interested to have
>> Apache Ignite extensions as part of Apache Bahir project.
>>
>> I have also requested for contributor access in Jira for Apache Bahir
>> project so that I can create issues and assign to myself. I can help with
>> code reviews as well.
>>
>> Also my thoughts on releases specific to dependencies for Apache Ignite is
>> to do a fast follow up release for modules based on latest Apache Ignite
>> stable release.
>>
>> Here is the email thread for reference
>> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>>
>> I wanted to connect and get feedback on the proposal and if we are ok to
>> move the following Apache Ignite Extensions
>>
>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>>
>> Regards,
>> Saikat
>>
>>
>> On Fri, Oct 18, 2019 at 9:44 PM Saikat Maitra 
>> wrote:
>>
>> > Hello,
>> >
>> > We wanted to discuss on a proposal to move and support the Apache Ignite
>> > integrations as separate Ignite Extensions as discussed here
>> >
>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Pub-Sub-Streamer-Implementation-td43944.html
>> > .
>> >
>> > The reason we wanted to move our Apache Ignite integration as  separate
>> > Extensions is this will help us to manage and maintain separate
>> lifecycle
>> > for Apache Ignite integrations.
>> >
>> > All the integrations will continue to be part of ASF and we will  keep
>> > supporting and developing in accordance with ASF vision and practices.
>> >
>> > We are considering following two choices for moving to Apache Ignite
>> > Extensions:
>> >
>> > 1. Reach out to Apache Bahir community and propose to make Ignite
>> > Extensions a separate module as part of Apache Bahir project.
>> >
>> > https://bahir.apache.org/
>> >
>> >
>> >
>> https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces96
>> >
>> >
>> > 2. Reach out to Apache Incubator community and request for a new project
>> > for Ignite Extensions.
>> >
>> > Please review and share feedback on our proposal.
>> >
>> > Warm Regards,
>> > Saikat
>> >
>>
>

[jira] [Created] (IGNITE-12324) improper message in BinaryObjectException is printed when accessing fieldOrder method binary field with binary object of unregistered type

[Kirill Tkalenko (Jira)]

Kirill Tkalenko created IGNITE-12324:


 Summary: improper message in BinaryObjectException is printed when 
accessing fieldOrder method binary field with binary object of unregistered type
 Key: IGNITE-12324
 URL: https://issues.apache.org/jira/browse/IGNITE-12324
 Project: Ignite
  Issue Type: Improvement
Reporter: Kirill Tkalenko
Assignee: Kirill Tkalenko
 Fix For: 2.8


ScenarioScenario1. Create `BinaryObject` instance with `time` field of type 
`TimeValue`.2. Extract `BinaryField` `time` from binary object3. Modify binary 
object internal array so that it would reference non existing binary type4. 
Invoke `fieldOrder` (or `value`) of binary field passing modified binary 
object5. Expect that `BinaryObjectException` is thrown5.1 Expect that message 
contains information about expected and actual binary type id, expected type 
name, field id, field name, expected field type.5.2. Expect that for actual 
type name message contains 'null'
Actual: \{noformat}class org.apache.ignite.binary.BinaryObjectException: Failed 
to get binary type details [typeId=-1291121110] at 
org.apache.ignite.internal.binary.BinaryTypeProxy.target(BinaryTypeProxy.java:116)
 at 
org.apache.ignite.internal.binary.BinaryTypeProxy.typeName(BinaryTypeProxy.java:75)
 at 
org.apache.ignite.internal.binary.BinaryFieldImpl.fieldOrder(BinaryFieldImpl.java:291)
 at 
org.apache.ignite.internal.binary.BinaryFieldImpl.value(BinaryFieldImpl.java:109)\{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DISCUSS] Proposal for Ignite Extensions as a separate Bahir module or Incubator project

[Alexey Zinoviev]

Please, give me permissions too, I'd glad to help with this modules
migration and support part of them in future, but also we need not only
contributor but a few Committer permissions to merge In repository in other
side it could be very long proccess.

Could you ask Bahir Community about that?

ср, 23 окт. 2019 г., 2:31 Saikat Maitra :

> Hi,
>
> I discussed with Apache Bahir community and they are interested to have
> Apache Ignite extensions as part of Apache Bahir project.
>
> I have also requested for contributor access in Jira for Apache Bahir
> project so that I can create issues and assign to myself. I can help with
> code reviews as well.
>
> Also my thoughts on releases specific to dependencies for Apache Ignite is
> to do a fast follow up release for modules based on latest Apache Ignite
> stable release.
>
> Here is the email thread for reference
> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>
> I wanted to connect and get feedback on the proposal and if we are ok to
> move the following Apache Ignite Extensions
>
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>
> Regards,
> Saikat
>
>
> On Fri, Oct 18, 2019 at 9:44 PM Saikat Maitra 
> wrote:
>
> > Hello,
> >
> > We wanted to discuss on a proposal to move and support the Apache Ignite
> > integrations as separate Ignite Extensions as discussed here
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSS-Pub-Sub-Streamer-Implementation-td43944.html
> > .
> >
> > The reason we wanted to move our Apache Ignite integration as  separate
> > Extensions is this will help us to manage and maintain separate lifecycle
> > for Apache Ignite integrations.
> >
> > All the integrations will continue to be part of ASF and we will  keep
> > supporting and developing in accordance with ASF vision and practices.
> >
> > We are considering following two choices for moving to Apache Ignite
> > Extensions:
> >
> > 1. Reach out to Apache Bahir community and propose to make Ignite
> > Extensions a separate module as part of Apache Bahir project.
> >
> > https://bahir.apache.org/
> >
> >
> >
> https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces96
> >
> >
> > 2. Reach out to Apache Incubator community and request for a new project
> > for Ignite Extensions.
> >
> > Please review and share feedback on our proposal.
> >
> > Warm Regards,
> > Saikat
> >
>