Re: Shall tajo
+1 on both counts. Perhaps 'branch-1.x' would be informative. Also, let's wait a couple of days to let others on the list to express their opinion, perhaps different and/or complimentary. And Jay - thanks for sharing your work: I owe you a look into this, but life last month wasn't overly relaxed in terms of spare time. Thanks, cos On Fri, Aug 30, 2019 at 11:30AM, Youngwoo Kim (김영우) wrote: > Sounds great! > > I like the idea that reshaping big data architecture at cloud native > deployments. > What about creating 'branch-1' from current status for existing users and > traditional architectures? So, we can make progress from master branch for > the future Bigtop releases. > > Jay, > I feel like we should create a JIRA issue or shared doc to keep discussions > on detailed plan and design docs for Bigtop NG. > > Thanks, > Youngwoo > > On Fri, Aug 30, 2019 at 10:28 AM Jun HE wrote: > > > Super +1 for Bigtop 2.0 with modern components and K8S support! I cannot > > wait to see Bigtop running on K8S... > > So can someone help to create a branch-2.0 now? :P > > > > > > Konstantin Boudnik 于2019年8月30日周五 上午2:24写道： > > > > > Aah, k8s - thanks Jay! > > > > > > BTW, I believe we can do this in a less formal stuff (ie no VOTE): we > > have > > > a > > > discussion going on the version of Bigtop. How about we make it 2.0 and > > > trim > > > the BOM into the needed shape and form? If there's enough drive in the > > > community to continue with 1.x line (1.4 and so on), it can be done as a > > > parallel effort. > > > > > > Thoughts? > > > Cos > > > > > > On Fri, Aug 30, 2019 at 01:26AM, Evans Ye wrote: > > > > I'm super +1 with K8S stuff and the core components idea! > > > > > > > > I'm with Cos and Jun. Removing those stuffs can ease our CI resource > > > > utilization and yield more stable CI results. This is what I'm thinking > > > how > > > > to proceed based on the previous feedback that a drop of supporting > > > > component should be carefully discussed > > > > > > > > 1. Spin up a vote to drop project XXX (if confident enough, put > > multiple > > > > components here) > > > > 2. If +3 binding votes with no -1 votes, it passes. > > > > 3. Put up the PR for code removing > > > > 4. Refine our CI settings (I can definitely help with this part) > > > > > > > > Best, > > > > Evans > > > > > > > > > > > > > > > > > > > > Jay Vyas 於 2019年8月29日 週四 下午7:26寫道： > > > > > > > > > First of all I’m all for dropping the old stuff. > > > > > > > > > > 1) My opinion - to further cos point - I think people running old > > stuff > > > > > don’t really need version updates, or if they do, they don’t > > > constitute a > > > > > large audience , or should contribute them ok their own. > > > > > > > > > > 2) surprise surprise :):) here’s my k8s native. View - we should move > > > the > > > > > old components into sustaining mode, and rebuild a new bigtop focus > > > solely > > > > > on spark , NiFi , Presto (with a lot of attention to minimal > > standalone > > > > > Hadoop w/ a Hive connector ) and target it at kubernetes native > > > deployments > > > > > :). > > > > > > > > > > I can make it so the k8s part is an impl detail so users can be > > mostly > > > > > decoupled from it . > > > > > > > > > > I’ve been integration testing various things in the bigdata ecosystem > > > on > > > > > k8s and there is a lot of demand without anyone owning the > > integration. > > > > > > > > > > > On Aug 29, 2019, at 2:39 AM, Konstantin Boudnik > > > wrote: > > > > > > > > > > > > I am not sure about Giraph, Tajo and some others, but Sqoop seems > > to > > > be > > > > > user > > > > > > around by people. So, if it isn't much of the burden for us - and > > it > > > > > seems > > > > > > pretty stable at the moment - I'd leave it. > > > > > > > > > > > > What I would think would makes sense to spend some of our efforts > > on > > > is > > > > > on > > > > > > adding modern tooling like Nifi/Airflow into the mix. As you said - > > > > > things > > > > > > move forward pretty fast, and we seem to be sticking to the some of > > > the > > > > > old > > > > > > stuff. > > > > > > > > > > > > Thanks for starting this discussion! > > > > > > Cos > > > > > > > > > > > >> On Wed, Aug 28, 2019 at 04:12PM, Jun HE wrote: > > > > > >> Hi, folks, > > > > > >> > > > > > >> I went through current components Bigtop is supporting, and I > > > noticed > > > > > that > > > > > >> these upstream projects haven't been released for quite a while: > > > > > >> Apache Tajo: > > > > > >>last release: release-0.11.3-rc0, May 11, 2016 > > > > > >> Apache Apex: > > > > > >>last release: v3.7.0, Apr 27, 2018 > > > > > >> Apache Giraph: > > > > > >>last release: rel/1.2.0-RC1, Oct 13 2016 > > > > > >> Apache Hama: > > > > > >>last release: 0.7.1-RC2, Mar 12, 2016 > > > > > >> Apache Sqoop: > > > > > >>last release: release-1.4.7-rc0, Dec 6, 2017; > > > release-1.99.7-rc1, Jul > > > > > >> 20, 2016 > > > > > >> > > > > > >> And some of them se
Re: Shall tajo
Sounds great! I like the idea that reshaping big data architecture at cloud native deployments. What about creating 'branch-1' from current status for existing users and traditional architectures? So, we can make progress from master branch for the future Bigtop releases. Jay, I feel like we should create a JIRA issue or shared doc to keep discussions on detailed plan and design docs for Bigtop NG. Thanks, Youngwoo On Fri, Aug 30, 2019 at 10:28 AM Jun HE wrote: > Super +1 for Bigtop 2.0 with modern components and K8S support! I cannot > wait to see Bigtop running on K8S... > So can someone help to create a branch-2.0 now? :P > > > Konstantin Boudnik 于2019年8月30日周五 上午2:24写道： > > > Aah, k8s - thanks Jay! > > > > BTW, I believe we can do this in a less formal stuff (ie no VOTE): we > have > > a > > discussion going on the version of Bigtop. How about we make it 2.0 and > > trim > > the BOM into the needed shape and form? If there's enough drive in the > > community to continue with 1.x line (1.4 and so on), it can be done as a > > parallel effort. > > > > Thoughts? > > Cos > > > > On Fri, Aug 30, 2019 at 01:26AM, Evans Ye wrote: > > > I'm super +1 with K8S stuff and the core components idea! > > > > > > I'm with Cos and Jun. Removing those stuffs can ease our CI resource > > > utilization and yield more stable CI results. This is what I'm thinking > > how > > > to proceed based on the previous feedback that a drop of supporting > > > component should be carefully discussed > > > > > > 1. Spin up a vote to drop project XXX (if confident enough, put > multiple > > > components here) > > > 2. If +3 binding votes with no -1 votes, it passes. > > > 3. Put up the PR for code removing > > > 4. Refine our CI settings (I can definitely help with this part) > > > > > > Best, > > > Evans > > > > > > > > > > > > > > > Jay Vyas 於 2019年8月29日 週四 下午7:26寫道： > > > > > > > First of all I’m all for dropping the old stuff. > > > > > > > > 1) My opinion - to further cos point - I think people running old > stuff > > > > don’t really need version updates, or if they do, they don’t > > constitute a > > > > large audience , or should contribute them ok their own. > > > > > > > > 2) surprise surprise :):) here’s my k8s native. View - we should move > > the > > > > old components into sustaining mode, and rebuild a new bigtop focus > > solely > > > > on spark , NiFi , Presto (with a lot of attention to minimal > standalone > > > > Hadoop w/ a Hive connector ) and target it at kubernetes native > > deployments > > > > :). > > > > > > > > I can make it so the k8s part is an impl detail so users can be > mostly > > > > decoupled from it . > > > > > > > > I’ve been integration testing various things in the bigdata ecosystem > > on > > > > k8s and there is a lot of demand without anyone owning the > integration. > > > > > > > > > On Aug 29, 2019, at 2:39 AM, Konstantin Boudnik > > wrote: > > > > > > > > > > I am not sure about Giraph, Tajo and some others, but Sqoop seems > to > > be > > > > user > > > > > around by people. So, if it isn't much of the burden for us - and > it > > > > seems > > > > > pretty stable at the moment - I'd leave it. > > > > > > > > > > What I would think would makes sense to spend some of our efforts > on > > is > > > > on > > > > > adding modern tooling like Nifi/Airflow into the mix. As you said - > > > > things > > > > > move forward pretty fast, and we seem to be sticking to the some of > > the > > > > old > > > > > stuff. > > > > > > > > > > Thanks for starting this discussion! > > > > > Cos > > > > > > > > > >> On Wed, Aug 28, 2019 at 04:12PM, Jun HE wrote: > > > > >> Hi, folks, > > > > >> > > > > >> I went through current components Bigtop is supporting, and I > > noticed > > > > that > > > > >> these upstream projects haven't been released for quite a while: > > > > >> Apache Tajo: > > > > >>last release: release-0.11.3-rc0, May 11, 2016 > > > > >> Apache Apex: > > > > >>last release: v3.7.0, Apr 27, 2018 > > > > >> Apache Giraph: > > > > >>last release: rel/1.2.0-RC1, Oct 13 2016 > > > > >> Apache Hama: > > > > >>last release: 0.7.1-RC2, Mar 12, 2016 > > > > >> Apache Sqoop: > > > > >>last release: release-1.4.7-rc0, Dec 6, 2017; > > release-1.99.7-rc1, Jul > > > > >> 20, 2016 > > > > >> > > > > >> And some of them seem to be in slow development: > > > > >> Apache Tajo: > > > > >>last commit: Jul 13, 2018 > > > > >> Apache Apex: > > > > >>last commit: Jun 20, 2018 > > > > >> Apache Hama: > > > > >>last commit: Jul 30, 2018 > > > > >> > > > > >> So I'm wondering whether we should continue support for these > > components > > > > >> (or part of them) in next/future releases. > > > > >> > > > > >> I understand that similar topics were discussed before. But, you > > know, > > > > this > > > > >> is an quickly evolving world, maybe it worth another revisit now? > ;) > > > > >> > > > > >> Regards, > > > > >> > > > > >> Jun > > > > > > >
Re: Shall tajo
Here’s where I’m at . https://github.com/jayunit100/apachecon-2019-bigtop3x/ It’s a lot easier then packaging all this stuff, but for most people will accomplish a similar end goal ( a big Data cloud in a box) and mostly a matter of curation of charts and images, and integration testing . So I see the amount of code to maintain for bigtop becoming minimal, and most of our job being curation and integration testing of canonical releases of these images and helm charts / operator yamls. This will be a significant paradigm shift if we do it , but it will be something that we can maintain even if we have a small community , that adds differntiated value to the asf and cncf ecosystems for sure. Working with the minio folks on how presto integration will work. Come to apachecon Vegas and we can talk more about a plan for the future . > On Aug 29, 2019, at 9:28 PM, Jun HE wrote: > > Super +1 for Bigtop 2.0 with modern components and K8S support! I cannot > wait to see Bigtop running on K8S... > So can someone help to create a branch-2.0 now? :P > > > Konstantin Boudnik 于2019年8月30日周五 上午2:24写道： > >> Aah, k8s - thanks Jay! >> >> BTW, I believe we can do this in a less formal stuff (ie no VOTE): we have >> a >> discussion going on the version of Bigtop. How about we make it 2.0 and >> trim >> the BOM into the needed shape and form? If there's enough drive in the >> community to continue with 1.x line (1.4 and so on), it can be done as a >> parallel effort. >> >> Thoughts? >> Cos >> >>> On Fri, Aug 30, 2019 at 01:26AM, Evans Ye wrote: >>> I'm super +1 with K8S stuff and the core components idea! >>> >>> I'm with Cos and Jun. Removing those stuffs can ease our CI resource >>> utilization and yield more stable CI results. This is what I'm thinking >> how >>> to proceed based on the previous feedback that a drop of supporting >>> component should be carefully discussed >>> >>> 1. Spin up a vote to drop project XXX (if confident enough, put multiple >>> components here) >>> 2. If +3 binding votes with no -1 votes, it passes. >>> 3. Put up the PR for code removing >>> 4. Refine our CI settings (I can definitely help with this part) >>> >>> Best, >>> Evans >>> >>> >>> >>> >>> Jay Vyas 於 2019年8月29日 週四 下午7:26寫道： >>> First of all I’m all for dropping the old stuff. 1) My opinion - to further cos point - I think people running old stuff don’t really need version updates, or if they do, they don’t >> constitute a large audience , or should contribute them ok their own. 2) surprise surprise :):) here’s my k8s native. View - we should move >> the old components into sustaining mode, and rebuild a new bigtop focus >> solely on spark , NiFi , Presto (with a lot of attention to minimal standalone Hadoop w/ a Hive connector ) and target it at kubernetes native >> deployments :). I can make it so the k8s part is an impl detail so users can be mostly decoupled from it . I’ve been integration testing various things in the bigdata ecosystem >> on k8s and there is a lot of demand without anyone owning the integration. > On Aug 29, 2019, at 2:39 AM, Konstantin Boudnik >> wrote: > > I am not sure about Giraph, Tajo and some others, but Sqoop seems to >> be user > around by people. So, if it isn't much of the burden for us - and it seems > pretty stable at the moment - I'd leave it. > > What I would think would makes sense to spend some of our efforts on >> is on > adding modern tooling like Nifi/Airflow into the mix. As you said - things > move forward pretty fast, and we seem to be sticking to the some of >> the old > stuff. > > Thanks for starting this discussion! > Cos > >> On Wed, Aug 28, 2019 at 04:12PM, Jun HE wrote: >> Hi, folks, >> >> I went through current components Bigtop is supporting, and I >> noticed that >> these upstream projects haven't been released for quite a while: >> Apache Tajo: >> last release: release-0.11.3-rc0, May 11, 2016 >> Apache Apex: >> last release: v3.7.0, Apr 27, 2018 >> Apache Giraph: >> last release: rel/1.2.0-RC1, Oct 13 2016 >> Apache Hama: >> last release: 0.7.1-RC2, Mar 12, 2016 >> Apache Sqoop: >> last release: release-1.4.7-rc0, Dec 6, 2017; >> release-1.99.7-rc1, Jul >> 20, 2016 >> >> And some of them seem to be in slow development: >> Apache Tajo: >> last commit: Jul 13, 2018 >> Apache Apex: >> last commit: Jun 20, 2018 >> Apache Hama: >> last commit: Jul 30, 2018 >> >> So I'm wondering whether we should continue support for these >> components >> (or part of them) in next/future releases. >> >> I understand that similar topics were discussed before. But, you >> know, this >> is an quickly evolving world, maybe it worth another revi
Re: Shall tajo
Super +1 for Bigtop 2.0 with modern components and K8S support! I cannot wait to see Bigtop running on K8S... So can someone help to create a branch-2.0 now? :P Konstantin Boudnik 于2019年8月30日周五 上午2:24写道： > Aah, k8s - thanks Jay! > > BTW, I believe we can do this in a less formal stuff (ie no VOTE): we have > a > discussion going on the version of Bigtop. How about we make it 2.0 and > trim > the BOM into the needed shape and form? If there's enough drive in the > community to continue with 1.x line (1.4 and so on), it can be done as a > parallel effort. > > Thoughts? > Cos > > On Fri, Aug 30, 2019 at 01:26AM, Evans Ye wrote: > > I'm super +1 with K8S stuff and the core components idea! > > > > I'm with Cos and Jun. Removing those stuffs can ease our CI resource > > utilization and yield more stable CI results. This is what I'm thinking > how > > to proceed based on the previous feedback that a drop of supporting > > component should be carefully discussed > > > > 1. Spin up a vote to drop project XXX (if confident enough, put multiple > > components here) > > 2. If +3 binding votes with no -1 votes, it passes. > > 3. Put up the PR for code removing > > 4. Refine our CI settings (I can definitely help with this part) > > > > Best, > > Evans > > > > > > > > > > Jay Vyas 於 2019年8月29日 週四 下午7:26寫道： > > > > > First of all I’m all for dropping the old stuff. > > > > > > 1) My opinion - to further cos point - I think people running old stuff > > > don’t really need version updates, or if they do, they don’t > constitute a > > > large audience , or should contribute them ok their own. > > > > > > 2) surprise surprise :):) here’s my k8s native. View - we should move > the > > > old components into sustaining mode, and rebuild a new bigtop focus > solely > > > on spark , NiFi , Presto (with a lot of attention to minimal standalone > > > Hadoop w/ a Hive connector ) and target it at kubernetes native > deployments > > > :). > > > > > > I can make it so the k8s part is an impl detail so users can be mostly > > > decoupled from it . > > > > > > I’ve been integration testing various things in the bigdata ecosystem > on > > > k8s and there is a lot of demand without anyone owning the integration. > > > > > > > On Aug 29, 2019, at 2:39 AM, Konstantin Boudnik > wrote: > > > > > > > > I am not sure about Giraph, Tajo and some others, but Sqoop seems to > be > > > user > > > > around by people. So, if it isn't much of the burden for us - and it > > > seems > > > > pretty stable at the moment - I'd leave it. > > > > > > > > What I would think would makes sense to spend some of our efforts on > is > > > on > > > > adding modern tooling like Nifi/Airflow into the mix. As you said - > > > things > > > > move forward pretty fast, and we seem to be sticking to the some of > the > > > old > > > > stuff. > > > > > > > > Thanks for starting this discussion! > > > > Cos > > > > > > > >> On Wed, Aug 28, 2019 at 04:12PM, Jun HE wrote: > > > >> Hi, folks, > > > >> > > > >> I went through current components Bigtop is supporting, and I > noticed > > > that > > > >> these upstream projects haven't been released for quite a while: > > > >> Apache Tajo: > > > >>last release: release-0.11.3-rc0, May 11, 2016 > > > >> Apache Apex: > > > >>last release: v3.7.0, Apr 27, 2018 > > > >> Apache Giraph: > > > >>last release: rel/1.2.0-RC1, Oct 13 2016 > > > >> Apache Hama: > > > >>last release: 0.7.1-RC2, Mar 12, 2016 > > > >> Apache Sqoop: > > > >>last release: release-1.4.7-rc0, Dec 6, 2017; > release-1.99.7-rc1, Jul > > > >> 20, 2016 > > > >> > > > >> And some of them seem to be in slow development: > > > >> Apache Tajo: > > > >>last commit: Jul 13, 2018 > > > >> Apache Apex: > > > >>last commit: Jun 20, 2018 > > > >> Apache Hama: > > > >>last commit: Jul 30, 2018 > > > >> > > > >> So I'm wondering whether we should continue support for these > components > > > >> (or part of them) in next/future releases. > > > >> > > > >> I understand that similar topics were discussed before. But, you > know, > > > this > > > >> is an quickly evolving world, maybe it worth another revisit now? ;) > > > >> > > > >> Regards, > > > >> > > > >> Jun > > > >
Re: Shall tajo
Aah, k8s - thanks Jay! BTW, I believe we can do this in a less formal stuff (ie no VOTE): we have a discussion going on the version of Bigtop. How about we make it 2.0 and trim the BOM into the needed shape and form? If there's enough drive in the community to continue with 1.x line (1.4 and so on), it can be done as a parallel effort. Thoughts? Cos On Fri, Aug 30, 2019 at 01:26AM, Evans Ye wrote: > I'm super +1 with K8S stuff and the core components idea! > > I'm with Cos and Jun. Removing those stuffs can ease our CI resource > utilization and yield more stable CI results. This is what I'm thinking how > to proceed based on the previous feedback that a drop of supporting > component should be carefully discussed > > 1. Spin up a vote to drop project XXX (if confident enough, put multiple > components here) > 2. If +3 binding votes with no -1 votes, it passes. > 3. Put up the PR for code removing > 4. Refine our CI settings (I can definitely help with this part) > > Best, > Evans > > > > > Jay Vyas 於 2019年8月29日 週四 下午7:26寫道： > > > First of all I’m all for dropping the old stuff. > > > > 1) My opinion - to further cos point - I think people running old stuff > > don’t really need version updates, or if they do, they don’t constitute a > > large audience , or should contribute them ok their own. > > > > 2) surprise surprise :):) here’s my k8s native. View - we should move the > > old components into sustaining mode, and rebuild a new bigtop focus solely > > on spark , NiFi , Presto (with a lot of attention to minimal standalone > > Hadoop w/ a Hive connector ) and target it at kubernetes native deployments > > :). > > > > I can make it so the k8s part is an impl detail so users can be mostly > > decoupled from it . > > > > I’ve been integration testing various things in the bigdata ecosystem on > > k8s and there is a lot of demand without anyone owning the integration. > > > > > On Aug 29, 2019, at 2:39 AM, Konstantin Boudnik wrote: > > > > > > I am not sure about Giraph, Tajo and some others, but Sqoop seems to be > > user > > > around by people. So, if it isn't much of the burden for us - and it > > seems > > > pretty stable at the moment - I'd leave it. > > > > > > What I would think would makes sense to spend some of our efforts on is > > on > > > adding modern tooling like Nifi/Airflow into the mix. As you said - > > things > > > move forward pretty fast, and we seem to be sticking to the some of the > > old > > > stuff. > > > > > > Thanks for starting this discussion! > > > Cos > > > > > >> On Wed, Aug 28, 2019 at 04:12PM, Jun HE wrote: > > >> Hi, folks, > > >> > > >> I went through current components Bigtop is supporting, and I noticed > > that > > >> these upstream projects haven't been released for quite a while: > > >> Apache Tajo: > > >>last release: release-0.11.3-rc0, May 11, 2016 > > >> Apache Apex: > > >>last release: v3.7.0, Apr 27, 2018 > > >> Apache Giraph: > > >>last release: rel/1.2.0-RC1, Oct 13 2016 > > >> Apache Hama: > > >>last release: 0.7.1-RC2, Mar 12, 2016 > > >> Apache Sqoop: > > >>last release: release-1.4.7-rc0, Dec 6, 2017; release-1.99.7-rc1, Jul > > >> 20, 2016 > > >> > > >> And some of them seem to be in slow development: > > >> Apache Tajo: > > >>last commit: Jul 13, 2018 > > >> Apache Apex: > > >>last commit: Jun 20, 2018 > > >> Apache Hama: > > >>last commit: Jul 30, 2018 > > >> > > >> So I'm wondering whether we should continue support for these components > > >> (or part of them) in next/future releases. > > >> > > >> I understand that similar topics were discussed before. But, you know, > > this > > >> is an quickly evolving world, maybe it worth another revisit now? ;) > > >> > > >> Regards, > > >> > > >> Jun > > signature.asc Description: Digital signature
Re: Shall tajo
I'm super +1 with K8S stuff and the core components idea! I'm with Cos and Jun. Removing those stuffs can ease our CI resource utilization and yield more stable CI results. This is what I'm thinking how to proceed based on the previous feedback that a drop of supporting component should be carefully discussed 1. Spin up a vote to drop project XXX (if confident enough, put multiple components here) 2. If +3 binding votes with no -1 votes, it passes. 3. Put up the PR for code removing 4. Refine our CI settings (I can definitely help with this part) Best, Evans Jay Vyas 於 2019年8月29日 週四 下午7:26寫道： > First of all I’m all for dropping the old stuff. > > 1) My opinion - to further cos point - I think people running old stuff > don’t really need version updates, or if they do, they don’t constitute a > large audience , or should contribute them ok their own. > > 2) surprise surprise :):) here’s my k8s native. View - we should move the > old components into sustaining mode, and rebuild a new bigtop focus solely > on spark , NiFi , Presto (with a lot of attention to minimal standalone > Hadoop w/ a Hive connector ) and target it at kubernetes native deployments > :). > > I can make it so the k8s part is an impl detail so users can be mostly > decoupled from it . > > I’ve been integration testing various things in the bigdata ecosystem on > k8s and there is a lot of demand without anyone owning the integration. > > > On Aug 29, 2019, at 2:39 AM, Konstantin Boudnik wrote: > > > > I am not sure about Giraph, Tajo and some others, but Sqoop seems to be > user > > around by people. So, if it isn't much of the burden for us - and it > seems > > pretty stable at the moment - I'd leave it. > > > > What I would think would makes sense to spend some of our efforts on is > on > > adding modern tooling like Nifi/Airflow into the mix. As you said - > things > > move forward pretty fast, and we seem to be sticking to the some of the > old > > stuff. > > > > Thanks for starting this discussion! > > Cos > > > >> On Wed, Aug 28, 2019 at 04:12PM, Jun HE wrote: > >> Hi, folks, > >> > >> I went through current components Bigtop is supporting, and I noticed > that > >> these upstream projects haven't been released for quite a while: > >> Apache Tajo: > >>last release: release-0.11.3-rc0, May 11, 2016 > >> Apache Apex: > >>last release: v3.7.0, Apr 27, 2018 > >> Apache Giraph: > >>last release: rel/1.2.0-RC1, Oct 13 2016 > >> Apache Hama: > >>last release: 0.7.1-RC2, Mar 12, 2016 > >> Apache Sqoop: > >>last release: release-1.4.7-rc0, Dec 6, 2017; release-1.99.7-rc1, Jul > >> 20, 2016 > >> > >> And some of them seem to be in slow development: > >> Apache Tajo: > >>last commit: Jul 13, 2018 > >> Apache Apex: > >>last commit: Jun 20, 2018 > >> Apache Hama: > >>last commit: Jul 30, 2018 > >> > >> So I'm wondering whether we should continue support for these components > >> (or part of them) in next/future releases. > >> > >> I understand that similar topics were discussed before. But, you know, > this > >> is an quickly evolving world, maybe it worth another revisit now? ;) > >> > >> Regards, > >> > >> Jun >
Re: Shall tajo
First of all I’m all for dropping the old stuff. 1) My opinion - to further cos point - I think people running old stuff don’t really need version updates, or if they do, they don’t constitute a large audience , or should contribute them ok their own. 2) surprise surprise :):) here’s my k8s native. View - we should move the old components into sustaining mode, and rebuild a new bigtop focus solely on spark , NiFi , Presto (with a lot of attention to minimal standalone Hadoop w/ a Hive connector ) and target it at kubernetes native deployments :). I can make it so the k8s part is an impl detail so users can be mostly decoupled from it . I’ve been integration testing various things in the bigdata ecosystem on k8s and there is a lot of demand without anyone owning the integration. > On Aug 29, 2019, at 2:39 AM, Konstantin Boudnik wrote: > > I am not sure about Giraph, Tajo and some others, but Sqoop seems to be user > around by people. So, if it isn't much of the burden for us - and it seems > pretty stable at the moment - I'd leave it. > > What I would think would makes sense to spend some of our efforts on is on > adding modern tooling like Nifi/Airflow into the mix. As you said - things > move forward pretty fast, and we seem to be sticking to the some of the old > stuff. > > Thanks for starting this discussion! > Cos > >> On Wed, Aug 28, 2019 at 04:12PM, Jun HE wrote: >> Hi, folks, >> >> I went through current components Bigtop is supporting, and I noticed that >> these upstream projects haven't been released for quite a while: >> Apache Tajo: >>last release: release-0.11.3-rc0, May 11, 2016 >> Apache Apex: >>last release: v3.7.0, Apr 27, 2018 >> Apache Giraph: >>last release: rel/1.2.0-RC1, Oct 13 2016 >> Apache Hama: >>last release: 0.7.1-RC2, Mar 12, 2016 >> Apache Sqoop: >>last release: release-1.4.7-rc0, Dec 6, 2017; release-1.99.7-rc1, Jul >> 20, 2016 >> >> And some of them seem to be in slow development: >> Apache Tajo: >>last commit: Jul 13, 2018 >> Apache Apex: >>last commit: Jun 20, 2018 >> Apache Hama: >>last commit: Jul 30, 2018 >> >> So I'm wondering whether we should continue support for these components >> (or part of them) in next/future releases. >> >> I understand that similar topics were discussed before. But, you know, this >> is an quickly evolving world, maybe it worth another revisit now? ;) >> >> Regards, >> >> Jun
