Sounds great! I like the idea that reshaping big data architecture at cloud native deployments. What about creating 'branch-1' from current status for existing users and traditional architectures? So, we can make progress from master branch for the future Bigtop releases.
Jay, I feel like we should create a JIRA issue or shared doc to keep discussions on detailed plan and design docs for Bigtop NG. Thanks, Youngwoo On Fri, Aug 30, 2019 at 10:28 AM Jun HE <ju...@apache.org> wrote: > Super +1 for Bigtop 2.0 with modern components and K8S support! I cannot > wait to see Bigtop running on K8S... > So can someone help to create a branch-2.0 now? :P > > > Konstantin Boudnik <c...@apache.org> 于2019年8月30日周五 上午2:24写道： > > > Aah, k8s - thanks Jay! > > > > BTW, I believe we can do this in a less formal stuff (ie no VOTE): we > have > > a > > discussion going on the version of Bigtop. How about we make it 2.0 and > > trim > > the BOM into the needed shape and form? If there's enough drive in the > > community to continue with 1.x line (1.4 and so on), it can be done as a > > parallel effort. > > > > Thoughts? > > Cos > > > > On Fri, Aug 30, 2019 at 01:26AM, Evans Ye wrote: > > > I'm super +1 with K8S stuff and the core components idea! > > > > > > I'm with Cos and Jun. Removing those stuffs can ease our CI resource > > > utilization and yield more stable CI results. This is what I'm thinking > > how > > > to proceed based on the previous feedback that a drop of supporting > > > component should be carefully discussed > > > > > > 1. Spin up a vote to drop project XXX (if confident enough, put > multiple > > > components here) > > > 2. If +3 binding votes with no -1 votes, it passes. > > > 3. Put up the PR for code removing > > > 4. Refine our CI settings (I can definitely help with this part) > > > > > > Best, > > > Evans > > > > > > > > > > > > > > > Jay Vyas <jayunit100.apa...@gmail.com> 於 2019年8月29日 週四 下午7:26寫道： > > > > > > > First of all I’m all for dropping the old stuff. > > > > > > > > 1) My opinion - to further cos point - I think people running old > stuff > > > > don’t really need version updates, or if they do, they don’t > > constitute a > > > > large audience , or should contribute them ok their own. > > > > > > > > 2) surprise surprise :):) here’s my k8s native. View - we should move > > the > > > > old components into sustaining mode, and rebuild a new bigtop focus > > solely > > > > on spark , NiFi , Presto (with a lot of attention to minimal > standalone > > > > Hadoop w/ a Hive connector ) and target it at kubernetes native > > deployments > > > > :). > > > > > > > > I can make it so the k8s part is an impl detail so users can be > mostly > > > > decoupled from it . > > > > > > > > I’ve been integration testing various things in the bigdata ecosystem > > on > > > > k8s and there is a lot of demand without anyone owning the > integration. > > > > > > > > > On Aug 29, 2019, at 2:39 AM, Konstantin Boudnik <c...@apache.org> > > wrote: > > > > > > > > > > I am not sure about Giraph, Tajo and some others, but Sqoop seems > to > > be > > > > user > > > > > around by people. So, if it isn't much of the burden for us - and > it > > > > seems > > > > > pretty stable at the moment - I'd leave it. > > > > > > > > > > What I would think would makes sense to spend some of our efforts > on > > is > > > > on > > > > > adding modern tooling like Nifi/Airflow into the mix. As you said - > > > > things > > > > > move forward pretty fast, and we seem to be sticking to the some of > > the > > > > old > > > > > stuff. > > > > > > > > > > Thanks for starting this discussion! > > > > > Cos > > > > > > > > > >> On Wed, Aug 28, 2019 at 04:12PM, Jun HE wrote: > > > > >> Hi, folks, > > > > >> > > > > >> I went through current components Bigtop is supporting, and I > > noticed > > > > that > > > > >> these upstream projects haven't been released for quite a while: > > > > >> Apache Tajo: > > > > >> last release: release-0.11.3-rc0, May 11, 2016 > > > > >> Apache Apex: > > > > >> last release: v3.7.0, Apr 27, 2018 > > > > >> Apache Giraph: > > > > >> last release: rel/1.2.0-RC1, Oct 13 2016 > > > > >> Apache Hama: > > > > >> last release: 0.7.1-RC2, Mar 12, 2016 > > > > >> Apache Sqoop: > > > > >> last release: release-1.4.7-rc0, Dec 6, 2017; > > release-1.99.7-rc1, Jul > > > > >> 20, 2016 > > > > >> > > > > >> And some of them seem to be in slow development: > > > > >> Apache Tajo: > > > > >> last commit: Jul 13, 2018 > > > > >> Apache Apex: > > > > >> last commit: Jun 20, 2018 > > > > >> Apache Hama: > > > > >> last commit: Jul 30, 2018 > > > > >> > > > > >> So I'm wondering whether we should continue support for these > > components > > > > >> (or part of them) in next/future releases. > > > > >> > > > > >> I understand that similar topics were discussed before. But, you > > know, > > > > this > > > > >> is an quickly evolving world, maybe it worth another revisit now? > ;) > > > > >> > > > > >> Regards, > > > > >> > > > > >> Jun > > > > > > >
