+1, assuming we're keeping RTC on the branch (, Jakob said pedantically).
On 12 February 2015 at 08:25, Milinda Pathirage <mpath...@umail.iu.edu> wrote:
> I am +1 for this approach.
>
> Milinda
>
> On Thu, Feb 12, 2015 at 11:22 AM, Chris Riccomini <criccom...@apache.org>
> wrote:
>
>> Hey Julian,
>>
>> > The rule of thumb I’ve learned integrating Calcite into Hive is that only
>> a branch should depend on snapshot versions of another component
>>
>> This is very practical. I hadn't considered the Calcite dependency issue
>> when I pushed for merging into master rather than a branch.
>>
>> Based on this, it seems like the best thing to do is:
>>
>> 1) Work off of Calcite SNAPSHOT Maven publication.
>> 2) Create a samza-sql branch.
>> 3) Migrate all existing samza-sql commits from master into the samza-sql
>> branch.
>>
>> Sorry for the churn on this all. I hadn't considered the binary dependency
>> issue. Is everyone OK with this?
>>
>> Cheers,
>> Chris
>>
>> On Wed, Feb 11, 2015 at 10:51 AM, Julian Hyde <jul...@hydromatic.net>
>> wrote:
>>
>> > This seems more like a branch than a classifier. Calcite is developing in
>> > a branch, and would produce snapshots from that branch. The rule of thumb
>> > I’ve learned integrating Calcite into Hive is that only a branch should
>> > depend on snapshot versions of another component. (Hive broke this rule
>> and
>> > got into a sticky situation where their trunk depended on a Calcite
>> > snapshot and they couldn’t release because Calcite was stuck in
>> pre-release
>> > vote limbo.)
>> >
>> > I’m going to try to produce maven artifacts with {groupId:
>> > “org.apache.calcite”, version: "1.1.0-stream-incubating-SNAPSHOT”,
>> > artifactId: “calcite-core”, “calcite-avatica”, etc.} and push them to
>> > Apache nexus. I do recommend that you use it on a branch.
>> >
>> > However, I don’t think there would be a problem getting the functionality
>> > into calcite’s master branch in time for Calcite’s next release, in
>> about 4
>> > weeks. (We commit to releasing approximately monthly.) The functionality
>> > and APIs would be marked “experimental, subject to change” but at least
>> you
>> > would be able to merge your work to master at that point. Doubtless you
>> > would need new features & bug fixes in Calcite after that, and so work
>> that
>> > depended on that would have to stay in a branch until Calcite was
>> released.
>> >
>> > Julian
>> >
>> > On Feb 11, 2015, at 10:34 AM, Felix GV <fville...@linkedin.com.INVALID>
>> > wrote:
>> >
>> > > I guess one concern with Calcite SNAPSHOT releases is that they imply
>> > they'll be included in the next release, which may not be the case if
>> > they're still considered very experimental.
>> > >
>> > > Alternatively, perhaps Julian can create a new version name with a
>> > classifier for streaming.
>> > >
>> > >
>> >
>> http://maven.apache.org/plugins/maven-deploy-plugin/examples/deploying-with-classifiers.html
>> > >
>> > > Then he could release SNAPSHOT artifacts for both the main/master code
>> > base as well as the streaming branch, no matter how experimental it is...
>> > >
>> > > --
>> > >
>> > > Felix GV
>> > > Data Infrastructure Engineer
>> > > Distributed Data Systems
>> > > LinkedIn
>> > >
>> > > f...@linkedin.com
>> > > linkedin.com/in/felixgv
>> > >
>> > > ________________________________________
>> > > From: Chris Riccomini [criccom...@apache.org]
>> > > Sent: Wednesday, February 11, 2015 10:23 AM
>> > > To: Chris Riccomini
>> > > Cc: dev@samza.apache.org
>> > > Subject: Re: [DISCUSS] SQL workflow
>> > >
>> > > Hey Julian,
>> > >
>> > > It looks like Calcite is at least setup to publish to Maven:
>> > >
>> > > https://repository.apache.org/#nexus-search;quick~calcite
>> > >
>> > > Julian, can you publish SNAPSHOT releases with your streaming changes
>> to
>> > it?
>> > >
>> > > Thanks!
>> > > Chris
>> > >
>> > > On Wed, Feb 11, 2015 at 10:08 AM, Chris Riccomini <
>> criccom...@apache.org
>> > >
>> > > wrote:
>> > >
>> > >> Hey Milinda,
>> > >>
>> > >> I've just committed SAMZA-484, so you should be able to re-base and
>> get
>> > >> all the latest code.
>> > >>
>> > >>> But we need to add Calcite as a dependency and Calcite streaming
>> > >> support is only in Julian's branch
>> > >>
>> > >> We'll have to have Julian publish a release to Apache. This can be a
>> > >> SNAPSHOT to Apache's SNAPSHOT repo, or a real Calcite release. We
>> can't
>> > >> check in binaries to the repo (licensing, release complexity, and just
>> > >> generally a bad idea). We'll have to coordinate with him.
>> > >>
>> > >> Cheers,
>> > >> Chris
>> > >>
>> > >> On Wed, Feb 11, 2015 at 9:43 AM, Milinda Pathirage <
>> > mpath...@umail.iu.edu>
>> > >> wrote:
>> > >>
>> > >>> No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add Calcite
>> > as a
>> > >>> dependency and Calcite streaming support is only in Julian's branch (
>> > >>> https://github.com/julianhyde/incubator-calcite/tree/chi). What can
>> > we do
>> > >>> about that?
>> > >>>
>> > >>> Milinda
>> > >>>
>> > >>> On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini <
>> > criccom...@apache.org>
>> > >>> wrote:
>> > >>>
>> > >>>> Hey Milinda,
>> > >>>>
>> > >>>> That'd be great. Do you have any dependency on SAMZA-484? I haven't
>> > had
>> > >>> a
>> > >>>> chance to commit that one yet.
>> > >>>>
>> > >>>> Cheers,
>> > >>>> Chris
>> > >>>>
>> > >>>> On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <
>> > >>> mpath...@umail.iu.edu>
>> > >>>> wrote:
>> > >>>>
>> > >>>>> /need/want
>> > >>>>>
>> > >>>>> Milinda
>> > >>>>>
>> > >>>>> On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <
>> > >>>> mpath...@umail.iu.edu
>> > >>>>>>
>> > >>>>> wrote:
>> > >>>>>
>> > >>>>>> Hi Chris,
>> > >>>>>>
>> > >>>>>> Do you need me to create the SAMZA-483 patch against latest master
>> > >>> with
>> > >>>>>> SAMZA-482 patch? I think that will make it easier to review the
>> > >>> patch?
>> > >>>>>>
>> > >>>>>> Thanks
>> > >>>>>> Milinda
>> > >>>>>>
>> > >>>>>> On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <
>> > >>>> criccom...@apache.org>
>> > >>>>>> wrote:
>> > >>>>>>
>> > >>>>>>> Cool. Looks like we've got consensus. I'll move forward with RTC
>> on
>> > >>>> some
>> > >>>>>>> of
>> > >>>>>>> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
>> > >>>>>>>
>> > >>>>>>> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <yanfang...@gmail.com>
>> > >>>> wrote:
>> > >>>>>>>
>> > >>>>>>>> +1 on this.
>> > >>>>>>>>
>> > >>>>>>>> Fang, Yan
>> > >>>>>>>> yanfang...@gmail.com
>> > >>>>>>>> +1 (206) 849-4108
>> > >>>>>>>>
>> > >>>>>>>> On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
>> > >>>>> criccom...@apache.org>
>> > >>>>>>>> wrote:
>> > >>>>>>>>
>> > >>>>>>>>> Hey all,
>> > >>>>>>>>>
>> > >>>>>>>>> Are we +1 on this? I think Jakob was the only one who was
>> > >>> curious
>> > >>>>>>> about
>> > >>>>>>>> it.
>> > >>>>>>>>>
>> > >>>>>>>>> Cheers,
>> > >>>>>>>>> Chris
>> > >>>>>>>>>
>> > >>>>>>>>> On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <nickpa...@gmail.com>
>> > >>>> wrote:
>> > >>>>>>>>>
>> > >>>>>>>>>> Hi, Jakob,
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>>> Eh? Not sure what this means...
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> I mean SAMZA-484 depends on SAMZA-482, and neither are
>> > >>>>> committed.
>> > >>>>>>> So
>> > >>>>>>>>>> Navina
>> > >>>>>>>>>>> is having to post Yi's patch, as well as her own, on the
>> > >>> JIRA.
>> > >>>>> It
>> > >>>>>>>> makes
>> > >>>>>>>>>> it
>> > >>>>>>>>>>> really hard to do code reviews because you can't tell
>> > >>> whether
>> > >>>> Yi
>> > >>>>>>> made
>> > >>>>>>>>> the
>> > >>>>>>>>>>> changes or Navina did.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>> Just to add to the point. It is also difficult to always see
>> > >>> a
>> > >>>>> long
>> > >>>>>>>> list
>> > >>>>>>>>> of
>> > >>>>>>>>>> changed files if the RB request is always based on the
>> > >>> master.
>> > >>>> It
>> > >>>>> is
>> > >>>>>>>>>> possible to have RB request based on another RB request (I
>> > >>> have
>> > >>>>>>> tried
>> > >>>>>>>> it
>> > >>>>>>>>>> before). But what happens if the base RB request is
>> > >>>>>>>> cancelled/discarded?
>> > >>>>>>>>> RB
>> > >>>>>>>>>> is not designed to track the revision changes in a dependency
>> > >>>>> chain.
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>> Cheers,
>> > >>>>>>>>>>> Chris
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <
>> > >>>> jgho...@gmail.com
>> > >>>>>>
>> > >>>>>>>>> wrote:
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>>> I want to avoid branches,
>> > >>>>>>>>>>>> Just curious, any reason for this?
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>> and I also want to avoid revision control over JIRA
>> > >>>>>>>>>>>> Eh? Not sure what this means...
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Thanks,
>> > >>>>>>>>>>>> jg
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> On 4 February 2015 at 17:11, Chris Riccomini <
>> > >>>>>>>> criccom...@apache.org>
>> > >>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>> Hey all,
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> @Jakob, yeah I was thinking we'll follow our normal
>> > >>> flow.
>> > >>>>>>> RTC. I
>> > >>>>>>>>> just
>> > >>>>>>>>>>>>> wanted to set expectation that the code committed
>> > >>> might be
>> > >>>>>>> not up
>> > >>>>>>>>> to
>> > >>>>>>>>>>> our
>> > >>>>>>>>>>>>> normal quality initially (missing docs, no tests, etc).
>> > >>>>> Until
>> > >>>>>>> the
>> > >>>>>>>>>>> quality
>> > >>>>>>>>>>>>> is raised, we should think of this module as
>> > >>> experimental.
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> @Milinda, awesome! Thanks. :)
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> Cheers,
>> > >>>>>>>>>>>>> Chris
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
>> > >>>>>>>>>>>> mpath...@umail.iu.edu>
>> > >>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Hi Chris,
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Hope we no longer need the SQL API. I'll create a RB
>> > >>> for
>> > >>>>>>> Calcite
>> > >>>>>>>>>>>>>> integration.
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Thanks
>> > >>>>>>>>>>>>>> Milinda
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
>> > >>>>>>>>>>> criccom...@apache.org>
>> > >>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> I think so. There was some RB downtime, but it just
>> > >>> got
>> > >>>>>>> fixed.
>> > >>>>>>>>> Yi,
>> > >>>>>>>>>>>>>> Navina,
>> > >>>>>>>>>>>>>>> Milinda, can you make sure your JIRAs have up to
>> > >>> date
>> > >>>>> RBs?
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:24 AM, sriram <
>> > >>>>>>> sriram....@gmail.com
>> > >>>>>>>>>
>> > >>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> Can we have updated RBs for all the three sub
>> > >>> tasks
>> > >>>>>>> before
>> > >>>>>>>> we
>> > >>>>>>>>>>>> commit?
>> > >>>>>>>>>>>>>>> This
>> > >>>>>>>>>>>>>>>> would help us to review even after we commit.
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
>> > >>>>>>>>>>>>>> criccom...@apache.org>
>> > >>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Hey all,
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Yi, Navina, and Milinda have been working on
>> > >>>>> SAMZA-390
>> > >>>>>>>>>>> sub-tickets
>> > >>>>>>>>>>>>>>>> related
>> > >>>>>>>>>>>>>>>>> to SQL operators. We're getting to the point
>> > >>> where
>> > >>>>> the
>> > >>>>>>>>> amount
>> > >>>>>>>>>> of
>> > >>>>>>>>>>>> work
>> > >>>>>>>>>>>>>>>>> floating around is quite large, and some tickets
>> > >>>>> build
>> > >>>>>>> off
>> > >>>>>>>>> of
>> > >>>>>>>>>>>> others.
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> I'm proposing that we commit this work into a
>> > >>>>> samza-sql
>> > >>>>>>>>>>> submodule
>> > >>>>>>>>>>>> on
>> > >>>>>>>>>>>>>>>>> master, and treat this module as experimental. I
>> > >>>> want
>> > >>>>>>> to
>> > >>>>>>>>> avoid
>> > >>>>>>>>>>>>>>> branches,
>> > >>>>>>>>>>>>>>>>> and I also want to avoid revision control over
>> > >>>> JIRA.
>> > >>>>>>> This
>> > >>>>>>>>>> means
>> > >>>>>>>>>>>> that
>> > >>>>>>>>>>>>>>>> there
>> > >>>>>>>>>>>>>>>>> will probably be a fair amount of commits/JIRAs
>> > >>> on
>> > >>>>> this
>> > >>>>>>>>> module
>> > >>>>>>>>>>> as
>> > >>>>>>>>>>>> we
>> > >>>>>>>>>>>>>>>>> iterate, but I think that's OK.
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Does this sound good to everyone?
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Cheers,
>> > >>>>>>>>>>>>>>>>> Chris
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> --
>> > >>>>>>>>>>>>>> Milinda Pathirage
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> PhD Student | Research Assistant
>> > >>>>>>>>>>>>>> School of Informatics and Computing | Data to Insight
>> > >>>>> Center
>> > >>>>>>>>>>>>>> Indiana University
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> twitter: milindalakmal
>> > >>>>>>>>>>>>>> skype: milinda.pathirage
>> > >>>>>>>>>>>>>> blog: http://milinda.pathirage.org
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>
>> > >>>>>>
>> > >>>>>>
>> > >>>>>>
>> > >>>>>> --
>> > >>>>>> Milinda Pathirage
>> > >>>>>>
>> > >>>>>> PhD Student | Research Assistant
>> > >>>>>> School of Informatics and Computing | Data to Insight Center
>> > >>>>>> Indiana University
>> > >>>>>>
>> > >>>>>> twitter: milindalakmal
>> > >>>>>> skype: milinda.pathirage
>> > >>>>>> blog: http://milinda.pathirage.org
>> > >>>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>> --
>> > >>>>> Milinda Pathirage
>> > >>>>>
>> > >>>>> PhD Student | Research Assistant
>> > >>>>> School of Informatics and Computing | Data to Insight Center
>> > >>>>> Indiana University
>> > >>>>>
>> > >>>>> twitter: milindalakmal
>> > >>>>> skype: milinda.pathirage
>> > >>>>> blog: http://milinda.pathirage.org
>> > >>>>>
>> > >>>>
>> > >>>
>> > >>>
>> > >>>
>> > >>> --
>> > >>> Milinda Pathirage
>> > >>>
>> > >>> PhD Student | Research Assistant
>> > >>> School of Informatics and Computing | Data to Insight Center
>> > >>> Indiana University
>> > >>>
>> > >>> twitter: milindalakmal
>> > >>> skype: milinda.pathirage
>> > >>> blog: http://milinda.pathirage.org
>> > >>>
>> > >>
>> > >>
>> >
>> >
>>
>
>
>
> --
> Milinda Pathirage
>
> PhD Student | Research Assistant
> School of Informatics and Computing | Data to Insight Center
> Indiana University
>
> twitter: milindalakmal
> skype: milinda.pathirage
> blog: http://milinda.pathirage.org
