Re: CALCITE-2458 Use of Kotlin for unit tests
+0.5 I haven't used Kotlin but I've had enough exposure that I'm very interested in the idea. The main reason for the lack of a +1 is a concern that it may prove a barrier to new contributors but if others are for it, it's the excuse I've been looking for to finally write some Kotlin. I think test code is just the place to introduce something like this. Anything that makes test code less verbose is a good way of encouraging more tests to be written :) -- Michael Mior mm...@apache.org Le lun. 17 sept. 2018 à 15:21, Vladimir Sitnikov < sitnikov.vladi...@gmail.com> a écrit : > Hi, > > I think it time for us to enable use of Kotlin in unit tests. > There are lots of language features (e.g. name parameters, data classes, > multiline strings, infix functions, nullability, smart casts, etc) that > would make code simpler to read and write. > > Note: the change is related to use of Kotlin in test-code only. > Kotlin use for production classes might require a bit more of thought (e.g. > the set of dependencies, etc), so let's settle on Kotlin for tests first. > > More details can be found in: > https://issues.apache.org/jira/browse/CALCITE-2458 (created Aug 8) > PR: https://github.com/apache/calcite/pull/786 (created Aug 9) > > If no-one objects within a week, I'll assume lazy consensus and commit it. > > It might be nice to gather a bit more feedback on the change though. What > do you think of the change? > > If you struggle what to say, you might find the below options useful: > > [ ] ++1: 'Wow! I like this! Let's do it!' > [ ] +1: 'Let's do it!' > [ ] +0.9: 'This is a cool idea and i like it' > > Vladimir >
Re: CALCITE-2458 Use of Kotlin for unit tests
-0 I kind of share Michael's concern that it might be a barrier, and the provided patch doesn't really showcase the announced benefits, and I would be fairly careful when introducing a new language into the project (the patch for example allows compiling production code with kotlin notably). Is there a good test class which is fairly complex in Java but could be written nicely in Kotlin? What about IDE support (editor, debugger, ...)? No direct exposure to Kotlin, but I did some good chunk of Scala where the same kind of benefits were advertised, but I remember some of my colleagues new to the language struggling with contribution (although you could read the code as an english sentence...), and so just being careful. On Mon, Sep 17, 2018 at 12:47 PM Michael Mior wrote: > +0.5 I haven't used Kotlin but I've had enough exposure that I'm very > interested in the idea. The main reason for the lack of a +1 is a concern > that it may prove a barrier to new contributors but if others are for it, > it's the excuse I've been looking for to finally write some Kotlin. I think > test code is just the place to introduce something like this. Anything that > makes test code less verbose is a good way of encouraging more tests to be > written :) > > -- > Michael Mior > mm...@apache.org > > > Le lun. 17 sept. 2018 à 15:21, Vladimir Sitnikov < > sitnikov.vladi...@gmail.com> a écrit : > > > Hi, > > > > I think it time for us to enable use of Kotlin in unit tests. > > There are lots of language features (e.g. name parameters, data classes, > > multiline strings, infix functions, nullability, smart casts, etc) that > > would make code simpler to read and write. > > > > Note: the change is related to use of Kotlin in test-code only. > > Kotlin use for production classes might require a bit more of thought > (e.g. > > the set of dependencies, etc), so let's settle on Kotlin for tests first. > > > > More details can be found in: > > https://issues.apache.org/jira/browse/CALCITE-2458 (created Aug 8) > > PR: https://github.com/apache/calcite/pull/786 (created Aug 9) > > > > If no-one objects within a week, I'll assume lazy consensus and commit > it. > > > > It might be nice to gather a bit more feedback on the change though. What > > do you think of the change? > > > > If you struggle what to say, you might find the below options useful: > > > > [ ] ++1: 'Wow! I like this! Let's do it!' > > [ ] +1: 'Let's do it!' > > [ ] +0.9: 'This is a cool idea and i like it' > > > > Vladimir > > >
Re: CALCITE-2458 Use of Kotlin for unit tests
I have been using Kotlin for a few months and really like it. It doesn't stray too far from Java except for a few places chosen to give maximum benefit. I think that Scala went too far, and produced a language in which you can easily write dense, obscure programs; Kotlin has learned from Scala's mistakes. But I set a high bar to adopting a new framework or language. Once code is written, it is in the project forever, and needs to be maintained. A large majority of contributors need to be comfortable reading, editing, and running such code. I have regrets about allowing Mockito into Calcite (it is only used in the Druid adapter) and JMockit into SQLLine. I am not familiar with these frameworks, don't have time to learn, and consequently cannot modify the tests. (I can't even figure out how to even run the JMockit-based test in SQLLine.) So, I'd like to hear from Calcite contributors - would you be prepared to take the time to learn Kotlin? (If you know what is a companion object and a receiver, I'd say you're an advanced Kotlin user.) On Mon, Sep 17, 2018 at 5:50 PM Laurent Goujon wrote: > > -0 I kind of share Michael's concern that it might be a barrier, and the > provided patch doesn't really showcase the announced benefits, and I would > be fairly careful when introducing a new language into the project (the > patch for example allows compiling production code with kotlin notably). Is > there a good test class which is fairly complex in Java but could be > written nicely in Kotlin? What about IDE support (editor, debugger, ...)? > > No direct exposure to Kotlin, but I did some good chunk of Scala where the > same kind of benefits were advertised, but I remember some of my colleagues > new to the language struggling with contribution (although you could read > the code as an english sentence...), and so just being careful. > > On Mon, Sep 17, 2018 at 12:47 PM Michael Mior wrote: > > > +0.5 I haven't used Kotlin but I've had enough exposure that I'm very > > interested in the idea. The main reason for the lack of a +1 is a concern > > that it may prove a barrier to new contributors but if others are for it, > > it's the excuse I've been looking for to finally write some Kotlin. I think > > test code is just the place to introduce something like this. Anything that > > makes test code less verbose is a good way of encouraging more tests to be > > written :) > > > > -- > > Michael Mior > > mm...@apache.org > > > > > > Le lun. 17 sept. 2018 à 15:21, Vladimir Sitnikov < > > sitnikov.vladi...@gmail.com> a écrit : > > > > > Hi, > > > > > > I think it time for us to enable use of Kotlin in unit tests. > > > There are lots of language features (e.g. name parameters, data classes, > > > multiline strings, infix functions, nullability, smart casts, etc) that > > > would make code simpler to read and write. > > > > > > Note: the change is related to use of Kotlin in test-code only. > > > Kotlin use for production classes might require a bit more of thought > > (e.g. > > > the set of dependencies, etc), so let's settle on Kotlin for tests first. > > > > > > More details can be found in: > > > https://issues.apache.org/jira/browse/CALCITE-2458 (created Aug 8) > > > PR: https://github.com/apache/calcite/pull/786 (created Aug 9) > > > > > > If no-one objects within a week, I'll assume lazy consensus and commit > > it. > > > > > > It might be nice to gather a bit more feedback on the change though. What > > > do you think of the change? > > > > > > If you struggle what to say, you might find the below options useful: > > > > > > [ ] ++1: 'Wow! I like this! Let's do it!' > > > [ ] +1: 'Let's do it!' > > > [ ] +0.9: 'This is a cool idea and i like it' > > > > > > Vladimir > > > > >
Re: CALCITE-2458 Use of Kotlin for unit tests
Julian>I'd like to hear from Calcite contributors - would you be prepared Julian>to take the time to learn Kotlin? We do not force everybody to write tests in Kotlin. However, writing tests in Kotlin might be a fun way of studying both Kotlin and Calcite which might be a great option for contributors. That is exactly what Michael is saying: Michael> it's the excuse I've been looking for to finally write some Kotlin. TL;DR of below: I would agree that adding a new technology is a thing. However I would disagree with both of the provided examples (SQLLine+JMockit; Druid+Mockito) after I have studied the relevant code. Julian>Once code is written, it is in the project forever The code can be removed. Julian>I can't even figure out how to even run the Julian>JMockit-based test in SQLLine. Julian, would you please care explain a bit more here? Your words might sound like "I can't figure out how to even run Kotlin-based test in Calcite", however as my PR shows, regular `mvn test` just works. I can easily run and debug tests from within IDEA. I've just downloaded SQLLine code, and SqlLineArgsTest runs as a part of mvn test, so what's the problem? Note: there's a failure (see https://github.com/julianhyde/sqlline/issues/154 ) which is NOT related to JMockit. Julian>I have regrets about allowing Mockito into Calcite (it is only used in Julian>the Druid adapter) DruidQueryFilterTest is the only test that uses Mockito. Apparently the test is trying to test STATIC DruidJsonFilter.toDruidFilters method, however due to some coding issues that method still requires certain (unknown) bits of DruidQuery which is extremely hard to instantiate. Does that mean Mockito is to blame there? I wouldn't say so. Mockito helps there to cut corners. Would it be better if we've spend 2-4+months on DruidQuery/DruidJsonFilter code design and review that could be testable without Mockito/JMockit? I doubt so. On top of that, the whole Druid thing is of much-much higher barrier to me rather than simple things like Mockito/JMockit/Java/Kotlin. Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
Il mar 18 set 2018, 09:42 Vladimir Sitnikov ha scritto: > Julian>I'd like to hear from Calcite contributors - would you be prepared > Julian>to take the time to learn Kotlin? > > We do not force everybody to write tests in Kotlin. > However, writing tests in Kotlin might be a fun way of studying both Kotlin > and Calcite which might be a great option for contributors. > > That is exactly what Michael is saying: > > Michael> it's the excuse I've been looking for to finally write some > Kotlin. > > > TL;DR of below: I would agree that adding a new technology is a thing. > However I would disagree with both of the provided examples > (SQLLine+JMockit; Druid+Mockito) after I have studied the relevant code. > > Julian>Once code is written, it is in the project forever > > The code can be removed. > > Julian>I can't even figure out how to even run the > Julian>JMockit-based test in SQLLine. > > Julian, would you please care explain a bit more here? > > Your words might sound like "I can't figure out how to even run > Kotlin-based test in Calcite", > however as my PR shows, regular `mvn test` just works. I can easily run and > debug tests from within IDEA. > > I've just downloaded SQLLine code, and SqlLineArgsTest runs as a part of > mvn test, so what's the problem? > Note: there's a failure (see > https://github.com/julianhyde/sqlline/issues/154 ) which is NOT related to > JMockit. > > Julian>I have regrets about allowing Mockito into Calcite (it is only used > in > Julian>the Druid adapter) > > DruidQueryFilterTest is the only test that uses Mockito. > Apparently the test is trying to test STATIC DruidJsonFilter.toDruidFilters > method, however > due to some coding issues that method still requires certain (unknown) bits > of DruidQuery which is extremely hard to instantiate. > > Does that mean Mockito is to blame there? I wouldn't say so. > Mockito helps there to cut corners. > > Would it be better if we've spend 2-4+months on DruidQuery/DruidJsonFilter > code design and review that could be testable without Mockito/JMockit? > I doubt so. > > On top of that, the whole Druid thing is of much-much higher barrier to me > rather than > simple things like Mockito/JMockit/Java/Kotlin. > I think that using Mockito/PowerMock nowadays is standard practice in modern Java libraries and application development. It is not easy but it is a very useful tool. So +10 from me. I have never used Kotlin, but when you introduce it, it is like forever, if you what to drop it you have to rewrite tests and no one has time and resources to spend on such activities. I don't know if Kotlin is enough widespread to say that it won't be a problem for new contributors. When you are new you already have to enter the world of new concepts and learn how the test suite are organised... I sponsor the idea of having domain specific test languages like QUIDEM which make it simpler to write complex assertions in this SQL/Calcite environment. So from me it is more like a -0 for Kotlin now My two cents Enrico > Vladimir > -- -- Enrico Olivelli
Re: CALCITE-2458 Use of Kotlin for unit tests
Vladimir, You are accusing Enrico of being a hypocrite. Ad hominem attacks have no place on this list. Please withdraw the remark and apologize. Your point has merit, but there is a way to make it without making it personal. Julian > On Sep 19, 2018, at 8:18 AM, Vladimir Sitnikov > wrote: > > Enrico>I don't know if Kotlin is enough widespread to say that it won't be a > Enrico>problem for new contributors. > > Enrico>I sponsor the idea of having domain specific test languages like > QUIDEM > Enrico>which make it simpler > > It is truly wonderful how you question "if Kotlin is widespread or not", > and sponsor QUIDEM at the same time. > I'm absolutely sure QUIDEM is NOT enough widespread, and it does create > problems for new contributors. > > Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
Just ran across this thread. I'm not as active on Calcite these days due to other demands on my time but this thread seems important enough to chime in. My standard response is -1 to introducing new languages. They are frequently of the moment (and their features are often adopted by other languages). They also tend be a barrier to additional adoption and contribution (although sometimes they engage a new community). They also risk lack of adoption (for example, in the Arrow project typically only a small set of contributors work on each language meaning if someone goes on vacation--or stops contributing--that part of the codebase is virtually unmaintained for some period of time). However, the biggest issue to me is that the current contributors are frequently required to maintain other people's code and thus all of a sudden a bunch of contributors will probably start to see part of the codebase as less approachable than they previously did. We already have this problem in Calcite because there is so much code managed by a very small set of contributors. Engineers always have good intentions and most people like the idea of learning a new language. I have no doubt that most of the people on this list (including myself) would like to learn Kotlin. At the same time, engineers are also almost always overly optimistic (again, including myself) and don't really have the time to learn a new language at any depth. My suggestion might be (at most) to do a trial run with a very constrained use case that is a tertiary part of the codebase. For example, maybe do it in one connector (for example the pig connector) and see how that works for some period of time. If people start to see the benefit there, then maybe it makes sense to adopt more broadly. Because of my lack of overall contribution to development on Calcite these days, I don't want to entirely block this proposal so I'll put my formal vote on this commit at -0.9. By this I mean I'd be -1 if I was more engaged but don't think it would be fair to block this from moving forward since others work way more on the codebase than I do atm. On Mon, Sep 17, 2018 at 12:21 PM Vladimir Sitnikov < sitnikov.vladi...@gmail.com> wrote: > Hi, > > I think it time for us to enable use of Kotlin in unit tests. > There are lots of language features (e.g. name parameters, data classes, > multiline strings, infix functions, nullability, smart casts, etc) that > would make code simpler to read and write. > > Note: the change is related to use of Kotlin in test-code only. > Kotlin use for production classes might require a bit more of thought (e.g. > the set of dependencies, etc), so let's settle on Kotlin for tests first. > > More details can be found in: > https://issues.apache.org/jira/browse/CALCITE-2458 (created Aug 8) > PR: https://github.com/apache/calcite/pull/786 (created Aug 9) > > If no-one objects within a week, I'll assume lazy consensus and commit it. > > It might be nice to gather a bit more feedback on the change though. What > do you think of the change? > > If you struggle what to say, you might find the below options useful: > > [ ] ++1: 'Wow! I like this! Let's do it!' > [ ] +1: 'Let's do it!' > [ ] +0.9: 'This is a cool idea and i like it' > > Vladimir >
Re: CALCITE-2458 Use of Kotlin for unit tests
Julian>You are accusing Enrico of being a hypocrite. Ad hominem attacks have no place on this list. I apologize if <> reads as accusing. The only reasoning Enrico provided was 1) "widespread or not" issue. This really surprised me, and I noted that QUIDEM is way less widespread. 2) "if you what to drop it you have to rewrite tests and no one has time and resources to spend on such activities." Even though that is true, it does not look like a valid reason to reject Kotlin. Julian>Please withdraw the remark and apologize. Your point has merit, but there is a way to make it without making it personal. I'm afraid I can't understand what you say here. Would you please elaborate? Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
Il mer 19 set 2018, 19:49 Vladimir Sitnikov ha scritto: > Julian>You are accusing Enrico of being a hypocrite. Ad hominem attacks > have no place on this list. > > I apologize if < widespread or not", and sponsor QUIDEM at the same time>> reads as > accusing. > > The only reasoning Enrico provided was > 1) "widespread or not" issue. This really surprised me, and I noted that > QUIDEM is way less widespread. > 2) "if you what to drop it you have to rewrite tests and no one has time > and resources to spend on such activities." Even though that is true, it > does not look like a valid reason to reject Kotlin. > Vladimir I am sorry, maybe my English was not clear. I was not accusing at all. I would like to learn Kotlin, I am not questioning about it. I was just trying to say that maybe Kotlin is still a new language and contributors and reviewers will need to learn it together with Calcite. About Quidem...to me it is an useful mechanism to write tests in Calcite domain, it is far different from a complete programming language. I appreciate very much that this community wants to be modern Enrico > Julian>Please withdraw the remark and apologize. Your point has merit, but > there is a way to make it without making it personal. > > I'm afraid I can't understand what you say here. Would you please > elaborate? > > Vladimir > -- -- Enrico Olivelli
Re: CALCITE-2458 Use of Kotlin for unit tests
> On Sep 19, 2018, at 10:49 AM, Vladimir Sitnikov > wrote: > > Julian>Please withdraw the remark and apologize. Your point has merit, but > there is a way to make it without making it personal. > > I'm afraid I can't understand what you say here. Would you please elaborate? You can attack his arguments without attacking him. You could have said, for instance, > I do not agree that widespread adoption is a necessary criterion for adopting > a > framework. As you noted, we have adopted Quidem, and it has very limited > adoption. And then you could explain what are the key criteria, in your opinion. Julian
Re: CALCITE-2458 Use of Kotlin for unit tests
Thanks everybody for your comments. I've received no strong objections so far, so I assume lazy consensus. I've merged the PR in https://git-wip-us.apache.org/repos/asf?p=calcite.git;a=commit;h=55d86646fe598d215bd53dbeac2bbbf32a9506f9 Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
That wasn’t consensus. Lazy consensus is sometimes described as “silence gives assent”. There was not silence, there are 12 messages on this thread, several of them negative. Julian > On Sep 24, 2018, at 5:43 PM, Vladimir Sitnikov > wrote: > > Thanks everybody for your comments. > > I've received no strong objections so far, so I assume lazy consensus. > I've merged the PR in > https://git-wip-us.apache.org/repos/asf?p=calcite.git;a=commit;h=55d86646fe598d215bd53dbeac2bbbf32a9506f9 > > Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
The following is for the reference purposes, and there's not a single intend to accuse anybody of doing something (e.g. miscalculations or whatever). My calculations might be off so please correct me if you think that is important. 1) Calcite includes 16 committers + 19 PMC. Most of them were silent. 2) http://mail-archives.apache.org/mod_mbox/calcite-dev/201809.mbox/thread?2 shows (as of time when I create this message) that the thread has 14 messages 5 of which were generated by me 3) Apache voting (see https://www.apache.org/foundation/voting.html#votes-on-code-modification ) explicitly recommends to use whole numbers for voting on code modifications. 4) No-one killed a proposal with a veto 5) Most of replies were around +-0, except Michael who was +0.5 and Jacques who was -0.9 (note that he explicitly declares he does not veto the change) To my best knowledge: Michael Mior +0.5 (binding) Laurent Goujon -0 (binding) Enrico Olivelli -0 Jacques Nadeau -0.9 (binding) Vladimir Sitnikov ++1 (binding) 6) I'm sure we don't really want to discuss this. I suggest we don't spend time on splitting hairs here. Apparently there's not that many people who care enough to spend time on this discussion. Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
You said upfront you were going to use lazy consensus. The discussion didn’t reach consensus. You went ahead anyway. > On Sep 25, 2018, at 2:12 AM, Vladimir Sitnikov > wrote: > > The following is for the reference purposes, and there's not a single > intend to accuse anybody of doing something (e.g. miscalculations or > whatever). > My calculations might be off so please correct me if you think that is > important. > > 1) Calcite includes 16 committers + 19 PMC. Most of them were silent. > > 2) http://mail-archives.apache.org/mod_mbox/calcite-dev/201809.mbox/thread?2 > shows (as of time when I create this message) that the thread has 14 > messages 5 of which were generated by me > > 3) Apache voting (see > https://www.apache.org/foundation/voting.html#votes-on-code-modification ) > explicitly recommends to use whole numbers for voting on code modifications. > > 4) No-one killed a proposal with a veto > > 5) Most of replies were around +-0, except Michael who was +0.5 and Jacques > who was -0.9 (note that he explicitly declares he does not veto the change) > To my best knowledge: > Michael Mior +0.5 (binding) > Laurent Goujon -0 (binding) > Enrico Olivelli -0 > Jacques Nadeau -0.9 (binding) > Vladimir Sitnikov ++1 (binding) > > 6) I'm sure we don't really want to discuss this. I suggest we don't spend > time on splitting hairs here. > Apparently there's not that many people who care enough to spend time on > this discussion. > > Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
Julian>You said upfront you were going to use lazy consensus. I'm afraid I fail to understand what do you mean by consensus. It would probably help if you provided a definition of "consensus being reached" vs "consensus not being reached". We can't (and shouldn't) wait forever for some unknown future opinions. I see "in-doubt" replies, however I don't see "-1" votes. CALCITE-2458 was created on August, 8. It is more than enough to collect objections. Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
Since this is an Apache project, we should use the Apache definition of "lazy consensus." ( https://www.apache.org/foundation/glossary.html#LazyConsensus) > A decision-making policy which assumes general consent if no responses are posted within a defined period. Per this definition, I agree with Julian that lazy consensus was *not* reached on this decision. You're correct that there were no -1 votes, but there were responses, some with reasonable objections. -- Michael Mior mm...@apache.org Le mar. 25 sept. 2018 à 10:08, Vladimir Sitnikov < sitnikov.vladi...@gmail.com> a écrit : > Julian>You said upfront you were going to use lazy consensus. > > I'm afraid I fail to understand what do you mean by consensus. > It would probably help if you provided a definition of "consensus being > reached" vs "consensus not being reached". > > We can't (and shouldn't) wait forever for some unknown future opinions. > > I see "in-doubt" replies, however I don't see "-1" votes. > > CALCITE-2458 was created on August, 8. > It is more than enough to collect objections. > > Vladimir >
Re: CALCITE-2458 Use of Kotlin for unit tests
Apache/voting> A decision-making policy which assumes general consent if no responses are posted within a defined period. Does that mean "absolutely no responses"? I hope no. Does that mean "a single -0 comment destroys lazy consensus"? I hope no. Does that mean "a single +0 comment destroys lazy consensus"? I hope no. Does that mean "any vote less than +1 kills the idea"? I hope no. I read "lazy consensus" more like "whoever does not object counts as the one who approves the idea". E.g. in OpenOffice they even suggest to just commit things and roll back later if there are objections: https://openoffice.apache.org/docs/governance/lazyConsensus.html OpenOffice/lazyConsensus>We have a time machine (Subversion), this means that as long as you commit (or submit patches) early and often the community has plenty of opportunity to indicate disapproval Michael>some with reasonable objections Michael, I can go one by one and discuss each and every "objection", however they do sound very weak. On the other hand, the expression was "-0" in most of the cases which I read as "fear of unknown" rather than true objection. Ex1) Just a bit of an example: Calcite codebase has "Pig" adapter. The tests there are written in, well, Pig language. Was there a all-way-long-research regarding the possibility of inclusion of Pig language to Calcite codebase? I'm sure there were zero researches given. Do Pig test disturb Calcite community? Of course it does. CI Job fails every now and then (see https://issues.apache.org/jira/browse/CALCITE-1751, https://issues.apache.org/jira/browse/CALCITE-1598, etc, etc) Even current CI Job fails VERY often due to some weird results of Pig tests. This greatly reduces the value of Apache Jenkins build reports. The strongest negative vote was from Jacques whose "standard response is -1 to introducing new languages". I wonder what Jacques would say regarding "Pig language introduction to Calcite codebase". Does "Pig language" create a barrier for contributors? I doubt so. Pig is invisible to most of the contributors as long as the tests pass. Unfortunately, Pig fails the build extremely often in builds.apache.org. Ex2) Quidem. As you might know, I've created mat-calcite-plugin, and my co-workers use that tool quite often to analyze Java heap dumps when they troubleshoot OOM and issues alike. Of course they happen to run into Calcite defects/glitches, and what they say me is they struggle/fear to touch Calcite codebase for the following reasons: 1) Existing tests are quite cryptic. Quidem is something that is hard to understand for a person with Java/SQL background. 2) It is not clear how to create a "simple" test case. It's not clear where the test belongs, what are the options, and so on. I can continue. I don't really have intention to blame specific parts of the codebase (especially the code I've created :) ), however I don't buy "Kotlin might impose a barrier for contributors" as a true objection. Kotlin is much closer to Java than say Pig/Cassandra/Geode/CSV. If Kotlin required a specific/cryptic setup from the contributor, then, yes it would be an strong objection to consider. If Kotlin reduced build times twice, then it would again be a strong objection. However, there was not a single strong objection given. Could we please stop making fuzz out of thin air like "voting whether should we enable use of Kotlin in unit tests or not" and proceed with development? PS. I do remember "vetoing" case is still pending, and it was a simple case with trivial technical reasoning. Of course current case is a bit more complicated, however "Kotlin for tests" is very safe bet. Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
1) I don't really understand the comparison with Pig. Calcite has a Pig adapter as a feature and writing tests in Pig is necessary to support that feature. (Same is true with other adapters.) That's not the case with Kotlin. If Calcite had a Kotlin API as a feature, of course we would have Kotlin code. 2) I'm surprised that others find the quidem tests cryptic but if anything, this seems to be another argument against Kotlin. > Could we please stop making fuzz out of thin air like "voting whether > should we enable use of Kotlin in unit tests or not" and proceed with > development? I'm not sure who was calling for a vote (I wasn't and I haven't seen that in this thread) but there seem to be others who don't see this as "out of thin air." I still maintain my +0.5 and I'm fine with the change but I think if we're going to use the term "lazy consensus" we should agree on what it means. -- Michael Mior mm...@apache.org Le mar. 25 sept. 2018 à 16:41, Vladimir Sitnikov < sitnikov.vladi...@gmail.com> a écrit : > Apache/voting> A decision-making policy which assumes general consent if no > responses > are posted within a defined period. > > Does that mean "absolutely no responses"? I hope no. > Does that mean "a single -0 comment destroys lazy consensus"? I hope no. > Does that mean "a single +0 comment destroys lazy consensus"? I hope no. > Does that mean "any vote less than +1 kills the idea"? I hope no. > > I read "lazy consensus" more like "whoever does not object counts as the > one who approves the idea". > E.g. in OpenOffice they even suggest to just commit things and roll back > later if there are objections: > https://openoffice.apache.org/docs/governance/lazyConsensus.html > > OpenOffice/lazyConsensus>We have a time machine (Subversion), this means > that as long as you commit (or submit patches) early and often the > community has plenty of opportunity to indicate disapproval > > Michael>some with reasonable objections > > Michael, I can go one by one and discuss each and every "objection", > however they do sound very weak. > On the other hand, the expression was "-0" in most of the cases which I > read as "fear of unknown" rather than true objection. > > Ex1) > Just a bit of an example: Calcite codebase has "Pig" adapter. The tests > there are written in, well, Pig language. > Was there a all-way-long-research regarding the possibility of inclusion of > Pig language to Calcite codebase? > I'm sure there were zero researches given. > > > Do Pig test disturb Calcite community? Of course it does. > CI Job fails every now and then (see > https://issues.apache.org/jira/browse/CALCITE-1751, > https://issues.apache.org/jira/browse/CALCITE-1598, etc, etc) > Even current CI Job fails VERY often due to some weird results of Pig > tests. > This greatly reduces the value of Apache Jenkins build reports. > > The strongest negative vote was from Jacques whose "standard response is -1 > to introducing new languages". > I wonder what Jacques would say regarding "Pig language introduction to > Calcite codebase". > > Does "Pig language" create a barrier for contributors? I doubt so. Pig is > invisible to most of the contributors as long as the tests pass. > Unfortunately, Pig fails the build extremely often in builds.apache.org. > > Ex2) > Quidem. > > As you might know, I've created mat-calcite-plugin, and my co-workers use > that tool quite often to analyze Java heap dumps when they troubleshoot OOM > and issues alike. > Of course they happen to run into Calcite defects/glitches, and what they > say me is they struggle/fear to touch Calcite codebase for the following > reasons: > 1) Existing tests are quite cryptic. Quidem is something that is hard to > understand for a person with Java/SQL background. > 2) It is not clear how to create a "simple" test case. It's not clear where > the test belongs, what are the options, and so on. > > I can continue. I don't really have intention to blame specific parts of > the codebase (especially the code I've created :) ), however I don't buy > "Kotlin might impose a barrier for contributors" as a true objection. > Kotlin is much closer to Java than say Pig/Cassandra/Geode/CSV. > > If Kotlin required a specific/cryptic setup from the contributor, then, yes > it would be an strong objection to consider. > If Kotlin reduced build times twice, then it would again be a strong > objection. > However, there was not a single strong objection given. > > Could we please stop making fuzz out of thin air like "voting whether > should we enable use of Kotlin in unit tests or not" and proceed with > development? > > PS. I do remember "vetoing" case is still pending, and it was a simple case > with trivial technical reasoning. > Of course current case is a bit more complicated, however "Kotlin for > tests" is very safe bet. > > Vladimir >
Re: CALCITE-2458 Use of Kotlin for unit tests
Michael>if we're going to use the term "lazy consensus" we should agree on what it means Could you clarify what it means? Does that mean "absolutely no responses"? Does that mean "a single -0 comment destroys lazy consensus"? Does that mean "a single +0 comment destroys lazy consensus"? Does that mean "any vote less than +1 kills the idea"? Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
I can clarify my *interpretation* of what it means. My interpretation may not be the definition we collectively agree on. I assume lazy consensus is employed when a change is expected to be potentially controversial. If no objections are raised, then it's safe to assume the change creates no controversy and lazy interpretation. If objections are raised, then I believe lazy consensus has *not* been reached. I wouldn't interpret these objections as a veto, but I think objections raised during lazy consensus should be addressed before moving forward. -- Michael Mior mm...@apache.org Le mer. 26 sept. 2018 à 04:51, Vladimir Sitnikov < sitnikov.vladi...@gmail.com> a écrit : > Michael>if we're going to use the term "lazy consensus" we should agree on > what it means > > Could you clarify what it means? > > Does that mean "absolutely no responses"? > Does that mean "a single -0 comment destroys lazy consensus"? > Does that mean "a single +0 comment destroys lazy consensus"? > Does that mean "any vote less than +1 kills the idea"? > > Vladimir >
Re: CALCITE-2458 Use of Kotlin for unit tests
Consensus is a central concept in the Apache Way. If we’re not all agreed what consensus is, we need to talk. And, not unrelated, I found Vladimir’s use of a -1 to veto a commit deeply disturbing. Not because I was on the receiving end of the vote, but because it indicated a failure to find consensus. It is the only -1 we have ever had and I would like it to be the last. Julian > On Sep 26, 2018, at 9:09 AM, Michael Mior wrote: > > I can clarify my *interpretation* of what it means. My interpretation may > not be the definition we collectively agree on. I assume lazy consensus is > employed when a change is expected to be potentially controversial. If no > objections are raised, then it's safe to assume the change creates no > controversy and lazy interpretation. If objections are raised, then I > believe lazy consensus has *not* been reached. I wouldn't interpret these > objections as a veto, but I think objections raised during lazy consensus > should be addressed before moving forward. > > -- > Michael Mior > mm...@apache.org > > > Le mer. 26 sept. 2018 à 04:51, Vladimir Sitnikov < > sitnikov.vladi...@gmail.com> a écrit : > >> Michael>if we're going to use the term "lazy consensus" we should agree on >> what it means >> >> Could you clarify what it means? >> >> Does that mean "absolutely no responses"? >> Does that mean "a single -0 comment destroys lazy consensus"? >> Does that mean "a single +0 comment destroys lazy consensus"? >> Does that mean "any vote less than +1 kills the idea"? >> >> Vladimir >>
Re: CALCITE-2458 Use of Kotlin for unit tests
Julian>Consensus is a central concept in the Apache Way https://www.apache.org/foundation/voting.html suggests to use whole numbers when it comes to "code changes". "Vetos" section is quite clear that vetos must be withdrawn before code modification could be applied. That provides an understandable machinery: 1) +1 is positive 2) -1 is negative (it is invalid and it has no weight without justification) Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
Per “An alternative to voting that is sometimes used to measure the acceptability of something is the concept of lazy consensus.” Lazy consensus is not a vote. You determine the results not by adding up integers but by reading people’s concerns. The dictionary definition of consensus is “unanimity”. > On Sep 27, 2018, at 4:53 AM, Vladimir Sitnikov > wrote: > > Julian>Consensus is a central concept in the Apache Way > > https://www.apache.org/foundation/voting.html suggests to use whole numbers > when it comes to "code changes". > "Vetos" section is quite clear that vetos must be withdrawn before code > modification could be applied. > That provides an understandable machinery: > 1) +1 is positive > 2) -1 is negative (it is invalid and it has no weight without justification) > > Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
Julian>Lazy consensus is not a vote https://www.apache.org/foundation/voting.html#votes-on-code-modification Apache/voting> Unless a *vote has been declared as using lazy consensus* , It implies that "vote can use lazy consensus". Julian>The dictionary definition of consensus is “unanimity” https://www.apache.org/foundation/glossary.html#LazyConsensus Apache/glossary> Also called '*lazy* *approval*'. Julian>unanimity It is not possible. Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
My point is that the doc is not entirely consistent. But you are choosing to focus on the definition of a “code vote” and entirely ignoring the definition (and notion) of consensus. In Apache votes are a formality, after consensus has been reached, so consensus is more important than votes. I am convinced that consensus (and unanimity) is possible. To achieve it, people need to listen to each other’s concerns in good faith, and then compromise. If we are operating on the basis of consensus, all of this stuff will be a lot more fun for everyone. Julian > On Sep 27, 2018, at 9:17 AM, Vladimir Sitnikov > wrote: > > Julian>Lazy consensus is not a vote > > https://www.apache.org/foundation/voting.html#votes-on-code-modification > Apache/voting> Unless a *vote has been declared as using lazy consensus* , > > It implies that "vote can use lazy consensus". > > Julian>The dictionary definition of consensus is “unanimity” > > https://www.apache.org/foundation/glossary.html#LazyConsensus > Apache/glossary> Also called '*lazy* *approval*'. > > Julian>unanimity > > It is not possible. > > Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
>In Apache votes are a formality, after consensus has been reached, so consensus is more important than votes The very first line of https://www.apache.org/foundation/voting.html reads Apache/voting> Because one of the fundamental aspects of accomplishing things within the Apache framework is doing so by *consensus*, there obviously needs to be a way to tell *whether it has been reached*. *This is done by voting*. Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
The Apache way is to reach consensus without voting whenever possible. Julian On Sep 29, 2018, at 12:32 AM, Vladimir Sitnikov wrote: >> In Apache votes are a formality, after consensus has been reached, so > consensus is more important than votes > > The very first line of https://www.apache.org/foundation/voting.html reads > > Apache/voting> Because one of the fundamental aspects of accomplishing > things within the Apache framework is doing so by *consensus*, there > obviously needs to be a way to tell *whether it has been reached*. *This is > done by voting*. > > Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
Hello, I would like to note that this change have also altered that now all tests in the core module are now compiled by the kotlin compiler. As I think Calcite is still a java library: I think that it would be better to compile and run the main tests by the supported platform's tools - and probably move the kotlin tests into a separate submodule. Right now it is not possible to compile the core module's tests without kotlin support: because the only kotlin test which contains "hello world" have made into the CalcitesSuite.java. regards, Zoltan On 9/24/18 11:43 PM, Vladimir Sitnikov wrote: Thanks everybody for your comments. I've received no strong objections so far, so I assume lazy consensus. I've merged the PR in https://git-wip-us.apache.org/repos/asf?p=calcite.git;a=commit;h=55d86646fe598d215bd53dbeac2bbbf32a9506f9 Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
>I would like to note that this change have also altered that now all tests in the core module are now compiled by the kotlin compiler. What do you mean by that? >Right now it is not possible to compile the core module's tests without kotlin support Why do you want that? Even if Kotlin tests are moved to a separate module, it would be impossible to compile that module "without kotlin support". Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
On 10/3/18 12:50 PM, Vladimir Sitnikov wrote: I would like to note that this change have also altered that now all tests in the core module are now compiled by the kotlin compiler. What do you mean by that? If I understand it correctly it works by wrapping around the java compiler somewhere inside the plugin; to enable bi directional callability between kotlin and java classes. ...or I'm interpreting the output of the below commands incorrectly? rm -r core/target/{test-,}classes/ mvn install -pl core -DskipTests -Dcheckstyle.skip org.jetbrains.kotlin:kotlin-maven-plugin:1.2.71:test-compile find core/target/test-classes -name *.class|head Right now it is not possible to compile the core module's tests without kotlin support Why do you want that? Even if Kotlin tests are moved to a separate module, it would be impossible to compile that module "without kotlin support". I tend to only work with a single module; which I have something to do...in Calcite that's usually just core; I don't see any reason to load all the others - it will just compile them all the time - and it usually just fills my workspace with stuff I don't need - if it was a separate module: I wouldn't ever load the kotlin test module...only if it breaks... Zoltan
Re: CALCITE-2458 Use of Kotlin for unit tests
> mvn install -pl core -DskipTests -Dcheckstyle.skip org.jetbrains.kotlin:kotlin-maven-plugin:1.2.71:test-compile What do you mean by "install" here? "install" here builds the full project with all the plugins, and it includes regular javac stuff. Vladimir
Re: CALCITE-2458 Use of Kotlin for unit tests
On 10/3/18 2:35 PM, Vladimir Sitnikov wrote: > mvn install -pl core -DskipTests -Dcheckstyle.skip org.jetbrains.kotlin:kotlin-maven-plugin:1.2.71:test-compile What do you mean by "install" here? "install" here builds the full project with all the plugins, and it includes regular javac stuff. yes; that install was misplaced...my mistake; sorry - if I run the java compiler it cames back complaining that it doesn't know what the testkt is. But still: are there any benefits that kotlin is added to the core module?
Re: CALCITE-2458 Use of Kotlin for unit tests
Zoltan>if I run the java compiler it cames back complaining that it doesn't know what the testkt is. In other words, Kotlin compiles *.kt files, and javac compiles *.java files. As for me it looks great. Zoltan>But still: are there any benefits that kotlin is added to the core module? Some of them are highlighted in CALCITE-2458 description. On top of that, having "core" tests in "core" module simplifies development flow: one can run tests from within IDE (e.g. hack relevant class and run relevant test), one can just use `mvn test` to run the tests, and so on. Building "just two modules" with maven is a pain. Maven modules are processed sequentially, so adding more modules would pile up overhead. Then there's "release artifacts", and what sense would it make "core-tests" artifact published to Central? Vladimir
***UNCHECKED*** Re: CALCITE-2458 Use of Kotlin for unit tests
Enrico>I don't know if Kotlin is enough widespread to say that it won't be a Enrico>problem for new contributors. Enrico>I sponsor the idea of having domain specific test languages like QUIDEM Enrico>which make it simpler It is truly wonderful how you question "if Kotlin is widespread or not", and sponsor QUIDEM at the same time. I'm absolutely sure QUIDEM is NOT enough widespread, and it does create problems for new contributors. Vladimir