Re: Have we considered static type checking for our python libs?

2022-01-26 Thread Berenguer Blasi
We'll have node reusage in dtests soon once CI is green again and 16951 merges (in review). #justfyi On 27/1/22 0:28, Joshua McKenzie wrote: I meant concretely from actually working on the tests, not a theoretical list. I think that's fair. I'll see if I can't whip up an example of

Re: Have we considered static type checking for our python libs?

2022-01-26 Thread Joshua McKenzie
> > I meant concretely from actually working on the tests, not a theoretical > list. I think that's fair. I'll see if I can't whip up an example of how it'd mutate a single test and the API boundary between that and ccm. Should be a quick but helpful exercise in seeing how it'd materially change

Re: Have we considered static type checking for our python libs?

2022-01-26 Thread Brandon Williams
On Wed, Jan 26, 2022 at 2:17 PM Joshua McKenzie wrote: >> >> I'm unable to think of an instance where typing in python could have helped >> me, > > Discoverability, IDE integration, more primitives to describe your intent in > your code to other maintainers, and shifting a certain class of

Re: Have we considered static type checking for our python libs?

2022-01-26 Thread Joshua McKenzie
> > I'm unable to think of an instance where typing in python could have > helped me, Discoverability, IDE integration, more primitives to describe your intent in your code to other maintainers, and shifting a certain class of runtime errors to being "merge-time" errors all come to mind. There's

Re: Have we considered static type checking for our python libs?

2022-01-26 Thread bened...@apache.org
Python execution is obviously not a bottleneck, but cluster startup/shutdown and lengthy waits due to lack of visibility to state changes in the tests quite probably are. The python tests are also IME quite rudimentary compared to their java equivalents, and more brittle, due to worse tooling

Re: Have we considered static type checking for our python libs?

2022-01-26 Thread Brandon Williams
On Wed, Jan 26, 2022 at 7:43 AM bened...@apache.org wrote: > I might even venture to predict that it might payoff with lower development > overhead, as we can run our tests much more quickly, and debug failures much > more easily. I don't think in practice these will happen at all, let alone

Re: Have we considered static type checking for our python libs?

2022-01-26 Thread bened...@apache.org
I don’t think this support would be very hard to add, if this is the only stumbling block. From: Andrés de la Peña Date: Wednesday, 26 January 2022 at 14:10 To: dev@cassandra.apache.org Subject: Re: Have we considered static type checking for our python libs? Last time I ported dtests during

Re: Have we considered static type checking for our python libs?

2022-01-26 Thread Andrés de la Peña
Last time I ported dtests during the 4.0 quality test epic there wasn't support for virtual nodes in in-jvm dtests. We have many Python dtests depending on vnodes that can't be totally ported if we still don't have support for vnodes, I don't know if it's still the case. On Wed, 26 Jan 2022 at

Re: Have we considered static type checking for our python libs?

2022-01-26 Thread Joshua McKenzie
Could be a very fruitful source of LHF tickets to highlight in the biweekly email and would be pretty trivial to integrate this into the build lead role (getting an epic and jira tickets created to port tests over, etc). we can run our tests much more quickly, and debug failures much more easily.

Re: Have we considered static type checking for our python libs?

2022-01-26 Thread bened...@apache.org
We could set this as a broad goal of the project, and like the build lead role could each volunteer to adopt a test every X weeks. We would have migrated in no time, I expect, with this kind of concerted effort, and might not even notice a significant penalty to other ongoing work. Last time I

Re: Have we considered static type checking for our python libs?

2022-01-26 Thread Joshua McKenzie
> > I have yet to encounter this class of problem in the dtests. It's more about development velocity and convenience than about preventing defects in our case, since we're not abusing duck-typing everywhere. Every time I have to work on python dtests (for instance, when doing build lead work and

Re: Have we considered static type checking for our python libs?

2022-01-26 Thread bened...@apache.org
I was sort of hoping we would retire the python dtests before long, at least in large part (probably not ever entirely, but 99%). I think many of them could be migrated to in-jvm dtests without much effort. I would hate to expend loads of effort modernising them when the same effort could see

Re: Have we considered static type checking for our python libs?

2022-01-26 Thread Brandon Williams
I have yet to encounter this class of problem in the dtests. On Wed, Jan 26, 2022 at 6:59 AM Joshua McKenzie wrote: > > Relevant links: > 1) Optional static typing for python: > https://docs.python.org/3/library/typing.html > 2) Mypy static type checker for python:

Have we considered static type checking for our python libs?

2022-01-26 Thread Joshua McKenzie
Relevant links: 1) Optional static typing for python: https://docs.python.org/3/library/typing.html 2) Mypy static type checker for python: https://github.com/python/mypy So the question - has anyone given any serious thought to introducing type hints and a static type checker in ccm and python