Re: Reformat src files with `erlfmt` on `main`

2021-06-03 Thread Bessenyei Balázs Donát
Thank you for relaying Paul's approval, @Robert Newson !

> asks if we've confirmed that the abstract syntax tree is unaffected (i.e, the 
> changes are purely cosmetic and make no difference to the compiled artifacts).

The script format_all [1] gets md5sum for the binaries of the files to
be changed and does the same after formatting and reports whether the
hashes match or not. If the script is written correctly, it's proven
that the changes are purely cosmetic and make no difference to the
artifacts.


Donat


[1]: 
https://github.com/apache/couchdb/pull/3568/files#diff-f2ea7984714072018b8a19daabd2dbc8df9f662af94849ffa7eb60b54bc711f0



On Thu, Jun 3, 2021 at 3:20 PM Robert Newson  wrote:
>
> As Paul's involuntary amanuensis he says "go for it" but asks if we've 
> confirmed that the abstract syntax tree is unaffected (i.e, the changes are 
> purely cosmetic and make no difference to the compiled artifacts).
>
> B.
>
> > On 2 Jun 2021, at 09:18, Bessenyei Balázs Donát  wrote:
> >
> >> My only nose-wrinkle was at `->` being placed on its own line under some
> > circumstances
> > I counted too many occurrences of that to add ignores for them (and people
> > would probably forget adding them on new code which would result in a mixed
> > state).
> > If there are no objections, I'll go ahead with merging it with the
> > controversial `->`s on newlines  (as advantages seem to outweigh this
> > drawback). As I mentioned earlier, if we can get a config option or change
> > to erlfmt, we can always do a quick reformat.
> >
> >> local git hook
> > I couldn't find a nice way to do it, so I can open a ticket to do that
> > later. The PR adds it to CI and people can run the checks (and the
> > formatter) themselves locally.
> >
> > I have not received a +>=0 from Paul, but as it's been more than a week now
> > I'll merge the PR assuming consent. (The PR is already approved on GitHub.)
> > The change is not irreversible and I'd be happy to either revert or adjust
> > if necessary.
> >
> > Thank you all for the support and the contribution!
> >
> >
> > Donat
> >
> >
> > On Fri, May 28, 2021 at 4:31 PM Ilya Khlopotov  wrote:
> >
> >>> Can it also be set up as a local git hook etc?
> >>
> >> Few complications here:
> >> 1) CouchDB codebase is not 100% resides in a single repository
> >> 2) Which hook manager to use given differences in platforms we support and
> >> the fact that none of the hook managers support multiple repositories.
> >> There are multiple options:
> >>
> >> - https://github.com/frankywahl/super_hooks
> >> - https://github.com/Arkweid/lefthook
> >> - https://github.com/pre-commit/pre-commit
> >>
> >> Do we need a separate ML discussion which hook manager to use?
> >>
> >> Another option is to update configure or rebar.config.script to place
> >> files (or links) in `.git/hooks/pre-commit`.
> >>
> >> Best regards,
> >> iilyak
> >>
> >> On 2021/05/21 12:25:53, Robert Newson  wrote:
> >>> Hi,
> >>>
> >>> My only nose-wrinkle was at `->` being placed on its own line under some
> >> circumstances. The rest looked good. I agree that uniformity of formatting
> >> is a very good thing and this reformat is long overdue.
> >>>
> >>> Agree with Donat that the formatting should be enforced by CI tools so
> >> there’s no backsliding. Can it also be set up as a local git hook etc?
> >>>
> >>> B.
> >>>
> >>>> On 21 May 2021, at 12:46, Bessenyei Balázs Donát 
> >> wrote:
> >>>>
> >>>> Hi All,
> >>>>
> >>>> I believe I've only seen +>=0s so far so I intend to (in the following
> >> order):
> >>>> * wait for an ok from @Robert Newson and @Paul J. Davis
> >>>> * add `erlfmt-ignore`s if necessary to #3568
> >>>> * add a check to CI (ideally via `make`) to ensure `erlfmt` is +1 on
> >>>> the PRs in #3568
> >>>> * create a PR for 3.x analogous to #3568
> >>>>
> >>>> Please let me know if I missed anything.
> >>>>
> >>>>
> >>>> Donat
> >>>>
> >>>>
> >>>> On Fri, May 21, 2021 at 2:27 AM Joan Touzet  wrote:
> >>>>>
> >>>>> In general I am +0.5 on the entire thing, but would like to see Bob
> >>>>> Newson or Paul Davis speak up. In the past the

Re: Reformat src files with `erlfmt` on `main`

2021-06-02 Thread Bessenyei Balázs Donát
> My only nose-wrinkle was at `->` being placed on its own line under some
circumstances
I counted too many occurrences of that to add ignores for them (and people
would probably forget adding them on new code which would result in a mixed
state).
If there are no objections, I'll go ahead with merging it with the
controversial `->`s on newlines  (as advantages seem to outweigh this
drawback). As I mentioned earlier, if we can get a config option or change
to erlfmt, we can always do a quick reformat.

> local git hook
I couldn't find a nice way to do it, so I can open a ticket to do that
later. The PR adds it to CI and people can run the checks (and the
formatter) themselves locally.

I have not received a +>=0 from Paul, but as it's been more than a week now
I'll merge the PR assuming consent. (The PR is already approved on GitHub.)
The change is not irreversible and I'd be happy to either revert or adjust
if necessary.

Thank you all for the support and the contribution!


Donat


On Fri, May 28, 2021 at 4:31 PM Ilya Khlopotov  wrote:

> > Can it also be set up as a local git hook etc?
>
> Few complications here:
> 1) CouchDB codebase is not 100% resides in a single repository
> 2) Which hook manager to use given differences in platforms we support and
> the fact that none of the hook managers support multiple repositories.
> There are multiple options:
>
> - https://github.com/frankywahl/super_hooks
> - https://github.com/Arkweid/lefthook
> - https://github.com/pre-commit/pre-commit
>
> Do we need a separate ML discussion which hook manager to use?
>
> Another option is to update configure or rebar.config.script to place
> files (or links) in `.git/hooks/pre-commit`.
>
> Best regards,
> iilyak
>
> On 2021/05/21 12:25:53, Robert Newson  wrote:
> > Hi,
> >
> > My only nose-wrinkle was at `->` being placed on its own line under some
> circumstances. The rest looked good. I agree that uniformity of formatting
> is a very good thing and this reformat is long overdue.
> >
> > Agree with Donat that the formatting should be enforced by CI tools so
> there’s no backsliding. Can it also be set up as a local git hook etc?
> >
> > B.
> >
> > > On 21 May 2021, at 12:46, Bessenyei Balázs Donát 
> wrote:
> > >
> > > Hi All,
> > >
> > > I believe I've only seen +>=0s so far so I intend to (in the following
> order):
> > > * wait for an ok from @Robert Newson and @Paul J. Davis
> > > * add `erlfmt-ignore`s if necessary to #3568
> > > * add a check to CI (ideally via `make`) to ensure `erlfmt` is +1 on
> > > the PRs in #3568
> > > * create a PR for 3.x analogous to #3568
> > >
> > > Please let me know if I missed anything.
> > >
> > >
> > > Donat
> > >
> > >
> > > On Fri, May 21, 2021 at 2:27 AM Joan Touzet  wrote:
> > >>
> > >> In general I am +0.5 on the entire thing, but would like to see Bob
> > >> Newson or Paul Davis speak up. In the past they've been the most vocal
> > >> about code formatting standards, and I'd at least like to see a +0
> from
> > >> both of them.
> > >>
> > >> -Joan
> > >>
> > >> On 20/05/2021 11:53, Ilya Khlopotov wrote:
> > >>> Good idea Donat!!!
> > >>>
> > >>> Even though I disagree with some of the choices made by erlfmt I
> appreciate consistency it provides.
> > >>> The choices are logical. I really love that every decision is
> documented and properly discussed. I did read PR in its entirety and in
> fact was not even noticed the ugly `->` in the beginning of the line closer
> to the end of the review process.
> > >>> I do believe our wetware would adjust in no time to new formatting.
> Given how easy it is to reason about. I agree with Donat's observation that
> we are spending too much time and emphasis on formatting issues every time
> we review PRs. I do believe it is a machine job to provide consistent
> formatting. We humans are better at other things. All in all I vote for
> adopting `erlfmt` for both 3.x and main.
> > >>>
> > >>> Also thank you Donat for providing validation scripts to make sure
> the re-formatted code compiles to the same beam files.
> > >>>
> > >>> Best regards,
> > >>> iilyak
> > >>>
> > >>>
> > >>> On 2021/05/18 18:13:14, Bessenyei Balázs Donát 
> wrote:
> > >>>> Hi dev@couchdb,
> > >>>>
> > >>>> To eliminate the need for formatting-related comments and thus
> 

Re: Reformat src files with `erlfmt` on `main`

2021-05-21 Thread Bessenyei Balázs Donát
Hi All,

I believe I've only seen +>=0s so far so I intend to (in the following order):
* wait for an ok from @Robert Newson and @Paul J. Davis
* add `erlfmt-ignore`s if necessary to #3568
* add a check to CI (ideally via `make`) to ensure `erlfmt` is +1 on
the PRs in #3568
* create a PR for 3.x analogous to #3568

Please let me know if I missed anything.


Donat


On Fri, May 21, 2021 at 2:27 AM Joan Touzet  wrote:
>
> In general I am +0.5 on the entire thing, but would like to see Bob
> Newson or Paul Davis speak up. In the past they've been the most vocal
> about code formatting standards, and I'd at least like to see a +0 from
> both of them.
>
> -Joan
>
> On 20/05/2021 11:53, Ilya Khlopotov wrote:
> > Good idea Donat!!!
> >
> > Even though I disagree with some of the choices made by erlfmt I appreciate 
> > consistency it provides.
> > The choices are logical. I really love that every decision is documented 
> > and properly discussed. I did read PR in its entirety and in fact was not 
> > even noticed the ugly `->` in the beginning of the line closer to the end 
> > of the review process.
> > I do believe our wetware would adjust in no time to new formatting. Given 
> > how easy it is to reason about. I agree with Donat's observation that we 
> > are spending too much time and emphasis on formatting issues every time we 
> > review PRs. I do believe it is a machine job to provide consistent 
> > formatting. We humans are better at other things. All in all I vote for 
> > adopting `erlfmt` for both 3.x and main.
> >
> > Also thank you Donat for providing validation scripts to make sure the 
> > re-formatted code compiles to the same beam files.
> >
> > Best regards,
> > iilyak
> >
> >
> > On 2021/05/18 18:13:14, Bessenyei Balázs Donát  wrote:
> >> Hi dev@couchdb,
> >>
> >> To eliminate the need for formatting-related comments and thus
> >> unnecessary cycles in PRs, I've invested a little time to see if we
> >> could use a formatter on `main` [1].
> >> The PR reformats `.erl` files in `src` and the script [2] included
> >> shows that the compiled binaries match "before" and "after".
> >> The formatter used in the PR is `erlfmt` [3] which is an opinionated
> >> [4] tool so it's more of a "take it or leave it" as-is. (We could try
> >> using other formatters if we want in case people want formatting but
> >> think the choices `erlfmt` makes are unacceptable.)
> >> Some members of the CouchDB dev community already left some great
> >> comments on the PR and I haven't seen any strong opposition so far,
> >> but I wanted to make sure more people are aware of this.
> >> If you have any questions, comments or concerns (or objections),
> >> please let me know.
> >>
> >>
> >> Thank you,
> >>
> >> Donat
> >>
> >>
> >> [1]: https://github.com/apache/couchdb/pull/3568
> >> [2]: 
> >> https://github.com/apache/couchdb/pull/3568/files#diff-7adfbc2d8dba4d4ff49ff2b760b81c006097f20f412ea2007f899042fd0de98a
> >> [3]: https://github.com/WhatsApp/erlfmt
> >> [4]: 
> >> https://github.com/WhatsApp/erlfmt#comparison-with-other-erlang-formatters
> >>


Reformat src files with `erlfmt` on `main`

2021-05-18 Thread Bessenyei Balázs Donát
Hi dev@couchdb,

To eliminate the need for formatting-related comments and thus
unnecessary cycles in PRs, I've invested a little time to see if we
could use a formatter on `main` [1].
The PR reformats `.erl` files in `src` and the script [2] included
shows that the compiled binaries match "before" and "after".
The formatter used in the PR is `erlfmt` [3] which is an opinionated
[4] tool so it's more of a "take it or leave it" as-is. (We could try
using other formatters if we want in case people want formatting but
think the choices `erlfmt` makes are unacceptable.)
Some members of the CouchDB dev community already left some great
comments on the PR and I haven't seen any strong opposition so far,
but I wanted to make sure more people are aware of this.
If you have any questions, comments or concerns (or objections),
please let me know.


Thank you,

Donat


[1]: https://github.com/apache/couchdb/pull/3568
[2]: 
https://github.com/apache/couchdb/pull/3568/files#diff-7adfbc2d8dba4d4ff49ff2b760b81c006097f20f412ea2007f899042fd0de98a
[3]: https://github.com/WhatsApp/erlfmt
[4]: https://github.com/WhatsApp/erlfmt#comparison-with-other-erlang-formatters


Re: GSoC 2021 participation

2021-04-05 Thread Bessenyei Balázs Donát
Sounds great, thank you for the details, Adam!
As I'm a PMC member at Apache Flume, I think me helping out with a
Debezium integration is a really good fit for me as I should be
somewhat familiar with streaming log data.
I've posted at dev@community ([1]). Let's see how it goes.


Donat

[1]: 
https://lists.apache.org/thread.html/rc3a6bc4ca017a76c2e53b94d3faaa2f088963cbe56b608b2e6e4485c%40%3Cdev.community.apache.org%3E

On Sat, Apr 3, 2021 at 12:32 AM Adam Kocoloski  wrote:
>
> Sure thing.
>
> https://debezium.io is an open-source project largely developed by Red Hat 
> that standardizes change data capture across a variety of data sources. It 
> ships with connectors for Oracle, SQL Server, MySQL, PostgreSQL, and MongoDB, 
> and typically records changes into topics in Apache Kafka.
>
> CouchDB, of course, has a change capture feed as a public API endpoint, so a 
> connector should be a smaller effort than consuming the WAL from a 
> traditional RDMS. Integrating with Debezium would provide an easy way to 
> translate the _changes feed into a Kafka topic which plugs us into a much 
> larger ecosystem of tools and alleviates the need for every consumer of data 
> in CouchDB to build a bespoke “follower” of the _changes feed.
>
> Happy to dig up answers to other questions and to make connections with Kafka 
> / Debezium experts where it makes sense. Cheers,
>
> Adam
>
> > On Apr 2, 2021, at 5:44 PM, Bessenyei Balázs Donát  
> > wrote:
> >
> > Thank you for sharing the idea, Adam. I like it!
> > Can you please share some details?
> >
> >
> > Donat
> >
> > On Wed, Mar 31, 2021 at 2:14 AM Adam Kocoloski  wrote:
> >>
> >> I would bias towards ecosystem integrations, e.g. a metrics exporter for 
> >> Prometheus (I think one is in the works), or a  Debezium connector for the 
> >> _changes feed (I haven’t seen any movement here). I suppose the Debezium 
> >> connector would ideally be part of the Debezium project itself, but that 
> >> feels like one that is reasonably self-contained and would provide a good 
> >> return on investment. Happy to help expand on that one a bit if you think 
> >> it’s worthwhile.
> >>
> >> Adam
> >>
> >>> On Mar 30, 2021, at 4:42 AM, Bessenyei Balázs Donát  
> >>> wrote:
> >>>
> >>> Wow, I didn't realize we are that far into the timeline.
> >>>
> >>> I actually don't have a well-scoped idea for GSoC this year.
> >>> Does anyone have good candidates for a project?
> >>>
> >>>
> >>> Donat
> >>>
> >>> On Mon, Mar 29, 2021 at 6:23 PM Joan Touzet  wrote:
> >>>>
> >>>> https://community.apache.org/gsoc.html might be of help.
> >>>>
> >>>> If you are looking to talk to other Apache projects that have done this
> >>>> before, you could reach out on the dev@community.a.o list.
> >>>>
> >>>>
> >>>> https://lists.apache.org/thread.html/r189a563fe003ad8f0e4c298e18fad4da8d4b2854bd2a5d741ae3ac45%40%3Cdev.community.apache.org%3E
> >>>>
> >>>> Note the dependency on JIRA:
> >>>>
> >>>>> All ASF projects are invited to submit their ideas to their issue 
> >>>>> tracker, please be sure to add the labels “gsoc2021” and “mentor” so 
> >>>>> that we can automatically include them in our list of subjects. If your 
> >>>>> project does not use JIRA please contact d...@community.apache.org.
> >>>>
> >>>> so you'll need to post at dev@community to get included in the master 
> >>>> list.
> >>>>
> >>>> Student applications start tomorrow for 2 weeks, so you'll need to get a
> >>>> move on... If I'm around on chat I can try and help a bit.
> >>>>
> >>>> -Joan
> >>>>
> >>>> On 29/03/2021 01:20, Bessenyei Balázs Donát wrote:
> >>>>> If there are any projects that don't exceed my CouchDB / erlang / JS
> >>>>> knowledge, I'd make sure I'm available enough to support someone doing
> >>>>> a GSoC with us.
> >>>>> What's the workflow here? Do we have to apply as a project? Do we have
> >>>>> to propose projects?
> >>>>> I did look at "Prospective ASF mentors: read this" of [1], but I don't
> >>>>> see what it looks like for a project. Do we need a vote here?
> >>>>>
> >>>>>
> >>>>> Donat

Re: Removing "node" field from replicator "/_scheduler/{jobs | docs}"

2021-04-02 Thread Bessenyei Balázs Donát
I support removing obsolete fields from responses.
I also support tracking API changes.


Donat

On Fri, Apr 2, 2021 at 10:23 PM Robert Newson  wrote:
>
> +1 to removing “node” on main (but not 3.x branch).
>
> B.
>
> > On 2 Apr 2021, at 21:11, Ilya Khlopotov  wrote:
> >
> > Hi,
> >
> > In FDB world there wouldn't be erlang mesh as far as I can tell. In this 
> > situation the `node` field in the response from `/_scheduler/jobs`  and 
> > `/_scheduler/docs` doesn't make sense.
> >
> > We could either remove the field or set it to `None`. I propose complete 
> > removal.
> >
> > I also propose to establish a formal process to track API changes formally. 
> > Sooner or latter we would need to compile list of changes between versions. 
> > In case of a rewrite on top of FDB I suspect the archeology process 
> > wouldn't be easy. We could create a github issue template which would set 
> > necessary labels for example.
> >
> > Best regards,
> > iilyak
> >
>


Re: GSoC 2021 participation

2021-04-02 Thread Bessenyei Balázs Donát
Thank you for sharing the idea, Adam. I like it!
Can you please share some details?


Donat

On Wed, Mar 31, 2021 at 2:14 AM Adam Kocoloski  wrote:
>
> I would bias towards ecosystem integrations, e.g. a metrics exporter for 
> Prometheus (I think one is in the works), or a  Debezium connector for the 
> _changes feed (I haven’t seen any movement here). I suppose the Debezium 
> connector would ideally be part of the Debezium project itself, but that 
> feels like one that is reasonably self-contained and would provide a good 
> return on investment. Happy to help expand on that one a bit if you think 
> it’s worthwhile.
>
> Adam
>
> > On Mar 30, 2021, at 4:42 AM, Bessenyei Balázs Donát  
> > wrote:
> >
> > Wow, I didn't realize we are that far into the timeline.
> >
> > I actually don't have a well-scoped idea for GSoC this year.
> > Does anyone have good candidates for a project?
> >
> >
> > Donat
> >
> > On Mon, Mar 29, 2021 at 6:23 PM Joan Touzet  wrote:
> >>
> >> https://community.apache.org/gsoc.html might be of help.
> >>
> >> If you are looking to talk to other Apache projects that have done this
> >> before, you could reach out on the dev@community.a.o list.
> >>
> >>
> >> https://lists.apache.org/thread.html/r189a563fe003ad8f0e4c298e18fad4da8d4b2854bd2a5d741ae3ac45%40%3Cdev.community.apache.org%3E
> >>
> >> Note the dependency on JIRA:
> >>
> >>> All ASF projects are invited to submit their ideas to their issue 
> >>> tracker, please be sure to add the labels “gsoc2021” and “mentor” so that 
> >>> we can automatically include them in our list of subjects. If your 
> >>> project does not use JIRA please contact d...@community.apache.org.
> >>
> >> so you'll need to post at dev@community to get included in the master list.
> >>
> >> Student applications start tomorrow for 2 weeks, so you'll need to get a
> >> move on... If I'm around on chat I can try and help a bit.
> >>
> >> -Joan
> >>
> >> On 29/03/2021 01:20, Bessenyei Balázs Donát wrote:
> >>> If there are any projects that don't exceed my CouchDB / erlang / JS
> >>> knowledge, I'd make sure I'm available enough to support someone doing
> >>> a GSoC with us.
> >>> What's the workflow here? Do we have to apply as a project? Do we have
> >>> to propose projects?
> >>> I did look at "Prospective ASF mentors: read this" of [1], but I don't
> >>> see what it looks like for a project. Do we need a vote here?
> >>>
> >>>
> >>> Donat
> >>>
> >>> [1]: https://community.apache.org/gsoc.html
> >>>
> >>> On Sun, Mar 28, 2021 at 11:52 PM Joan Touzet  wrote:
> >>>>
> >>>> The ASF often ends up doing GSoC. I don't think we've ever had the
> >>>> sponsor within the project for it (or for Outreachy, for that matter).
> >>>>
> >>>> The most critical part is being available on a regular basis for proper
> >>>> mentoring. If you don't think you can get that into your schedule, don't
> >>>> volunteer. Assume you will get zero support from any other developer
> >>>> (not true, but best to plan for the worst case situation...)
> >>>>
> >>>> The second most critical part is to come up with a self-contained
> >>>> project that makes sense for CouchDB. The most obvious thing to me would
> >>>> be Fauxton work, esp. as it falls into the "sweet spot" of JS
> >>>> development. I dunno how good of a target main is, given how in flux it
> >>>> is; others might have a better take on that. There's also this PR that
> >>>> never finished up:
> >>>>
> >>>>  https://github.com/apache/couchdb/issues/1254
> >>>>
> >>>> These topics are all probably too big, but maybe one of them could be
> >>>> cut down to something summer-sized:
> >>>>
> >>>>  https://github.com/apache/couchdb/projects/1
> >>>>
> >>>> Thanks for taking on this initiative! I know for a fact I won't have
> >>>> time this summer, or I'd agree to join you.
> >>>>
> >>>> -Joan
> >>>>
> >>>> On 28/03/2021 15:59, Bessenyei Balázs Donát wrote:
> >>>>> Hi All,
> >>>>>
> >>>>> I've just seen that the ASF is accepted as a mentoring organisation
> >>>>> for GSoC 2021.
> >>>>> Is CouchDB interested in participating?
> >>>>> I've never done a GSoC before, but I'd certainly be interested. I'd be
> >>>>> happy to help a student contribute to CouchDB.
> >>>>>
> >>>>> What do you all think?
> >>>>>
> >>>>>
> >>>>> Thank you,
> >>>>> Donat
> >>>>>
>


Re: [DISCUSS] API versioning redux

2021-03-30 Thread Bessenyei Balázs Donát
> It'd be like a Python 2 -> 3 transition

I think the simile is spot on.
But that shouldn't be a problem, it's more like evolution.

> your user base plummet

Looking at [1] and for lack of a better idea for reference [2], the
user base issues are not obvious to me.

Having said that, I'm okay with dropping the idea, I just want to make
sure it's also considered.


[1]: https://www.jetbrains.com/lp/python-developers-survey-2020/
[2]: https://www.tiobe.com/tiobe-index//

On Tue, Mar 30, 2021 at 7:20 PM Joan Touzet  wrote:
>
>
>
> On 30/03/2021 12:57, Bessenyei Balázs Donát wrote:
> > If we are bumping major anyways, can we just go "clean slate" and
> > promise replication compatibility only?
>
> Basically: watch your user base plummet. It'd be like a Python 2 -> 3
> transition, or a Perl 5 -> 6 one. It would take years of concerted
> effort to recover.
>
> This is why I'd thought pushing that off to 5.0, with 4.0 being the
> transitional release, would be best. But I have no strong opinion.
>
> > All that while we'd publish a migration guide that explains the
> > differences and helps users adjust their implementations, but this way
> > we'd get rid of some potential complexity.
> >
> > If we do want to go with API versioning, can someone help me with a
> > worked example that shows how and when an endpoint gets a new `v_`,
> > how that affects other endpoints, which versions of CouchDB need
> > updating for such a change and when do we get to drop previous
> > versions of the API?
>
> I'd like to see this too.
>
> > Thank you,
> > Donat
> >
> > On Mon, Mar 29, 2021 at 6:28 PM Joan Touzet  wrote:
> >>
> >> On 22/02/2021 20:48, Adam Kocoloski wrote:
> >>> Hi all,
> >>>
> >>> Several times in recent months we’ve had discussions on this list about 
> >>> behavior changes in CouchDB 4.0 that boil down to tradeoffs between
> >>>
> >>> a) maintaining compatibility with current CouchDB APIs, and
> >>> b) capitalizing on the transactional semantics of FoundationDB
> >>>
> >>> An example would be the support for large result sets in a view response: 
> >>> do we stitch together multiple transactions to support very large 
> >>> responses, or provide new pagination APIs and guarantee that each 
> >>> response is generated from a single isolated snapshot?
> >>>
> >>> I’d like to suggest that we “have our cake and eat it, too” by 
> >>> introducing versioned APIs as previously suggested by Ilya[1]. We’d have 
> >>> a V3 API that strives to maximize compatibility with CouchDB 3.x, and a 
> >>> V4 API that still looks and feels like CouchDB, but introduces breaking 
> >>> changes where necessary to provide well-defined transactional semantics. 
> >>> I think this explicit approach to API evolution could make life easier 
> >>> for client library maintainers, and also free us up to improve the API 
> >>> without needing to be quite as careful about in-place backwards 
> >>> compatibility.
> >>>
> >>> [1]: 
> >>> https://lists.apache.org/thread.html/rcc742c0fdca0363bb338b54526045720868597ea35ee6842aef174e0%40%3Cdev.couchdb.apache.org%3E
> >>>
> >>> What do you think? Cheers, Adam
> >>
> >> In general, +1, as long as the concerns raised by Bob are addressed.
> >> Namely: if I point a CouchDB 1.x-3.x client at a future CouchDB 4.0, I
> >> shouldn't be *too* surprised, and as much as possible should "just work."
> >>
> >> I've long believed that 4.0 would be a transitional version, and 5.0
> >> would break things completely... so if 5.0 (or whatever) stops accepting
> >> "v3" syntax, or "default" changes to "v4", fine by me.
> >>
> >> Also it'd be great if this was advertised in the feature strings shown
> >> in `GET /` in some intelligent way. Maybe an array of API versions
> >> supported?
> >>
> >> -Joan


Re: [DISCUSS] API versioning redux

2021-03-30 Thread Bessenyei Balázs Donát
If we are bumping major anyways, can we just go "clean slate" and
promise replication compatibility only?
All that while we'd publish a migration guide that explains the
differences and helps users adjust their implementations, but this way
we'd get rid of some potential complexity.

If we do want to go with API versioning, can someone help me with a
worked example that shows how and when an endpoint gets a new `v_`,
how that affects other endpoints, which versions of CouchDB need
updating for such a change and when do we get to drop previous
versions of the API?


Thank you,
Donat

On Mon, Mar 29, 2021 at 6:28 PM Joan Touzet  wrote:
>
> On 22/02/2021 20:48, Adam Kocoloski wrote:
> > Hi all,
> >
> > Several times in recent months we’ve had discussions on this list about 
> > behavior changes in CouchDB 4.0 that boil down to tradeoffs between
> >
> > a) maintaining compatibility with current CouchDB APIs, and
> > b) capitalizing on the transactional semantics of FoundationDB
> >
> > An example would be the support for large result sets in a view response: 
> > do we stitch together multiple transactions to support very large 
> > responses, or provide new pagination APIs and guarantee that each response 
> > is generated from a single isolated snapshot?
> >
> > I’d like to suggest that we “have our cake and eat it, too” by introducing 
> > versioned APIs as previously suggested by Ilya[1]. We’d have a V3 API that 
> > strives to maximize compatibility with CouchDB 3.x, and a V4 API that still 
> > looks and feels like CouchDB, but introduces breaking changes where 
> > necessary to provide well-defined transactional semantics. I think this 
> > explicit approach to API evolution could make life easier for client 
> > library maintainers, and also free us up to improve the API without needing 
> > to be quite as careful about in-place backwards compatibility.
> >
> > [1]: 
> > https://lists.apache.org/thread.html/rcc742c0fdca0363bb338b54526045720868597ea35ee6842aef174e0%40%3Cdev.couchdb.apache.org%3E
> >
> > What do you think? Cheers, Adam
>
> In general, +1, as long as the concerns raised by Bob are addressed.
> Namely: if I point a CouchDB 1.x-3.x client at a future CouchDB 4.0, I
> shouldn't be *too* surprised, and as much as possible should "just work."
>
> I've long believed that 4.0 would be a transitional version, and 5.0
> would break things completely... so if 5.0 (or whatever) stops accepting
> "v3" syntax, or "default" changes to "v4", fine by me.
>
> Also it'd be great if this was advertised in the feature strings shown
> in `GET /` in some intelligent way. Maybe an array of API versions
> supported?
>
> -Joan


Re: GSoC 2021 participation

2021-03-30 Thread Bessenyei Balázs Donát
Wow, I didn't realize we are that far into the timeline.

I actually don't have a well-scoped idea for GSoC this year.
Does anyone have good candidates for a project?


Donat

On Mon, Mar 29, 2021 at 6:23 PM Joan Touzet  wrote:
>
> https://community.apache.org/gsoc.html might be of help.
>
> If you are looking to talk to other Apache projects that have done this
> before, you could reach out on the dev@community.a.o list.
>
>
> https://lists.apache.org/thread.html/r189a563fe003ad8f0e4c298e18fad4da8d4b2854bd2a5d741ae3ac45%40%3Cdev.community.apache.org%3E
>
> Note the dependency on JIRA:
>
> > All ASF projects are invited to submit their ideas to their issue tracker, 
> > please be sure to add the labels “gsoc2021” and “mentor” so that we can 
> > automatically include them in our list of subjects. If your project does 
> > not use JIRA please contact d...@community.apache.org.
>
> so you'll need to post at dev@community to get included in the master list.
>
> Student applications start tomorrow for 2 weeks, so you'll need to get a
> move on... If I'm around on chat I can try and help a bit.
>
> -Joan
>
> On 29/03/2021 01:20, Bessenyei Balázs Donát wrote:
> > If there are any projects that don't exceed my CouchDB / erlang / JS
> > knowledge, I'd make sure I'm available enough to support someone doing
> > a GSoC with us.
> > What's the workflow here? Do we have to apply as a project? Do we have
> > to propose projects?
> > I did look at "Prospective ASF mentors: read this" of [1], but I don't
> > see what it looks like for a project. Do we need a vote here?
> >
> >
> > Donat
> >
> > [1]: https://community.apache.org/gsoc.html
> >
> > On Sun, Mar 28, 2021 at 11:52 PM Joan Touzet  wrote:
> >>
> >> The ASF often ends up doing GSoC. I don't think we've ever had the
> >> sponsor within the project for it (or for Outreachy, for that matter).
> >>
> >> The most critical part is being available on a regular basis for proper
> >> mentoring. If you don't think you can get that into your schedule, don't
> >> volunteer. Assume you will get zero support from any other developer
> >> (not true, but best to plan for the worst case situation...)
> >>
> >> The second most critical part is to come up with a self-contained
> >> project that makes sense for CouchDB. The most obvious thing to me would
> >> be Fauxton work, esp. as it falls into the "sweet spot" of JS
> >> development. I dunno how good of a target main is, given how in flux it
> >> is; others might have a better take on that. There's also this PR that
> >> never finished up:
> >>
> >>   https://github.com/apache/couchdb/issues/1254
> >>
> >> These topics are all probably too big, but maybe one of them could be
> >> cut down to something summer-sized:
> >>
> >>   https://github.com/apache/couchdb/projects/1
> >>
> >> Thanks for taking on this initiative! I know for a fact I won't have
> >> time this summer, or I'd agree to join you.
> >>
> >> -Joan
> >>
> >> On 28/03/2021 15:59, Bessenyei Balázs Donát wrote:
> >>> Hi All,
> >>>
> >>> I've just seen that the ASF is accepted as a mentoring organisation
> >>> for GSoC 2021.
> >>> Is CouchDB interested in participating?
> >>> I've never done a GSoC before, but I'd certainly be interested. I'd be
> >>> happy to help a student contribute to CouchDB.
> >>>
> >>> What do you all think?
> >>>
> >>>
> >>> Thank you,
> >>> Donat
> >>>


Re: 3.2 release?

2021-03-29 Thread Bessenyei Balázs Donát
Following up on my open PR sweep:
@Joan: I see that you've asked on a couple of PRs back in Oct 2020 if
they are to be merged and some of them were left open.
Is there any objection to closing them all? Or let's ignore them for now?


Donat

On Mon, Mar 29, 2021 at 11:06 AM Bessenyei Balázs Donát
 wrote:
>
> I think I have read access to the COUCHDB namespace in Confluence, but not 
> edit.
>
> I'll skim through `main` to see if we need to / want to backport
> things from there.
> I can also bump every PR targeted for 3.x to see if people want to get
> that content in there.
>
> Release notes: I'm really _not_ good with docs, so I'm happy there's
> someone who's doing that.
>
>
> Donat
>
> On Sun, Mar 28, 2021 at 9:20 PM Joan Touzet  wrote:
> >
> > Great! Excited to get help.
> >
> > Here's the release procedure:
> >
> >   https://cwiki.apache.org/confluence/display/COUCHDB/Release+Procedure
> >
> > Make sure you have wiki access, your Apache creds should get you in.
> >
> > The most valuable thing you can do is make sure all of the PRs that we
> > want to land, have landed. Review all in-progress PRs and see if any
> > more need to land on 3.x. Also, look for any important backports from
> > main, though with fdb finally having landed, this will be more limited
> > than previously. This usually takes a week or two.
> >
> > I get the most enjoyment out of doing the release notes, personally, but
> > if you'd like to do it instead, go for it. Have a look at the last few
> > releases to see the level of detail I've been including. (Plus at least
> > one joke, which may or may not be funny.) I'll try and start on this
> > tomorrow.
> >
> > -Joan
> >
> > On 28/03/2021 14:30, Bessenyei Balázs Donát wrote:
> > > +1
> > >
> > > Let me know how I can help.
> > >
> > >
> > > Donat
> > >
> > > On Sun, Mar 28, 2021 at 7:03 PM Joan Touzet  wrote:
> > >>
> > >> Hi everyone,
> > >>
> > >> We have a number of high profile improvements to 3.x since 3.1.1 came
> > >> out. I think it's about time for 3.2. Does everyone agree?
> > >>
> > >> If so, we should start a clock for ~2 weeks to finish closing out any
> > >> last minute improvements, fixes, tweaks, etc. anyone wants to get in.
> > >>
> > >> -Joan "we did it before and we can do it again" Touzet


Re: 3.2 release?

2021-03-29 Thread Bessenyei Balázs Donát
I think I have read access to the COUCHDB namespace in Confluence, but not edit.

I'll skim through `main` to see if we need to / want to backport
things from there.
I can also bump every PR targeted for 3.x to see if people want to get
that content in there.

Release notes: I'm really _not_ good with docs, so I'm happy there's
someone who's doing that.


Donat

On Sun, Mar 28, 2021 at 9:20 PM Joan Touzet  wrote:
>
> Great! Excited to get help.
>
> Here's the release procedure:
>
>   https://cwiki.apache.org/confluence/display/COUCHDB/Release+Procedure
>
> Make sure you have wiki access, your Apache creds should get you in.
>
> The most valuable thing you can do is make sure all of the PRs that we
> want to land, have landed. Review all in-progress PRs and see if any
> more need to land on 3.x. Also, look for any important backports from
> main, though with fdb finally having landed, this will be more limited
> than previously. This usually takes a week or two.
>
> I get the most enjoyment out of doing the release notes, personally, but
> if you'd like to do it instead, go for it. Have a look at the last few
> releases to see the level of detail I've been including. (Plus at least
> one joke, which may or may not be funny.) I'll try and start on this
> tomorrow.
>
> -Joan
>
> On 28/03/2021 14:30, Bessenyei Balázs Donát wrote:
> > +1
> >
> > Let me know how I can help.
> >
> >
> > Donat
> >
> > On Sun, Mar 28, 2021 at 7:03 PM Joan Touzet  wrote:
> >>
> >> Hi everyone,
> >>
> >> We have a number of high profile improvements to 3.x since 3.1.1 came
> >> out. I think it's about time for 3.2. Does everyone agree?
> >>
> >> If so, we should start a clock for ~2 weeks to finish closing out any
> >> last minute improvements, fixes, tweaks, etc. anyone wants to get in.
> >>
> >> -Joan "we did it before and we can do it again" Touzet


Re: GSoC 2021 participation

2021-03-28 Thread Bessenyei Balázs Donát
If there are any projects that don't exceed my CouchDB / erlang / JS
knowledge, I'd make sure I'm available enough to support someone doing
a GSoC with us.
What's the workflow here? Do we have to apply as a project? Do we have
to propose projects?
I did look at "Prospective ASF mentors: read this" of [1], but I don't
see what it looks like for a project. Do we need a vote here?


Donat

[1]: https://community.apache.org/gsoc.html

On Sun, Mar 28, 2021 at 11:52 PM Joan Touzet  wrote:
>
> The ASF often ends up doing GSoC. I don't think we've ever had the
> sponsor within the project for it (or for Outreachy, for that matter).
>
> The most critical part is being available on a regular basis for proper
> mentoring. If you don't think you can get that into your schedule, don't
> volunteer. Assume you will get zero support from any other developer
> (not true, but best to plan for the worst case situation...)
>
> The second most critical part is to come up with a self-contained
> project that makes sense for CouchDB. The most obvious thing to me would
> be Fauxton work, esp. as it falls into the "sweet spot" of JS
> development. I dunno how good of a target main is, given how in flux it
> is; others might have a better take on that. There's also this PR that
> never finished up:
>
>   https://github.com/apache/couchdb/issues/1254
>
> These topics are all probably too big, but maybe one of them could be
> cut down to something summer-sized:
>
>   https://github.com/apache/couchdb/projects/1
>
> Thanks for taking on this initiative! I know for a fact I won't have
> time this summer, or I'd agree to join you.
>
> -Joan
>
> On 28/03/2021 15:59, Bessenyei Balázs Donát wrote:
> > Hi All,
> >
> > I've just seen that the ASF is accepted as a mentoring organisation
> > for GSoC 2021.
> > Is CouchDB interested in participating?
> > I've never done a GSoC before, but I'd certainly be interested. I'd be
> > happy to help a student contribute to CouchDB.
> >
> > What do you all think?
> >
> >
> > Thank you,
> > Donat
> >


GSoC 2021 participation

2021-03-28 Thread Bessenyei Balázs Donát
Hi All,

I've just seen that the ASF is accepted as a mentoring organisation
for GSoC 2021.
Is CouchDB interested in participating?
I've never done a GSoC before, but I'd certainly be interested. I'd be
happy to help a student contribute to CouchDB.

What do you all think?


Thank you,
Donat


Re: 3.2 release?

2021-03-28 Thread Bessenyei Balázs Donát
+1

Let me know how I can help.


Donat

On Sun, Mar 28, 2021 at 7:03 PM Joan Touzet  wrote:
>
> Hi everyone,
>
> We have a number of high profile improvements to 3.x since 3.1.1 came
> out. I think it's about time for 3.2. Does everyone agree?
>
> If so, we should start a clock for ~2 weeks to finish closing out any
> last minute improvements, fixes, tweaks, etc. anyone wants to get in.
>
> -Joan "we did it before and we can do it again" Touzet


Re: [VOTE] Set a finite default for max_attachment_size

2021-02-02 Thread Bessenyei Balázs Donát
Hi Joan,

> Point of order - when we do 72-hour votes, it's best to not count
weekends in that 72-hours.

I remembered the 72 hours is 72 so that everyone has an opportunity to
chime in regardless of their location / timezone. (Ie. so that there's
at least a fraction of a weekday included for everyone.) The closest
reference I can find is [1].
Nevertheless, I'll do my best to start my next votes on a Monday or
Tuesday so that the weekends don't mess things up.
Thank you for letting me know about this!


Donat

[1]: 
https://www.apache.org/foundation/voting.html#expressing-votes-1-0-1-and-fractions


Re: [VOTE] Set a finite default for max_attachment_size

2021-02-01 Thread Bessenyei Balázs Donát
This vote is now closed as there were three +1s, one +0 and no -1s and
the 72 hours is up. I'll merge the PR.

Thanks to all who voted!


Donat


On Mon, Feb 1, 2021 at 7:52 PM Eric Avdey  wrote:
>
> Ok, fair enough, +0 from me with a note that I'd still prefer to see this 
> limit aligned with 4.x limits, so users wouldn't have to adjust to this 
> change twice.
>
>
> Eric
>
> > On Feb 1, 2021, at 14:47, Nick Vatamaniuc  wrote:
> >
> > I am +1 to lowering as it's better than infinity.
> >
> > But I also see Eric's point. I was surprised a while back just like
> > Eric that I could successfully upload >1GB-sized files.  So why not
> > 0.5GB or 2GB? I am thinking 2GB was (is?) a common limit on some OSes
> > and file systems (FAT32) since they use ints for file size and
> > offsets. Since our attachment won't be saved as is in the file systems
> > inside a .couch file 2GB may be too high, so 1GB as a limit makes
> > sense to me.
> >
> > -Nick
> >
> > On Mon, Feb 1, 2021 at 1:25 PM Paul Davis  
> > wrote:
> >>
> >> +1
> >>
> >> Default unlimited seems like an oversight regardless of what we change it 
> >> to.
> >>
> >> On Mon, Feb 1, 2021 at 11:59 AM Eric Avdey  wrote:
> >>>
> >>> Maybe I didn't express myself clear enough. Setting some finit default is 
> >>> not a purpose, it's what you are doing and I'm asking what the reason for 
> >>> this change. In other words I'm not asking what are you doing, I'm asking 
> >>> why are you doing this.
> >>>
> >>> Introducing a new limit will be a breaking change to anoyone who uploads 
> >>> attachments larger than that limit, obviously, so "assumed 1G is large 
> >>> enough" sounds really arbitrary to me without any factual support for 
> >>> that assumption.
> >>>
> >>>
> >>> Eric
> >>>
> >>>
> >>>> On Feb 1, 2021, at 13:15, Bessenyei Balázs Donát  
> >>>> wrote:
> >>>>
> >>>> The purpose of this vote / PR is to set _some_ finite default. I went
> >>>> with 1G as I assumed that would not break anyone's production system.
> >>>> I'd support decreasing that limit over time.
> >>>>
> >>>> The vote has been open for 72 hours now, but I believe it still needs
> >>>> two more +1s to pass.
> >>>>
> >>>>
> >>>> Donat
> >>>>
> >>>> On Thu, Jan 28, 2021 at 10:44 PM Eric Avdey  wrote:
> >>>>>
> >>>>> This got me curious and I tried to upload Ubuntu image as an 
> >>>>> attachment. Interestingly CouchDB 3.x accepted first 1.4G of 2.8G file 
> >>>>> and then returned proper 201 response with a new doc revision, which I 
> >>>>> certanly didn't expect. Should say, that 1.4G seems suspiciously 
> >>>>> similar to a normal memory limit for a 32 bit process.
> >>>>>
> >>>>> Putting this aside, I agree that uploading large attachments is an 
> >>>>> anti-pattern and 1G seems excessive, hence my question. I'd expect this 
> >>>>> number to be based on something and correlating it with a  technical 
> >>>>> limit in 4.x makes a lot of sense to me.
> >>>>>
> >>>>>
> >>>>> Eric
> >>>>>
> >>>>>
> >>>>>> On Jan 28, 2021, at 16:02, Robert Newson  wrote:
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> I think a gigabyte is _very_ generous given our experience of this 
> >>>>>> feature in practice.
> >>>>>>
> >>>>>> In 4.x attachment size will necessarily be much more restrictive, so 
> >>>>>> it seems prudent to move toward that limit.
> >>>>>>
> >>>>>> I don’t think many folks (hopefully no one!) is routinely inserting 
> >>>>>> attachments over 1 gib today, I’d be fairly surprised if it even works.
> >>>>>>
> >>>>>> B.
> >>>>>>
> >>>>>>> On 28 Jan 2021, at 19:42, Eric Avdey  wrote:
> >>>>>>>
> >>>>>>> There is no justification neither here or on the PR for this change, 
> >>>>>>> i.e. why this is done. Original infinity default was set to preserve 
> >>>>>>> previous behaviour, this change will inadvertently break workflow for 
> >>>>>>> users who upload large attachment and haven't set explicit default, 
> >>>>>>> so why is it fine to do now? There might be some discussion around 
> >>>>>>> this somewhere, but it'd be nice to include it here for sake of 
> >>>>>>> people like me who's out of the loop.
> >>>>>>>
> >>>>>>> Also 1G limit seems arbitrary - how was it choosen?
> >>>>>>>
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Eric
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> On Jan 28, 2021, at 01:46, Bessenyei Balázs Donát 
> >>>>>>>>  wrote:
> >>>>>>>>
> >>>>>>>> Hi All,
> >>>>>>>>
> >>>>>>>> In https://github.com/apache/couchdb/pull/3347 I'm proposing to set a
> >>>>>>>> finite default for max_attachment_size .
> >>>>>>>> The PR is approved, but as per Ilya's request, I'd like to call for a
> >>>>>>>> lazy majority vote here.
> >>>>>>>> The vote will remain open for at least 72 hours from now.
> >>>>>>>>
> >>>>>>>> Please let me know if you have any questions, comments or concerns.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Donat
> >>>>>>>
> >>>>>>
> >>>>>
> >>>
>


Re: [VOTE] Set a finite default for max_attachment_size

2021-02-01 Thread Bessenyei Balázs Donát
The purpose of this vote / PR is to set _some_ finite default. I went
with 1G as I assumed that would not break anyone's production system.
I'd support decreasing that limit over time.

The vote has been open for 72 hours now, but I believe it still needs
two more +1s to pass.


Donat

On Thu, Jan 28, 2021 at 10:44 PM Eric Avdey  wrote:
>
> This got me curious and I tried to upload Ubuntu image as an attachment. 
> Interestingly CouchDB 3.x accepted first 1.4G of 2.8G file and then returned 
> proper 201 response with a new doc revision, which I certanly didn't expect. 
> Should say, that 1.4G seems suspiciously similar to a normal memory limit for 
> a 32 bit process.
>
> Putting this aside, I agree that uploading large attachments is an 
> anti-pattern and 1G seems excessive, hence my question. I'd expect this 
> number to be based on something and correlating it with a  technical limit in 
> 4.x makes a lot of sense to me.
>
>
> Eric
>
>
> > On Jan 28, 2021, at 16:02, Robert Newson  wrote:
> >
> > Hi,
> >
> > I think a gigabyte is _very_ generous given our experience of this feature 
> > in practice.
> >
> > In 4.x attachment size will necessarily be much more restrictive, so it 
> > seems prudent to move toward that limit.
> >
> > I don’t think many folks (hopefully no one!) is routinely inserting 
> > attachments over 1 gib today, I’d be fairly surprised if it even works.
> >
> > B.
> >
> >> On 28 Jan 2021, at 19:42, Eric Avdey  wrote:
> >>
> >> There is no justification neither here or on the PR for this change, i.e. 
> >> why this is done. Original infinity default was set to preserve previous 
> >> behaviour, this change will inadvertently break workflow for users who 
> >> upload large attachment and haven't set explicit default, so why is it 
> >> fine to do now? There might be some discussion around this somewhere, but 
> >> it'd be nice to include it here for sake of people like me who's out of 
> >> the loop.
> >>
> >> Also 1G limit seems arbitrary - how was it choosen?
> >>
> >>
> >> Thanks,
> >> Eric
> >>
> >>
> >>
> >>> On Jan 28, 2021, at 01:46, Bessenyei Balázs Donát  
> >>> wrote:
> >>>
> >>> Hi All,
> >>>
> >>> In https://github.com/apache/couchdb/pull/3347 I'm proposing to set a
> >>> finite default for max_attachment_size .
> >>> The PR is approved, but as per Ilya's request, I'd like to call for a
> >>> lazy majority vote here.
> >>> The vote will remain open for at least 72 hours from now.
> >>>
> >>> Please let me know if you have any questions, comments or concerns.
> >>>
> >>>
> >>> Donat
> >>
> >
>


[VOTE] Set a finite default for max_attachment_size

2021-01-27 Thread Bessenyei Balázs Donát
Hi All,

In https://github.com/apache/couchdb/pull/3347 I'm proposing to set a
finite default for max_attachment_size .
The PR is approved, but as per Ilya's request, I'd like to call for a
lazy majority vote here.
The vote will remain open for at least 72 hours from now.

Please let me know if you have any questions, comments or concerns.


Donat


Re: [DISCUSS] Removing erlang 19 support

2021-01-25 Thread Bessenyei Balázs Donát
Thank you all for the input!

I'll remove erlang 19 for `couchdb-config` and we'll be able to refer
to this thread when we have to remove it anywhere else.


Donat

On Sat, Jan 23, 2021 at 10:09 PM Adam Kocoloski  wrote:
>
> Ah, good research there Joan. +1 to Donat’s suggestion to drop support for 19 
> from me.
>
> Adam
>
> > On Jan 22, 2021, at 4:49 PM, Joan Touzet  wrote:
> >
> > On 2021-01-22 4:37 p.m., Robert Newson wrote:
> >> Iteresting. I’m actually surprised at the inversion here (that CouchDB 
> >> is dependent on  IBM to confirm CouchDB’s stability). I’ve always agonised 
> >> over even the perception that IBM/Cloudant is calling the shots. I 
> >> appreciate the reassurance that running at scale provides, of course, I 
> >> just don’t think it can be our official position.
> >
> > It's a tough one. I was pretty aggressive on CouchDB running the very 
> > latest until the scheduler collapse problem surfaced. After that, there was 
> > another problem (I can't recall) that was pretty serious too. I took a 
> > wait-and-see attitude at that point, and after I didn't see IBM move 
> > forward to a newer release, didn't move forward ourselves. Looks like we 
> > ended up in deadlock!
> >
> > However! See your own comments on this:
> >
> > https://github.com/apache/couchdb/issues/3115#issuecomment-729031967
> >
> > I knew there was something at the back of my head on this. Guess we're both 
> > getting old ;)
> >
> >> On the core point of the thread, it seems there’s no barrier to dropping 
> >> Erlang 19 support, so I think we can go to a VOTE thread, perhaps best to 
> >> wait till Monday for others to chime in on this discussion though.
> >
> > More important is that we already committed changes on the main repo re: 
> > Erlang 19 about 14 months ago by Paul:
> >
> > https://github.com/apache/couchdb/commit/3594f2f1fc16903c1c383ebaf205d31c9c17fb3a
> >
> > I think that makes Donat's request pretty straightforward.
> >
> >> I also think that IBM Cloudant’s chosen Erlang release is in part 
> >> influenced by CouchDB’s lack of support for later versions and requirement 
> >> of compatible with older releases, which now appears illusory.
> >
> > If we're ready to move to 21 or 22 as a default, we're ready. Let's hope 
> > the serious issues in 21/22 are at least mitigated. I'm happy to make the 
> > 3.3 release (or whatever is next) use the very latest version of 21 or 22 
> > from GitHub, subject to community recommendations and encouragement. 23 is 
> > still a WIP: https://github.com/apache/couchdb/issues/3115
> >
> > -Jon
> >
> >> B.
> >>> On 22 Jan 2021, at 21:19, Joan Touzet  wrote:
> >>>
> >>> On 22/01/2021 15:48, Robert Newson wrote:
> >>>> I’m +1 on dropping Erlang 19 support. Erlang is now on major release 23.
> >>>
> >>> No problem here.
> >>>
> >>>> I’d further advocate a general policy of supporting only the most recent 
> >>>> 2 or 3 major releases of Erlang/OTP.
> >>>>
> >>>> The main (I think only?) reason to keep compatibility so far back is 
> >>>> because of the versions supported by some OS’es. I don’t feel that is a 
> >>>> strong reason given the couchdb install, in common with Erlang-based 
> >>>> projects, is self-contained. The existence of Erlang Solutions packages 
> >>>> for all common platforms is also a factor.
> >>>
> >>> That hasn't been the case for at least 2 years, if not longer.
> >>>
> >>> As the release engineer, I've been focused on stability for everyone.
> >>> This is largely driven by knowing that IBM/Cloudant largely run CouchDB
> >>> on 20.x at scale. Standing on the shoulders of giants, our releases run
> >>> the latest 20.x release at the time of binary generation.
> >>>
> >>> A few times recently issues cropped up in 21 and 22 that we didn't
> >>> encounter in our user base because, at scale, we are deployed on
> >>> 20.3.8.something. Some of these issues were non-trivial. (I'm off today,
> >>> so I don't have the time to dig into the specifics until Monday.)
> >>>
> >>> So my $0.02 is that: if IBM/Cloudant is ready to move to something newer
> >>> at scale, I'm ready to release binaries on a newer Erlang by default.
> >>>
> >>> The alternative (running newer Erlangs in the binary distributions than
> 

[DISCUSS] Removing erlang 19 support

2021-01-22 Thread Bessenyei Balázs Donát
Hi All,

CI for https://github.com/apache/couchdb-config appears to be broken.
I wanted to fix it in
https://github.com/apache/couchdb-config/pull/34/files , but I'm
getting issues with erlang 19. Are we okay with dropping 19 support
there?

On a different note: are we okay with dropping erlang 19 support
overall in couch project(s)?


Thank you,
Donat


Re: [DISCUSS] VS Code dev container?

2021-01-18 Thread Bessenyei Balázs Donát
I'm not a VS Code user, but I like the idea a lot!

For this change to lower the bar for new contributors, I think it
would be the best to have the change in-tree with a few lines of
description in the Readme.
One additional benefit I haven't seen mentioned would be the
Codespaces [1] integration. That should help us lower the bar even
further.


Donat

[1]: https://github.com/features/codespaces

On Mon, Jan 18, 2021 at 2:19 AM Adam Kocoloski  wrote:
>
> Hi folks,
>
> I don’t know how many of you use VS Code as your preferred editor, but I 
> found myself setting up a new laptop recently and I thought I’d try out their 
> support for containerized development environments[1] to get my CouchDB rig 
> up and running. I started with the 3.x branch this weekend and came up with 
> the following:
>
> https://github.com/apache/couchdb/compare/3.x...3.x-devcontainer
>
> With this .devcontainer folder a VS Code user who clones CouchDB gets a 
> prompt to mount the codebase inside a container with all the dependencies 
> required to build CouchDB, run the test suite, build the docs, etc. The 
> source code lives outside the container, and the data files produced by a dev 
> cluster do as well. I think this could help lower the bar for new 
> contributors to get involved in the project. Some additional details:
>
> - The .devcontainer also installs the Erlang Language Server extension into 
> the container, so the setup comes complete with syntax coloring, 
> auto-complete, module outlines, etc.
>
> - I used an official Elixir image as the base for this image (given that we 
> need Elixir for the test suite these days). Elixir builds off the the 
> official Erlang images, which in turn are based on Debian. I initially tried 
> to use the images we maintain for our CI setup but found some challenges 
> getting those to work with Erlang LS out of the box and this ended up being a 
> simpler route. Open to discussion on this one.
>
> - I defaulted to Erlang 22 / Debian Buster and SpiderMonkey 60 installed from 
> Debian. This means a user needs to add `—spidermonkey-version 60` during 
> ./configure. I expect this will add a little friction for new users. Would it 
> be OK to have configure inherit $SM_VSN from the external environment in 
> addition to the command-line to smooth this over?
>
> I‘m polishing up another devcontainer for the main branch that uses Docker 
> Compose to run FoundationDB using the official FDB image alongside the 
> CouchDB layer, but I wanted to take the pulse of the folks here to see if 
> these container configs make sense in-tree and whether there are opinions 
> about how they should be done. I intentionally opted for simplicity on this 
> front rather than trying to reuse too much of our CI or production image code.
>
> Cheers, Adam
>
> [1]: https://code.visualstudio.com/docs/remote/containers
>
>


Re: [ANNOUNCE] Bessenyei Balázs Donát elected as CouchDB committer

2021-01-14 Thread Bessenyei Balázs Donát
Dear All,

Thank you for the invitation and the welcome! It's a great honour.

@Joan: my username is bessbd where I can pick and it isn't already in
use. People usually call me Donat.


Donat


On Thu, Jan 14, 2021 at 10:09 PM Nick Vatamaniuc  wrote:
>
> Congrats Donat and welcome!
>
> -Nick
>
>
> On Thu, Jan 14, 2021 at 3:55 PM Joan Touzet  wrote:
> >
> > Congratulations Bessenyei! Do you have a nickname (other than bessbd)?
> >
> > -Joan
> >
> > On 2021-01-14 3:48 p.m., Robert Newson wrote:
> > > Dear community,
> > >
> > > I am pleased to announce that the CouchDB Project Management Committee 
> > > has elected Bessenyei Balázs Donát as a CouchDB committer.
> > >
> > > Apache ID: bessbd
> > >
> > > Committers are given a binding vote in certain project decisions, as well 
> > > as write access to public project infrastructure.
> > >
> > > This election was made in recognition of Donát's commitment to the 
> > > project. We mean this in the sense of being loyal to the project and its 
> > > interests.
> > >
> > > Please join me in extending a warm welcome to Donát!
> > >
> > > On behalf of the CouchDB PMC,
> > >
> > > Robert Newson
> > >


Re: CouchDB participating in Hacktoberfest 2020?

2020-10-07 Thread Bessenyei Balázs Donát
Hi Alessio,

Oh, that's great. I've just found it in the Slack messages.

I'd be happy to assist with the spam and supporting new contributors.
Mind that I don't think I can label or merge any PRs in CouchDB,
though.


Donat

On Wed, 7 Oct 2020 at 11:15, Alessio 'Blaster' Biancalana
 wrote:
>
> Hi :-)
> We spoke a bit about that on Slack, totally informally.
>
> If anyone volunteers into managing the spam that arrives it can be done, I
> was about volunteering for that but I'm afraid actually this year spam
> became a real problem. Maybe we could try :-)
>
> Alessio
>
> On Wed, Oct 7, 2020 at 10:57 AM Bessenyei Balázs Donát 
> wrote:
>
> > Hi All,
> >
> > I was wondering if Apache CouchDB is interested in participating in
> > Hacktoberfest 2020.
> > Previously it was an opt-in-by-default thing, but due to recent
> > changes ( https://hacktoberfest.digitalocean.com/hacktoberfest-update
> > ),
> > it is now required to add a topic to the GH project or label PRs
> > accordingly ( https://hacktoberfest.digitalocean.com/details#details
> > ).
> >
> > I believe it's a little effort to join something great / cool and the
> > increased visibility + potential contributions should be worthwhile.
> >
> >
> > Thank you,
> > Donat
> >
> > PS. please let me know if there is anything I can do to help.
> >


CouchDB participating in Hacktoberfest 2020?

2020-10-07 Thread Bessenyei Balázs Donát
Hi All,

I was wondering if Apache CouchDB is interested in participating in
Hacktoberfest 2020.
Previously it was an opt-in-by-default thing, but due to recent
changes ( https://hacktoberfest.digitalocean.com/hacktoberfest-update
),
it is now required to add a topic to the GH project or label PRs
accordingly ( https://hacktoberfest.digitalocean.com/details#details
).

I believe it's a little effort to join something great / cool and the
increased visibility + potential contributions should be worthwhile.


Thank you,
Donat

PS. please let me know if there is anything I can do to help.


Re: [DISCUSS] Prometheus endpoint in CouchDB 4.x

2020-09-22 Thread Bessenyei Balázs Donát
Hi Peng Hui (and Garren),

I think the Prometheus support would be an awesome feature!
>From the e-mail it's not clear to me what would happen to the already
existing endpoints mentioned (`_stats`, `_system` and
`_active_tasks`).
Are they going to be deprecated / removed? Or does the info become
available in multiple formats (and places)?


Thank you,
Donat

On Tue, 22 Sep 2020 at 14:55, jiangph  wrote:
>
> Hey all,
>
> We would like to add a Prometheus metrics endpoint for CouchDB and wanted to 
> see if the community would be interested in us contributing this to CouchDB 
> 4.x.
>
> Prometheus is a CNCF open-source project and the Prometheus metrics endpoint 
> format is supported by many monitoring tools. Its data model is based around 
> having a metric name which then contains a label name and a label value:
>
> {=, ...}
>
> And it supports the Counter, Gauge, Histogram, and Summary metric types.
>
> The idea for the new Prometheus endpoint, /_metrics, would be that the 
> endpoint is a consolidation of the _stats [1],  _system [2], and 
> _active_tasks [3] endpoints.
>
> For _stats and _system, the conversion from JSON to Prometheus-based format 
> seems to be straightforward.
>
> JSON format:
> {
>  "value": {
>   "min": 0,
>   "max": 0,
>   "arithmetic_mean": 0,
>   "geometric_mean": 0,
>   "harmonic_mean": 0,
>   "median": 0,
>   "variance": 0,
>   "standard_deviation": 0,
> ...
> "percentile": [
>[
> 50,
> 0
>],
>[
> 75,
> 0
>],
>[
> 90,
> 0
>],
>[
> 95,
> 0
>],
>[
> 99,
> 0
>],
>[
> 999,
> 0
>]
>   ],
>   "histogram": [
>[
> 0,
> 0
>]
>   ],
> }
>
> Prometheus-based format:
>
> couchdb_stats{value="min"} 0
> couchdb_stats{value="max"} 0
> couchdb_stats{value="percentile50"} 0
> couchdb_stats{value="percentile75"} 0
> couchdb_stats{value="percentile95"} 0
>
> For _active_tasks, the change will be a bit more complicated, and some fields 
> will be added to labels and tags.
>
> JSON format:
>
> {
> "checkpointed_source_seq": 68585,
> "continuous": false,
> "doc_id": null,
> "doc_write_failures": 0,
> "docs_read": 4524,
> "docs_written": 4524,
> "missing_revisions_found": 4524,
> "pid": "<0.1538.5>",
> "progress": 44,
> "replication_id": "9bc1727d74d49d9e157e260bb8bbd1d5",
> "revisions_checked": 4524,
> "source": "mailbox",
> "source_seq": 154419,
> "started_on": 1376116644,
> "target": "http://mailsrv:5984/mailbox ",
> "type": "replication",
> "updated_on": 1376116651
> }
>
> Prometheus-based would look something like:
>
> format:couchdb_active_task{type="replication", source="mailbox", 
> target="http://mailsrv:5984/mailbox ", 
> docs_count = "docs_read"} 4524
> couchdb_active_task{type="replication", source="mailbox", 
> target="http://mailsrv:5984/mailbox ", 
> docs_count = "docs_written"} 4524
> couchdb_active_task{type="replication", source="mailbox", 
> target="http://mailsrv:5984/mailbox ", 
> docs_count = "missing_revisions_found"} 4524
>
>
> Best regards,
> Garren Smith
> Peng Hui Jiang
>
> [1] 
> https://docs.couchdb.org/en/latest/api/server/common.html#node-node-name-stats
>  
> 
> [2] https://docs.couchdb.org/en/latest/api/server/common.html#active-tasks 
> 
> [3] 
> https://docs.couchdb.org/en/latest/api/server/common.html#node-node-name-system
>  
> 


Re: [VOTE] Release Apache CouchDB 3.1.1 (RC2)

2020-09-16 Thread Bessenyei Balázs Donát
> Why should we remove this? I don’t think it is controversial.

Does that mean 3.1.1 will be released with the query parameter
`buffer_response`?

Nit:
> 3.1.1 will not be cut unless 3 +1 votes, minimum, arrive from committers, 
> with no blockers by convention.

According to http://www.apache.org/legal/release-policy.html#release-approval
I thought it has to be 3 PMC members (which by the way I think is
already passed if Nick, Glynn and Joan (RM) are all +1).
Or is there a different process in CouchDB?


Thank you,
Donat


On Tue, 15 Sep 2020 at 20:32, Joan Touzet  wrote:
>
> FYI, Linux packages in test form are available now at
>
> https://repo-nightly.couchdb.org/3.x/
>
> As always, subject to removal, do not use in production, etc.
>
> Windows build will probably land tomorrow, I'm fairly busy today.
>
> -Joan
>
> On 15/09/2020 08:26, Will Holley wrote:
> > Environment:
> >RHEL 8
> >Elixir: 1.9.1
> >Erlang: 20.3.8.25
> >
> > CPU Architectures: amd64, ppc64le, s390x
> >
> > Sig: ok
> > Checksums: ok
> > Configure, make & make check: ok
> > Build release, add admin & start: ok
> > Used Fauxton to:
> >- configure cluster: ok
> >- verify install: ok
> >- create dbs: ok
> >- create docs: ok
> >- create replications between dbs on same cluster: ok
> >- links to docs: ok
> >
> > +1
> >
> > On Tue, 15 Sep 2020 at 05:20, Nick Vatamaniuc  wrote:
> >
> >> Environment:
> >>Ubuntu 18.04.5, x86_64
> >>$ asdf current
> >>  elixir 1.9.4-otp-22
> >>  erlang 22.2.3
> >>   $ cat /etc/apt/sources.list.d/couchdb-bintray.list
> >>  deb https://apache.bintray.com/couchdb-deb bionic main
> >>
> >> Sig: ok
> >> Checksums: ok
> >> Configure, make & make check: ok
> >> Build release, add admin & start: ok
> >> Used Fauxton to:
> >>- configure cluster: ok
> >>- verify install: ok
> >>- create dbs: ok
> >>- create docs: ok
> >>- create replications between dbs on same cluster: ok
> >>- links to docs: ok
> >>
> >> +1
> >>
> >> Thank you for creating the release, Joan!
> >>
> >> -Nick
> >>
> >> On Mon, Sep 14, 2020 at 10:15 PM Joan Touzet  wrote:
> >>>
> >>> Hi everyone,
> >>>
> >>> There have been no votes on this release. Are people available to try it
> >>> out? Tomorrow, I will be able to complete the binary builds and submit
> >>> my own vote.
> >>>
> >>> Remember, 3.1.1 will not be cut unless 3 +1 votes, minimum, arrive from
> >>> committers, with no blockers by convention. The Mac situation needs to
> >>> be investigated. It's not clear to me that there is consensus on
> >>> blocking the release on the new request header toggle, but if there is,
> >>> I could cut a 3.1.1-RC3 _without_ that change tomorrow if consensus
> >>> materialises overnight.
> >>>
> >>> Note that I will be on holiday starting end of the week for 2.5 weeks.
> >>> If 3.1.1 doesn't cut from this RC, for whatever reason, the next
> >>> opportunity I will have to turn the crank will be 5 October 2020.
> >>>
> >>> -Joan
> >>>
> >>> On 2020-09-11 6:53 p.m., Joan Touzet wrote:
>  Dear community,
> 
>  I would like to propose that we release Apache CouchDB 3.1.1.
> 
>  Changes since the last round:
> 
>    https://github.com/apache/couchdb/compare/3.1.1-RC1...3.1.1-RC2
> 
>  Candidate release notes:
> 
>    https://docs.couchdb.org/en/latest/whatsnew/3.1.html
> 
>  We encourage the whole community to download and test these release
>  artefacts so that any critical issues can be resolved before the
> >> release
>  is made. Everyone is free to vote on this release, so dig right in!
>  (Only PMC members have binding votes, but they depend on community
>  feedback to gauge if an official release is ready to be made.)
> 
>  The release artefacts we are voting on are available here:
> 
>    https://dist.apache.org/repos/dist/dev/couchdb/source/3.1.1/rc.2/
> 
>  There, you will find a tarball, a GPG signature, and SHA256/SHA512
>  checksums.
> 
>  Please follow the test procedure here:
> 
> 
> 
> >> https://cwiki.apache.org/confluence/display/COUCHDB/Testing+a+Source+Release
> 
> 
>  Please remember that "RC2" is an annotation. If the vote passes, these
>  artefacts will be released as Apache CouchDB 3.1.1.
> 
>  Because of the weekend, this vote will remain open until 5PM ET
> >> (UTC-4),
>  Tuesday, 15 September 2020.
> 
>  Please cast your votes now.
> 
>  Thanks,
>  Joan "once more unto the breech, dear friends" Touzet
> >>
> >


Re: [ANNOUNCE] Glynn Bird joins the PMC

2020-09-15 Thread Bessenyei Balázs Donát
Congrats, Glynn!


Donat

On Tue, 15 Sep 2020 at 15:14, Garren Smith  wrote:
>
> Congratulations Glynn and welcome.
>
> On Tue, Sep 15, 2020 at 2:12 PM Andy Wenk  wrote:
>
> > Welcome Glynn ;-)
> >
> > Best
> >
> > Andy
> > --
> > Andy Wenk
> > Hamburg
> >
> > GPG fingerprint C32E 275F BCF3 9DF6 4E55  21BD 45D3 5653 77F9 3D29
> >
> >
> >
> > > On 15. Sep 2020, at 04:21, Nick Vatamaniuc  wrote:
> > >
> > > Congrats, Glynn!
> > >
> > > -Nick
> > >
> > > On 9/14/20 10:07 PM, Joan Touzet wrote:
> > >> Congratulations Glynn - and welcome!
> > >> -Joan
> > >> On 2020-09-14 12:22 p.m., Michelle P wrote:
> > >>>
> > >>> Dear community,
> > >>>
> > >>> I am delighted to announce that Glynn Bird joins the Apache CouchDB
> > Project Management Committee today.
> > >>>
> > >>> Glynn has made outstanding, sustained contributions to the project.
> > This appointment is an official acknowledgement of their position within
> > the community, and our trust in their ability to provide oversight for the
> > project.
> > >>>
> > >>> Everybody, please join me in congratulating Glynn!
> > >>>
> > >>> On behalf of the CouchDB PMC,
> > >>> Michelle
> > >>>
> > >
> >
> >


Re: [VOTE] Release Apache CouchDB 3.1.1 (RC2)

2020-09-14 Thread Bessenyei Balázs Donát
Hi All,

I've checked SHAsums and GPG signature and they are correct.
I've also done some end-to-end comparison to the latest docker image
and there seems to be no regression.

And I'm interested to see the discussion addressing Richard Ellis'
concerns as any post-release changes to the "location" of the new
control (qs -> header) would require a major version (in semver-land).


Thank you,
Donat


On Sat, 12 Sep 2020 at 00:53, Joan Touzet  wrote:
>
> Dear community,
>
> I would like to propose that we release Apache CouchDB 3.1.1.
>
> Changes since the last round:
>
>  https://github.com/apache/couchdb/compare/3.1.1-RC1...3.1.1-RC2
>
> Candidate release notes:
>
>  https://docs.couchdb.org/en/latest/whatsnew/3.1.html
>
> We encourage the whole community to download and test these release
> artefacts so that any critical issues can be resolved before the release
> is made. Everyone is free to vote on this release, so dig right in!
> (Only PMC members have binding votes, but they depend on community
> feedback to gauge if an official release is ready to be made.)
>
> The release artefacts we are voting on are available here:
>
>  https://dist.apache.org/repos/dist/dev/couchdb/source/3.1.1/rc.2/
>
> There, you will find a tarball, a GPG signature, and SHA256/SHA512
> checksums.
>
> Please follow the test procedure here:
>
>
> https://cwiki.apache.org/confluence/display/COUCHDB/Testing+a+Source+Release
>
> Please remember that "RC2" is an annotation. If the vote passes, these
> artefacts will be released as Apache CouchDB 3.1.1.
>
> Because of the weekend, this vote will remain open until 5PM ET (UTC-4),
> Tuesday, 15 September 2020.
>
> Please cast your votes now.
>
> Thanks,
> Joan "once more unto the breech, dear friends" Touzet


Re: Per Doc Access

2020-08-04 Thread Bessenyei Balázs Donát
On Tue, 4 Aug 2020 at 13:10, Jan Lehnardt  wrote:
>
> Ah, there might be a misconception. Per-doc-access databases are not “more 
> secure”
> than regular databases. They are a trade-off between additional 
> access-control for
> additional CPU and disk resources. But it’s not a case of having a regular 
> db-as-
> we-know-and-use-it-today and enabling per-doc-access and now it is more 
> secure,
> it behaves differently and your app needs to account for that.

I didn't mean it would make the product more secure out-of-the-box. I
was just referring to the principle of least privilege ([1]) - as in
people would not be able to create "free for all" databases by
accident (forgetting to supply the enable flag). Please let me know if
I misunderstood the feature somehow.

> I don’t mind adding a global off switch that overrides the on-when-specified 
> case
> to disable all per-doc-access creations.

Awesome, thank you!


Donat

[1] https://en.wikipedia.org/wiki/Principle_of_least_privilege


Re: Per Doc Access

2020-08-04 Thread Bessenyei Balázs Donát
On Tue, 4 Aug 2020 at 12:34, Jan Lehnardt  wrote:
>
> The Erlang tests are already exclusively using the HTTP API. I don’t plan
> to rewrite those to Elixir, but documentation on how to use this will be
> written before this is merged.

The documentation before merge sounds like a good idea, thank you for that!

>
> > - what do you think about making this feature toggleable via an ini option?
>
> I could be persuaded to provide a default toggle like we have for `q` as an
> ini option, but I want to make sure people know this is opt-in behaviour, so
> I’m on the fence on allowing this to be the default on database creation.

Are you suggesting a three-state flag, such as "always-on" (for the
security-conscious people and environments),
"per-db-user-defined-on-create-but-off-when-unspecified" (opt-in, most
flexible), "always-off" ("compatibility mode")?
That would be neat.


Thank you,

Donat


Re: Per Doc Access

2020-08-04 Thread Bessenyei Balázs Donát
Hey Jan,

I've skimmed through the PR and the RFC and it looks good to me on a first read!
I have two questions (requests, maybe) though:
- do you think adding some "system-level" (elixir, I guess) tests
would be valuable so that a more formalized specification (and
verification) of the behavior is available? (They certainly help me
understand functionality a lot.)
- what do you think about making this feature toggleable via an ini option?

Also, I've noticed there are a couple of TODO comments in the code.
Are you planning to remove them before merging?
Anyway, I'm (non-binding) +1 on the change!


Thank you,

Donat

On Mon, 3 Aug 2020 at 17:29, Jan Lehnardt  wrote:
>
> *bump* Hey all, it’d be great to get at least some cursory feedback on this.
>
> Best
> Jan
> —
>
> > On 26. Jul 2020, at 20:28, Jan Lehnardt  wrote:
> >
> > Hey all,
> >
> > I’m happy to present the first PR worth sharing for introducing 
> > per-doc-access control to the 3.x codebase.
> >
> >https://github.com/apache/couchdb/pull/3038
> >
> > There are few odds and ends left to do, but this is in good enough shape to 
> > get wider review on approach and implementation so far.
> >
> > My hope would be to include this in a future 3.2.0 release before embarking 
> > on reimplementing this for 4.x, which should be considerably simpler.
> >
> > The PR and linked resources have most of the information relevant to this.
> >
> > Please review, test and critique heavily, and let me know any questions you 
> > might have.
> >
> > This concludes a couple of weeks worth of effort spread across multiple 
> > years. It all started with the developer summit in Boston and Adam’s 
> > initial presentation of this design. I hope this makes it justice.
> >
> > Best
> > Jan
> > —
> > Professional Support for Apache CouchDB:
> > https://neighbourhood.ie/couchdb-support/
> >
>


Re: [DISCUSS] couchdb 4.0 transactional semantics

2020-07-22 Thread Bessenyei Balázs Donát
On Tue, 21 Jul 2020 at 18:45, Jan Lehnardt  wrote:
> I’m not sure why a URL parameter vs. a path makes a big difference?
>
> Do you have an example?
>
> Best
> Jan
> —

Oh, sure! OpenAPI Generator [1] and et al. for example generate Java
methods (like [2] out of spec [3]) per path per verb.
Java's type safety and the way methods are currently generated don't
really provide an easy way to retrieve multiple kinds of responses, so
having them separate would help a lot there.


Donat

PS. I'm getting self-conscious about discussing this in this thread.
Should I open a new one?


[1] https://openapi-generator.tech/
[2] 
https://github.com/OpenAPITools/openapi-generator/blob/c49d8fd/samples/client/petstore/java/okhttp-gson/src/main/java/org/openapitools/client/api/PetApi.java#L606
[3] 
https://github.com/OpenAPITools/openapi-generator/blob/c49d8fd/samples/client/petstore/java/okhttp-gson/api/openapi.yaml#L208


Re: [DISCUSS] couchdb 4.0 transactional semantics

2020-07-21 Thread Bessenyei Balázs Donát
On Tue, 21 Jul 2020 at 17:42, Jan Lehnardt  wrote:
> We rather don’t like to break things just because we can :)
>
> Do you have anything specific in mind?
>
> Best
> Jan
> —
>

I'm not suggesting that breaking changes should be introduced just for
the fun of it :)
Anyway, an example could be the changes feed [1]: it returns JSON,
line-by-line JSON or EventSource responses (for `normal`, `continuous`
and `eventsource` modes, respectively).
This makes integration and tooling around it difficult. One potential
fix to that could be separating the feed into different URLs (such as
`_changes`, `_changes/_continuous` and `_changes/_eventsource`).

Let me know what you think.


Donat

[1] https://docs.couchdb.org/en/stable/api/database/changes.html


Re: [DISCUSS] couchdb 4.0 transactional semantics

2020-07-21 Thread Bessenyei Balázs Donát
I think being able to leverage FoundationDB's serializability is an
awesome idea! +1 (non-binding) on all 4 points.
I also support the idea of changing the API in backwards-incompatible
ways if that makes things more convenient / streamlined. I wonder,
does this mean other, backwards-incompatible changes are also welcome
in the next major? (Given that replicator-compatibility (from later on
in this thread) is expected.)


Thank you,

Donat


On Thu, 16 Jul 2020 at 18:26, Paul Davis  wrote:
>
> From what I'm reading it sounds like we have general consensus on a few 
> things:
>
> 1. A single CouchDB API call should map to a single FDB transaction
> 2. We absolutely do not want to return a valid JSON response to any
> streaming API that hit a transaction boundary (because data
> loss/corruption)
> 3. We're willing to change the API requirements so that 2 is not an issue.
> 4. None of this applies to continuous changes since that API call was
> never a single snapshot.
>
> If everyone generally agrees with that summarization, my suggestion
> would be that we just revisit the new pagination APIs and make them
> the only behavior rather than having them be opt-in. I believe those
> APIs already address all the concerns in this thread and the only
> reason we kept the older versions with `restart_tx` was to maintain
> API backwards compatibility at the expense of a slight change to
> semantics of snapshots. However, if there's a consensus that the
> semantics are more important than allowing a blanket `GET
> /db/_all_docs` I think it'd make the most sense to just embrace the
> pagination APIs that already exist and were written to cover these
> issues.
>
> The only thing I'm not 100% on is how to deal with non-continuous
> replications. I.e., the older single shot replication. Do we go back
> with patches to older replicators to allow 4.0 compatibility? Just
> declare that you have to mediate a replication on the newer of the two
> CouchDB deployments? Sniff the replicator's UserAgent and behave
> differently on 4.x for just that special case?
>
> Paul
>
> On Wed, Jul 15, 2020 at 7:25 PM Adam Kocoloski  wrote:
> >
> > Sorry, I also missed that you quoted this specific bit about eagerly 
> > requesting a new snapshot. Currently the code will just react to the 
> > transaction expiring, then wait till it acquires a new snapshot if 
> > “restart_tx” is set (which can take a couple of milliseconds on a 
> > FoundationDB cluster that is deployed across multiple AZs in a cloud 
> > Region) and then proceed.
> >
> > Adam
> >
> > > On Jul 15, 2020, at 6:54 PM, Adam Kocoloski  wrote:
> > >
> > > Right now the code has an internal “restart_tx” flag that is used to 
> > > automatically request a new snapshot if the original one expires and 
> > > continue streaming the response. It can be used for all manner of 
> > > multi-row responses, not just _changes.
> > >
> > > As this is a pretty big change to the isolation guarantees provided by 
> > > the database Bob volunteered to elevate the issue to the mailing list for 
> > > a deeper discussion.
> > >
> > > Cheers, Adam
> > >
> > >> On Jul 15, 2020, at 11:38 AM, Joan Touzet  wrote:
> > >>
> > >> I'm having trouble following the thread...
> > >>
> > >> On 14/07/2020 14:56, Adam Kocoloski wrote:
> > >>> For cases where you’re not concerned about the snapshot isolation (e.g. 
> > >>> streaming an entire _changes feed), there is a small performance 
> > >>> benefit to requesting a new FDB transaction asynchronously before the 
> > >>> old one actually times out and swapping over to it. That’s a pattern 
> > >>> I’ve seen in other FDB layers but I’m not sure we’ve used it anywhere 
> > >>> in CouchDB yet.
> > >>
> > >> How does _changes work right now in the proposed 4.0 code?
> > >>
> > >> -Joan
> > >
> >