Re: [prometheus-developers] [VOTE] Rename blackbox_exporter to prober

['Sylvain Rabot' via Prometheus Developers]

No.

> On 20 Jan 2022, at 15:41, Julien Pivotto  wrote:
> 
> Dear Prometheans,
> 
> As per our governance, I'd like to cast a vote to rename the Blackbox
> Exporter to Prober.
> This vote is based on the following thread:
> https://groups.google.com/g/prometheus-developers/c/advMjgmJ1E4/m/A0abCsUrBgAJ
> 
> Any Prometheus team member is eligible to vote, and votes for the
> community are welcome too, but do not formally count in the result.
> 
> Here is the content of the vote:
> 
>> We want to rename Blackbox Exporter to Prober.
> 
> I explicitly leave out the "how" out of this vote. If this vote passes,
> a specific issue will be created in the blackbox exporter repository
> explaining how I plan to work and communicate on this change. I will
> make sure that enough time passes so that as many people as possible can
> give their input on the "how".
> 
> The vote is open until February 3rd. If the vote comes positive before
> next week's dev summit, the "how" can also be discussed during the dev
> summit, and I would use that discussion as input for the previously
> mentioned github issue.
> 
> -- 
> Julien Pivotto
> @roidelapluie
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to prometheus-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/prometheus-developers/20220120144119.GA522055%40hydrogen.

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/7648BD93-A509-4EB7-BF12-E2961BD284BF%40abstraction.fr.

Re: [prometheus-developers] Lazy consensus: Merging options

[Sylvain Rabot]

Squashing and rebasing also sucks for people who sign their commits.

On Thu, 3 Dec 2020 at 16:55, Sylvain Rabot  wrote:

> I also don’t like squashing.
>
> I don’t count the hours lost because I thought a commit referenced in a
> merged PR was not in a tag because the squash generated a new commit id.
>
> On 3 Dec 2020, at 15:27, Frederic Branczyk  wrote:
>
> 
> I don’t like squash merging, don’t think I’ve ever used rebase merging but
> don’t feel too strongly about it. Merge commit is my preference.
>
> On Thu 3. Dec 2020 at 15:06, Julien Pivotto 
> wrote:
>
>> On 03 Dec 14:59, Bartłomiej Płotka wrote:
>> > I am ok with this proposal.
>> >
>> > Long term I would even vote for squash only, but we discussed this in
>> the
>> > past.
>>
>> How would you merge release branches in master?
>>
>> >
>> > Kind Regards,
>> > Bartek Płotka (@bwplotka)
>> >
>> >
>> > On Thu, 3 Dec 2020 at 14:20, Brian Brazil <
>> brian.bra...@robustperception.io>
>> > wrote:
>> >
>> > > On Thu, 3 Dec 2020 at 13:15, Ben Kochie  wrote:
>> > >
>> > >> I'd like to adjust our defaults for GitHub merging settings:
>> > >>
>> > >> Right now, we allow all three modes for PR merges.
>> > >> * Merge commits
>> > >> * Squash merging
>> > >> * Rebase merging
>> > >>
>> > >> Proposal: Remove rebase merging (aka fast-forward merges) so that we
>> > >> stick to merge/squash and merge.
>> > >>
>> > >
>> > > I use rebase merges sometimes to keep the history clean from
>> > > unnecessary merge commits, so I'd like it to hang around.
>> > >
>> > > Brian
>> > >
>> > >
>> > >>
>> > >> [image: image.png]
>> > >>
>> > >> --
>> > >> You received this message because you are subscribed to the Google
>> Groups
>> > >> "Prometheus Developers" group.
>> > >> To unsubscribe from this group and stop receiving emails from it,
>> send an
>> > >> email to prometheus-developers+unsubscr...@googlegroups.com.
>> > >> To view this discussion on the web visit
>> > >>
>> https://groups.google.com/d/msgid/prometheus-developers/CABbyFmp0X26pjfvyATvaUxH9p_nwBh0QSMgtJGNzfDLnZJjdMQ%40mail.gmail.com
>> > >> <
>> https://groups.google.com/d/msgid/prometheus-developers/CABbyFmp0X26pjfvyATvaUxH9p_nwBh0QSMgtJGNzfDLnZJjdMQ%40mail.gmail.com?utm_medium=email&utm_source=footer
>> >
>> > >> .
>> > >>
>> > >
>> > >
>> > > --
>> > > Brian Brazil
>> > > www.robustperception.io
>> > >
>> > > --
>> > > You received this message because you are subscribed to the Google
>> Groups
>> > > "Prometheus Developers" group.
>> > > To unsubscribe from this group and stop receiving emails from it,
>> send an
>> > > email to prometheus-developers+unsubscr...@googlegroups.com.
>> > > To view this discussion on the web visit
>> > >
>> https://groups.google.com/d/msgid/prometheus-developers/CAHJKeLpwuPY6iE0k7zRP8PFAGTrEx9hYzx6j%3DQT8p4hLQVF6-w%40mail.gmail.com
>> > > <
>> https://groups.google.com/d/msgid/prometheus-developers/CAHJKeLpwuPY6iE0k7zRP8PFAGTrEx9hYzx6j%3DQT8p4hLQVF6-w%40mail.gmail.com?utm_medium=email&utm_source=footer
>> >
>> > > .
>> > >
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups "Prometheus Developers" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an email to prometheus-developers+unsubscr...@googlegroups.com.
>> > To view this discussion on the web visit
>> https://groups.google.com/d/msgid/prometheus-developers/CAMssQwahWTP3uPQuEDcu8jQB_EBDe5AOKXrJYd6%2Bad-wqOpEFQ%40mail.gmail.com
>> .
>>
>>
>>
>> --
>> Julien Pivotto
>> @roidelapluie
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Prometheus Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to prometheus-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/prometheus-developers/20201203140641.GA543460%40oxygen
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-developers/CAOs1Umx8Ug%2BaT3sr7VEw7ryngY4Fm7Fzzdp7z6QO6ODpWXa7mQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/prometheus-developers/CAOs1Umx8Ug%2BaT3sr7VEw7ryngY4Fm7Fzzdp7z6QO6ODpWXa7mQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
>

-- 
Sylvain Rabot 

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CADjtP1EZbOzod29V309WqA%2B1S7U7cn-BRFxTPs4kszRO4w5ctA%40mail.gmail.com.

Re: [prometheus-developers] Lazy consensus: Merging options

[Sylvain Rabot]

I also don’t like squashing.

I don’t count the hours lost because I thought a commit referenced in a merged 
PR was not in a tag because the squash generated a new commit id.

> On 3 Dec 2020, at 15:27, Frederic Branczyk  wrote:
> 
> 
> I don’t like squash merging, don’t think I’ve ever used rebase merging but 
> don’t feel too strongly about it. Merge commit is my preference. 
> 
>> On Thu 3. Dec 2020 at 15:06, Julien Pivotto  
>> wrote:
>> On 03 Dec 14:59, Bartłomiej Płotka wrote:
>> > I am ok with this proposal.
>> > 
>> > Long term I would even vote for squash only, but we discussed this in the
>> > past.
>> 
>> How would you merge release branches in master?
>> 
>> > 
>> > Kind Regards,
>> > Bartek Płotka (@bwplotka)
>> > 
>> > 
>> > On Thu, 3 Dec 2020 at 14:20, Brian Brazil 
>> > 
>> > wrote:
>> > 
>> > > On Thu, 3 Dec 2020 at 13:15, Ben Kochie  wrote:
>> > >
>> > >> I'd like to adjust our defaults for GitHub merging settings:
>> > >>
>> > >> Right now, we allow all three modes for PR merges.
>> > >> * Merge commits
>> > >> * Squash merging
>> > >> * Rebase merging
>> > >>
>> > >> Proposal: Remove rebase merging (aka fast-forward merges) so that we
>> > >> stick to merge/squash and merge.
>> > >>
>> > >
>> > > I use rebase merges sometimes to keep the history clean from
>> > > unnecessary merge commits, so I'd like it to hang around.
>> > >
>> > > Brian
>> > >
>> > >
>> > >>
>> > >> [image: image.png]
>> > >>
>> > >> --
>> > >> You received this message because you are subscribed to the Google 
>> > >> Groups
>> > >> "Prometheus Developers" group.
>> > >> To unsubscribe from this group and stop receiving emails from it, send 
>> > >> an
>> > >> email to prometheus-developers+unsubscr...@googlegroups.com.
>> > >> To view this discussion on the web visit
>> > >> https://groups.google.com/d/msgid/prometheus-developers/CABbyFmp0X26pjfvyATvaUxH9p_nwBh0QSMgtJGNzfDLnZJjdMQ%40mail.gmail.com
>> > >> 
>> > >> .
>> > >>
>> > >
>> > >
>> > > --
>> > > Brian Brazil
>> > > www.robustperception.io
>> > >
>> > > --
>> > > You received this message because you are subscribed to the Google Groups
>> > > "Prometheus Developers" group.
>> > > To unsubscribe from this group and stop receiving emails from it, send an
>> > > email to prometheus-developers+unsubscr...@googlegroups.com.
>> > > To view this discussion on the web visit
>> > > https://groups.google.com/d/msgid/prometheus-developers/CAHJKeLpwuPY6iE0k7zRP8PFAGTrEx9hYzx6j%3DQT8p4hLQVF6-w%40mail.gmail.com
>> > > 
>> > > .
>> > >
>> > 
>> > -- 
>> > You received this message because you are subscribed to the Google Groups 
>> > "Prometheus Developers" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an 
>> > email to prometheus-developers+unsubscr...@googlegroups.com.
>> > To view this discussion on the web visit 
>> > https://groups.google.com/d/msgid/prometheus-developers/CAMssQwahWTP3uPQuEDcu8jQB_EBDe5AOKXrJYd6%2Bad-wqOpEFQ%40mail.gmail.com.
>> 
>> 
>> 
>> -- 
>> Julien Pivotto
>> @roidelapluie
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Prometheus Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to prometheus-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/prometheus-developers/20201203140641.GA543460%40oxygen.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to prometheus-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/prometheus-developers/CAOs1Umx8Ug%2BaT3sr7VEw7ryngY4Fm7Fzzdp7z6QO6ODpWXa7mQ%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/3E965868-F280-4C81-9895-D6BBFF469458%40abstraction.fr.

Re: [prometheus-developers] Enable Dependabot on prometheus repos

[Sylvain Rabot]

It is as long as you don’t vendor deps.

> On 8 Jun 2020, at 14:10, Simon Pasquier  wrote:
> 
> +1 for the React codebase.
> FWIW last time I looked at it, dependabot wasn't really ready for
> managing Go dependencies.
> 
>> On Mon, Jun 8, 2020 at 7:11 AM Julien Pivotto
>>  wrote:
>> 
>> I +1 this
>> 
>>> Le lun. 8 juin 2020 à 07:09, Ben Kochie  a écrit :
>>> 
>>> I'd like to enable Dependabot on prometheus/prometheus, specifically to 
>>> automatically handle the React UI version bumps. This will deal with the 
>>> small version bumps for all the random security vulnerabilities that pop up.
>>> 
>>> The bot will create PRs like this one:
>>> 
>>> https://github.com/prometheus-community/monaco-promql/pull/12
>>> 
>>> --
>>> You received this message because you are subscribed to the Google Groups 
>>> "Prometheus Developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to prometheus-developers+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/prometheus-developers/CABbyFmrdAwW8Qp0%2BemgwQQKWNpPVUA1RrRVuOXAndbM3VpPHLQ%40mail.gmail.com.
>> 
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Prometheus Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to prometheus-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/prometheus-developers/CAFJ6V0rngch9PCYup_svH4nYSNyZ2VeMUnFbym%3DS4qOSsyvPVw%40mail.gmail.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to prometheus-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/prometheus-developers/CAM6RFu4zFvwzS1DB9O8eUEoHikukX5fape9p9s6%2BFi66ycSxkw%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CE29ECDF-675F-4174-BCAB-66E3C69747FB%40abstraction.fr.

Re: [prometheus-developers] Removing Vendor

[Sylvain Rabot]

Several maintainers have given their thoughts on the subjects now.

What would be the next step ?

Should this be put to a vote ?

Regards.

On Mon, 9 Mar 2020 at 16:26, Julien Pivotto  wrote:

> Hi there,
>
> Sylvain has open a pull request to remove the vendor directory. We had
> larger threads before on versioning etc, but I would like to start a new
> thread about this particular topic to see if we have a consensus.
>
> https://github.com/prometheus/prometheus/pull/6949
>
> One of the blocker I have is that I would like to be 100% sure that the
> checks that are run by CNCF on licence compatibility have full support
> for this, e.g. they can fetch the modules and scan the licenses.
>
> --
>  (o-Julien Pivotto
>  //\Open-Source Consultant
>  V_/_   Inuits - https://www.inuits.eu
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-developers/20200309152603.GA25698%40oxygen
> .
>


-- 
Sylvain Rabot 

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CADjtP1GOLZQ3-WM5GXJjaouCp0T_BUq%3DsueihUAyeEX6Pf31yA%40mail.gmail.com.

Re: [prometheus-developers] Removing Vendor

[Sylvain Rabot]

I wish I would have been able to write this argument.

Thanks Bartek.

On Mon, 9 Mar 2020 at 20:59, Bartłomiej Płotka  wrote:

> Unpopular opinion: IMO vendoring can be long term a really bad pattern.
> The go.mod, go modules or any other dependency management tool or dep
> caching (e.g GOPROXY) was created *exactly* to avoid copying all the
> dependencies to your repository. (:
>
> So I would be happy to remove vendor dir and work towards safe caching
> dependencies in some other repository, GOPROXY etc.
>
> > I'm talking about vendoring in general. We'd want a good reason to
> change our build system in this way, and I'm not aware of any reason why
> we'd want to change this with our current build system.
>
> *Fair. Would be nice to enumerate the downsides:*
>
> *  We update deps mostly when adding some feature etc. This means
> usually us being lazy and doing feature code + committing thousands of
> files. Fun to review. Splitting by commit does not really help. Rarely we
> review like this and still it's super noisy, it's easy to miss a meaningful
> change in our codebase hidden among 300k lines of codes. ):
> * Repo size dramatically larger. Cloning etc operations e.g. CI checkout.
> * Git stats totally malformed (lines changed).
> * We host in our repo **not our code**.
>
> > Having it locally is handy for debugging, and it means we at least have
> a copy and a clear record of what changed when. It also more easily allows
> for local development.
>
> I don't get it. go.mod and go.sum are versioned so those already give the
> clear record. For local development `go mod vendor` locally is a one-off
> step (local vendor will keep being updated for lifetime of your local
> repo), so I don't any problem with this.
>
> I think the main argument for having a vendor folder is to be safe on a
> malicious act of dependency actor, which removes or compromises the
> dependency. Anyone can replace the code for a given tag anytime really (:
> Given that it never happened, not sure if worth pursuing. Wonder why no one
> thought about some separate repo solution just for deps then (:
>
> Bartek
>
> On Mon, 9 Mar 2020 at 18:19, Sylvain Rabot  wrote:
>
>> On Mon, 9 Mar 2020 at 16:54, Brian Brazil <
>> brian.bra...@robustperception.io> wrote:
>>
>>> On Mon, 9 Mar 2020 at 15:41, Julien Pivotto 
>>> wrote:
>>>
>>>> On 09 Mar 15:30, Brian Brazil wrote:
>>>> > On Mon, 9 Mar 2020 at 15:26, Julien Pivotto 
>>>> wrote:
>>>> >
>>>> > > Hi there,
>>>> > >
>>>> > > Sylvain has open a pull request to remove the vendor directory. We
>>>> had
>>>> > > larger threads before on versioning etc, but I would like to start
>>>> a new
>>>> > > thread about this particular topic to see if we have a consensus.
>>>> > >
>>>> > > https://github.com/prometheus/prometheus/pull/6949
>>>> > >
>>>> > > One of the blocker I have is that I would like to be 100% sure that
>>>> the
>>>> > > checks that are run by CNCF on licence compatibility have full
>>>> support
>>>> > > for this, e.g. they can fetch the modules and scan the licenses.
>>>> > >
>>>> >
>>>> > The question here is that as this is all internal, does this make
>>>> sense for
>>>> > our build&review processes?
>>>>
>>>>
>>>> Can you clarify the question. Are you speaking about removing vendoring
>>>> or license check?
>>>>
>>>
>>> I'm talking about vendoring in general. We'd want a good reason to
>>> change our build system in this way, and I'm not aware of any reason why
>>> we'd want to change this with our current build system.
>>>
>>
>>>
>>>> > I don't recall offhand anyone in -team having issues with our current
>>>> state
>>>> > (beyond a brief discussion if we still wanted it when we switched to
>>>> go-mod
>>>> > - we do).
>>>>
>>>> I am all for keeping vendor but I have not really strong arguments
>>>> against removing it.
>>>>
>>>> I often use it to grep some dependencies sources but I could still run
>>>> go mod vendor locally.
>>>>
>>>
>>> Having it locally is handy for debugging, and it means we at least have
>>> a copy and a clear record of what changed when.

Re: [prometheus-developers] Removing Vendor

[Sylvain Rabot]

On Mon, 9 Mar 2020 at 16:54, Brian Brazil 
wrote:

> On Mon, 9 Mar 2020 at 15:41, Julien Pivotto 
> wrote:
>
>> On 09 Mar 15:30, Brian Brazil wrote:
>> > On Mon, 9 Mar 2020 at 15:26, Julien Pivotto 
>> wrote:
>> >
>> > > Hi there,
>> > >
>> > > Sylvain has open a pull request to remove the vendor directory. We had
>> > > larger threads before on versioning etc, but I would like to start a
>> new
>> > > thread about this particular topic to see if we have a consensus.
>> > >
>> > > https://github.com/prometheus/prometheus/pull/6949
>> > >
>> > > One of the blocker I have is that I would like to be 100% sure that
>> the
>> > > checks that are run by CNCF on licence compatibility have full support
>> > > for this, e.g. they can fetch the modules and scan the licenses.
>> > >
>> >
>> > The question here is that as this is all internal, does this make sense
>> for
>> > our build&review processes?
>>
>>
>> Can you clarify the question. Are you speaking about removing vendoring
>> or license check?
>>
>
> I'm talking about vendoring in general. We'd want a good reason to change
> our build system in this way, and I'm not aware of any reason why we'd want
> to change this with our current build system.
>

>
>> > I don't recall offhand anyone in -team having issues with our current
>> state
>> > (beyond a brief discussion if we still wanted it when we switched to
>> go-mod
>> > - we do).
>>
>> I am all for keeping vendor but I have not really strong arguments
>> against removing it.
>>
>> I often use it to grep some dependencies sources but I could still run
>> go mod vendor locally.
>>
>
> Having it locally is handy for debugging, and it means we at least have a
> copy and a clear record of what changed when. It also more easily allows
> for local development.
>

I personally don't have any trouble doing any of that without vendoring.

Also, it's much easier to keep track of what happened to the dependencies
in the history of the go.mod than in the vendor/ dir.

I admit it can be handy to grep in the vendor/ dir although I personally
hate it when I'm looking for something and I get submerged by results in
the vendor/ dir.


> --
> Brian Brazil
> www.robustperception.io
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-developers/CAHJKeLqvvbvXvfMaDJomBM10hs2pLkiY6Q9fyuQZfo537b%2Bm6g%40mail.gmail.com
> <https://groups.google.com/d/msgid/prometheus-developers/CAHJKeLqvvbvXvfMaDJomBM10hs2pLkiY6Q9fyuQZfo537b%2Bm6g%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>


-- 
Sylvain Rabot 

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CADjtP1Fwqi_rn_27vZRgfUUwUj1QBGQeyPnVnZY8CHimtWcYvQ%40mail.gmail.com.

Re: [prometheus-developers] Prometheus violating Go modules semantic versioning rules - what to do?

[Sylvain Rabot]

Ok so I've been talking to Duco about this and he convinced me that using
in repo replace directives with relative paths is not a good idea.

If he has the time he can explain it much better than I could.

On Fri, 28 Feb 2020 at 12:04, Sylvain Rabot  wrote:

> After giving it more search, it came to (my) light that go mod handles
> "special" git tags for go mod sub-versioning.
>
>
> https://github.com/golang/go/wiki/Modules#what-are-multi-module-repositories
>
> The tag for version 1.2.3 of module "my-repo/foo/rop" is "foo/rop/v1.2.3".
>>
>
> I.E, "go get github.com/prometheus/prometheus/tsdb/v2@v2.13.0" would
> resolve "github.com/prometheus/prometheus.git" at tag "tsdb/v2.13.0"
>
> With that in mind, I believe having multiple independent go modules in the
> same git repo is OK.
>
> With this we can leverage the initial reason of having those "core"
> modules inside prometheus/prometheus (no sync overhead) and a decent
> independent development oriented life cycle.
>
> We currently have release shepherds, I think that if we were to implement
> this, we would need a shepherd per module responsible for releasing
> versions of "his" module(s).
>
> I made a Draft PR to get a sense of it:
> https://github.com/prometheus/prometheus/pull/6888/commits/44149825edb0b5c92885f362b6f3ae1643e2f451
>
> It's pretty light if we exclude removing vendoring altogether (because I
> don't believe vendoring makes sense anymore with the latest versions of go
> and because each module having its own go mod needs its own vendor/ dir).
>
> The only problem I encountered came from promu. In the case we want to
> build a binary in a module that has its own go.mod file (like tsdb), the go
> build command needs to be run from within the dir of the module (because of
> the relative paths in the go.mod replace directive I assume). I already
> have a patch for this (
> https://github.com/sylr/promu/commit/c6e6a2bcdf5d50338b1ee70022c40152a8992c82
> ).
>
> Regards.
>
> On Fri, 7 Feb 2020 at 18:12, Bjoern Rabenstein  wrote:
>
>> I think it became already obvious from the discussion so far: Semantic
>> versioning for the user of the Prometheus server is completely
>> different from semantic versioning for the user of Go packages
>> contained in the Prometheus code base.
>>
>> Conflating both will be bad for both parties.
>>
>> We could have a _separate_ versioning, using vx.y.z tags for the Go
>> module versioning and a new tag naming schema (e.g. release-x.y.z) for
>> user releases.
>>
>> Besides the obvious concerns about this, I also think that semantic
>> versioning for a large number of Go packages together is problematic
>> in any case. Any breaking change in any one package will bump the
>> major version for _all_ packages, leading to new import paths in the
>> Go modules world, which can lead to interoperability problems for
>> indirect users of the packages. (They could find themselves in the
>> situation of assigning a v2/model.Label spit out by one of their
>> direct dependencies to a v3/model.Label required by another of their
>> direct dependencies.)
>>
>> That's why I think Duco's Modularize project has some
>> merits. Alternatively (as I said before), I could see better tooling
>> for maintaining multiple Go modules within the same Git repo as a
>> viable (and less invasive) solution.
>>
>> Therefore, I'd prefer to see how Modularize turns out in practice (or
>> if alternatively somebody comes up with tooling and/or easy-to-follow
>> best practices for multi-module repos) and then reconsider what we can
>> do in prometheus/prometheus.
>>
>> --
>> Björn Rabenstein
>> [PGP-ID] 0x851C3DA17D748D03
>> [email] bjo...@rabenste.in
>>
>
>
> --
> Sylvain Rabot 
>


-- 
Sylvain Rabot 

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CADjtP1EgLt--JMvyZrw%3DKyxd%3Da3WJ%3DoaWCsR0Euc5HQwBb7_GA%40mail.gmail.com.

Re: [prometheus-developers] Prometheus violating Go modules semantic versioning rules - what to do?

[Sylvain Rabot]

After giving it more search, it came to (my) light that go mod handles
"special" git tags for go mod sub-versioning.

https://github.com/golang/go/wiki/Modules#what-are-multi-module-repositories

The tag for version 1.2.3 of module "my-repo/foo/rop" is "foo/rop/v1.2.3".
>

I.E, "go get github.com/prometheus/prometheus/tsdb/v2@v2.13.0" would
resolve "github.com/prometheus/prometheus.git" at tag "tsdb/v2.13.0"

With that in mind, I believe having multiple independent go modules in the
same git repo is OK.

With this we can leverage the initial reason of having those "core" modules
inside prometheus/prometheus (no sync overhead) and a decent independent
development oriented life cycle.

We currently have release shepherds, I think that if we were to implement
this, we would need a shepherd per module responsible for releasing
versions of "his" module(s).

I made a Draft PR to get a sense of it:
https://github.com/prometheus/prometheus/pull/6888/commits/44149825edb0b5c92885f362b6f3ae1643e2f451

It's pretty light if we exclude removing vendoring altogether (because I
don't believe vendoring makes sense anymore with the latest versions of go
and because each module having its own go mod needs its own vendor/ dir).

The only problem I encountered came from promu. In the case we want to
build a binary in a module that has its own go.mod file (like tsdb), the go
build command needs to be run from within the dir of the module (because of
the relative paths in the go.mod replace directive I assume). I already
have a patch for this (
https://github.com/sylr/promu/commit/c6e6a2bcdf5d50338b1ee70022c40152a8992c82
).

Regards.

On Fri, 7 Feb 2020 at 18:12, Bjoern Rabenstein  wrote:

> I think it became already obvious from the discussion so far: Semantic
> versioning for the user of the Prometheus server is completely
> different from semantic versioning for the user of Go packages
> contained in the Prometheus code base.
>
> Conflating both will be bad for both parties.
>
> We could have a _separate_ versioning, using vx.y.z tags for the Go
> module versioning and a new tag naming schema (e.g. release-x.y.z) for
> user releases.
>
> Besides the obvious concerns about this, I also think that semantic
> versioning for a large number of Go packages together is problematic
> in any case. Any breaking change in any one package will bump the
> major version for _all_ packages, leading to new import paths in the
> Go modules world, which can lead to interoperability problems for
> indirect users of the packages. (They could find themselves in the
> situation of assigning a v2/model.Label spit out by one of their
> direct dependencies to a v3/model.Label required by another of their
> direct dependencies.)
>
> That's why I think Duco's Modularize project has some
> merits. Alternatively (as I said before), I could see better tooling
> for maintaining multiple Go modules within the same Git repo as a
> viable (and less invasive) solution.
>
> Therefore, I'd prefer to see how Modularize turns out in practice (or
> if alternatively somebody comes up with tooling and/or easy-to-follow
> best practices for multi-module repos) and then reconsider what we can
> do in prometheus/prometheus.
>
> --
> Björn Rabenstein
> [PGP-ID] 0x851C3DA17D748D03
> [email] bjo...@rabenste.in
>


-- 
Sylvain Rabot 

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CADjtP1FkVUSFx8hJpe10pTS3d4P_5NKHThu8nORKvyYJHCCavA%40mail.gmail.com.

Re: [prometheus-developers] Reduce list of GOOS/GOARCH crossbuilt for each PR

[Sylvain Rabot]

So I tried to reproduce what has been done in the golang-builder docker
image on the circleci host to be able to build with cgo but I'm stuck on an
error I do not understand:

https://app.circleci.com/jobs/github/prometheus/node_exporter/7835

~ building up to 1 concurrent crossbuilds
~ building up to 1 concurrent binaries
< building platform darwin/amd64
< builing binary darwin/amd64 node_exporter
+ GOOS=darwin GOARCH=amd64 go build -o .build/darwin-amd64/node_exporter
-ldflags "-X github.com/prometheus/common/version.Version=0.18.1 -X
github.com/prometheus/common/version.Revision=eba0002f1fba4f032722566699dc9036107a9a43
-X github.com/prometheus/common/version.Branch=pull/1606 -X
github.com/prometheus/common/version.BuildUser=circleci@default-5cf5bcf5-1942-4bca-b1f3-b262e5474fe1
-X github.com/prometheus/common/version.BuildDate=20200216-20:57:30
-extldflags '-static'" -mod=vendor -tags 'netgo static_build'
github.com/prometheus/node_exporter
# crypto/x509
ld: warning: object file (/tmp/go-build262275209/b077/_cgo_main.o) was
built for newer OSX version (10.10) than being linked (10.6)
ld: warning: object file (/tmp/go-build262275209/b077/_x001.o) was built
for newer OSX version (10.10) than being linked (10.6)
ld: warning: object file (/tmp/go-build262275209/b077/_x002.o) was built
for newer OSX version (10.10) than being linked (10.6)
# github.com/prometheus/node_exporter
/usr/local/go/pkg/tool/linux_amd64/link: running o64-clang failed: exit
status 1
ld: library not found for -lcrt0.o
clang: error: linker command failed with exit code 1 (use -v to see
invocation)

If anyone has insights about this I would be very grateful.

On Sun, 16 Feb 2020 at 14:19, Sylvain Rabot  wrote:

>
> On Sun, 16 Feb 2020 at 13:41, Tobias Schmidt  wrote:
>
>>
>>
>> On Sun, Feb 16, 2020 at 11:38 AM Sylvain Rabot 
>> wrote:
>>
>>> I've also been looking into it.
>>>
>>> The sharing the go build cache would be the ultimate solution but for
>>> prometheus, for all current goos/goarch, it accounts for more than 6GB and
>>> I believe that is too much for circleci to handle (they recommend keeping
>>> cache under 500MB).
>>>
>>> However I made a PR on promu that replace current crossbuild process
>>> with one that (amongst other things) does not use docker anymore and it
>>> seems that it *halves* the duration of building all current goos/goarch:
>>>
>>
>> How does this work for node_exporter with its CGO usage?
>>
>
> I don't know, I'll try it. But if it does not work, I moved the docker
> cross-building in a separate promu command ("promu crossbuild-docker") so
> we could still use that for node_exporter.
>
>
>>
>>
>>>
>>> - https://github.com/prometheus/promu/pull/177
>>> - https://github.com/prometheus/prometheus/pull/6824
>>> -
>>> https://app.circleci.com/github/prometheus/prometheus/pipelines/838ffe3c-b738-430b-8cf0-c707842e7e8f/workflows/dd50dabc-04e3-4b48-8a36-c85456668c2f
>>>
>>> Regards.
>>>
>>> On Fri, 14 Feb 2020 at 14:48, Simon Pasquier 
>>> wrote:
>>>
>>>> FWIW I'm looking into this. From my understanding, the bulk of the
>>>> issue is that "promu crossbuild" can't reuse the Go build cache
>>>> between CI runs hence it rebuilds everything from scratch every time.
>>>>
>>>> On Thu, Feb 13, 2020 at 5:36 PM Ben Kochie  wrote:
>>>> >
>>>> > Part of the problem is that we run the builds in serial. We have a
>>>> lot more compute capacity in our CircleCI for running more docker tasks in
>>>> parallel, but promu doesn't know how to distribute the work.
>>>> >
>>>> > But I also think we could cut the build step down for PRs. But I
>>>> think we should keep the full build in master.
>>>> >
>>>> > On Wed, Feb 12, 2020 at 3:50 PM Chris Marchbanks <
>>>> csmarchba...@gmail.com> wrote:
>>>> >>
>>>> >> I also support this, waiting 2-3 hours for the build job to finish
>>>> is frustrating. I know that building on 32 bit architectures does not catch
>>>> all issues, specifically the alignment bug using the atomic package.
>>>> Perhaps add at least one 32 bit build on the pull request though?
>>>> >>
>>>> >> Is it worth it to build everything on every master, or should a
>>>> build all job be added to the nightly build? I agree that we should build
>>>> everything on a cadence more frequent than a release.
>>>> >>
&

Re: [prometheus-developers] Moving prometheus/tsdb to prometheus-junkyard/tsdb

[Sylvain Rabot]

Thank you.

On Sun, 16 Feb 2020 at 16:15, Julien Pivotto  wrote:

> All this seems at least unrelated to the move of an archived repo to the
> junkyard after 200 days. I will test moving a repo back and forth before
> moving tsdb, just to be 100÷ sure that we can do it, so we can close the
> discussion here.
>
>
> - Original Message -
> From: Sylvain Rabot 
> To: Bartłomiej Płotka 
> Cc: Julien Pivotto , prometheus-developers <
> prometheus-developers@googlegroups.com>, Duco van Amstel <
> duco.vanams...@gmail.com>
> Sent: Sun, 16 Feb 2020 15:55:59 +0100 (CET)
> Subject: Re: [prometheus-developers] Moving prometheus/tsdb to
> prometheus-junkyard/tsdb
>
> On Sun, 16 Feb 2020 at 15:08, Sylvain Rabot 
> wrote:
>
> > On Sun, 16 Feb 2020 at 14:34, Bartłomiej Płotka 
> > wrote:
> >
> >> I fully agree with your arguments here. Since both Cortex and Thanos are
> >> very both active in Prometheus Team, it's easier for us to understand
> that
> >> potential obstacles each version introduces. That's why I am looking
> >> forward for proper versioning in some way of another.
> >>
> >> If we did move prematurely... Maybe, but we acknowledged that only
> >> Prometheus, Thanos, Cortex, Loki and Conprof are using TSDB. Any other
> >> project, if using, did not give any objections on public discussions if
> to
> >> move TSDB or not. Since it's was very painful for Prometheus we went
> ahead.
> >> If you think we were wrong or we could improve something in the process,
> >> let us know. (:
> >>
> >
> > Björn comment about waiting for the outcome of the go module compliance
> > has been ignored (
> >
> https://github.com/prometheus/prometheus/pull/5805#issuecomment-516387796)
> > although 3 people, me included, sided with him with a thumb up.
> >
>
> To be fair he later green lighted it:
> https://github.com/prometheus/prometheus/pull/5805#issuecomment-520774612
>
>
> >
> >> In the same way decision to put TSDB now to junkyard was done in public
> >> and with several months ahead.
> >>
> >> Anyway I think when moved we can get back TSDB alone anytime, so this
> >> should answer your concern. However, in fact you can fork it and bring
> back
> >> as your fork anytime as well: it's open source. But we, as a team,
> agreed
> >> to maintain it inside Prometheus repo and IMO we see many benefits of
> it:
> >> e.g reduction of complexity around tons of unnecessary interfaces,
> velocity
> >> of release process and tsdb bug fixes! I think it was a good decision.
> (:
> >>
> >> Kind Regards,
> >> Bartek
> >>
> >> On Sun, 16 Feb 2020, 13:16 Sylvain Rabot, 
> wrote:
> >>
> >>> It wouldn't have, separate versioning can't solve merging upstreams
> that
> >>> have breaking changes.
> >>>
> >>> It's not about that, it's about respect towards people relying on TSDB.
> >>>
> >>> I'd like prometheus people to acknowledge that their project is growing
> >>> outside their reach. This is a recognition that they did well, very
> well,
> >>> and *they should be really proud of that and embrace it*. I believe
> >>> that embracing it means starting to respect common development rules
> and it
> >>> should start with go modules conventions and SEMVER.
> >>>
> >>> Like I said, TSDB is a cornerstone of the prometheus ecosystem.
> >>>
> >>> If we want to make the ecosystem thrive, I believe *major components
> >>> like TSDB* should have their own *development oriented lifecycle*
> >>> (prometheus/prometheus has an user oriented lifecycle) which respects
> both
> >>> go module conventions and SEMVER git tags to make life easier for third
> >>> parties that use them.
> >>>
> >>> That does not prevent anyone to make breaking changes to them and that
> >>> advertises those who rely on it what they can expect when managing
> their
> >>> prometheus dependencies.
> >>>
> >>> You as Thanos maintainer want to upgrade from TSDB v0.7.0 to v0.7.2,
> you
> >>> should have almost nothing to do. But, if you want to upgrade from
> v0.7.2
> >>> to v2.0.3, you would know that should roll up your sleeves because
> you're
> >>> in for some heavy lifting.
> >>>
> >>> Today if you want to upgrade prometheus in Thanos from 2.15.0 to 2.16.

Re: [prometheus-developers] Reduce list of GOOS/GOARCH crossbuilt for each PR

[Sylvain Rabot]

I'm not removing it, I am adding an in host crossbuild process.

Results I have show that doing the full cross-building on the host is 10
minutes faster (and not twice faster like I said previously) that the 2
hours it takes with docker on prometheus/prometheus (
https://app.circleci.com/jobs/github/prometheus/prometheus/25973).

The real improvement (without sharing go cache across builds) comes from
removing the "-a" flag in the go build flags.

On Sun, 16 Feb 2020 at 16:17, Julien Pivotto  wrote:

> What is the reasoning behind the removal of the docker build?
>
> - Original Message -----
> From: Sylvain Rabot 
> To: Tobias Schmidt 
> Cc: Simon Pasquier , Ben Kochie ,
> Chris Marchbanks , Matthias Rampke <
> m...@soundcloud.com>, Krasimir Georgiev , Prometheus
> Developers 
> Sent: Sun, 16 Feb 2020 14:19:55 +0100 (CET)
> Subject: Re: [prometheus-developers] Reduce list of GOOS/GOARCH crossbuilt
> for each PR
>
> On Sun, 16 Feb 2020 at 13:41, Tobias Schmidt  wrote:
>
> >
> >
> > On Sun, Feb 16, 2020 at 11:38 AM Sylvain Rabot 
> > wrote:
> >
> >> I've also been looking into it.
> >>
> >> The sharing the go build cache would be the ultimate solution but for
> >> prometheus, for all current goos/goarch, it accounts for more than 6GB
> and
> >> I believe that is too much for circleci to handle (they recommend
> keeping
> >> cache under 500MB).
> >>
> >> However I made a PR on promu that replace current crossbuild process
> with
> >> one that (amongst other things) does not use docker anymore and it seems
> >> that it *halves* the duration of building all current goos/goarch:
> >>
> >
> > How does this work for node_exporter with its CGO usage?
> >
>
> I don't know, I'll try it. But if it does not work, I moved the docker
> cross-building in a separate promu command ("promu crossbuild-docker") so
> we could still use that for node_exporter.
>
>
> >
> >
> >>
> >> - https://github.com/prometheus/promu/pull/177
> >> - https://github.com/prometheus/prometheus/pull/6824
> >> -
> >>
> https://app.circleci.com/github/prometheus/prometheus/pipelines/838ffe3c-b738-430b-8cf0-c707842e7e8f/workflows/dd50dabc-04e3-4b48-8a36-c85456668c2f
> >>
> >> Regards.
> >>
> >> On Fri, 14 Feb 2020 at 14:48, Simon Pasquier 
> wrote:
> >>
> >>> FWIW I'm looking into this. From my understanding, the bulk of the
> >>> issue is that "promu crossbuild" can't reuse the Go build cache
> >>> between CI runs hence it rebuilds everything from scratch every time.
> >>>
> >>> On Thu, Feb 13, 2020 at 5:36 PM Ben Kochie  wrote:
> >>> >
> >>> > Part of the problem is that we run the builds in serial. We have a
> lot
> >>> more compute capacity in our CircleCI for running more docker tasks in
> >>> parallel, but promu doesn't know how to distribute the work.
> >>> >
> >>> > But I also think we could cut the build step down for PRs. But I
> think
> >>> we should keep the full build in master.
> >>> >
> >>> > On Wed, Feb 12, 2020 at 3:50 PM Chris Marchbanks <
> >>> csmarchba...@gmail.com> wrote:
> >>> >>
> >>> >> I also support this, waiting 2-3 hours for the build job to finish
> is
> >>> frustrating. I know that building on 32 bit architectures does not
> catch
> >>> all issues, specifically the alignment bug using the atomic package.
> >>> Perhaps add at least one 32 bit build on the pull request though?
> >>> >>
> >>> >> Is it worth it to build everything on every master, or should a
> build
> >>> all job be added to the nightly build? I agree that we should build
> >>> everything on a cadence more frequent than a release.
> >>> >>
> >>> >> On Wed, Feb 12, 2020 at 1:58 AM 'Matthias Rampke' via Prometheus
> >>> Developers  wrote:
> >>> >>>
> >>> >>> I would build everything on master, that way we catch before
> >>> starting a release if there is something wrong.
> >>> >>>
> >>> >>> /MR
> >>> >>>
> >>> >>> On Wed, Feb 12, 2020 at 8:37 AM Sylvain Rabot <
> >>> sylv...@abstraction.fr> wrote:
> >>> >>>>
> >>> >>>> I did not say it but I was speaking of pro

Re: [prometheus-developers] Moving prometheus/tsdb to prometheus-junkyard/tsdb

[Sylvain Rabot]

On Sun, 16 Feb 2020 at 15:08, Sylvain Rabot  wrote:

> On Sun, 16 Feb 2020 at 14:34, Bartłomiej Płotka 
> wrote:
>
>> I fully agree with your arguments here. Since both Cortex and Thanos are
>> very both active in Prometheus Team, it's easier for us to understand that
>> potential obstacles each version introduces. That's why I am looking
>> forward for proper versioning in some way of another.
>>
>> If we did move prematurely... Maybe, but we acknowledged that only
>> Prometheus, Thanos, Cortex, Loki and Conprof are using TSDB. Any other
>> project, if using, did not give any objections on public discussions if to
>> move TSDB or not. Since it's was very painful for Prometheus we went ahead.
>> If you think we were wrong or we could improve something in the process,
>> let us know. (:
>>
>
> Björn comment about waiting for the outcome of the go module compliance
> has been ignored (
> https://github.com/prometheus/prometheus/pull/5805#issuecomment-516387796)
> although 3 people, me included, sided with him with a thumb up.
>

To be fair he later green lighted it:
https://github.com/prometheus/prometheus/pull/5805#issuecomment-520774612


>
>> In the same way decision to put TSDB now to junkyard was done in public
>> and with several months ahead.
>>
>> Anyway I think when moved we can get back TSDB alone anytime, so this
>> should answer your concern. However, in fact you can fork it and bring back
>> as your fork anytime as well: it's open source. But we, as a team, agreed
>> to maintain it inside Prometheus repo and IMO we see many benefits of it:
>> e.g reduction of complexity around tons of unnecessary interfaces, velocity
>> of release process and tsdb bug fixes! I think it was a good decision. (:
>>
>> Kind Regards,
>> Bartek
>>
>> On Sun, 16 Feb 2020, 13:16 Sylvain Rabot,  wrote:
>>
>>> It wouldn't have, separate versioning can't solve merging upstreams that
>>> have breaking changes.
>>>
>>> It's not about that, it's about respect towards people relying on TSDB.
>>>
>>> I'd like prometheus people to acknowledge that their project is growing
>>> outside their reach. This is a recognition that they did well, very well,
>>> and *they should be really proud of that and embrace it*. I believe
>>> that embracing it means starting to respect common development rules and it
>>> should start with go modules conventions and SEMVER.
>>>
>>> Like I said, TSDB is a cornerstone of the prometheus ecosystem.
>>>
>>> If we want to make the ecosystem thrive, I believe *major components
>>> like TSDB* should have their own *development oriented lifecycle*
>>> (prometheus/prometheus has an user oriented lifecycle) which respects both
>>> go module conventions and SEMVER git tags to make life easier for third
>>> parties that use them.
>>>
>>> That does not prevent anyone to make breaking changes to them and that
>>> advertises those who rely on it what they can expect when managing their
>>> prometheus dependencies.
>>>
>>> You as Thanos maintainer want to upgrade from TSDB v0.7.0 to v0.7.2, you
>>> should have almost nothing to do. But, if you want to upgrade from v0.7.2
>>> to v2.0.3, you would know that should roll up your sleeves because you're
>>> in for some heavy lifting.
>>>
>>> Today if you want to upgrade prometheus in Thanos from 2.15.0 to 2.16.0
>>> you have absolutely no idea what you are stepping into. Well, you
>>> personally might as you are quite involved in both projects but that is not
>>> the case for everyone.
>>>
>>> That's why I believe that the TSDB move was a bad thing and should never
>>> have happened. Now that is has, I just want to make sure it's not something
>>> we can't go back from. It might never happen, or it might, only time will
>>> tell.
>>>
>>> Regards.
>>>
>>>
>>>
>>>
>>>
>>> On Sun, 16 Feb 2020 at 13:23, Bartłomiej Płotka 
>>> wrote:
>>>
>>>> At least when it fails it fails immediately in build time :p
>>>>
>>>> So not sure how separate versioning would help you there. It would be a
>>>> new major version of tsdb module, sure. But what then? It's will require
>>>> extra manual bump anyway then you will have same work to be done, am I
>>>> right?
>>>>
>>>>
>>>> Kind Regards,
>>>> B

Re: [prometheus-developers] Moving prometheus/tsdb to prometheus-junkyard/tsdb

[Sylvain Rabot]

On Sun, 16 Feb 2020 at 14:34, Bartłomiej Płotka  wrote:

> I fully agree with your arguments here. Since both Cortex and Thanos are
> very both active in Prometheus Team, it's easier for us to understand that
> potential obstacles each version introduces. That's why I am looking
> forward for proper versioning in some way of another.
>
> If we did move prematurely... Maybe, but we acknowledged that only
> Prometheus, Thanos, Cortex, Loki and Conprof are using TSDB. Any other
> project, if using, did not give any objections on public discussions if to
> move TSDB or not. Since it's was very painful for Prometheus we went ahead.
> If you think we were wrong or we could improve something in the process,
> let us know. (:
>

Björn comment about waiting for the outcome of the go module compliance has
been ignored (
https://github.com/prometheus/prometheus/pull/5805#issuecomment-516387796)
although 3 people, me included, sided with him with a thumb up.


>
> In the same way decision to put TSDB now to junkyard was done in public
> and with several months ahead.
>
> Anyway I think when moved we can get back TSDB alone anytime, so this
> should answer your concern. However, in fact you can fork it and bring back
> as your fork anytime as well: it's open source. But we, as a team, agreed
> to maintain it inside Prometheus repo and IMO we see many benefits of it:
> e.g reduction of complexity around tons of unnecessary interfaces, velocity
> of release process and tsdb bug fixes! I think it was a good decision. (:
>
> Kind Regards,
> Bartek
>
> On Sun, 16 Feb 2020, 13:16 Sylvain Rabot,  wrote:
>
>> It wouldn't have, separate versioning can't solve merging upstreams that
>> have breaking changes.
>>
>> It's not about that, it's about respect towards people relying on TSDB.
>>
>> I'd like prometheus people to acknowledge that their project is growing
>> outside their reach. This is a recognition that they did well, very well,
>> and *they should be really proud of that and embrace it*. I believe that
>> embracing it means starting to respect common development rules and it
>> should start with go modules conventions and SEMVER.
>>
>> Like I said, TSDB is a cornerstone of the prometheus ecosystem.
>>
>> If we want to make the ecosystem thrive, I believe *major components
>> like TSDB* should have their own *development oriented lifecycle*
>> (prometheus/prometheus has an user oriented lifecycle) which respects both
>> go module conventions and SEMVER git tags to make life easier for third
>> parties that use them.
>>
>> That does not prevent anyone to make breaking changes to them and that
>> advertises those who rely on it what they can expect when managing their
>> prometheus dependencies.
>>
>> You as Thanos maintainer want to upgrade from TSDB v0.7.0 to v0.7.2, you
>> should have almost nothing to do. But, if you want to upgrade from v0.7.2
>> to v2.0.3, you would know that should roll up your sleeves because you're
>> in for some heavy lifting.
>>
>> Today if you want to upgrade prometheus in Thanos from 2.15.0 to 2.16.0
>> you have absolutely no idea what you are stepping into. Well, you
>> personally might as you are quite involved in both projects but that is not
>> the case for everyone.
>>
>> That's why I believe that the TSDB move was a bad thing and should never
>> have happened. Now that is has, I just want to make sure it's not something
>> we can't go back from. It might never happen, or it might, only time will
>> tell.
>>
>> Regards.
>>
>>
>>
>>
>>
>> On Sun, 16 Feb 2020 at 13:23, Bartłomiej Płotka 
>> wrote:
>>
>>> At least when it fails it fails immediately in build time :p
>>>
>>> So not sure how separate versioning would help you there. It would be a
>>> new major version of tsdb module, sure. But what then? It's will require
>>> extra manual bump anyway then you will have same work to be done, am I
>>> right?
>>>
>>>
>>> Kind Regards,
>>> Bartek
>>>
>>> On Sun, 16 Feb 2020, 12:21 Sylvain Rabot, 
>>> wrote:
>>>
>>>>
>>>> On Sun, 16 Feb 2020 at 13:01, Bartłomiej Płotka 
>>>> wrote:
>>>>
>>>>> Thanks for your opinion Sylvian! I agree. I totally see the versioning
>>>>> cycle being a problem. In fact as Thanos maintainers, together with Cortex
>>>>> maintainers, we are probably the biggest users of TSDB alone, so we feel
>>>>> the pain (wh

Re: [prometheus-developers] Reduce list of GOOS/GOARCH crossbuilt for each PR

[Sylvain Rabot]

On Sun, 16 Feb 2020 at 13:41, Tobias Schmidt  wrote:

>
>
> On Sun, Feb 16, 2020 at 11:38 AM Sylvain Rabot 
> wrote:
>
>> I've also been looking into it.
>>
>> The sharing the go build cache would be the ultimate solution but for
>> prometheus, for all current goos/goarch, it accounts for more than 6GB and
>> I believe that is too much for circleci to handle (they recommend keeping
>> cache under 500MB).
>>
>> However I made a PR on promu that replace current crossbuild process with
>> one that (amongst other things) does not use docker anymore and it seems
>> that it *halves* the duration of building all current goos/goarch:
>>
>
> How does this work for node_exporter with its CGO usage?
>

I don't know, I'll try it. But if it does not work, I moved the docker
cross-building in a separate promu command ("promu crossbuild-docker") so
we could still use that for node_exporter.


>
>
>>
>> - https://github.com/prometheus/promu/pull/177
>> - https://github.com/prometheus/prometheus/pull/6824
>> -
>> https://app.circleci.com/github/prometheus/prometheus/pipelines/838ffe3c-b738-430b-8cf0-c707842e7e8f/workflows/dd50dabc-04e3-4b48-8a36-c85456668c2f
>>
>> Regards.
>>
>> On Fri, 14 Feb 2020 at 14:48, Simon Pasquier  wrote:
>>
>>> FWIW I'm looking into this. From my understanding, the bulk of the
>>> issue is that "promu crossbuild" can't reuse the Go build cache
>>> between CI runs hence it rebuilds everything from scratch every time.
>>>
>>> On Thu, Feb 13, 2020 at 5:36 PM Ben Kochie  wrote:
>>> >
>>> > Part of the problem is that we run the builds in serial. We have a lot
>>> more compute capacity in our CircleCI for running more docker tasks in
>>> parallel, but promu doesn't know how to distribute the work.
>>> >
>>> > But I also think we could cut the build step down for PRs. But I think
>>> we should keep the full build in master.
>>> >
>>> > On Wed, Feb 12, 2020 at 3:50 PM Chris Marchbanks <
>>> csmarchba...@gmail.com> wrote:
>>> >>
>>> >> I also support this, waiting 2-3 hours for the build job to finish is
>>> frustrating. I know that building on 32 bit architectures does not catch
>>> all issues, specifically the alignment bug using the atomic package.
>>> Perhaps add at least one 32 bit build on the pull request though?
>>> >>
>>> >> Is it worth it to build everything on every master, or should a build
>>> all job be added to the nightly build? I agree that we should build
>>> everything on a cadence more frequent than a release.
>>> >>
>>> >> On Wed, Feb 12, 2020 at 1:58 AM 'Matthias Rampke' via Prometheus
>>> Developers  wrote:
>>> >>>
>>> >>> I would build everything on master, that way we catch before
>>> starting a release if there is something wrong.
>>> >>>
>>> >>> /MR
>>> >>>
>>> >>> On Wed, Feb 12, 2020 at 8:37 AM Sylvain Rabot <
>>> sylv...@abstraction.fr> wrote:
>>> >>>>
>>> >>>> I did not say it but I was speaking of prometheus/prometheus, I
>>> haven't checked others repos for their full cross-building time.
>>> >>>>
>>> >>>> I think we can come up with a minimal list of GOOS/GOARCH for PRs
>>> but, if you think building the complete list on tags only is not enough, we
>>> could do it on tags & master.
>>> >>>>
>>> >>>> If we were to choose to build the complete list for tags only I
>>> would suggest to build this for PRs:
>>> >>>>
>>> >>>> - linux/amd64
>>> >>>> - linux/386
>>> >>>> - linux/arm
>>> >>>> - linux/arm64
>>> >>>> - darwin/amd64
>>> >>>> - windows/amd64
>>> >>>> - freebsd/amd64
>>> >>>> - openbsd/amd64
>>> >>>> - netbsd/amd64
>>> >>>> - dragonfly/amd64
>>> >>>>
>>> >>>> If we were to choose to build the complete list for tags & master
>>> then I would suggest an even more reduced one:
>>> >>>>
>>> >>>> - linux/amd64
>>> >>>> - darwin/amd64
>>> >>>> - window

Re: [prometheus-developers] Moving prometheus/tsdb to prometheus-junkyard/tsdb

[Sylvain Rabot]

It wouldn't have, separate versioning can't solve merging upstreams that
have breaking changes.

It's not about that, it's about respect towards people relying on TSDB.

I'd like prometheus people to acknowledge that their project is growing
outside their reach. This is a recognition that they did well, very well,
and *they should be really proud of that and embrace it*. I believe that
embracing it means starting to respect common development rules and it
should start with go modules conventions and SEMVER.

Like I said, TSDB is a cornerstone of the prometheus ecosystem.

If we want to make the ecosystem thrive, I believe *major components like
TSDB* should have their own *development oriented lifecycle*
(prometheus/prometheus has an user oriented lifecycle) which respects both
go module conventions and SEMVER git tags to make life easier for third
parties that use them.

That does not prevent anyone to make breaking changes to them and that
advertises those who rely on it what they can expect when managing their
prometheus dependencies.

You as Thanos maintainer want to upgrade from TSDB v0.7.0 to v0.7.2, you
should have almost nothing to do. But, if you want to upgrade from v0.7.2
to v2.0.3, you would know that should roll up your sleeves because you're
in for some heavy lifting.

Today if you want to upgrade prometheus in Thanos from 2.15.0 to 2.16.0 you
have absolutely no idea what you are stepping into. Well, you personally
might as you are quite involved in both projects but that is not the case
for everyone.

That's why I believe that the TSDB move was a bad thing and should never
have happened. Now that is has, I just want to make sure it's not something
we can't go back from. It might never happen, or it might, only time will
tell.

Regards.





On Sun, 16 Feb 2020 at 13:23, Bartłomiej Płotka  wrote:

> At least when it fails it fails immediately in build time :p
>
> So not sure how separate versioning would help you there. It would be a
> new major version of tsdb module, sure. But what then? It's will require
> extra manual bump anyway then you will have same work to be done, am I
> right?
>
>
> Kind Regards,
> Bartek
>
> On Sun, 16 Feb 2020, 12:21 Sylvain Rabot,  wrote:
>
>>
>> On Sun, 16 Feb 2020 at 13:01, Bartłomiej Płotka 
>> wrote:
>>
>>> Thanks for your opinion Sylvian! I agree. I totally see the versioning
>>> cycle being a problem. In fact as Thanos maintainers, together with Cortex
>>> maintainers, we are probably the biggest users of TSDB alone, so we feel
>>> the pain (which is not really THAT big).
>>>
>>
>> Yeah I knew that you, most of all people, would feel that pain. But I beg
>> to differ on the "which is not really THAT big", I spent a couple of
>> afternoons trying to keep prometheus up to date in thanos and I do not keep
>> good memories of it.
>>
>>
>>> That's why we started a discussion around proper go module versioning to
>>> solve this:
>>> https://groups.google.com/forum/#!searchin/prometheus-developers/versioning%7Csort:date/prometheus-developers/F1Vp0rLk3TQ/XyXngVP8AAAJ
>>> Hopefully once solved, we can keep mono-repo like structure but have
>>> separate versioning for TSDB. My personal opinion is that the work we do
>>> with Duco with help here:
>>> https://github.com/Helcaraxan/modularise-example-core
>>>
>>> We will see it in the soon future.
>>>
>>> Kind Regards,
>>> Bartek
>>>
>>> On Sun, 16 Feb 2020 at 12:00, Julien Pivotto 
>>> wrote:
>>>
>>>> On 16 Feb 12:52, Sylvain Rabot wrote:
>>>> > I strongly believe that TSDB is a cornerstone of the prometheus
>>>> ecosystem
>>>> > (and not prometheus/prometheus alone) and as such should have its own
>>>> > lifecycle.
>>>> >
>>>> > I also believe the original reason for the move ("Keeping them in
>>>> sync,
>>>> > versioning etc, is a pain") should have been solved by tooling.
>>>> >
>>>> > I'm sure at one point people using TSDB outside of prometheus will
>>>> complain
>>>> > about the TSDB versioning being tied to prometheus.
>>>> >
>>>> > So I'd like to make sure we can go back because even if the move is
>>>> > considered safe now, I'm persuaded it only benefits internal
>>>> prometheus
>>>> > developments at the expense of the whole ecosystem.
>>>> >
>>>> > Regards.
>>>>
>>>> There are discussions in progress outside of this 

Re: [prometheus-developers] Moving prometheus/tsdb to prometheus-junkyard/tsdb

[Sylvain Rabot]

On Sun, 16 Feb 2020 at 13:01, Bartłomiej Płotka  wrote:

> Thanks for your opinion Sylvian! I agree. I totally see the versioning
> cycle being a problem. In fact as Thanos maintainers, together with Cortex
> maintainers, we are probably the biggest users of TSDB alone, so we feel
> the pain (which is not really THAT big).
>

Yeah I knew that you, most of all people, would feel that pain. But I beg
to differ on the "which is not really THAT big", I spent a couple of
afternoons trying to keep prometheus up to date in thanos and I do not keep
good memories of it.


> That's why we started a discussion around proper go module versioning to
> solve this:
> https://groups.google.com/forum/#!searchin/prometheus-developers/versioning%7Csort:date/prometheus-developers/F1Vp0rLk3TQ/XyXngVP8AAAJ
> Hopefully once solved, we can keep mono-repo like structure but have
> separate versioning for TSDB. My personal opinion is that the work we do
> with Duco with help here:
> https://github.com/Helcaraxan/modularise-example-core
>
> We will see it in the soon future.
>
> Kind Regards,
> Bartek
>
> On Sun, 16 Feb 2020 at 12:00, Julien Pivotto 
> wrote:
>
>> On 16 Feb 12:52, Sylvain Rabot wrote:
>> > I strongly believe that TSDB is a cornerstone of the prometheus
>> ecosystem
>> > (and not prometheus/prometheus alone) and as such should have its own
>> > lifecycle.
>> >
>> > I also believe the original reason for the move ("Keeping them in sync,
>> > versioning etc, is a pain") should have been solved by tooling.
>> >
>> > I'm sure at one point people using TSDB outside of prometheus will
>> complain
>> > about the TSDB versioning being tied to prometheus.
>> >
>> > So I'd like to make sure we can go back because even if the move is
>> > considered safe now, I'm persuaded it only benefits internal prometheus
>> > developments at the expense of the whole ecosystem.
>> >
>> > Regards.
>>
>> There are discussions in progress outside of this discussion.
>>
>> I would like to add that golang versioning totally tolerate multiple
>> modules in one git repo with different versioning schemes.
>>
>> https://github.com/hashicorp/consul/tree/master/api
>>
>> is a go module on its own, module github.com/hashicorp/consul/api
>> inside the github.com/hashicorp/consul repo.
>>
>> They have dedicated versions (see
>> https://github.com/hashicorp/consul/tree/api/v1.4.0): consul/api is
>> v1.4.0,
>> while consul is v1.7.0.
>>
>> So it looks like we could get the advantages of a single repo and a
>> dedicated module lifecycle if we need to.
>>
>> >
>> > On Sun, 16 Feb 2020 at 12:39, Bartłomiej Płotka 
>> wrote:
>> >
>> > > Hm, why would it need to have its own lifecycle?
>> > >
>> > > We waited for some time exactly to make sure that all is safe for the
>> move.
>> > >
>> > > Bartek
>> > >
>> > > On Sun, 16 Feb 2020, 11:25 Sylvain Rabot, 
>> wrote:
>> > >
>> > >> Hi,
>> > >>
>> > >> Before doing so, could we make sure this action can be reverted ?
>> > >>
>> > >> I'd hate that, if we realize TSDB needs its own lifecycle, we
>> wouldn't be
>> > >> able to reopen github.com/prometheus/tsdb because of some github
>> logic.
>> > >>
>> > >> Regards.
>> > >>
>> > >> On Fri, 14 Feb 2020 at 00:19, Julien Pivotto > >
>> > >> wrote:
>> > >>
>> > >>> Dear community,
>> > >>>
>> > >>> In the dev summit 2019/1, it was decided to move tsdb to the
>> junkyard:
>> > >>>
>> > >>>
>> https://docs.google.com/document/d/1NQIX78nwBhfLZD3pAb0PK-uBKYqnkzjjVhOQ-kIaEGU/edit
>> > >>>
>> > >>> The https://github.com/prometheus/tsdb has been moved in Augustus
>> 2019
>> > >>> inside
>> > >>> the prometheus repo itself:
>> > >>> https://github.com/prometheus/prometheus/tsdb
>> > >>>
>> > >>> I would like to trigger the move of the old repo to
>> > >>> https://github.com/prometheus-junkyard/tsdb at the end of this
>> month (29
>> > >>> February). That is 200 days after the repo has been merged in the
>> > >>> prometheus server repo.
>> > >>>
&

Re: [prometheus-developers] Moving prometheus/tsdb to prometheus-junkyard/tsdb

[Sylvain Rabot]

No worries :)

On Sun, 16 Feb 2020 at 13:01, Bartłomiej Płotka  wrote:

> s/Sylvian/Sylvain
>
> Sorry (:
>
> On Sun, 16 Feb 2020 at 12:01, Bartłomiej Płotka 
> wrote:
>
>> Thanks for your opinion Sylvian! I agree. I totally see the versioning
>> cycle being a problem. In fact as Thanos maintainers, together with Cortex
>> maintainers, we are probably the biggest users of TSDB alone, so we feel
>> the pain (which is not really THAT big).
>>
>> That's why we started a discussion around proper go module versioning to
>> solve this:
>> https://groups.google.com/forum/#!searchin/prometheus-developers/versioning%7Csort:date/prometheus-developers/F1Vp0rLk3TQ/XyXngVP8AAAJ
>> Hopefully once solved, we can keep mono-repo like structure but have
>> separate versioning for TSDB. My personal opinion is that the work we do
>> with Duco with help here:
>> https://github.com/Helcaraxan/modularise-example-core
>>
>> We will see it in the soon future.
>>
>> Kind Regards,
>> Bartek
>>
>> On Sun, 16 Feb 2020 at 12:00, Julien Pivotto 
>> wrote:
>>
>>> On 16 Feb 12:52, Sylvain Rabot wrote:
>>> > I strongly believe that TSDB is a cornerstone of the prometheus
>>> ecosystem
>>> > (and not prometheus/prometheus alone) and as such should have its own
>>> > lifecycle.
>>> >
>>> > I also believe the original reason for the move ("Keeping them in sync,
>>> > versioning etc, is a pain") should have been solved by tooling.
>>> >
>>> > I'm sure at one point people using TSDB outside of prometheus will
>>> complain
>>> > about the TSDB versioning being tied to prometheus.
>>> >
>>> > So I'd like to make sure we can go back because even if the move is
>>> > considered safe now, I'm persuaded it only benefits internal prometheus
>>> > developments at the expense of the whole ecosystem.
>>> >
>>> > Regards.
>>>
>>> There are discussions in progress outside of this discussion.
>>>
>>> I would like to add that golang versioning totally tolerate multiple
>>> modules in one git repo with different versioning schemes.
>>>
>>> https://github.com/hashicorp/consul/tree/master/api
>>>
>>> is a go module on its own, module github.com/hashicorp/consul/api
>>> inside the github.com/hashicorp/consul repo.
>>>
>>> They have dedicated versions (see
>>> https://github.com/hashicorp/consul/tree/api/v1.4.0): consul/api is
>>> v1.4.0,
>>> while consul is v1.7.0.
>>>
>>> So it looks like we could get the advantages of a single repo and a
>>> dedicated module lifecycle if we need to.
>>>
>>> >
>>> > On Sun, 16 Feb 2020 at 12:39, Bartłomiej Płotka 
>>> wrote:
>>> >
>>> > > Hm, why would it need to have its own lifecycle?
>>> > >
>>> > > We waited for some time exactly to make sure that all is safe for
>>> the move.
>>> > >
>>> > > Bartek
>>> > >
>>> > > On Sun, 16 Feb 2020, 11:25 Sylvain Rabot, 
>>> wrote:
>>> > >
>>> > >> Hi,
>>> > >>
>>> > >> Before doing so, could we make sure this action can be reverted ?
>>> > >>
>>> > >> I'd hate that, if we realize TSDB needs its own lifecycle, we
>>> wouldn't be
>>> > >> able to reopen github.com/prometheus/tsdb because of some github
>>> logic.
>>> > >>
>>> > >> Regards.
>>> > >>
>>> > >> On Fri, 14 Feb 2020 at 00:19, Julien Pivotto <
>>> roidelapl...@inuits.eu>
>>> > >> wrote:
>>> > >>
>>> > >>> Dear community,
>>> > >>>
>>> > >>> In the dev summit 2019/1, it was decided to move tsdb to the
>>> junkyard:
>>> > >>>
>>> > >>>
>>> https://docs.google.com/document/d/1NQIX78nwBhfLZD3pAb0PK-uBKYqnkzjjVhOQ-kIaEGU/edit
>>> > >>>
>>> > >>> The https://github.com/prometheus/tsdb has been moved in Augustus
>>> 2019
>>> > >>> inside
>>> > >>> the prometheus repo itself:
>>> > >>> https://github.com/prometheus/prometheus/tsdb
>>> > >>>
>>> > >>> I would like to trigger the mov

Re: [prometheus-developers] Moving prometheus/tsdb to prometheus-junkyard/tsdb

[Sylvain Rabot]

On Sun, 16 Feb 2020 at 13:00, Julien Pivotto  wrote:

> On 16 Feb 12:52, Sylvain Rabot wrote:
> > I strongly believe that TSDB is a cornerstone of the prometheus ecosystem
> > (and not prometheus/prometheus alone) and as such should have its own
> > lifecycle.
> >
> > I also believe the original reason for the move ("Keeping them in sync,
> > versioning etc, is a pain") should have been solved by tooling.
> >
> > I'm sure at one point people using TSDB outside of prometheus will
> complain
> > about the TSDB versioning being tied to prometheus.
> >
> > So I'd like to make sure we can go back because even if the move is
> > considered safe now, I'm persuaded it only benefits internal prometheus
> > developments at the expense of the whole ecosystem.
> >
> > Regards.
>
> There are discussions in progress outside of this discussion.
>

Yes, I know, I still do not understand why TSDB has been moved into
prometheus before those discussions have been sorted ...


>
> I would like to add that golang versioning totally tolerate multiple
> modules in one git repo with different versioning schemes.
>
> https://github.com/hashicorp/consul/tree/master/api
>
> is a go module on its own, module github.com/hashicorp/consul/api
> inside the github.com/hashicorp/consul repo.
>
> They have dedicated versions (see
> https://github.com/hashicorp/consul/tree/api/v1.4.0): consul/api is
> v1.4.0,
> while consul is v1.7.0.
>
> So it looks like we could get the advantages of a single repo and a
> dedicated module lifecycle if we need to.
>

I must admit I did not know that but after a first quick glance I'm sure it
will become a mess. Having several go modules in the same repo, which
ultimately, if their versions differ, will no longer reflect git tags ... I
wouldn't do that, ever.


>
> >
> > On Sun, 16 Feb 2020 at 12:39, Bartłomiej Płotka 
> wrote:
> >
> > > Hm, why would it need to have its own lifecycle?
> > >
> > > We waited for some time exactly to make sure that all is safe for the
> move.
> > >
> > > Bartek
> > >
> > > On Sun, 16 Feb 2020, 11:25 Sylvain Rabot, 
> wrote:
> > >
> > >> Hi,
> > >>
> > >> Before doing so, could we make sure this action can be reverted ?
> > >>
> > >> I'd hate that, if we realize TSDB needs its own lifecycle, we
> wouldn't be
> > >> able to reopen github.com/prometheus/tsdb because of some github
> logic.
> > >>
> > >> Regards.
> > >>
> > >> On Fri, 14 Feb 2020 at 00:19, Julien Pivotto 
> > >> wrote:
> > >>
> > >>> Dear community,
> > >>>
> > >>> In the dev summit 2019/1, it was decided to move tsdb to the
> junkyard:
> > >>>
> > >>>
> https://docs.google.com/document/d/1NQIX78nwBhfLZD3pAb0PK-uBKYqnkzjjVhOQ-kIaEGU/edit
> > >>>
> > >>> The https://github.com/prometheus/tsdb has been moved in Augustus
> 2019
> > >>> inside
> > >>> the prometheus repo itself:
> > >>> https://github.com/prometheus/prometheus/tsdb
> > >>>
> > >>> I would like to trigger the move of the old repo to
> > >>> https://github.com/prometheus-junkyard/tsdb at the end of this
> month (29
> > >>> February). That is 200 days after the repo has been merged in the
> > >>> prometheus server repo.
> > >>>
> > >>> GitHub says that "All links to the previous repository location are
> > >>> automatically redirected to the new location":
> > >>>
> > >>>
> https://help.github.com/en/github/administering-a-repository/transferring-a-repository
> > >>> which means that links to pull requests etc should still work after
> > >>> the rename.
> > >>>
> > >>> If someone has strong arguments against this, please reply to this
> > >>> message.
> > >>>
> > >>> --
> > >>>  (o-Julien Pivotto
> > >>>  //\Open-Source Consultant
> > >>>  V_/_   Inuits - https://www.inuits.eu
> > >>>
> > >>> --
> > >>> You received this message because you are subscribed to the Google
> > >>> Groups "Prometheus Developers" group.
> > >>> To unsubscribe from this group and stop receiving emails from it,
> send
> > >>> an email to prometheus-deve

Re: [prometheus-developers] Moving prometheus/tsdb to prometheus-junkyard/tsdb

[Sylvain Rabot]

I strongly believe that TSDB is a cornerstone of the prometheus ecosystem
(and not prometheus/prometheus alone) and as such should have its own
lifecycle.

I also believe the original reason for the move ("Keeping them in sync,
versioning etc, is a pain") should have been solved by tooling.

I'm sure at one point people using TSDB outside of prometheus will complain
about the TSDB versioning being tied to prometheus.

So I'd like to make sure we can go back because even if the move is
considered safe now, I'm persuaded it only benefits internal prometheus
developments at the expense of the whole ecosystem.

Regards.

On Sun, 16 Feb 2020 at 12:39, Bartłomiej Płotka  wrote:

> Hm, why would it need to have its own lifecycle?
>
> We waited for some time exactly to make sure that all is safe for the move.
>
> Bartek
>
> On Sun, 16 Feb 2020, 11:25 Sylvain Rabot,  wrote:
>
>> Hi,
>>
>> Before doing so, could we make sure this action can be reverted ?
>>
>> I'd hate that, if we realize TSDB needs its own lifecycle, we wouldn't be
>> able to reopen github.com/prometheus/tsdb because of some github logic.
>>
>> Regards.
>>
>> On Fri, 14 Feb 2020 at 00:19, Julien Pivotto 
>> wrote:
>>
>>> Dear community,
>>>
>>> In the dev summit 2019/1, it was decided to move tsdb to the junkyard:
>>>
>>> https://docs.google.com/document/d/1NQIX78nwBhfLZD3pAb0PK-uBKYqnkzjjVhOQ-kIaEGU/edit
>>>
>>> The https://github.com/prometheus/tsdb has been moved in Augustus 2019
>>> inside
>>> the prometheus repo itself:
>>> https://github.com/prometheus/prometheus/tsdb
>>>
>>> I would like to trigger the move of the old repo to
>>> https://github.com/prometheus-junkyard/tsdb at the end of this month (29
>>> February). That is 200 days after the repo has been merged in the
>>> prometheus server repo.
>>>
>>> GitHub says that "All links to the previous repository location are
>>> automatically redirected to the new location":
>>>
>>> https://help.github.com/en/github/administering-a-repository/transferring-a-repository
>>> which means that links to pull requests etc should still work after
>>> the rename.
>>>
>>> If someone has strong arguments against this, please reply to this
>>> message.
>>>
>>> --
>>>  (o-Julien Pivotto
>>>  //\Open-Source Consultant
>>>  V_/_   Inuits - https://www.inuits.eu
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Prometheus Developers" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to prometheus-developers+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/prometheus-developers/20200213231949.GA18483%40oxygen
>>> .
>>>
>>
>>
>> --
>> Sylvain Rabot 
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Prometheus Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to prometheus-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/prometheus-developers/CADjtP1GFSFug0WYQO0svnJv2%3D2Xgz-GSzfL1ThN_kHeoznOQsg%40mail.gmail.com
>> <https://groups.google.com/d/msgid/prometheus-developers/CADjtP1GFSFug0WYQO0svnJv2%3D2Xgz-GSzfL1ThN_kHeoznOQsg%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>

-- 
Sylvain Rabot 

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CADjtP1FHRxpDrOjg1_EOc9UnYmyzeT_mYHbmXu6Z1bj7B57UKA%40mail.gmail.com.

Re: [prometheus-developers] Moving prometheus/tsdb to prometheus-junkyard/tsdb

[Sylvain Rabot]

Hi,

Before doing so, could we make sure this action can be reverted ?

I'd hate that, if we realize TSDB needs its own lifecycle, we wouldn't be
able to reopen github.com/prometheus/tsdb because of some github logic.

Regards.

On Fri, 14 Feb 2020 at 00:19, Julien Pivotto  wrote:

> Dear community,
>
> In the dev summit 2019/1, it was decided to move tsdb to the junkyard:
>
> https://docs.google.com/document/d/1NQIX78nwBhfLZD3pAb0PK-uBKYqnkzjjVhOQ-kIaEGU/edit
>
> The https://github.com/prometheus/tsdb has been moved in Augustus 2019
> inside
> the prometheus repo itself: https://github.com/prometheus/prometheus/tsdb
>
> I would like to trigger the move of the old repo to
> https://github.com/prometheus-junkyard/tsdb at the end of this month (29
> February). That is 200 days after the repo has been merged in the
> prometheus server repo.
>
> GitHub says that "All links to the previous repository location are
> automatically redirected to the new location":
>
> https://help.github.com/en/github/administering-a-repository/transferring-a-repository
> which means that links to pull requests etc should still work after
> the rename.
>
> If someone has strong arguments against this, please reply to this
> message.
>
> --
>  (o-Julien Pivotto
>  //\Open-Source Consultant
>  V_/_   Inuits - https://www.inuits.eu
>
> --
> You received this message because you are subscribed to the Google Groups
> "Prometheus Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to prometheus-developers+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/prometheus-developers/20200213231949.GA18483%40oxygen
> .
>


-- 
Sylvain Rabot 

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CADjtP1GFSFug0WYQO0svnJv2%3D2Xgz-GSzfL1ThN_kHeoznOQsg%40mail.gmail.com.

Re: [prometheus-developers] Reduce list of GOOS/GOARCH crossbuilt for each PR

[Sylvain Rabot]

I've also been looking into it.

The sharing the go build cache would be the ultimate solution but for
prometheus, for all current goos/goarch, it accounts for more than 6GB and
I believe that is too much for circleci to handle (they recommend keeping
cache under 500MB).

However I made a PR on promu that replace current crossbuild process with
one that (amongst other things) does not use docker anymore and it seems
that it *halves* the duration of building all current goos/goarch:

- https://github.com/prometheus/promu/pull/177
- https://github.com/prometheus/prometheus/pull/6824
-
https://app.circleci.com/github/prometheus/prometheus/pipelines/838ffe3c-b738-430b-8cf0-c707842e7e8f/workflows/dd50dabc-04e3-4b48-8a36-c85456668c2f

Regards.

On Fri, 14 Feb 2020 at 14:48, Simon Pasquier  wrote:

> FWIW I'm looking into this. From my understanding, the bulk of the
> issue is that "promu crossbuild" can't reuse the Go build cache
> between CI runs hence it rebuilds everything from scratch every time.
>
> On Thu, Feb 13, 2020 at 5:36 PM Ben Kochie  wrote:
> >
> > Part of the problem is that we run the builds in serial. We have a lot
> more compute capacity in our CircleCI for running more docker tasks in
> parallel, but promu doesn't know how to distribute the work.
> >
> > But I also think we could cut the build step down for PRs. But I think
> we should keep the full build in master.
> >
> > On Wed, Feb 12, 2020 at 3:50 PM Chris Marchbanks 
> wrote:
> >>
> >> I also support this, waiting 2-3 hours for the build job to finish is
> frustrating. I know that building on 32 bit architectures does not catch
> all issues, specifically the alignment bug using the atomic package.
> Perhaps add at least one 32 bit build on the pull request though?
> >>
> >> Is it worth it to build everything on every master, or should a build
> all job be added to the nightly build? I agree that we should build
> everything on a cadence more frequent than a release.
> >>
> >> On Wed, Feb 12, 2020 at 1:58 AM 'Matthias Rampke' via Prometheus
> Developers  wrote:
> >>>
> >>> I would build everything on master, that way we catch before starting
> a release if there is something wrong.
> >>>
> >>> /MR
> >>>
> >>> On Wed, Feb 12, 2020 at 8:37 AM Sylvain Rabot 
> wrote:
> >>>>
> >>>> I did not say it but I was speaking of prometheus/prometheus, I
> haven't checked others repos for their full cross-building time.
> >>>>
> >>>> I think we can come up with a minimal list of GOOS/GOARCH for PRs
> but, if you think building the complete list on tags only is not enough, we
> could do it on tags & master.
> >>>>
> >>>> If we were to choose to build the complete list for tags only I would
> suggest to build this for PRs:
> >>>>
> >>>> - linux/amd64
> >>>> - linux/386
> >>>> - linux/arm
> >>>> - linux/arm64
> >>>> - darwin/amd64
> >>>> - windows/amd64
> >>>> - freebsd/amd64
> >>>> - openbsd/amd64
> >>>> - netbsd/amd64
> >>>> - dragonfly/amd64
> >>>>
> >>>> If we were to choose to build the complete list for tags & master
> then I would suggest an even more reduced one:
> >>>>
> >>>> - linux/amd64
> >>>> - darwin/amd64
> >>>> - windows/amd64
> >>>> - freebsd/amd64
> >>>>
> >>>> Regards.
> >>>>
> >>>> On Tue, 11 Feb 2020 at 23:17, Matthias Rampke 
> wrote:
> >>>>>
> >>>>> There are some exceptions like node exporter where it's important
> that all variants at least build, but that has a custom setup already.
> >>>>>
> >>>>> What would be a sufficient subset? Do we need to worry about
> endianness and 32 bit architectures, or would just building not catch
> issues specific to these anyway?
> >>>>>
> >>>>> /MR
> >>>>>
> >>>>> On Tue, 11 Feb 2020, 22:50 Krasimir Georgiev, 
> wrote:
> >>>>>>
> >>>>>> I think that is a very good idea.
> >>>>>>
> >>>>>> On Feb 11 2020, at 11:19 pm, Sylvain Rabot 
> wrote:
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>>
> >>>>>> I'm wondering if we could reduce the list of GOOS/GOARCH that

Re: [prometheus-developers] Reduce list of GOOS/GOARCH crossbuilt for each PR

[Sylvain Rabot]

I did not say it but I was speaking of prometheus/prometheus, I haven't
checked others repos for their full cross-building time.

I think we can come up with a minimal list of GOOS/GOARCH for PRs but, if
you think building the complete list on tags only is not enough, we could
do it on tags & master.

If we were to choose to build the complete list for tags only I would
suggest to build this for PRs:

- linux/amd64
- linux/386
- linux/arm
- linux/arm64
- darwin/amd64
- windows/amd64
- freebsd/amd64
- openbsd/amd64
- netbsd/amd64
- dragonfly/amd64

If we were to choose to build the complete list for tags & master then I
would suggest an even more reduced one:

- linux/amd64
- darwin/amd64
- windows/amd64
- freebsd/amd64

Regards.

On Tue, 11 Feb 2020 at 23:17, Matthias Rampke  wrote:

> There are some exceptions like node exporter where it's important that all
> variants at least build, but that has a custom setup already.
>
> What would be a sufficient subset? Do we need to worry about endianness
> and 32 bit architectures, or would just building not catch issues specific
> to these anyway?
>
> /MR
>
> On Tue, 11 Feb 2020, 22:50 Krasimir Georgiev,  wrote:
>
>> I think that is a very good idea.
>>
>> On Feb 11 2020, at 11:19 pm, Sylvain Rabot 
>> wrote:
>>
>> Hi,
>>
>>
>> I'm wondering if we could reduce the list of GOOS/GOARCH that are crossbuilt 
>> for every PR by circle-ci.
>>
>>
>> The building of the complete list seems like a waste of time & resources to 
>> me.
>>
>>
>> Maybe we could select a few and only build the complete list when building 
>> tags ?
>>
>>
>> Regards.
>>
>>
>> --
>> Sylvain Rabot 
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Prometheus Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to prometheus-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/prometheus-developers/CADjtP1FJKyVj_gq-hgVgyyVbJ%3D-pECFqcPK-QviXmKB1R-oAgg%40mail.gmail.com
>> <https://groups.google.com/d/msgid/prometheus-developers/CADjtP1FJKyVj_gq-hgVgyyVbJ%3D-pECFqcPK-QviXmKB1R-oAgg%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Prometheus Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to prometheus-developers+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/prometheus-developers/2508CDF1-CC2A-4AC6-B9EE-D68B53AFF166%40getmailspring.com
>> <https://groups.google.com/d/msgid/prometheus-developers/2508CDF1-CC2A-4AC6-B9EE-D68B53AFF166%40getmailspring.com?utm_medium=email&utm_source=footer>
>> .
>>
>

-- 
Sylvain Rabot 

-- 
You received this message because you are subscribed to the Google Groups 
"Prometheus Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to prometheus-developers+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/prometheus-developers/CADjtP1FUF8y_xBTavAbKOAfmii%2BUd8tezimVPXR0UZUzdor2NQ%40mail.gmail.com.