Re: [foreman-dev] Foreman instrumenting analysis

[Lukas Zapletal]

Statsd can be configured for remote transport, meaning that the
collecting agent (or aggregating process if you like) can run on
remote server (or container). It is recommended to run it either on
localhost or at least LAN, it is not a good idea to route the UDP
packets through complex environments tho as they can get lost. Also
creating a SPOF is not a good idea, but I've seen articles or comments
about having one central statsd collector for all hosts. Those people
had usually questions around scaleability because single point of
entry was getting overloaded.

There are some WIP patches for Prometheus as well giving a possibility
to have single HTTP REST endpoint for all subprocesses of a Rails app
server, but if you take a look into this (links are in my original
email) these are pretty hacky. One is creating a local shared memory
block for communication, the other is doing the same via serialized db
file. This is doing dozens of system calls per single measurement,
compared to just one or two for UDP datagram this is way too much
IMHO.

The question tho is if there is another protocol I am not aware of.
There are actually two which I both tested to be honest:

1) PCP trace API - http://pcp.io/man/man3/pmdatrace.3.html

PCP is a monitoring collecting daemon which is in most Linux distros
(and in RHEL as well) and it has a very simple API which uses TCP
connection for communication with trace agent (called PMDA trace). I
wrote a Ruby wrapper around this simple API
(https://github.com/lzap/ruby-pcptrace) and I have a working
prototype. Disadvantage is that in PCP world this API is seen as
legacy, might get removed in the future. Also aggregation is only done
for transaction type observation.

1) PCP MMV API - http://pcp.io/books/PCP_PG/html/id5213288nat.html

Another agent which uses memory mapped files for ultra-fast
communication. This is the fastest possible application
instrumentation I've seen, but it is a little bit of an overkill
primarily targeted to HPC environment. Also no aggregation is done and
there is no Ruby bindings at all. In both cases, a PCP daemon needs to
be running.

One question tho - isn't standard practice to have one container per
pod that will serve as monitoring endpoint? I am no expert with
Kubernetes, but I believe that's exactly what this technology is built
for - you can specify services and their dependency on each other. The
price we need to pay (an extra service) is balanced with better
reliability - I can imagine when Rails/Passenger stops responding you
won't be able to reach the monitoring endpoint as well thus we'd need
to maintain a separate web stack for that.

-- 
Later,
  Lukas @lzap Zapletal

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] FactoryGirl renamed to FactoryBot - plugin maintainer action needed

[Tomer Brisker]

Hello,
This has now been merged in Foreman core.
All plugins should make sure that their respective PRs pass tests and
merge them now. If you need assistance making this change in your
plugin, please reach out to us and we would be happy to help.

Keep in mind that any PRs that include test changes that use
FactoryGirl will need to be modified now to use FactoryBot, even if
previously the tests passed.

Tomer
On Wed, Oct 25, 2017 at 4:52 PM, Marek Hulan  wrote:
> Thanks for sending out the heads up. Please also note that such PRs will get
> out of date very quickly so make sure you have them up to date just before
> merge :-)
>
> --
> Marek
>
>
> On October 25, 2017 3:23:29 PM Andrew Kofink  wrote:
>
>> I went ahead and made an issue for Katello [1]. I'll submit a PR today
>> with
>> the changes.
>>
>> [1] http://projects.theforeman.org/issues/21458
>>
>> On Wed, Oct 25, 2017 at 9:12 AM, Tomer Brisker 
>> wrote:
>>
>>> Hello,
>>>
>>> The Thoughtbot team have decided to rename the factory_girl gem, which we
>>> depend on for tests, to factory_bot [1]. This led to our CI breaking due
>>> to
>>> the name change. For now, we have pinned factory_girl to the last working
>>> version to unblock CI, but we should prepare to change to the new name.
>>>
>>> This requires, basically, changing every occurrence of `FactoryGirl` to
>>> `FactoryBot`, and every `factory_girl` to `factory_bot`. Other then the
>>> name change, no other changes were made to the API.
>>>
>>> Marek has opened a PR[2] for fixing this in foreman-core, however once
>>> this is merged it will break tests for all plugins depending on the core
>>> factory_girl. To prevent prolonged breakage, all plugin maintainers
>>> should
>>> work to prepare a PR making the above mentioned change, so that it can be
>>> merged once core PR is merged.
>>>
>>> To give you some time to fix this and prevent breakage, the core PR will
>>> be merged one week from today, on November 1st. Please make sure you are
>>> prepared to correctly handle this change in your plugin. If needed, feel
>>> free to reach out to me, Marek, or any of the other core maintainers.
>>>
>>> ?[1] https://github.com/thoughtbot/factory_bot/issues/921?
>>> ?[2] https://github.com/theforeman/foreman/pull/4935?
>>>
>>> --
>>> Have a nice day,
>>> Tomer Brisker
>>> Red Hat Engineering
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "foreman-dev" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to foreman-dev+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>>
>> --
>> Andrew Kofink
>> akof...@redhat.com
>> IRC: akofink
>> Software Engineer
>> Red Hat Satellite
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to foreman-dev+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



-- 
Have a nice day,
Tomer Brisker
Red Hat Engineering

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Merge Permissions for Katello/katello

[Andrew Kofink]

Thanks everyone!

On Tue, Oct 31, 2017 at 9:47 PM, Justin Sherrill 
wrote:

> Given its been a number of days with no -1's, I've now granted you access!
>
> Congrats!
>
> Justin
>
> On Tue, Oct 31, 2017 at 9:36 PM, Eric D Helms 
> wrote:
>
>> +1
>>
>> On Oct 31, 2017 10:17 AM, "Walden Raines"  wrote:
>>
>>> +1, figured you already had merge permissions.
>>>
>>> On Tue, Oct 31, 2017 at 8:16 AM, Brad Buckingham 
>>> wrote:
>>>
 +1 to providing Andrew with merge permissions!!

 On Fri, Oct 27, 2017 at 10:25 AM, Andrew Kofink 
 wrote:

> Hello,
>
> I'm not sure why I waited so long to ask, but in light of other
> requests, I submit my own. Here are my merged PRs
> ,
> and here are other PRs I've helped review
> .
> Do you consent to giving me the mighty power of the Green Button?
>
> Andrew
>
> --
> Andrew Kofink
> akof...@redhat.com
> IRC: akofink
> Software Engineer
> Red Hat Satellite
>
> --
> You received this message because you are subscribed to the Google
> Groups "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

 --
 You received this message because you are subscribed to the Google
 Groups "foreman-dev" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to foreman-dev+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.

>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "foreman-dev" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to foreman-dev+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to foreman-dev+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Andrew Kofink
akof...@redhat.com
IRC: akofink
Software Engineer
Red Hat Satellite

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] FactoryGirl renamed to FactoryBot - plugin maintainer action needed

[Marek Hulán]

On středa 1. listopadu 2017 10:19:08 CET Tomer Brisker wrote:
> Hello,
> This has now been merged in Foreman core.
> All plugins should make sure that their respective PRs pass tests and
> merge them now. If you need assistance making this change in your
> plugin, please reach out to us and we would be happy to help.
> 
> Keep in mind that any PRs that include test changes that use
> FactoryGirl will need to be modified now to use FactoryBot, even if
> previously the tests passed.
> 

Hello all,

if you're uncertain whether your plugin is affected or there some action 
required from you, see the public pad [1]. We keep the status of the whole 
migration there. A lot of plugins have master branch tests broken because of 
rubocop update, for some plugins I sent a PR that should fix it too. This 
could reveal plugins that are no longer maintained :-)

[1] https://pad-katello.rhcloud.com/p/factory_bot_update

--
Marek

> Tomer
> 
> On Wed, Oct 25, 2017 at 4:52 PM, Marek Hulan  wrote:
> > Thanks for sending out the heads up. Please also note that such PRs will
> > get out of date very quickly so make sure you have them up to date just
> > before merge :-)
> > 
> > --
> > Marek
> > 
> > On October 25, 2017 3:23:29 PM Andrew Kofink  wrote:
> >> I went ahead and made an issue for Katello [1]. I'll submit a PR today
> >> with
> >> the changes.
> >> 
> >> [1] http://projects.theforeman.org/issues/21458
> >> 
> >> On Wed, Oct 25, 2017 at 9:12 AM, Tomer Brisker 
> >> 
> >> wrote:
> >>> Hello,
> >>> 
> >>> The Thoughtbot team have decided to rename the factory_girl gem, which
> >>> we
> >>> depend on for tests, to factory_bot [1]. This led to our CI breaking due
> >>> to
> >>> the name change. For now, we have pinned factory_girl to the last
> >>> working
> >>> version to unblock CI, but we should prepare to change to the new name.
> >>> 
> >>> This requires, basically, changing every occurrence of `FactoryGirl` to
> >>> `FactoryBot`, and every `factory_girl` to `factory_bot`. Other then the
> >>> name change, no other changes were made to the API.
> >>> 
> >>> Marek has opened a PR[2] for fixing this in foreman-core, however once
> >>> this is merged it will break tests for all plugins depending on the core
> >>> factory_girl. To prevent prolonged breakage, all plugin maintainers
> >>> should
> >>> work to prepare a PR making the above mentioned change, so that it can
> >>> be
> >>> merged once core PR is merged.
> >>> 
> >>> To give you some time to fix this and prevent breakage, the core PR will
> >>> be merged one week from today, on November 1st. Please make sure you are
> >>> prepared to correctly handle this change in your plugin. If needed, feel
> >>> free to reach out to me, Marek, or any of the other core maintainers.
> >>> 
> >>> ?[1] https://github.com/thoughtbot/factory_bot/issues/921?
> >>> ?[2] https://github.com/theforeman/foreman/pull/4935?
> >>> 
> >>> --
> >>> Have a nice day,
> >>> Tomer Brisker
> >>> Red Hat Engineering
> >>> 
> >>> --
> >>> You received this message because you are subscribed to the Google
> >>> Groups
> >>> "foreman-dev" group.
> >>> To unsubscribe from this group and stop receiving emails from it, send
> >>> an
> >>> email to foreman-dev+unsubscr...@googlegroups.com.
> >>> For more options, visit https://groups.google.com/d/optout.
> >> 
> >> --
> >> Andrew Kofink
> >> akof...@redhat.com
> >> IRC: akofink
> >> Software Engineer
> >> Red Hat Satellite
> >> 
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "foreman-dev" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an
> >> email to foreman-dev+unsubscr...@googlegroups.com.
> >> For more options, visit https://groups.google.com/d/optout.
> > 
> > --
> > You received this message because you are subscribed to the Google Groups
> > "foreman-dev" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to foreman-dev+unsubscr...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Merge permissions for Katello/katello

[Jonathon Turel]

At last! Thanks Justin.

On Tuesday, October 31, 2017 at 9:48:03 PM UTC-4, Justin Sherrill wrote:
>
> Given its been a number of days with no -1's, I've now granted you access!
>
> Congrats!
>
> Justin
>
> On Tue, Oct 31, 2017 at 9:37 PM, Eric D Helms  > wrote:
>
>> +1
>>
>> On Oct 31, 2017 9:10 AM, "Andrew Kofink" > 
>> wrote:
>>
>>> +1
>>>
>>> On Tue, Oct 31, 2017 at 8:17 AM, Brad Buckingham >> > wrote:
>>>
 +1 from me as well!

 On Fri, Oct 27, 2017 at 9:19 AM, John Mitsch >>> > wrote:

> +1 from me!
>
> John Mitsch
> Red Hat Engineering
> (860)-967-7285
> irc: jomitsch
>
> On Thu, Oct 26, 2017 at 4:00 PM, Jonathon Turel  > wrote:
>
>> Hey folks,
>>
>> The time has come (through some helpful suggestions) that I should 
>> make my request to have $subject bestowed upon me. If you'd like to see 
>> some of my contributions to Katello here are my merged PRs 
>> 
>>  
>> and PRs I've otherwise participated in 
>> .
>>  
>> Looking forward to contributing more as I continue to come up to speed. 
>> What do you all think?
>>
>> Thanks!
>>
>> Jonathon
>>
>> -- 
>> You received this message because you are subscribed to the Google 
>> Groups "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, 
>> send an email to foreman-dev...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> -- 
> You received this message because you are subscribed to the Google 
> Groups "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to foreman-dev...@googlegroups.com .
> For more options, visit https://groups.google.com/d/optout.
>

 -- 
 You received this message because you are subscribed to the Google 
 Groups "foreman-dev" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to foreman-dev...@googlegroups.com .
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>
>>>
>>> -- 
>>> Andrew Kofink
>>> ako...@redhat.com 
>>> IRC: akofink
>>> Software Engineer
>>> Red Hat Satellite
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "foreman-dev" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to foreman-dev...@googlegroups.com .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to foreman-dev...@googlegroups.com .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Foreman instrumenting analysis

[Eric D Helms]

On Wed, Nov 1, 2017 at 3:51 AM, Lukas Zapletal  wrote:

> Statsd can be configured for remote transport, meaning that the
> collecting agent (or aggregating process if you like) can run on
> remote server (or container). It is recommended to run it either on
> localhost or at least LAN, it is not a good idea to route the UDP
> packets through complex environments tho as they can get lost. Also
> creating a SPOF is not a good idea, but I've seen articles or comments
> about having one central statsd collector for all hosts. Those people
> had usually questions around scaleability because single point of
> entry was getting overloaded.
>
> There are some WIP patches for Prometheus as well giving a possibility
> to have single HTTP REST endpoint for all subprocesses of a Rails app
> server, but if you take a look into this (links are in my original
> email) these are pretty hacky. One is creating a local shared memory
> block for communication, the other is doing the same via serialized db
> file. This is doing dozens of system calls per single measurement,
> compared to just one or two for UDP datagram this is way too much
> IMHO.
>

Does Prometheus only not work in a multi-process Rails web server? Does it
work for a single process multi-threaded web server? This is an interesting
roadblock given you'd expect this to affect lots of webserver across
multiple languages out there.


>
> The question tho is if there is another protocol I am not aware of.
> There are actually two which I both tested to be honest:
>
> 1) PCP trace API - http://pcp.io/man/man3/pmdatrace.3.html
>
> PCP is a monitoring collecting daemon which is in most Linux distros
> (and in RHEL as well) and it has a very simple API which uses TCP
> connection for communication with trace agent (called PMDA trace). I
> wrote a Ruby wrapper around this simple API
> (https://github.com/lzap/ruby-pcptrace) and I have a working
> prototype. Disadvantage is that in PCP world this API is seen as
> legacy, might get removed in the future. Also aggregation is only done
> for transaction type observation.
>
> 1) PCP MMV API - http://pcp.io/books/PCP_PG/html/id5213288nat.html
>
> Another agent which uses memory mapped files for ultra-fast
> communication. This is the fastest possible application
> instrumentation I've seen, but it is a little bit of an overkill
> primarily targeted to HPC environment. Also no aggregation is done and
> there is no Ruby bindings at all. In both cases, a PCP daemon needs to
> be running.
>
> One question tho - isn't standard practice to have one container per
> pod that will serve as monitoring endpoint? I am no expert with
> Kubernetes, but I believe that's exactly what this technology is built
> for - you can specify services and their dependency on each other. The
> price we need to pay (an extra service) is balanced with better
> reliability - I can imagine when Rails/Passenger stops responding you
> won't be able to reach the monitoring endpoint as well thus we'd need
> to maintain a separate web stack for that.
>

Yes, standard practice is to think about one container per pod (in a
Kubernetes environment). However, there are patterns for things like log
aggregation and monitoring such as doing a sidecar container that ensures
co-location. The part I don't entirely get with sidecars is if I scale the
pod to say 5, I get 5 web applications and 5 monitoring containers and that
seems odd. Which I why I think the tendency is towards models where your
single process/application is the end point for your metrics to be scrapped
by an outside agent or services.

I agree you want the collector to be separate, but if your web application
is down what value would a monitoring endpoint being alive provide? The
application would be down, thus no metrics to serve up. The other exporters
such as the one exporting metrics about the underlying system would be
responsible for giving system metrics. In the Kube world, this is handled
by readiness and liveness probes for Kubenernetes to re-spin the container
if it stops responding.


>
> --
> Later,
>   Lukas @lzap Zapletal
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Eric D. Helms
Red Hat Engineering

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[foreman-dev] Propsing a move from Google Groups to Discourse

[Greg Sutcliffe]

Hi all,

As ever from me, this is long. Sorry about that, it's a habit. Here's
the TL;DR:

* What: move Google Groups -> Discourse
* Why: https://blog.discourse.org/category/use-cases/
* Can I try? - Scroll to the end for login details

So, as some of you know, I'm a big fan of the Discourse[1] forum
software - I use it for another community, and it's just lovely. I've
been testing it out recently with a view to using it for Foreman, and
I think it's time to explain my reasoning and ask for your thoughts.

# What?

Firstly, the "what" - what do I want to do? Simply put, I want to
migrate from Google Groups to Discourse. That means locking the groups
from further emails and using Discourse for our "written" discussions.
Obviosuly there's data migration that needs to happen there, but we'll
get to that.

Before all the die-hard mailing list fans stop reading here - please
keep reading. Discourse has options to interact entirely by email.
Your workflow may not be broken :)

# Why?

Why do I want this? The short version is "because anything is better
than Google Groups", but more seriously, I think Discourse is great.
The reasons are different for each of our mailing lists though, so let
me break it down:

## foreman-users

When it comes to supporting our users, what matters is that they can
ask a question, get a reply, and feel confident in that reply. For
those who do the replying, they need to be highlight and in some way
rewarded for the work they do.

The problem with a mailing list is that neither of these things is
really achievable. If a user (new to our list, who knows no-one) gets
2 different replies, who is (s)he to trust? A forum can display user
levels, and badges, making the developer reply stand out from the
other new user's reply. A mailing list has nothing - and worse, the
Groups API is so bad that I barely know who our mailing list regulars
are (I have to webscrape it using a crawler ...) so I can reach out to
them for help, questions etc.

Discourse also searches for likely similar topics *while* you're
typing yours. This should hopefully help users to find existing
support for their issue before they make duplicate posts :)

For hard-working supporters, Discourse provides automatic "trust
levels" (in additional to manually promoting admins and moderators).
These trust levels allow you to unlock extra powers as you participate
in the community, which helps to reward the people who do help out on
the boards. Gamifying? yes, but it works. It'll also help me know who
to speak to when there's some swag to send out ...

To make this worse (for Groups) there is a confounding of data in the
users-list today - we also use it for plugin announcements, and events
& CFPs, and so on. This is because making a new mailing list would
have a much smaller membership, making it not suitable for purpose.
Yes, you can say "hey we created a new mailing list for X, sign up!"
but the reality is that people just don't. Discourse gives us the
flexibility to create new boards for things as-and-when we feel the
need to.

Discourse also supports plugins (its a rails app), so we can look at
things like templates for new support issues, and so on. It also has
bots which can be used to deal with some things (I have not
investigated the bots too much yet).

## foreman-dev

In contrast to the -users list, which is primarily support and
notifications, dev is all about discussion. Here I think we'll see a
real benefit in a couple of areas.

Firstly, back when we discussed how to handle RFCs, one of the
criticisms of a mailing list was the lack of rich format support
(markdown, images, code snippets etc) - Discourse supports all of
that. Quality of discussions should improve, I hope.

Discourse also has "like" buttons which can be used in place of "+1"
emails, and hopefully would encourage some of the quiter voices in the
community to click the like button, even when they might not have
emailed a +1. There are also polling options, so taken together I
*hope* this will help us to resolve discussions better than we have
historically.

The ability to create dedicated boards for popular plugins is also
pretty nice, if we decide to go that way.

## foreman-announce

This list has historically been very low traffic. On Discourse, the
most useful feature for this purpose is the "global banner". This
allows a single post to be show at the top of *every* page, a global
pin if you like. Cleverly though, individual users can dismiss it once
they've seen it - so it's not that intrusive. This would be excellent
for new releases, but also CFPs etc, and would free up the announce
  board for more general things like plugin releases etc.

## General notes

Discourse has a lot of other things going for it. Here's a few I
found:

* Great set of metrics and APIs which allow us to do all sorts of
  interesting things that I simply cannot do with Google Groups
* Much better search support Private messages may be useful (although
  I can't really s

Re: [foreman-dev] Foreman instrumenting analysis

[Lukas Zapletal]

> Does Prometheus only not work in a multi-process Rails web server? Does it
> work for a single process multi-threaded web server? This is an interesting
> roadblock given you'd expect this to affect lots of webserver across
> multiple languages out there.

Any Rails app that has multiple processes needs currently to figure
out how to deliver data to the HTTP endpoint. E.g. store it in a
database or something, which is not the best approach.

Absolutely, it lacks quite important feature right there. It stems
from the design which is pull-based.

> Yes, standard practice is to think about one container per pod (in a
> Kubernetes environment). However, there are patterns for things like log
> aggregation and monitoring such as doing a sidecar container that ensures
> co-location. The part I don't entirely get with sidecars is if I scale the
> pod to say 5, I get 5 web applications and 5 monitoring containers and that
> seems odd. Which I why I think the tendency is towards models where your
> single process/application is the end point for your metrics to be scrapped
> by an outside agent or services.
>
> I agree you want the collector to be separate, but if your web application
> is down what value would a monitoring endpoint being alive provide? The
> application would be down, thus no metrics to serve up. The other exporters
> such as the one exporting metrics about the underlying system would be
> responsible for giving system metrics. In the Kube world, this is handled by
> readiness and liveness probes for Kubenernetes to re-spin the container if
> it stops responding.

In container world, monitoring agents are running on hosts, not in
containers themselves. And collector agents can be 1:1 or 1:N (e.g.
for each container host). I am not sure I follow you here. Why you
don't see added value again? Monitoring agent without any apps
connected is as useful as ssh deamon waiting for connections.

Let me put it this way - push approach seems to be more appropriate
for multi process Ruby application than pull approach. That's what we
are discussing here, unless there are better protocols/agents I am not
aware of.

Honestly, pull approach via simple HTTP REST API seems cleaner but it
is just not good fit and also it creates other unnecessary
responsibility on the app itself. You are working on containerizing
Foreman, so it is also actually against this effort.

Anyway, let me throw another integration. Collectd has an agent (or
plugin) that opens a local socket which can be used to receive data
from other applications. I wrote Ruby client library the other day
(https://github.com/lzap/collectd-uxsock) but I believe this make no
difference than statsd - you still need a local process to gather the
data.

-- 
Later,
  Lukas @lzap Zapletal

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Propsing a move from Google Groups to Discourse

[Greg Sutcliffe]

OK, some questions I expect to come up:

* Will the mailing list import be kept up to date?

Yes, I plan to update it every day or two. Happily it's incremental (no
7 hour import) but sadly it's not something that's easy to automate. 

It's also why I've not renamed the categories yet, or fixed split-users 
(where a person has used more than on email address) - since the import
recreates anything it can't find, we'd get duplicates.

The sync is one-way, nothing done on Discourse will go back to the
list, but I'd ask that you treat the list categories as read-only for
now, and play around in the Test category, in case it confuses the
importer.

* Can I use email?

Yes, entirely if you wish. The catgories will have direct in-bound
email address (only the Test category has one right now), and all
outgoing emails have a reply-to set. We've tested it, and things like
in-line replies, URLs, and formatting seem OK.

* Can I log in with GitHub (or other)?

Yes, but the catch is that while the accounts are locked, GitHub login
will still work (unlike using password reset) - so you may see
weirdness if your account is not unlocked yet. Try it if you like
though, it's enabled. If we migrate fully over, then all accounts will
get unlocked and this will work fully.

* Are there spam controls like Groups?

Yes, and we'll actually have a lot more control over automating this. I
can go into details if people want more detail

* Won't it be hard to get people to update addressbooks?

Yes. We can mititgate this. Firstly, we'll obviously give appropriate
notice to the community - I don't expect us to migrate next week.
Second, once the groups are "closed" we *could* consider redirecting
mails to the Google groups addresses to Discourse, or bounce them back
with an appropriate message to the user. The categories will have in-
bound addresses, so they just need to update 2 addresses in their
contacts.

Hit me with more!
Greg

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: This is a digitally signed message part

Re: [foreman-dev] Propsing a move from Google Groups to Discourse

[Greg Sutcliffe]

One more point, we're currently on a free tier of our SMTP provider,
which is limited to 6k emails / month & 200 mails / day.

I'm seeing if I can negotiate a free higher tier for a FOSS project,
but for now, go easy on the mail testing :D

Greg

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: This is a digitally signed message part

[foreman-dev] Building a Rails 5.1 SCL

[Eric D Helms]

In a previous thread [1], we discussed building an SCL vs. vendorizing gems
and the general consensus was to build an SCL. This thread is to outline a
starting plan for how to build and maintain a Rails 5.1 SCL. I invite and
appreciate comments towards this design.

I'll begin by laying out some overall goals for the effort, a general
proposal of information and finally a breakdown of the why for each of the
proposal items to better explain my thinking.


Goals

 * Stand alone Rails 5.1 SCL and repository
 * Owned and maintained by the Foreman community
 * Easy to update and maintain
 * Strive for automation and package tooling to reduce maintenance cost


Proposal

Build location: Copr
SCL Name: tfm-ror51
Git repository: theforeman/tfm-ror51-packaging
Hosted: yum.theforeman.org/rails/tfm-ror51
RPM Signing: signed by unique Foreman owned GPG key
Tooling Repo: create package tooling repository separate from
tfm-ror51-packaging repo
Tooling Technology: Ansible


Breakdown

Build Location

There has been discussion around moving our RPM builds to Copr and off of
Koji. This will require shifting our configuration and setup, testing and
working out kinks in Copr workflow. Building this Rails SCL provides us an
opportunity to use Copr from the start, get familiar with it and workout
tooling to interact with and manage a repository there. Therefore, I am
proposing to start with Copr from the get go and avoid Koji.

SCL Name

Given the Foreman community will own the SCL, and our SCL namespace is
'tfm' I am suggesting we follow convention by naming it tfm-ror51. Previous
Rails SCLs, were named similarly: rh-ror41, rh-ror42, sclo-ror42.

Git Repository

I am proposing we don't put this into foreman-packaging given the lifecycle
of the SCL will not follow that of Foreman. Further, with the goal to
create a stand-alone Rails SCL repository this should have its own
lifecycle.

Hosted

We could point at the direct Copr repository, and reduce our bandwidth.
However, since we own this Rails SCL I believe we should host it as such
officially. Further, this would allow us to do some pre-flight testing by
building a repository in Copr, running a test of either the SCL itself
and/or Foreman against the SCL updates before promoting.

RPM Signing

I am recommending here that we sign the SCL RPMs with a new GPG key
generated just for signing the SCL packages.

Tooling

With an eye towards foreman-packaging changes, I am proposing we create a
separate git repository for all package tooling. The goal here would to
build out new tooling to allow easier maintenance of the packaging and
repository.  This will allow the tooling to evolve independently of either
packaging repository.  Further, when applying this tooling to
foreman-packaging later, the tooling would not have to be duplicated across
branches.

Tooling Technology

I am proposing a centralization on Ansible as the tooling technology for a
number of reasons. First, I feel that it has a low barrier to entry while
providing a mix of high and low level interfaces. Secondly, Ansible allows
orchestrating and building out more complex and directed tasks. Third, a
team of us has some built out Ansible tooling that has been working well
for us in another area that would be easy to port for packaging management.
I personally think bash is complex to understand for complex actions and is
too simple for building up a set of cohesive tooling.

-- 
Eric D. Helms
Red Hat Engineering

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Propsing a move from Google Groups to Discourse

[Andrew Kofink]

I admit, I skimmed the prior emails (they were tl). I just wanted to ask if
discourse is searchable/is indexed from the wider internet. I've often
found relevant mailing list discussions by searching from Google, and I
really value that.

- Andrew

On Wed, Nov 1, 2017 at 12:04 PM, Greg Sutcliffe 
wrote:

> One more point, we're currently on a free tier of our SMTP provider,
> which is limited to 6k emails / month & 200 mails / day.
>
> I'm seeing if I can negotiate a free higher tier for a FOSS project,
> but for now, go easy on the mail testing :D
>
> Greg
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Andrew Kofink
akof...@redhat.com
IRC: akofink
Software Engineer
Red Hat Satellite

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Merge permission for theforemen/foreman-ansible-modules

[Ewoud Kohl van Wijngaarden]

+1 while I haven't looked at his ansible work that closely I have seen 
that he's already behaving like an excellent maintainer: good reviews, 
active and generally a good and friendly person to have around.


On Mon, Oct 30, 2017 at 11:46:34AM -0400, Andrew Kofink wrote:

+1 from me! We have benefited greatly from Matthias' contributions in
foreman-ansible-modules. If you'd like to see some of his work, here are
his merged PRs
,
and here are all the PRs he has helped to review

.

On Mon, Oct 30, 2017 at 11:34 AM, Matthias Dellweg  wrote:


Hello,
i was just encouraged, to ask for merge/push permission in
theforeman/foreman-ansible-modules.
I have contributed to this repository almost since the beginning (yes,
it's a very young one),
and did a hand full of reviews that lead to constructive discussions. The
collaboration with
Andrew, Eric and Evgeni have always been very fruitful from my perspective.
For co-inventing the DRY glue layer 'cement' (with fobheb), github
classifies me as the top
garbage collector with by far the most removed lines.

I ask you kindly to vote, whether i shall be entrusted with the power of
the merge.
Thanks for considering,
  Matthias


--
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Propsing a move from Google Groups to Discourse

[Greg Sutcliffe]

The site will be made public if/when we go live with it. I assume Google will 
index it then, I don't think it does any robots.txt stuff - certainly I'm using 
google to look up configuration questions on meta.discourse.org :)
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Merge permission for theforemen/foreman-ansible-modules

[Eric D Helms]

+1 from me as well (I'll aim to add you in the requisite 5 more days
depending on the further outcome)

On Wed, Nov 1, 2017 at 3:24 PM, Ewoud Kohl van Wijngaarden <
ew...@kohlvanwijngaarden.nl> wrote:

> +1 while I haven't looked at his ansible work that closely I have seen
> that he's already behaving like an excellent maintainer: good reviews,
> active and generally a good and friendly person to have around.
>
> On Mon, Oct 30, 2017 at 11:46:34AM -0400, Andrew Kofink wrote:
>
>> +1 from me! We have benefited greatly from Matthias' contributions in
>> foreman-ansible-modules. If you'd like to see some of his work, here are
>> his merged PRs
>> > g+repo%3Atheforeman%2Fforeman-ansible-modules+is%3Amerged&type=Issues>,
>> and here are all the PRs he has helped to review
>> > lweg+-author%3Aakofink+repo%3Atheforeman%2Fforeman-ansible
>> -modules+is%3Amerged&type=>
>> .
>>
>> On Mon, Oct 30, 2017 at 11:34 AM, Matthias Dellweg 
>> wrote:
>>
>> Hello,
>>> i was just encouraged, to ask for merge/push permission in
>>> theforeman/foreman-ansible-modules.
>>> I have contributed to this repository almost since the beginning (yes,
>>> it's a very young one),
>>> and did a hand full of reviews that lead to constructive discussions. The
>>> collaboration with
>>> Andrew, Eric and Evgeni have always been very fruitful from my
>>> perspective.
>>> For co-inventing the DRY glue layer 'cement' (with fobheb), github
>>> classifies me as the top
>>> garbage collector with by far the most removed lines.
>>>
>>> I ask you kindly to vote, whether i shall be entrusted with the power of
>>> the merge.
>>> Thanks for considering,
>>>   Matthias
>>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Eric D. Helms
Red Hat Engineering

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Propsing a move from Google Groups to Discourse

[Eric D Helms]

I skimmed the site over, played with configuration, clicked and tried to
play with various part of it. There are parts I'll have to get used to, and
there are parts I wish were done slightly different but overall I like the
advanced power behind it.

Thinking a loud, it seems like this might allow us to more easily expanded
topic areas in the future? For example, if we wanted to have infrastructure
focused/tagged emails, or plugins specific, or design discussions that are
marked as such?

On Wed, Nov 1, 2017 at 3:28 PM, Greg Sutcliffe 
wrote:

> The site will be made public if/when we go live with it. I assume Google
> will index it then, I don't think it does any robots.txt stuff - certainly
> I'm using google to look up configuration questions on meta.discourse.org
> :)
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Eric D. Helms
Red Hat Engineering

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Propsing a move from Google Groups to Discourse

[Greg Sutcliffe]

That's indeed one of my aims. One common piece of Discourse wisdom is not to 
have too many categories, so I don't for see one per plugin - but the system 
also has tags which would work well in the Support category (I.e. foreman-users)

A category for Infra might well make sense separate from dev discussions. I 
could also see this being limited by trust level.

I can also see us making more use of Announcements, for events and plugin 
releases.

Greg
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.