Re: [foreman-dev] Getting Foreman and Smart-Proxy to run in FIPS environment.

[Dmitri Dolguikh]

I added a bunch of (more) granular-level issues to
http://projects.theforeman.org/issues/3511 tracker.

Cheers,
-d

On Wed, Nov 22, 2017 at 2:13 PM, Dmitri Dolguikh <witlessb...@gmail.com>
wrote:

> On Wed, Nov 22, 2017 at 9:33 AM, James Shewey <jdshe...@gmail.com> wrote:
>
>> You may not be getting an ABRT because ruby was patched some time ago to
>> catch this ABRT and a handler was created to make this a non-fatal error
>> (ruby used to just core dump - see https://bugzilla.redhat.com/sh
>> ow_bug.cgi?id=717709). I suspect that is why you aren't seeing the
>> expected signal further down the stack.
>>
>
> This doesn't explain the behaviour I'm seeing -- an exception would be
> caught and logged, and foreman would continue to run. I'm seeing the app
> exit without any stacktraces or core-dumps. I'm not using system ruby
> either.
>
> -d
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Getting Foreman and Smart-Proxy to run in FIPS environment.

[Dmitri Dolguikh]

On Wed, Nov 22, 2017 at 9:33 AM, James Shewey  wrote:

> You may not be getting an ABRT because ruby was patched some time ago to
> catch this ABRT and a handler was created to make this a non-fatal error
> (ruby used to just core dump - see https://bugzilla.redhat.com/
> show_bug.cgi?id=717709). I suspect that is why you aren't seeing the
> expected signal further down the stack.
>

This doesn't explain the behaviour I'm seeing -- an exception would be
caught and logged, and foreman would continue to run. I'm seeing the app
exit without any stacktraces or core-dumps. I'm not using system ruby
either.

-d

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Getting Foreman and Smart-Proxy to run in FIPS environment.

[Dmitri Dolguikh]

Do you have some examples how such an abort looks like with and without GDB?

with gdb + .gdbinit:

Program received signal SIGABRT, Aborted.
0x7fb45991e1f7 in __GI_raise (sig=sig@entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
56  return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
$1 = (rb_vm_t *) 0x117ef20
* #
0x7fb45259accc :in `digest'
(irb):5:in `irb_binding'
0x7fb45a80b318 :in `eval'
/home/vagrant/.rvm/rubies/ruby-2.4.2/lib/ruby/2.4.0/irb/workspace.rb:87:in
`evaluate'
/home/vagrant/.rvm/rubies/ruby-2.4.2/lib/ruby/2.4.0/irb/context.rb:381:in
`evaluate'
/home/vagrant/.rvm/rubies/ruby-2.4.2/lib/ruby/2.4.0/irb.rb:493:in
`block (2 levels) in eval_input'
...

with gdb only:
Program received signal SIGABRT, Aborted.
0x7f080307f1f7 in __GI_raise (sig=sig@entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
56  return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
(gdb) bt
#0  0x7f080307f1f7 in __GI_raise (sig=sig@entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x7f08030808e8 in __GI_abort () at abort.c:90
#2  0x7f07f9f2498f in OpenSSLDie (file=file@entry=0x7f07fa066cd3
"md5_dgst.c", line=line@entry=82,
assertion=assertion@entry=0x7f07fa066cb0 "Digest MD5 forbidden in FIPS
mode!") at cryptlib.c:1002
#3  0x7f07f9f2b209 in MD5_Init (c=0x34af790) at md5_dgst.c:82
#4  0x7f07fbcfbea0 in algo_init (algo=0x7f07fa518d60 ,
pctx=0x34af790) at digest.c:586
#5  0x7f07fbcfbf4d in rb_digest_base_alloc (klass=54958520) at
digest.c:606
#6  0x7f0803e6a6de in rb_obj_alloc (klass=54958520) at object.c:1863
#7  0x7f07fbcfbd32 in rb_digest_class_s_digest (argc=0,
argv=0x7f0804387668, klass=54958520) at digest.c:472
#8  0x7f0803f5997b in call_cfunc_m1 (func=0x7f07fbcfbccc
, recv=54958520, argc=1,
argv=0x7f0804387660) at vm_insnhelper.c:1589
...



> Why I am asking. Once we start supporting FIPS, I wonder how we are
> gonna provide support for users and customers who are in FIPS mode and
> "someting is breaking apart". Installing GDB would be really the last
> thing to do. I wonder if there is any chance of using SystemTap or
> even ABRT (Automatic Bug Reporting Tool) data (stacktrace with debug
> info, core dump). Trying to find a workflow which we will use for
> retrospective analysis to find the Ruby code which is causing this.
>
> Was wondering if we can hook into ABRT and print stacktrace similarly
> as we do for:
>
> https://github.com/theforeman/foreman/blob/698e916ce208b5040b83a908a058c8
> 3c94d158ee/config/initializers/sig_ttin_trap.rb


I tried installing an abrt signal handler when I was preparing this note,
but it never gets called, possibly due to openssl calling exit immediately
after raising 'abrt' [1]. Similarly, rubygem-abrt isn't catching 'abrt'
signals raised by openssl. The only thing that worked for me was the
approach I documented.

-d

 [1]
http://docs.huihoo.com/doxygen/openssl/1.0.1c/cryptlib_8c_source.html#l00912

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Getting Foreman and Smart-Proxy to run in FIPS environment.

[Dmitri Dolguikh]

On Mon, Nov 20, 2017 at 2:05 AM, Lukas Zapletal  wrote:

> Perhaps - can you bit elaborate the GDB
> thing? Is this some kind of hook that use used for FIPS stack to
> report "mistakes" (e.g. signal or exception when you attempt to use
> md5 hash)? I wonder if there is a way to catch these without GDB, for
> example with SystemTap which would allow to catch these also in
> production. Perhaps I misinterpret how this works, I haven't opened
> the PDFs to be honest yet :-)
>

DGB + ruby-specific .gdbinit are used to report ruby callstack when ABRT is
caught. I didn't look for alternatives, seeing that gdb + .gdbinit
combination is pretty trivial to use. I don't really care what we use here
as long as we can get ruby callstack back.


>
> I understand you only investigated core and smart proxy and I assume
> plugin authors will need to do the similar audit themselves. Since we
> are dealing with security, I would like to propose to create kind of
> FIPS CHECKLIST somewhere (perhaps our wikipage) so we could follow it
> and also add own findings/places to look for. In security world it's
> much more useful to have blueprints or checklists, I would not like to
> reinvent the wheel when doing this for discovery or other plugins.
>

Sure, sounds like a good idea. I think it's going to consist of the list of
hash-functions and ciphers not to use and a recommendation to test changes
on a system with FIPS enabled.


>
> Once we settle down on your proposal, please do create tracker
> issue(s) so we can associate our audits or changes to it. If you find
> anything that needs to be done for all plugins, please create tracker
> issue as well.
>

Tracker issue for Foreman already exists:
http://projects.theforeman.org/issues/3511.

-d

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Discourse summary week 2 (ish)

[Dmitri Dolguikh]

Another +1 from me -- quite like the UI, email integration seems to be (or
will be?) good enough. As long as the amount of maintenance isn't too big,
I think the move is a good idea.
-d

On Wed, Nov 15, 2017 at 7:05 AM, John Mitsch  wrote:

> Also a +1 from me. Among other things, I like the categories, markdown
> support, and tags.
>
> John Mitsch
> Red Hat Engineering
> (860)-967-7285 <(860)%20967-7285>
> irc: jomitsch
>
> On Wed, Nov 15, 2017 at 9:59 AM, Sebastian Gräßl 
> wrote:
>
>> I am one of the (silent) +1 voices Greg mentioned.
>>
>> Discourse would be a very welcomed move for me, there are a number of
>> reasons, which Greg already mentioned as well.
>>
>> For me the one key reason is the possibility to have multiple categories
>> to have discussions and share information in a proper context.
>>
>>
>> On Wednesday, November 15, 2017 at 3:46:01 PM UTC+1, Greg Sutcliffe wrote:
>>>
>>> On 15/11/17 13:58, Lukas Zapletal wrote:
>>> > I AM STRONGLY AGAINST *MIGRATING* OUR MAILING LISTS TO ANY KIND OF
>>> > FORUM.
>>>
>>> I have included your opinion in both summaries. As a long standing
>>> member of the community, your vote does carry some weight - but shouting
>>> doesn't get you an extra one, and makes it appear like you're trying to
>>> drown out the debate (which I'm sure you don't intend).
>>>
>>> If however you feel you've been misrepresented in either of my summaries
>>> in some way, please let me know so I can correct that.
>>>
>>> > You do not have blessing. You gathered how many opinions? Two dozens?
>>> > How many of us are subscribed here?
>>>
>>> No one decides "blessing" or lack of it unilaterally, not me, or you.
>>> The community decides, and current feedback suggests far more in
>>> *cautious* favour than against - certainly enough to continue the
>>> discussion.
>>>
>>> One thing I have not yet done is post how I see the migration actually
>>> happening *if* we choose to do it. That may help alleviate fears for
>>> some, so I will try to get that posted shortly. My view of that process
>>> won't include a side-by-side site for reasons I already expressed to
>>> Ivan earlier. Again, this is all very much *if*, please don't assume
>>> this is already decided.
>>>
>>> > You gathered how many opinions? Two dozens? How many of us are
>>> > subscribed here?
>>>
>>> 670 addresses according to Google Groups, but I know that's not what you
>>> mean :)
>>>
>>> As with all our discussions, we can only count the opinions that have
>>> actually been stated. We don't wait forever on any decision, and we
>>> frequently take action on 5 votes or less.
>>>
>>> As with other large debates we've had in the past, I'm being quite
>>> reserved with pacing this debate, *precisely* because I know how
>>> intrusive it is. I'm spending some of my time trying to convince those
>>> who haven't said anything yet to contribute (even if they give it a -1)
>>> so that we can be more sure of our final decision, but for sure we
>>> *will* have to take a decision at some point.
>>>
>>> It's worth saying again, no final decisions have been taken, this is
>>> still a consultation for now - but we *do* need to consult with the
>>> users too. It's a far larger community, and I hope we'll get a nice
>>> collection of opinions from them to look through.
>>>
>>> Greg
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to foreman-dev+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Getting Foreman and Smart-Proxy to run in FIPS environment.

[Dmitri Dolguikh]

> Given we are targeting Rails 5.1 for the SCL we are building and it's the
> newest, does anything change here with using it?
>

The approach stays the same, I think. My bet would be the problems stay the
same too (i.e. use of MD5 in caches).


> I have heard that the teams that work on those backend projects we use
> are/have looked into FIPS compliancy so we don't need to worry about that
> aspect.
>
> Sounds like we still need to look at the installer to ensure Kafo, Puppet
> and the Puppet modules are all FIPS compliant?
>

Yes, although I'd imagine puppet itself is going to be ok, as long as
FIPS-compatible hash function is configured. Puppet modules have issues,
for example dhcpd and bind puppet modules might use MD5s, as those
dependencies are configured with MD5-based shared secrets.


I suppose we could tests for this? For example, if a user provides custom
> SSL certificates for the web server we could run a check and warn a user?
>

Before a certificate is installed, yes. Might get involved though (as most
things tend to when it comes to SSL/TLS and certificates).


> Is it possible to detect that a system is running in FIPS mode to only
> warn in that case?
>

Yes, it is possible to detect that a system is running in FIPS mode.


>
> How often do you see this CI job running? And I assume this would be
> temporary until we are fully FIPS compliant?
>

A few times a day at first -- to help resolve w/e issues we currently have.
After that -- w/e pattern/schedule we currently have or plan to have, in
other words, we run *all* CI in FIPS mode.


cheers,
-d

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[foreman-dev] Getting Foreman and Smart-Proxy to run in FIPS environment.

[Dmitri Dolguikh]

What is FIPS?
>From Wikipedia [1]: The Federal Information Processing Standard (FIPS)
Publication 140-2, (FIPS PUB 140-2), is a U.S. government computer
security standard used to approve cryptographic modules. The title is
Security Requirements for Cryptographic Modules.

What are Implications of FIPS 140-2 Support for Foreman, Katello, and
Smart-Proxy?
Linux system, or rather an SSL library in FIPS-compatible mode will
only have a set of ciphers and hash functions compatible with FIPS.
[2] contains the list of approved cryptographic functions, Oracle
graciously compiled the list of not approved ones, which is more
useful and can be found at [3].


OpenSSL in FIPS mode
My understanding is that only OpenSSL versions 1.0.1 and 1.0.2 have
FIPS 140-2 validated cryptographic modules. OpenSSL raises ABRT signal
when it receives a call to one of the unapproved ciphers/functions.


Foreman in FIPS mode
I haven’t looked at pulp, candlepin, qpid, goferd, etc, and at point
don’t know how and if these can be made to work in FIPS mode. All
tests I’ve done so far were against Rails 5.0, Considering the number
of dependencies, we will need to limit FIPS support to just one
version of Rails.

Rails and other (ruby) dependencies.
MD5 is used (hard-coded) in a few places in Rails, at this point I’m
quite certain that its use is constrained to various built-in caches.
I had to disable *all* Rails caches to be able to run Foreman in FIPS
mode. Additionally, strong ETAG’s cannot be used, I’m not sure if they
are used, or there are plans for them.
Spring uses MD5 to generate application ID, but will use one in
SPRING_APPLICATION_ID environment variable if it’s available.
Gravatar uses MD5 hashes in their urls, doesn’t look like other hashes
are supported.
I think apipie cache uses MD5, but I will need to verify this.

Foreman
app/services/password_crypt uses MD5 for grub2 passwords, which will
need to be switched to SHA512. MD5 will need to be removed from the
list of hash functions
SshKey#generate_fingerprint, call to SSHKey.fingerprint uses MD5

A note: with caching disabled, and issues above fixed, I was able to
get Foreman suite of tests to pass, and get Foreman to start.

Smart-Proxy
Smart-Proxy codebase appears to be compatible with FIPS (ran and
passed tests ok without any changes), but there are issues with
external depdencies.

DHCPD uses MD5-based omapi shared secret. DHCPD shared secret with
bind is also md5-based.
BIND when used with dhcpd uses MD5 hashes stored in TXT as host id.
Puppet needs to be run in FIPS mode (FIPS-compatible hash function
needs to be configured). I assume this covers all of puppet, including
mcollective, puppet run, puppetca.
BMC/IPMI authentication can use MD5 or lower based hashes, older
clients may not have newer hash functions.
Salt appears to use MD5 hashes by default, individual nodes must be
configured to use other hash_type

Any 3rd party SSL certificates that may need to be verified or decoded
by either Foreman or Smart-Proxy must be generated using
FIPS-compatible algorithms/hash functions.

How we can reach FIPS compatibility
The easiest first step would be to replace offending cryptographic and
hash functions in Foreman, and in Smart-Proxy case, 3rd party
configuration files with FIPS-compatible ones. Additionally, any new
code changes that employ MD5 or other non approved functions shouldn’t
be accepted.
The next step would be to create a CI job that will continuously
execute the the full suite of tests on a VM with FIPS mode enabled.
GDB configured with Ruby’s project .gdbinit [4] and a tiny batch [5]
of commands can be used to report on FIPS-related failures.
Considering the amount of dependencies Foreman and Smart-Proxy have, I
think would be useful to have all CI environments switched to run in
FIPS mode: this should increase the probability of discovering of new
FIPS-related issues before our users.
Lastly, a FIPS-compatible caching solution for Rails needs to be
found, if none exist, an existing one needs to be modified to support
FIPS.


Any feedback would be appreciated,
-d

[1] Wikipedia article on FIPS 140-2, https://en.wikipedia.org/wiki/
FIPS_140-2
[2] Approved Security Functions for FIPS 140-2,
https://csrc.nist.gov/csrc/media/publications/fips/140/2/final/documents/
fips1402annexa.pdf
[3] List of algorithms not approved for FIPS 140-2,
https://docs.oracle.com/cd/E36784_01/html/E54953/fips-notok-1.html
[4] Ruby project’s gdb helper functions,
https://github.com/ruby/ruby/blob/trunk/.gdbinit
[5] Catching SIGABRTs with gdb and ruby-specific .gdbinit,
https://gist.github.com/witlessbird/904fefb0031c2eda96da61bd19424c86

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] speeding up tests

[Dmitri Dolguikh]

On Tue, Nov 7, 2017 at 12:29 AM, Lukas Zapletal  wrote:

> We should start tracking test breakages in time, if we have a test
> that did not break for a year, lets delete it. As simple as that. It's
> added value is almost zero.
>


The only time tests should be deleted is when the functionality they verify
has been removed. Test fragility, long execution time, and complexity are
not the reasons to get rid of them, but the signs of badly factored code,
wrong level of testing, or both.


>
> This is not out of my head, I've heard that on a talk somewhere.
> Instead of deleting, we can move it to archive - run them on a nightly
> basis, if we find this too much. Frankly, I prefer deletion.
>
> Also, let's ditch unit tests in favor of integration tests. We have a
> lot of areas covered with both and this is extra work and slowing down
> things.
>
>
Let's not. Integration and higher-up tests give big gains in test coverage
quickly, but that comes at the price of fragility, slow execution, and
inability to pinpoint failures. Unit tests, in our case it will be model
and controller tests, should be the main suite of tests.

-d


> LZ
>
> On Tue, Nov 7, 2017 at 9:19 AM, Ohad Levy  wrote:
> > Hi,
> >
> > The challenge:
> >
> > As a developer, I feel under utilized waiting for tests, the current test
> > suite for foreman core takes over an hour, multiply it by number of
> really
> > simple code fixes based on comment and it takes up a good chunk of a day.
> >
> > I would like to suggest a short and long term solution, and would
> appreciate
> > your feedback.
> >
> > Short-term:
> >
> > Break down test matrix into more specific groups, this will allow to
> > re-trigger just a smaller part of the tests vs. the entire suit today,
> for
> > example:
> > * API tests
> > * DB migrations
> > * UI Integration tests
> > * ...
> >
> > Long term:
> >
> > Break down core into smaller repositories - This is probably a bigger
> topic
> > then just tests, but its worth raising in this context as well.. the
> > benefits i see are:
> >
> > * smaller repos - faster tests per repo (we would still need a plugin
> matrix
> > kind of tests done at some point)
> > * better review process - smaller repos would mean the maintainers of the
> > repo actually care for all/most of the pr's - today its not the case -
> many
> > PR's are handled by sub groups (e.g. anything under webpack folder is
> > "someone else")
> > * potentially better API between parts - e.g. we can create a schema
> > specific repo (not saying its a good idea) - which will always work with
> a
> > dummy rails app - in the long run - this will ensure our schema is
> resilient
> > to upgrades and model changes in core.
> >
> > I would even go further and enable auto merge after review + successful
> > tests, e.g.
> > 1. PR is sent
> > 2. repo tests are executed and pass
> > 3. reviewer approve the change
> > 4. larger test matrix is executed and pass
> > 5. code get auto merged.
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "foreman-dev" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to foreman-dev+unsubscr...@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Later,
>   Lukas @lzap Zapletal
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] speeding up tests

[Dmitri Dolguikh]

Break down test matrix into more specific groups, this will allow to
re-trigger just a smaller part of the tests vs. the entire suit today, for
example:

> * API tests
> * DB migrations
> * UI Integration tests
> * ...
>
>
Probably a re-hash of what most developers know, but I'm going to do this
anyway.

Automated tests can be split into two groups -- "developer" and
"everything-else" tests. The former are meant to be used often, even
continuously (and are also known as unit-tests). These tests are meant to
test individual pieces of functionality (in OOP -- classes) in isolation
from the rest of the system (in other words without any dependencies,
external or other parts of the system). The rest of automated tests, such
as integration, end-to-end (including acceptance) tests can be slow and
fragile, and are oftentimes executed at some intervals (e.g. once on hour).

Of course, in rails environment we have almost unit-tests (model tests),
almost-integration tests (controller tests), and a bunch of weird (but
useful) stuff. Not sure how useful it would be to try to break up rails
tests into smaller chunks (at least as the first step) -- models and
controllers are rather tightly coupled, for example. I think it would be
useful to be able to run *just* the rails tests however, as it will give
developers and CI an ability to quickly (or at least quicker) check if the
change breaks anything.

The rest of the tests can be executed periodically, but possibly with
different frequencies per suite of tests. One of the main and biggest
drawbacks of such an approach is that the HEAD of the development branch
can become unstable/broken. When it happens, it can be difficult to
determine what change caused the failure (I wonder if automated git-bisect
can help with this). Such an approach also requires a coordination of
efforts in locating the issue, fixing it, and in abstaining from committing
code while the build is broken .



> Long term:
>
> Break down core into smaller repositories - This is probably a bigger
> topic then just tests, but its worth raising in this context as well..
>

Can we talk about service architecture now?

-d

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Vendorizing or Building RPMs

[Dmitri Dolguikh]

> Today we package all required rubygems as RPMs and utilize a thirdparty
> Software Collection (SCL) for both the Ruby (rh-ruby22) and Rails stack
> (sclo-ror42). The team that manages the RHSCLs has already released Ruby
> 2.3 and RUby 2.4 SCLs and will continue to do so. However, there are no
> plans to release any further SCLs for the Rails stack. This means, in
> addition to our own dependencies, we need to build and manage the Rails
> stack for the versions we want. This adds more packaging burden on the RPM
> side. GIven this, we have a few options:
>
>1) Build and manage our own SCLs for every version of Rails from here
on
> out
>2) Vendorize Rails and thirdparty gems into a one or more base RPMs
> (e.g. an RPM for rails stack and one for thirdparty dependencies)

Not sure if this has already been considered: use bundler to package the
app. In this case all dependencies, including rails will be cached locally,
and we could then wrap the result into an rpm. Not sure how plugins would
fit into this though.

-d

On Tue, Oct 17, 2017 at 11:17 AM, Bryan Kearney 
wrote:

> On 10/16/2017 04:36 AM, Michael Moll wrote:
> > I don't really have stakes in RPMs, but I'd try to stick with option 1
> > and I even expect that there's some demand for Rails SCLs at least in
> > the RHEL/CentOS world by other projects, too.
> >
> > Regards
>
> I would suggest the decision be made based on the assumption that the
> foreman community owns all the work for options 1 and 2. I have no
> special instights, but I would hate to assume we could package based on
> another communities work. Assume this commuity owns it all, which way is
> better/easier/quicker/less error prone.
>
> -- bk
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Release process & permissions

[Dmitri Dolguikh]

Why not distribute the release process? Each component can have (probably
already does) multiple maintainers who are capable of doing most of the
leg-work. The role of the release nanny then becomes coordinating the
effort, making public announcements and such. Such an approach would help
avoid overly broad permissions too.

-d

On Thu, Sep 28, 2017 at 3:23 AM, Ewoud Kohl van Wijngaarden <
ew...@kohlvanwijngaarden.nl> wrote:

> To release a gem to rubygems I'd recommend looking at how voxpupuli
> implemented this using Travis[1]. The same can be done for puppet[2]. That
> means you can push a tag to git and the release is there.
>
> There are also tools that help you bump. For puppet there is
> puppet-blacksmith[3]. How to do annotated tags is in my notes[3]. A more
> generic script is bump2version[5] and I'm sure there are similar tools in
> the ruby world.
>
> IMHO these tools should be suggested in the plugin templates.
>
> [1]: https://github.com/voxpupuli/modulesync/blob/e53079030501756
> fc4111511fb9b6a239ffcb2a3/.travis.yml#L18-L27
> [2]: https://github.com/voxpupuli/puppet-nginx/blob/fc363114e509f
> 439b51ea22723a996ffca8a107d/.travis.yml#L48-L58
> [3]: https://github.com/voxpupuli/puppet-blacksmith
> [4]: http://pad-katello.rhcloud.com/p/releasing-modules
> [5]: https://github.com/c4urself/bump2version
>
> On Wed, Sep 27, 2017 at 08:08:46AM -0700, Daniel Lobato wrote:
>
>> For foreman-docker I had a process:
>>
>> https://github.com/theforeman/foreman-docker/blob/master/rel
>> ease/RELEASE.md
>> https://github.com/theforeman/foreman-docker/blob/master/rel
>> ease/rollout-release
>>
>> which I planned to implement in all theforeman org plugins ideally.
>> It didn't happen but it's a similar thing.
>>
>> On Wednesday, September 27, 2017 at 4:56:51 PM UTC+2, Timo Goebel wrote:
>>
>>>
>>> ... isn‘t continuous delivery old these days? Why aren‘t we doing it?
>>> I vote in favor of automating this. This ensures predictable results and
>>> hopefully makes this process easier. To be honest: The current process
>>> scares me.
>>> Same for plugin releases. They also are way too manual right now.
>>>
>>> Timo
>>>
>>> > On 27. Sep 2017, at 16:46, Daniel Lobato Garcia >>
>>> > wrote:
>>> >
>>> > Hi devs,
>>> >
>>> > After a few releases, and now that I'm trying to help someone else to
>>> > take over in case it's needed, I found a roadblock.
>>> >
>>> > Whoever is doing the release, needs to have **many** permissions.
>>> >
>>> > Otherwise, it doesn't make much sense for a person to take over release
>>> > responsibilities. For example, if Ondrej has to do 1.15.5, he would
>>> need
>>> > the following permissions (see at the end of the email).
>>> >
>>> > Of course there are alternatives:
>>> >
>>> > 1 is to have the release nanny be supervised by people who have
>>> 'earned'
>>> > these permissions. This is a bad idea because some of the tasks just
>>> > cannot be 'supervised'. The nanny would have to ask someone to tag
>>> > repositories, modify jenkins jobs, upload GPG signatures, post to the
>>> > mailing list, tag new builds in Koji...
>>> >
>>> > 2 is to extend http://ci.theforeman.org/view/Release%20pipeline/ and
>>> > make it a real pipeline from 0 to release completed. At this moment,
>>> > releases that are not the first RC1 are mostly automated by
>>> > https://github.com/dlobatog/foreman_release and
>>> > https://github.com/theforeman/tool_belt.
>>> >
>>> > My proposal is to go forward with 2. Give Jenkins permissions to do all
>>> > of the actions needed, and whoever is the release nanny, ideally only
>>> > has to make sure all of the steps are moving forward. If something
>>> > breaks, figure out how to fix it for the next release.
>>> >
>>> > This would mean making a few extra jobs before and after the current
>>> > release pipeline. In my opinion, it's the way to go to ensure anyone
>>> can
>>> > take over this responsibility.
>>> >
>>> > At this moment, we are in a situation where only people who
>>> > mostly have permissions everywhere can successfully do a release
>>> without
>>> > asking many people for favors.
>>> >
>>> > Personally if we complete this, I see it as a big win as it would dwarf
>>> > our bus factor for release managers & allow us to release at any pace
>>> we
>>> > desire (right now it's slow because we can't truly release things from
>>> > one day to the next due to the work involved).
>>> >
>>> > Thoughts?
>>> >
>>> > Here's the list of permissions:
>>> >
>>> > 
>>> >
>>> > Github:
>>> >  - Push in foreman, foreman-selinux, foreman-installer,
>>> >smart-proxy, foreman-infra, foreman-packaging
>>> >
>>> > Transifex -
>>> >  - Allow to change the auto-update URL to point to latest -stable
>>> >branch
>>> >
>>> > Redmine -
>>> >  - Create new "Found in Release" version
>>> >
>>> > Jenkins -
>>> >  - Modify jobs
>>> >  - Run jobs
>>> >
>>> > Koji -
>>> >  - Create tags
>>> >  - SSH access to update the mash scripts
>>> >  - Create 

Re: [foreman-dev] Plugin gems with a single author

[Dmitri Dolguikh]

On Sun, Aug 20, 2017 at 6:49 AM, Ewoud Kohl van Wijngaarden <
ew...@kohlvanwijngaarden.nl> wrote:

> On Fri, Aug 18, 2017 at 12:48:06PM +0100, Greg Sutcliffe wrote:
>
>> Dominic Cleal:
>> - smart_proxy_dns_route53
>>
>
> I'd like to get added to this plugin.
>
> Ewoud Kohl van Wijngaarden:
>> - smart_proxy_dns_powerdns
>>
>
> Looking for a co-owner here :)
>

You can add me.

Cheers,
-d


> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Taking over plugins - do we need a policy?

[Dmitri Dolguikh]

FWIW, the approach I arrived at for smart-proxy plugins:

 - Explain the original developer their options for hosting the plugin
(i.e. within Foreman github organization, and outside of it, and what
each option entails). The former implies that Foreman developers have
a say over technical matters that affect maintenance (for example test
coverage).
 - Should they choose to move their code into the Foreman
organization, ask them to grant a committer access to at least one of
the Foreman core developers, and rights to release gems.
 - Agree with them on commit and review process.


Cheers,
-d



On Tue, Jul 18, 2017 at 4:15 AM, Greg Sutcliffe
 wrote:
> Hi all,
>
> I've been thinking for a while about plugins, and how to continue them
> when original authors move on. It's only natural that developers will
> come and go, so we need to think about how to deal with this. We've got
> a few examples of this now, and have had others in the past.
>
> 1) I'm playing with Salt in my spare time at the moment, with a view to
> (maybe) taking on the foreman_salt plugin, since Stephen is no longer
> working on it. However, it's only chance that I know this fact -
> there's no easy way for an author to mark a plugin as "orphaned".
>
> 2) Some plugins are awaiting changes but the author hasn't responded
> (yet!). Foreman_bootdisk has some open PRs at the moment that fall into
> this category (PRs 42, 43 for example), and default_hostgroup has pen
> issues (oops!). Presumably we need a way to ping authors and find out
> if they're just AFK or have stepped away from the plugin entirely.
>
> 3) Some plugins are definitely abandoned. I recall Chris Pisano taking
> over the foreman_banner plugin last year and struggling to get in touch
> with the original author at all.
>
> For context, Fedora does have a policy for this that makes some sense:
>
> https://fedoraproject.org/wiki/Policy_for_nonresponsive_package_maintai
> ners
>
> That's quite heavy, but some of the points make sense. So, do we need
> to add something to our docs about this. My gut feeling is:
>
> * Yes, we do
> * Only applies to plugins in "theforeman" GitHub repo
> * We need to add extra maintainers to the Rubygems *before* the author
> leaves - Chris had a real issue here. This could be a requirement of
> getting aplugin packaged, for example.
> * We need to allow authors to "abandon" a plugin clearly (something
> like how the Arch User Repository does it maybe?)
>
> Thoughts?
> Greg
>
> --
> You received this message because you are subscribed to the Google Groups 
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Re: [RFC] - Proxies with multiple interfaces

[Dmitri Dolguikh]

On Tue, Apr 4, 2017 at 7:18 PM,   wrote:
>
> I have an alternative, although I am not sure it's better:
> We can introduce "reachability" concept to our subnets. Something that would
> say "You can get from subnet X to subnet Y".

That sounds very much like you are suggesting that Foreman maintains
routing tables. Manually or automatically? Either will escalate very
quickly.
-d

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Re: 1.15 branching in approximately 2 weeks

[Dmitri Dolguikh]

On Wed, Mar 15, 2017 at 8:10 AM, Tomas Strachota  wrote:
> On Tue, Mar 14, 2017 at 2:28 PM, Daniel Lobato  wrote:
>> We're planning to do the branching on all projects on March 28th -
>> please keep it in mind and try to get
>> things that would be important for 1.15 by then.
>>
>> I do not have commit access everywhere, so please check the following
>> list and reply if you
>> could take care of cherry-picking and branching:
>>
>> foreman - Daniel - dlobatog
>> foreman-installer - Stephen - stbenjam
>> smart-proxy - Daniel - dlobatog - (or Dmitri - witlessb if you prefer)

yes can do.
-d

>> foreman-selinux - Lukas - lzap
>> foreman-packaging - Eric - ehelms
>> theforeman.org - Daniel - dlobatog
>> translations - Daniel - dlobatog
>> community-templates - Daniel - dlobatog
>> hammer-cli - Tomas? Martin?
>> hammer-cli-foreman - Tomas? Martin?
>
> I'll take care of hammer branching and release.
>
>> puppet-foreman - ?
>> puppet-foreman_proxy - ?
>>
>>
>> On Fri, Mar 10, 2017 at 7:43 PM, Daniel Lobato Garcia
>>  wrote:
>>> As per 
>>> http://projects.theforeman.org/projects/foreman/wiki/Foreman_115_Schedule,
>>> 1.15 is due to be branched soon, in 2 weeks approximately.
>>>
>>> Please help to make develop more stable during this process, be it through
>>> thorough testing, bugfixing or improving documentation.
>>>
>>> Help with updating the manual would be appreciated, see the list of
>>> tasks in:
>>>   - https://github.com/theforeman/theforeman.org/issues/765
>>>
>>> Some translations are nearing 100% completion, we welcome contributions
>>> that help finishing them for the next release:
>>>   - https://www.transifex.com/foreman/foreman/foreman/
>>>
>>> If you have any critical fixes that you think should be included in 1.15
>>> in any of the core projects, please reply to this read to raise
>>> attention over them. Maintainers feel free to set the 1.15 label to
>>> any PR of that nature.
>>>
>>> Best,
>>>
>>> --
>>> Daniel Lobato Garcia
>>>
>>> @dLobatog
>>> blog.daniellobato.me
>>> daniellobato.me
>>>
>>> GPG: http://keys.gnupg.net/pks/lookup?op=get=0x7A92D6DD38D6DE30
>>> Keybase: https://keybase.io/elobato
>>
>>
>>
>> --
>> Daniel Lobato
>>
>> @dlobatog
>> daniellobato.me
>>
>> GPG: http://keys.gnupg.net/pks/lookup?op=get=0x7A92D6DD38D6DE30
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to foreman-dev+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Issues with support of IPv6/DHCPv6 in Smart-Proxy

[Dmitri Dolguikh]

On Fri, Mar 10, 2017 at 4:45 PM, Ewoud Kohl van Wijngaarden
<ew...@kohlvanwijngaarden.nl> wrote:
> On Fri, Mar 10, 2017 at 02:40:04PM +0000, Dmitri Dolguikh wrote:
>> After thinking about support for ipv6 + dhcpv6 in smart-poxy and
>> Foreman some more, I realized that:
>>
>>  - while host duid management is possible, it’s going to be somewhat
>> involved (quite a few different scenarios) and possibly fragile
>
> If I understand it correct then RFC6939 allows you to use the MAC
> address as well. Given this is very close to the IPv4 implementation I
> believe this is what should be explored first.
>

Option 79 is used to specify client link-layer address, but it
requires that either:
 - dhcp client populate the option, or
 - first hop dhcp relay populates the option

Which isn't guaranteed. For example neither NetworkManager nor
dhclient generate this option by default.

> ISC dhcpd 4.3 support this and Debian Jessie ships it, but EL7 doesn't.
>
> Do I understand it correct that besides DUID vs MAC the OMAPI protocol
> doesn't support setting IPv6 addresses?

Correct.

>
>>  - dhcp servers can be configured to update dns records automatically
>> using hostname (and domain name, if required) that client passes with
>> dhcp request (or names can be generated automatically)
>
> While possible, I'm not sure how many people use this in a datacenter
> context.

Neither do I. Would be interesting to learn.

>
>>  - Foreman's main requirement is to set correct pxe boot options for
>> managed hosts
>
> Correct, it's good not to lose focus on this to keep the scope small.
>
>>  - multiple dhcp servers can co-exist on the same network/network
>> segment, with responses selected by the client based on server
>> precedence
>>
>> This leads me to think that having a small dhcp server (that we can
>> interface with using a convenient set of api) dedicated to handling
>> pxe requests in a given network segment and ignoring all request is an
>> option that’s worth investigating. Turns out google has a project that
>> fits the bill (https://github.com/google/netboot). I’m trying to
>> establish a contact with project’s maintainers to find out about
>> dhcpv6 state (not currently implemented) and see if they need help.
>
> Also note that it should be packaged.

Nods.

-d

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Issues with support of IPv6/DHCPv6 in Smart-Proxy

[Dmitri Dolguikh]

After thinking about support for ipv6 + dhcpv6 in smart-poxy and
Foreman some more, I realized that:

 - while host duid management is possible, it’s going to be somewhat
involved (quite a few different scenarios) and possibly fragile
 - dhcp servers can be configured to update dns records automatically
using hostname (and domain name, if required) that client passes with
dhcp request (or names can be generated automatically)
 - Foreman's main requirement is to set correct pxe boot options for
managed hosts
 - multiple dhcp servers can co-exist on the same network/network
segment, with responses selected by the client based on server
precedence

This leads me to think that having a small dhcp server (that we can
interface with using a convenient set of api) dedicated to handling
pxe requests in a given network segment and ignoring all request is an
option that’s worth investigating. Turns out google has a project that
fits the bill (https://github.com/google/netboot). I’m trying to
establish a contact with project’s maintainers to find out about
dhcpv6 state (not currently implemented) and see if they need help.

 Cheers,
-d

On Wed, Mar 8, 2017 at 11:53 AM, Lukas Zapletal <l...@redhat.com> wrote:
> We can always try to offload the work on platform team, I've been
> pretty successful in getting some work done in iPXE project for
> example. The only issue is this can take some time until it gets to
> the other team backlog.
>
> To be honest, I don't know which path you should take. I'd vote for
> the cleanest one, that is trying to push proper IPv6 into ISC. If
> there's no politics behind it, I don't believe they would not want to
> have this kind of feature. On the other hand, they've been resisting
> quite long time now (IPv6 is old how much, 20 years? :-)
>
> Since you already did great amount of investigation in the IPv6 field,
> do you want to give dnsmasq a look in this regard? If this turns out
> to be a good workaround, maybe the ISC IPv6 would not be so hot topic
> then (when I finish with my provider).
>
> LZ
>
> On Tue, Mar 7, 2017 at 5:37 PM, Dmitri Dolguikh <witlessb...@gmail.com> wrote:
>> On Tue, Mar 7, 2017 at 4:28 PM, Perry Gagne <pga...@redhat.com> wrote:
>>> Wouldn't it be better to add support for  fixed-address6 or is there some
>>> issue with doing that?
>>
>> It looks to me like ISC aren’t really interested in extending
>> functionality of dhcpd. On top of this, it may actually be more work
>> than it looks: both memory db serialization and omapi need updates
>> (the latter accepts fixed-addresses only atm). I have a pretty
>> high-level understanding of dhcpd codebase, but even at this level it
>> looks like omapi-related change might be quite big in scope.
>>
>> -d
>>
>>>
>>> On Tue, Mar 7, 2017 at 11:27 AM, Dmitri Dolguikh <witlessb...@gmail.com>
>>> wrote:
>>>>
>>>> On Tue, Mar 7, 2017 at 4:22 PM, Perry Gagne <pga...@redhat.com> wrote:
>>>> >> This is definitely the case as far as ipv6-specific dhcpd
>>>> >> configuration goes. The code I linked to is responsible for
>>>> >> persistence of in-memory host records, and it doesn’t support ipv6 at
>>>> >> all.
>>>> >
>>>> > Maybe I am confused. Are you trying to add support for fixed-address6 or
>>>> > trying to work around the support not being there?
>>>>
>>>> The latter.
>>>> -d
>>>>
>>>> >
>>>> > On Mon, Mar 6, 2017 at 11:32 AM, Dmitri Dolguikh <witlessb...@gmail.com>
>>>> > wrote:
>>>> >>
>>>> >> On Mon, Mar 6, 2017 at 4:11 PM, Perry Gagne <pga...@redhat.com> wrote:
>>>> >> >
>>>> >> > Doesn't ISC DHCPD use " fixed-address6", "range6", etc for DHCPv6.
>>>> >> > The
>>>> >> > code
>>>> >> > you linked seems to be related to "fixed-address" which is the ipv4
>>>> >> > variant.
>>>> >> >
>>>> >>
>>>> >> This is definitely the case as far as ipv6-specific dhcpd
>>>> >> configuration goes. The code I linked to is responsible for
>>>> >> persistence of in-memory host records, and it doesn’t support ipv6 at
>>>> >> all.
>>>> >>
>>>> >> > Another question I have, does DHCP really need to be the one
>>>> >> > assigning
>>>> >> > the
>>>> >> > address? Should DHCP just be providing config info, and leave

Re: [foreman-dev] Issues with support of IPv6/DHCPv6 in Smart-Proxy

[Dmitri Dolguikh]

On Tue, Mar 7, 2017 at 4:28 PM, Perry Gagne <pga...@redhat.com> wrote:
> Wouldn't it be better to add support for  fixed-address6 or is there some
> issue with doing that?

It looks to me like ISC aren’t really interested in extending
functionality of dhcpd. On top of this, it may actually be more work
than it looks: both memory db serialization and omapi need updates
(the latter accepts fixed-addresses only atm). I have a pretty
high-level understanding of dhcpd codebase, but even at this level it
looks like omapi-related change might be quite big in scope.

-d

>
> On Tue, Mar 7, 2017 at 11:27 AM, Dmitri Dolguikh <witlessb...@gmail.com>
> wrote:
>>
>> On Tue, Mar 7, 2017 at 4:22 PM, Perry Gagne <pga...@redhat.com> wrote:
>> >> This is definitely the case as far as ipv6-specific dhcpd
>> >> configuration goes. The code I linked to is responsible for
>> >> persistence of in-memory host records, and it doesn’t support ipv6 at
>> >> all.
>> >
>> > Maybe I am confused. Are you trying to add support for fixed-address6 or
>> > trying to work around the support not being there?
>>
>> The latter.
>> -d
>>
>> >
>> > On Mon, Mar 6, 2017 at 11:32 AM, Dmitri Dolguikh <witlessb...@gmail.com>
>> > wrote:
>> >>
>> >> On Mon, Mar 6, 2017 at 4:11 PM, Perry Gagne <pga...@redhat.com> wrote:
>> >> >
>> >> > Doesn't ISC DHCPD use " fixed-address6", "range6", etc for DHCPv6.
>> >> > The
>> >> > code
>> >> > you linked seems to be related to "fixed-address" which is the ipv4
>> >> > variant.
>> >> >
>> >>
>> >> This is definitely the case as far as ipv6-specific dhcpd
>> >> configuration goes. The code I linked to is responsible for
>> >> persistence of in-memory host records, and it doesn’t support ipv6 at
>> >> all.
>> >>
>> >> > Another question I have, does DHCP really need to be the one
>> >> > assigning
>> >> > the
>> >> > address? Should DHCP just be providing config info, and leave the
>> >> > address up
>> >> > to SLAAC? [1]
>> >>
>> >> It is possible to use autoconf for IPv6 address (self-)assignment and
>> >> rely on dhcp for host configuration only. I’m not sure how useful
>> >> autoconf is in larger networks (anyone has experience with this?), but
>> >> it has a potential for making our life harder.
>> >>
>> >> -d
>> >>
>> >> >
>> >> >
>> >> > [1] https://tools.ietf.org/html/rfc3736
>> >> >
>> >>
>> >> --
>> >> You received this message because you are subscribed to the Google
>> >> Groups
>> >> "foreman-dev" group.
>> >> To unsubscribe from this group and stop receiving emails from it, send
>> >> an
>> >> email to foreman-dev+unsubscr...@googlegroups.com.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "foreman-dev" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an
>> > email to foreman-dev+unsubscr...@googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to foreman-dev+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Issues with support of IPv6/DHCPv6 in Smart-Proxy

[Dmitri Dolguikh]

On Tue, Mar 7, 2017 at 4:22 PM, Perry Gagne <pga...@redhat.com> wrote:
>> This is definitely the case as far as ipv6-specific dhcpd
>> configuration goes. The code I linked to is responsible for
>> persistence of in-memory host records, and it doesn’t support ipv6 at
>> all.
>
> Maybe I am confused. Are you trying to add support for fixed-address6 or
> trying to work around the support not being there?

The latter.
-d

>
> On Mon, Mar 6, 2017 at 11:32 AM, Dmitri Dolguikh <witlessb...@gmail.com>
> wrote:
>>
>> On Mon, Mar 6, 2017 at 4:11 PM, Perry Gagne <pga...@redhat.com> wrote:
>> >
>> > Doesn't ISC DHCPD use " fixed-address6", "range6", etc for DHCPv6. The
>> > code
>> > you linked seems to be related to "fixed-address" which is the ipv4
>> > variant.
>> >
>>
>> This is definitely the case as far as ipv6-specific dhcpd
>> configuration goes. The code I linked to is responsible for
>> persistence of in-memory host records, and it doesn’t support ipv6 at
>> all.
>>
>> > Another question I have, does DHCP really need to be the one assigning
>> > the
>> > address? Should DHCP just be providing config info, and leave the
>> > address up
>> > to SLAAC? [1]
>>
>> It is possible to use autoconf for IPv6 address (self-)assignment and
>> rely on dhcp for host configuration only. I’m not sure how useful
>> autoconf is in larger networks (anyone has experience with this?), but
>> it has a potential for making our life harder.
>>
>> -d
>>
>> >
>> >
>> > [1] https://tools.ietf.org/html/rfc3736
>> >
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to foreman-dev+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Issues with support of IPv6/DHCPv6 in Smart-Proxy

[Dmitri Dolguikh]

On Tue, Mar 7, 2017 at 1:46 PM, Lukas Zapletal <l...@redhat.com> wrote:
> On Mon, Mar 6, 2017 at 4:53 PM, Dmitri Dolguikh <witlessb...@gmail.com> wrote:
>> - It is difficult to determine host’s client id (dhcp unique identifier, AKA 
>> DUID)
>
> What you mean by that? If sysadmins let DUID to change, they are
> breaking IPv6 specs and servers will mis-behave. What exactly are you
> trying to solve here. We can provide some tools to help them (e.g.
> sending DUIDs alongside with other facts so they can easily recover
> them if they reboot or something like that). Do you have some
> automatic DUID management in mind?

There are a few issues here:
 - /etc/machine-id is generated during system install (each new clean
install on the same hardware will yield a different machine-id).
 - NetworkManager above 0.9.7.997 derives DUID based on
/etc/machine-id. We cannot calculate DUID prior to system installation
unless we manage /etc/machine-id.
 - sysconfig-based networking by default will use DUID-LLT (link-layer
address plus timestamp) in stateful mode, but it can be overridden in
the interface config file. We will need to manage interface config
files in order to have persistent DUIDs.
 - Windows hosts use yet another way of configuring DUIDs
 - As DUID value is dependent on how network interfaces are configured, we:
  - may need to ask the user how they plan to manage networking and
generate appropriate configuration (probably too error-prone)
  - create config files for both NetworkManager and sysconfig

>
> NM calculates DUID from system-id, we can generate both in advance
> when creating new host and provide them via Kickstart or snippet
> indeed if this makes things easier.

Yes, also see above.

>
>> - PXE uses client id (UUID/GUID) that is different from that of the host.
>
> Is this a real issue? We already deal with two allocations per PXE
> booted system with IPv4 (even three if you use discovery) and it
> works.

Less so for IPv6. It’s something that we may need to manage however,
esp. if we decide to rely on dns record management by dhcp server.

>
> Anyway, I am writing dnsmasq provider in my spare time. Does dnsmasq
> makes it any better? I was primarly targetting to improve out-of-box
> experience on IPv4 networks, but if dnsmasq makes it easier (I haven't
> looked into IPv6 at all), I can speedup my coding and provide the
> plugin earlier. Dnsmasq can work as DHCP or DHCP Proxy and it's in
> RHEL. Even if it won't provide a workaround or replacement, we could
> send a patch or something as it has smaller and modern codebase.
> Writing own DHCP proxy? Meeeh. Just a thought.

I don’t know much about dnsmasq. Most people use isc dhcpd or dhcpy6d
(https://dhcpy6d.ifw-dresden.de/) for dhcp6.

-d

>
> --
> Later,
>   Lukas @lzap Zapletal
>
> --
> You received this message because you are subscribed to the Google Groups 
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Issues with support of IPv6/DHCPv6 in Smart-Proxy

[Dmitri Dolguikh]

On Mon, Mar 6, 2017 at 4:11 PM, Perry Gagne  wrote:
>
> Doesn't ISC DHCPD use " fixed-address6", "range6", etc for DHCPv6. The code
> you linked seems to be related to "fixed-address" which is the ipv4 variant.
>

This is definitely the case as far as ipv6-specific dhcpd
configuration goes. The code I linked to is responsible for
persistence of in-memory host records, and it doesn’t support ipv6 at
all.

> Another question I have, does DHCP really need to be the one assigning the
> address? Should DHCP just be providing config info, and leave the address up
> to SLAAC? [1]

It is possible to use autoconf for IPv6 address (self-)assignment and
rely on dhcp for host configuration only. I’m not sure how useful
autoconf is in larger networks (anyone has experience with this?), but
it has a potential for making our life harder.

-d

>
>
> [1] https://tools.ietf.org/html/rfc3736
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] RFC idea: Templates rendering only through Proxies

[Dmitri Dolguikh]

I think this is a good idea; and it can be pushed into separating
templates into a standalone service which would handle the whole
template lifecycle, not just serving of pre-rendered templates (if
eventually).

Also agreed re: “built” ping, it’s a special call, although the
responsibility for collecting this data can be moved to smart-proxy
too.

OT: HA smart-proxy setups are hard b/c proxy wasn’t engineered for HA,
and different HA setups require different approaches (HA dhcp is
different from HA dns, etc). It very much solvable, but it’s a fair
bit of work and so far I was unable to get a clear answer about how
important is to have such a feature.

-d

On Thu, Jan 26, 2017 at 3:21 PM, Lukas Zapletal  wrote:
> On Thu, Jan 26, 2017 at 11:16 AM, Timo Goebel  wrote:
>> How do you want to handle the "built"-Url? I think, you still need a
>> callback to Foreman when a Host has finished provisioning. That basically
>> voids most of the advantages (like improving reliability - btw: a Foreman ha
>> is easy vs. a smart-proxy ha setup is hard).
>
> Yeah, sure, I noticed this too, but this built call is special - no
> output, no need of determining template API endpoint etc. It's a very
> simple case.
>
>> I do think, it's a good idea that Foreman knows the current state of an
>> installation. We could even show valueable information on the provisioning
>> progress or potential errors like with [2]. That would be a huge benefit for
>> users, at least from my experience.
>
> My proposal does not change that, I believe. Template contents audit
> is something I miss from the day one when I started with foreman and
> pre-rendering could basically help with that - we could put whole
> template content into audit record during build, so we can be sure
> clients see the exactly same.
>
> --
> Later,
>   Lukas @lzap Zapletal
>
> --
> You received this message because you are subscribed to the Google Groups 
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[foreman-dev] Re: Drop 1.9.3 from smart-proxy test matrix

[Dmitri Dolguikh]

I opened a PR in foreman-infra that removes ruby 1.9.3 from test 
matrix: https://github.com/theforeman/foreman-infra/pull/256.
-d

On Monday, November 7, 2016 at 11:41:13 AM UTC, Dmitri Dolguikh wrote:
>
> Awhile ago when we announced end of support for ruby 1.8.7 for 
> smart-proxy, we also announced that we’ll likely drop support for 
> 1.9.3 too. 
>
> While we are no longer running tests against 1.8.7 in smart-proxy 
> development branch, we still have ruby 1.9.3 in the test matrix. I’d 
> like to remove 1.9.3 from the matrix but want to see if there are any 
> concerns with that first. 
>
> Thanks, 
> -Dmitri 
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[foreman-dev] Drop 1.9.3 from smart-proxy test matrix

[Dmitri Dolguikh]

Awhile ago when we announced end of support for ruby 1.8.7 for
smart-proxy, we also announced that we’ll likely drop support for
1.9.3 too.

While we are no longer running tests against 1.8.7 in smart-proxy
development branch, we still have ruby 1.9.3 in the test matrix. I’d
like to remove 1.9.3 from the matrix but want to see if there are any
concerns with that first.

Thanks,
-Dmitri

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Re: foreman-proxy rest api

[Dmitri Dolguikh]

Here’s a partial list of smart-proxy api:
http://projects.theforeman.org/projects/smart-proxy/wiki/API. Please
keep in mind that the list doesn’t include any of the non-core plugin
apis.

Cheers,
-Dmitri


On Fri, Sep 2, 2016 at 3:29 PM, 'Gennady Voronkov' via foreman-dev
 wrote:
>
>>You can create new endpoints via the plugin API, extending the smart
>>proxy using a new rubygem.
>>http://projects.theforeman.org/projects/foreman/wiki/Smart-Proxy_Plugins
>>contains more information and links to some existing examples.
>>--
>>Dominic Cleal
>>dom...@cleal.org
>
> thanks a lot, may you know all endpoints for foreman-proxy? or tell me
> please how to find out?
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Deprecate EL6?

[Dmitri Dolguikh]

I’d like to see development of new features stopped on EL6:
 - no more random build breakages due to dependencies not supporting
1.8.7 and/or 1.9.3
 - older releases are still available for those who prefer to stay on EL6
 - Backup/restore procedures are documented, support for config file
migrations in smart-proxy has been available for quite awhile now

I have at least one feature planned for smart-proxy that requires
dependencies not available on 1.8.7 (at all). At this point converting
the smart-proxy to use SCL isn’t worth it, and it would be great not
to have an additional 3-months wait.

-d

On Fri, Jul 29, 2016 at 8:08 AM, Marek Hulán  wrote:
> On Thursday 28 of July 2016 16:34:30 Greg Sutcliffe wrote:
>> So I had a read back through the thread, and drew a few conclusions. The
>> history seems to be:
>>
>> * The thread was opened ~10 weeks ago, and went quiet ~6 weeks ago
>> * At that time, there were 5 upvotes (Dominic, Daniel, Ewoud, Michael,
>> Duncan), 1 downvote (BK), and 1 unsure (Ohad, who said he had "mixed
>> feelings). Others commented but did not express a preference.
>> * We then jump forward to a few days ago, with all the discussion that's
>> happened since.
>>
>> It's hard to deny that it looked OK to go ahead with, as of 3 days ago;
>> 5-to-1 is something we wouldn't normally challenge, and 6 weeks is plenty
>> of time to speak up about concerns.
>>
>> However, I do take issue with comparing it to the previous OS deprecations,
>> which by-and-large have been Debian/Ubuntu deprecations. These (a) don't
>> impact as large a part of the community, (b) have a shorter shelf-life than
>> EL (implying users are more comfortable with upgrading), and (c) have a
>> simpler packaging structure. They're also unlikely to be using any of the
>> RPM-centric plugins we offer, which (of course) will be more affected by
>> dropping EL6 (bear in mind that according to the 2016 survey, some 30% of
>> our community use Katello - that's not insignificant). As such, while I
>> agree no fallout occurred in the past, I think we do have to be more
>> careful this time.
>>
>> More than anything, I think it's the sudden switch from "dead thread" to
>> "it's done" that has raised tempers, rather than the decision itself. I
>> think it would have been reasonable to follow up with a "this seems
>> decided, I'll do it in a few days" type of email - similar to how the
>> branching happens. We have a similar "final call" approach in the RFCs, in
>> case anyone lost sight of the thread in the noise.
>>
>> In short then, we are where we are. I think these concerns *should* have
>> been raised earlier, but now that they *have* been raised, I think they're
>> valid, and we should finish the discussion. As it stands today, we have
>> around 5-to-4 in favour of dropping EL6 in 1.13, but that's not any kind of
>> consensus.
>
> Count me in for EL6 support for at least another release. If there's some
> extra effort needed with which I can help somehow, I'm happy to do so. I
> thought we solved main problems on smart-proxy using puppet API and SCL for
> dynflow plugin.
>
> --
> Marek
>
>>
>> Can we get a (re)statement of the pros/cons from each side, to inform the
>> choices of those on the fence? As of 10th May it seemed that the support of
>> EL6 was only causing minor issues - is that still true? Other than
>> preparation time (which is valid) are there other concerns for dropping it?
>>
>> (PS - More generally, we seem to have an issue with "closing"
>> discussions/RFCs with a firm decision; many discussions die out in this
>> way. Thoughts on how we can improve that are welcome, and I will post my
>> own, but lets start a new thread for that)
>
> --
> You received this message because you are subscribed to the Google Groups 
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[foreman-dev] Vendor options in MS dhcp provider

[Dmitri Dolguikh]

Support for vendor options is currently broken in MS dhcp provider:
 - Vendor options have to be created before they can be assigned
values. We create vendor classes if they don’t exist, but don’t do the
same for options.
 - A lesser problem is the way we name vendor classes in MS dhcp
differs from how we do the same in isc dhcp, it’s possible to end up
with multiple vendor classes representing the same set of vendor
options.

Both of these can be fixed, but how important is support for
sun-specific vendor options, esp. in Windows environments? I haven’t
seen complains regarding issue #1, which probably means that vendor
options can be dropped altogether on Windows.

Thoughts or concerns?

Cheers,
-Dmitri

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Revamping the news section of the website frontpage - design discussion

[Dmitri Dolguikh]

I think the home page we have atm contains *way* too much information
(three screens or even more?).  Some of it is duplicated/too
similar/should be grouped together (why do we have “introduction”,
“quckstart”, and “get started”? Should we have both “documentation”
and “learn more” links?. Do we need a dedicated “videos" link at the
top?). I don’t the videos currently located on the home page should be
there either.

I think it would be useful to show important announcements (releases,
CVE's, etc) and upcoming events on the home page, if only a limited
number.

Having said all that, I don’t know how long it will take to figure out
our home page content and layout, so an intermediate solution is
probably warranted. Perhaps that one line that Daniel suggested?

>
> (a) does the proposed design work for you?

parts of it, but not the whole thing.

> (b) if not, what would be your counter-proposal for highlighting news/events
> etc?

I think it should take a minimal amount of space (as I think that the
home page should fit on one screen)

> (c) should we keep the top banner for occasional use?

If we have a home page that fits on one screen the information about
events/announcements spot will always be visible, which would make
such a banner redundant.


Cheers,
-d

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.