[foreman-dev] Infra/Deployment/Platform Roadmap Update

2017-10-17 Thread Eric D Helms
A week or two back, I sent out a gathered Roadmap of tasks in
Infra/Deployment/Platform. The following is a report of updates on some of
those endeavors until I can move this to a better location for facilitating
updates and tracking.


   - Rails 5.X


   - Updates
  - Discussion on vendorizing vs. building SCL
  https://groups.google.com/forum/#!topic/foreman-dev/xJyxMx1lXy4
  - Notice of current state of core running on 5.1:
  https://groups.google.com/forum/#!topic/foreman-dev/kCMCCNZUN4w
   - Prior


   - need rh-ror50 or custom built SCL


   - consider whether we introduce a new SCL (e.g. tfm-ruby23) to separate
  RPMs built against old SCL vs. new


   - comment mmoll: We would need to upgrade Ruby (to 2.4 or 2.5) later,
  but I'd expect Rails 5.0 and even 5.1 to fully work also on Ruby 2.2


   - commend ekohl: if package names remained equal then it would simplify
  the installer/docs


   - should we jump to 5.1.latest and build the SCL instead? rh-ror50 is
  the last Rails SCL from RHSCL team


   - comment mmoll: with a little help from core and plugin devs, a move to
  Rails 5.1 for 1.17 feels achievable.


   - dlobatog: +1 - introducing the SCL to retire it quite soon will be a
  pain for upgrades and 5.1 for 1.17 doesn't seem that far off.


   - Jenkins Migration


   - Updates
  - Jenkins now has HTTPS interface: https://ci.theforeman.org/
  - redirect from HTTP needs adding
   - Prior
  - Migrate Jenkins master to EL7


   - add https interface to Jenkins


   - Jenkins Job Updates


   - Updates
  - Foreman nightly release pipeline created and has replaced former
  Foreman nightly jobs
   - Prior
  - Migrate jobs to pipelines


   - What is the benefit for this effort?


   - modern approach, more secure, provides more efficient jobs, jobs that
  are protected against crashes and restarts


   - Move all jobs into JJB


   - Update JJB code location within git for discoverability


   - Update jobs to run tests with all plugins installed


   - Update hammer core tests to run tests also for the major plugins (at
  least foreman and katello)


   - Add job for running hammer integration tests against live
  foreman/katello


   - Running Container Stack


   - Updates
  - No updates
   - Prior
  - address Github issues created from initial merge


   - remove current hacks in deployment


   - build up test suite for verifying container stack


   - add Jenkins job to build containers nightly


   - find way to continuously test container deployment


   - Merging katello-packaging to foreman-packaging


   - Updates
  - No updates
   - Prior
  - develop and agree on strategy for moving packages


   - move packages


   - any chance of moving Katello tags into foreman-plugins directly?


   - yes, but we need to solve other problems first:


   - updated yum repository structure (see below)


   - individual package release pipelines


   - Release automation


   - Updates
  - No updates
   - Prior
  - Using tool_belt & foreman_release to do the
cherry_picking/tagging/building/signing
  automatically


   - Update http://projects.theforeman.org/projects/
  foreman/wiki/Release_Process to document how it should work


   - Moving Katello puppet modules to foreman


   - Updates
  - foreman-module-sync being synchronized with Katello's
  - Ewoud working on migration steps to undertake in the next week
   - Prior
  - move modules to theforeman Organization


   - add puppet modules to foreman-installer-modulesync


   - Merging katello and foreman installers


   - Updates
  - No updates
   - Prior
  - Move all checks/hooks


   - Add katello modules


   - Move bin/{foreman-proxy-certs-generate,katello-certs-check}


   - Migrate scenarios


   - Sort out the packaging


   - Add deprecation notices to katello-installer / wipe master branch


   - Updated yum repository structure


   - Updates
  - No updates
   - Prior
  - Email thread discussing re-structure of repositories


   - agree on layout


   - re-factor mash scripts for new deployment


   - re-factor sync scripts to yum/deb repositories


   - update foreman release RPM for new repositories


   - Move package building from Koji to Copr


   - Updates
  - No updates
   - Prior
  - Phase 1: Submit builds in paralel - only rubygems and nodejs


   - Phase 2: Submit builds in paralel - foreman-core packages


   - Phase 3: Migrate to Copr


   - Multi-server service deployment


   - Migration off of Openshift V2


   - Redmine -- moved to stand alone server and off of Openshift entirely


   - prprocessor -- still running on Openshift V2


   - etherpad -- moved from katello to foreman openshift v2 instance


-- 
Eric D. Helms
Red Hat Engineering

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop

Re: [foreman-dev] The road to Rails 5.1

2017-10-17 Thread Eric D Helms
Thanks for the update Michael. I just want to point interested parties to
the RPM side of the discussions that are on going over in
https://groups.google.com/forum/#!topic/foreman-dev/xJyxMx1lXy4

On Tue, Oct 17, 2017 at 4:32 PM, Michael Moll  wrote:

> Hi,
>
> while the original plan was to switch to Rails 5.0 soon and then begin
> 5.1 work, it's a major downside that RPMs would be broken for a
> potentially longer period, so I closed
> https://github.com/theforeman/foreman/pull/4867 and like to draw the
> attention of interested parties to
> https://github.com/theforeman/foreman/pull/4836 where all the changes
> that are queued up lead to a still 100% green Rails 5.0 test run and
> leave 21 test failures with 5.1.
>
> I guess two of iNečas' recently opened PRs will fix some of these.
>
> Besides the missing changes to get green tests one major problem is that
> there's no Rails 5 compatible version of turbolinks-classic, so either
> Foreman needs to move to turbolinks 5 or a forked turbolinks-classic gem
> would be needed, if turbolinks should be kept.
>
> In the meanwhile Rails 5.1 should be packaged for RPM in some way... :)
>
> Regards
> --
> Michael Moll
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Eric D. Helms
Red Hat Engineering

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [foreman-dev] Vendorizing or Building RPMs

2017-10-17 Thread Eric D Helms
Plugins do present the most complicated part of this process given they
would each potentially be requiring a few independent gems separate from
the core stack and we would not want bloat. Seems we'd need either:

 1) a single gem stack that plugins contribute to and is renegenrated with
all them enabled
 2) a method for plugins to generate the bundle install, and pick out just
the gems they need to contribute on top of the core stack

On Tue, Oct 17, 2017 at 6:42 PM, Dmitri Dolguikh 
wrote:

> > Today we package all required rubygems as RPMs and utilize a thirdparty
> > Software Collection (SCL) for both the Ruby (rh-ruby22) and Rails stack
> > (sclo-ror42). The team that manages the RHSCLs has already released Ruby
> > 2.3 and RUby 2.4 SCLs and will continue to do so. However, there are no
> > plans to release any further SCLs for the Rails stack. This means, in
> > addition to our own dependencies, we need to build and manage the Rails
> > stack for the versions we want. This adds more packaging burden on the
> RPM
> > side. GIven this, we have a few options:
> >
> >1) Build and manage our own SCLs for every version of Rails from here
> on
> > out
> >2) Vendorize Rails and thirdparty gems into a one or more base RPMs
> > (e.g. an RPM for rails stack and one for thirdparty dependencies)
>
> Not sure if this has already been considered: use bundler to package the
> app. In this case all dependencies, including rails will be cached locally,
> and we could then wrap the result into an rpm. Not sure how plugins would
> fit into this though.
>
> -d
>
> On Tue, Oct 17, 2017 at 11:17 AM, Bryan Kearney 
> wrote:
>
>> On 10/16/2017 04:36 AM, Michael Moll wrote:
>> > I don't really have stakes in RPMs, but I'd try to stick with option 1
>> > and I even expect that there's some demand for Rails SCLs at least in
>> > the RHEL/CentOS world by other projects, too.
>> >
>> > Regards
>>
>> I would suggest the decision be made based on the assumption that the
>> foreman community owns all the work for options 1 and 2. I have no
>> special instights, but I would hate to assume we could package based on
>> another communities work. Assume this commuity owns it all, which way is
>> better/easier/quicker/less error prone.
>>
>> -- bk
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to foreman-dev+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Eric D. Helms
Red Hat Engineering

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [foreman-dev] Vendorizing or Building RPMs

2017-10-17 Thread Dmitri Dolguikh
> Today we package all required rubygems as RPMs and utilize a thirdparty
> Software Collection (SCL) for both the Ruby (rh-ruby22) and Rails stack
> (sclo-ror42). The team that manages the RHSCLs has already released Ruby
> 2.3 and RUby 2.4 SCLs and will continue to do so. However, there are no
> plans to release any further SCLs for the Rails stack. This means, in
> addition to our own dependencies, we need to build and manage the Rails
> stack for the versions we want. This adds more packaging burden on the RPM
> side. GIven this, we have a few options:
>
>1) Build and manage our own SCLs for every version of Rails from here
on
> out
>2) Vendorize Rails and thirdparty gems into a one or more base RPMs
> (e.g. an RPM for rails stack and one for thirdparty dependencies)

Not sure if this has already been considered: use bundler to package the
app. In this case all dependencies, including rails will be cached locally,
and we could then wrap the result into an rpm. Not sure how plugins would
fit into this though.

-d

On Tue, Oct 17, 2017 at 11:17 AM, Bryan Kearney 
wrote:

> On 10/16/2017 04:36 AM, Michael Moll wrote:
> > I don't really have stakes in RPMs, but I'd try to stick with option 1
> > and I even expect that there's some demand for Rails SCLs at least in
> > the RHEL/CentOS world by other projects, too.
> >
> > Regards
>
> I would suggest the decision be made based on the assumption that the
> foreman community owns all the work for options 1 and 2. I have no
> special instights, but I would hate to assume we could package based on
> another communities work. Assume this commuity owns it all, which way is
> better/easier/quicker/less error prone.
>
> -- bk
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[foreman-dev] The road to Rails 5.1

2017-10-17 Thread Michael Moll
Hi,

while the original plan was to switch to Rails 5.0 soon and then begin
5.1 work, it's a major downside that RPMs would be broken for a
potentially longer period, so I closed
https://github.com/theforeman/foreman/pull/4867 and like to draw the
attention of interested parties to
https://github.com/theforeman/foreman/pull/4836 where all the changes
that are queued up lead to a still 100% green Rails 5.0 test run and
leave 21 test failures with 5.1.

I guess two of iNečas' recently opened PRs will fix some of these.

Besides the missing changes to get green tests one major problem is that
there's no Rails 5 compatible version of turbolinks-classic, so either
Foreman needs to move to turbolinks 5 or a forked turbolinks-classic gem
would be needed, if turbolinks should be kept.

In the meanwhile Rails 5.1 should be packaged for RPM in some way... :)

Regards
-- 
Michael Moll

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [foreman-dev] Vendorizing or Building RPMs

2017-10-17 Thread Bryan Kearney
On 10/16/2017 04:36 AM, Michael Moll wrote:
> I don't really have stakes in RPMs, but I'd try to stick with option 1
> and I even expect that there's some demand for Rails SCLs at least in
> the RHEL/CentOS world by other projects, too.
> 
> Regards

I would suggest the decision be made based on the assumption that the
foreman community owns all the work for options 1 and 2. I have no
special instights, but I would hate to assume we could package based on
another communities work. Assume this commuity owns it all, which way is
better/easier/quicker/less error prone.

-- bk

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[foreman-dev] Place to put slide decks

2017-10-17 Thread Lukas Zapletal
Hey,

I have a couple of slide decks about Foreman, mostly recycled
material. At OSS Europe in Prague I am giving "the standard" talk
about Foreman on Monday, will edit the latest version of the slides
but I was wondering - do we have a place to put those OpenOffice.org
files in?

If not, shall we create another git repository? :-)

I am putting the event into our events.yaml tomorrow.

-- 
Later,
  Lukas @lzap Zapletal

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [foreman-dev] Turned off autoupdate at transifex

2017-10-17 Thread Bryan Kearney
I have just re-enabled this.

-- bk


On 10/17/2017 09:51 AM, Lukas Zapletal wrote:
> I would like to know what was the reason to do that. It used to work
> fine for discovery so far.
> 
> I just pushed updated POT into git, I am about to branch off 10.0
> tomorrow. You told me you are going to re-enable this again? Thanks
> 
> LZ
> 
> On Thu, Oct 5, 2017 at 4:18 PM, Lukas Zapletal  wrote:
>> Bryan, I don't understand. Why this was a problem? My understanding is
>> that this feature takes latest and greatest source POT file and puts
>> that for translations.
>>
>> In my workflow, I just pull translations back into PO files when do
>> doing a release. This was a good workflow I think. Or maybe I am
>> missing something.
>>
>> LZ
>>
>> On Thu, Oct 5, 2017 at 2:49 PM, Bryan Kearney  
>> wrote:
>>> I have turned off autoupdatnig of transifex resources. This was a feature
>>> for transifex to pull in the latest stings from github. This would work if
>>> core and all plugin writers are syncing the strings at every release.
>>>
>>> If you release core or a plugin and you are not doing that as part of your
>>> release process can you bug me off line? I would like to make sure that I
>>> did not mess anyone up, and that strings are flowing correctly from project
>>> <-> transifex <-> Red Hat.
>>>
>>> -- bk
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "foreman-dev" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to foreman-dev+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>> --
>> Later,
>>   Lukas @lzap Zapletal
> 
> 
> 

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [foreman-dev] Turned off autoupdate at transifex

2017-10-17 Thread Lukas Zapletal
I would like to know what was the reason to do that. It used to work
fine for discovery so far.

I just pushed updated POT into git, I am about to branch off 10.0
tomorrow. You told me you are going to re-enable this again? Thanks

LZ

On Thu, Oct 5, 2017 at 4:18 PM, Lukas Zapletal  wrote:
> Bryan, I don't understand. Why this was a problem? My understanding is
> that this feature takes latest and greatest source POT file and puts
> that for translations.
>
> In my workflow, I just pull translations back into PO files when do
> doing a release. This was a good workflow I think. Or maybe I am
> missing something.
>
> LZ
>
> On Thu, Oct 5, 2017 at 2:49 PM, Bryan Kearney  wrote:
>> I have turned off autoupdatnig of transifex resources. This was a feature
>> for transifex to pull in the latest stings from github. This would work if
>> core and all plugin writers are syncing the strings at every release.
>>
>> If you release core or a plugin and you are not doing that as part of your
>> release process can you bug me off line? I would like to make sure that I
>> did not mess anyone up, and that strings are flowing correctly from project
>> <-> transifex <-> Red Hat.
>>
>> -- bk
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to foreman-dev+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>
> --
> Later,
>   Lukas @lzap Zapletal



-- 
Later,
  Lukas @lzap Zapletal

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [foreman-dev] Proposed drop of supporting ruby 2.0 in hammer

2017-10-17 Thread Tomas Strachota
Thanks,
I removed ruby 2.0.0 from configuration matrix of the jenkins jobs
relavant to hammer. And created two redmine issues for removing the
old compatibility code:
* http://projects.theforeman.org/issues/21359
* http://projects.theforeman.org/issues/21360

T.

On Tue, Oct 10, 2017 at 2:44 PM, Andrew Kofink  wrote:
> +1 out with the old
>
> On Tue, Oct 10, 2017 at 8:00 AM, Michael Moll  wrote:
>>
>> On Tue, Oct 10, 2017 at 01:45:42PM +0200, Ewoud Kohl van Wijngaarden
>> wrote:
>> > On Tue, Oct 10, 2017 at 01:21:36PM +0200, Tomas Strachota wrote:
>> > >we recently encountered a compatibility issue with older version of
>> > >Clamp that we use on ruby 2.0 installations. Latest Clamp releases
>> > >require ruby 2.1+. See [1] for some more details.
>> > >
>> > >The easiest solution seems to be dropping ruby 2.0 support, which was
>> > >eol 2016-02-24 anyway. We use scl with ruby 2.2 on rpm based distros,
>> > >so we should be safe there.
>> > Support for Trusty has been dropped in 1.16 and 1.17 will drop Jessie.
>> > Focussing on 2.1+ or 2.2+ should be no problem.
>>
>> exactly.
>> --
>> Michael Moll
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to foreman-dev+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --
> Andrew Kofink
> akof...@redhat.com
> IRC: akofink
> Associate Software Engineer
> Red Hat Satellite
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [foreman-dev] foreman_tasks V1 endpoint works but V2 returns 404

2017-10-17 Thread rajesh . erasani
Tried lower cased v2 as well. Still the same.

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [foreman-dev] foreman_tasks V1 endpoint works but V2 returns 404

2017-10-17 Thread rajesh . erasani

Based on the understanding from following documentation’s v2 should work. Any 
other ideas?

https://theforeman.org/manuals/1.15/index.html#5.1.6APIVersioning 

https://theforeman.org/api/1.15

https://access.redhat.com/documentation/en-us/red_hat_satellite/6.0/html/api_guide/chap-foreman_tasks

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [foreman-dev] Sharing and idea: VMWare VIX instead of DHCP for vCenter VM's provisioning

2017-10-17 Thread Ewoud Kohl van Wijngaarden

On Mon, Oct 16, 2017 at 11:39:56AM -0700, Evgeny Vasilchenko wrote:

I'd like to share an idea for provisioning of VMware guest VM's with
Foreman.

The idea required programming new type of Foreman proxy - unfortunately, I
neither have time or enough skills for that.

*The goal:*

  - Allow configuration of Linux (and maybe Windows) VM network interfaces
  with VMWare VIX's API 
  instead of DHCP

*Facts:*

  - VMWare offers a free VM management API called *VIX*
  - https://blogs.vmware.com/vix/2008/07/what-is-vix-and.html
 - *VIX is an API that lets you programmatically control the products
 that host VMware VMs, and control the VMs themselves.*
- it works with many VMWare products including vCenter
 -
- VIX package can be installed on both

*Linux and Windows *
  - VIX has a wrapper utility called *vmrun*
  (https://www.vmware.com/support/developer/vix-api/vix112_vmrun_command.pdf)
  which offers a CLI for controlling VM's running - e.g. starting, stopping,
  pausing, deleting, etc...

  - vmrun can be installed on a *Foreman Smart-Proxy* (AKA Sat6 capsule)
  *host* and
 - run any command or script *INSIDE* of a guest VM running on a
 vCenter computing resource - i

*.e. it can configure whatever, including network interfaces. *
  - in order to run something inside of VM - vmrun only need to
  know (beside of vCenter credentials of course)
  - *path to .VMX file* (already known after VM provisioning)
   - guest OS username (i.e. root, administrator) and password
- command line or script name to be executed

In fact VIX API and vmrun can do way more that this - I feel it simply must
be integrated into Foreman Proxy.
I've been using it for various VCenter related automation tasks and like
it's versatility and power.

Any ideas?


Would a VMware VIX backend to Remote Execution be feasable and useful?

--
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [foreman-dev] foreman_tasks V1 endpoint works but V2 returns 404

2017-10-17 Thread Greg Sutcliffe
On Mon, 2017-10-16 at 22:15 -0700, rajesh.eras...@gmail.com wrote:
> # curl --user sledge:hammer -H "Content-Type:application/json" -k htt
> ps://ind2q00katello01.qa.local/foreman_tasks/api/V2/tasks/3574500b-
> 0394-4a94-9f86-8ff1890ceadb

I think this is a character-case issue - it's *v2* not *V2*. I can
replicate the 404 using V2.

Greg

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [foreman-dev] Vendorizing or Building RPMs

2017-10-17 Thread Ivan Necas
I'm ok with the vendorizing approach, as long as it makes the release
process more
effective than it is right now.

Could you expand more on how the core dependencies, plugins and
plugin-dependencies work?
Perhaps this is something we could align with the debian packaging as well to
address some of the issues that we are facing.

-- Ivan

On Sat, Oct 14, 2017 at 8:24 PM, Eric D Helms  wrote:
> Note: This is specific to RPMs because as far as I know the Debian process
> is different and uses gems directly. Please correct me and contribute any
> information with respect to Debian that I miss. I believe the Nodejs part of
> the email does apply to Debian.
>
> This proposal is in two parts: gems and node packages. The general issue
> applies to both, the project has a need to be able to move and update
> dependencies more quickly given the speed at which platforms like Rails,
> React and Webpack update.
>
>
> Node/NPM Packages
>
> Today, we package nearly all nodejs packages that are required to either
> build the UI or runtime UI dependencies into packages. At current count,
> there are 78 nodejs packages listed in Koji as RPMs. Over the past few
> months, the move to React and updating UI within Foreman core and branching
> into plugins has led to multiple, and at times, compounding packaging and
> thus nightly breakages. One of those breakages lasted over a week with
> multiple developers spending many hours tracking down the issue. The speed
> at which these changes are made makes keeping up successful nightly builds
> difficult. The git repository gets well ahead of the packaging which leads
> to compounding issues. This has also been the cause of stable version
> release delays.
>
> Proposal:
> Deprecate nodejs packages in favor of foreman-assets or a new RPM
> foreman-node-modules that contains a source tarball of node_modules/
> packaged into a simple RPM that is used as a build dependency. This tarball
> would be regenerated, and the package bumped, as dependency updates are
> needed.
>
>
> Ruby Gems
>
> Today we package all required rubygems as RPMs and utilize a thirdparty
> Software Collection (SCL) for both the Ruby (rh-ruby22) and Rails stack
> (sclo-ror42). The team that manages the RHSCLs has already released Ruby 2.3
> and RUby 2.4 SCLs and will continue to do so. However, there are no plans to
> release any further SCLs for the Rails stack. This means, in addition to our
> own dependencies, we need to build and manage the Rails stack for the
> versions we want. This adds more packaging burden on the RPM side. GIven
> this, we have a few options:
>
>1) Build and manage our own SCLs for every version of Rails from here on
> out
>2) Vendorize Rails and thirdparty gems into a one or more base RPMs (e.g.
> an RPM for rails stack and one for thirdparty dependencies)
>
> Option 2, to vendorize, is a deviation from our prior practices in the area
> of production deployment. Thus, I am reaching out to the community to get
> feedback. One interesting consideration for vendorizing is when containers
> are considered and having the ability to build them using 'bundle install'
> versus using RPM based installation.
>
> In theory vendorizing allows the community to move faster, keep up to date
> with dependencies more easily and reduces the burden on RPM packaging to
> keep nightly builds flowing. However, this does stray from the standard RPM
> based installation benefits typically associated to it.
>
>
> Please consider carefully, and weigh in even if it is simply to +1 or -1 a
> given idea. Deployment is a large part of our project and the more voices
> weighing in the better.
>
> Thanks,
> Eriic
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [foreman-dev] Vendorizing or Building RPMs

2017-10-17 Thread Lukas Zapletal
I agree with Tomer here, ditto.

On the other hand, the way how rubygems are being packaged makes
package patches (RPM patch file sets) really hard (in short - it's a
hack). Since I also touch packaging work here and there, I know how
much work is to maintain full stack like Ruby on Rails. So I can
imagine having RoR vendored completely, if and only if this goes into
downstream without much changes (I see this as possible problem).
There are advantages of this approach, we could easily ship more gems,
e.g. development gems, without much effort. Every gem vendored would
need to be reviewed (changelog and license) for sure, just like
others.

So I give Node vendoring +1 and both B1 and B2 proposals also +1 +1.

LZ

On Tue, Oct 17, 2017 at 9:56 AM, Tomer Brisker  wrote:
> Thanks for bringing up the discussion!
>
> As one who has been bitten multiple times by the issue of building RPMs for
> node modules, a definite +1 (or more like +1000) from me to vendoring the
> entire node_modules directory.
> This will save a lot of time and effort on the part of maintainers that will
> no longer have to wrestle with koji for every module, as well as reduce the
> load on the builders since we won't be wasting cycles building RPMs for the
> sole purpose of consuming them during our assets compilation.
>
> However, I'm not sure if this approach should also apply to ruby gems. In
> general, the number of gems and the frequency in which they are updates are
> both lower, and unlike the node RPMs they are actually used during runtime
> and not only during build time, making sense for them to be separate RPMs
> that can be in some cases even be upgraded without upgrading the core
> packages (which again doesn't apply to node modules which will require a
> recompile anyways).
>
> On Mon, Oct 16, 2017 at 4:50 PM, Michael Moll  wrote:
>>
>> Hi,
>>
>> On Mon, Oct 16, 2017 at 02:36:27PM +0100, Greg Sutcliffe wrote:
>> > > Ruby Gems
>> > >
>> > > Option 2, to vendorize, is a deviation from our prior practices in
>> > > the area of production deployment. Thus, I am reaching out to the
>> > > community to get feedback. One interesting consideration for
>> > > vendorizing is when containers are considered and having the ability
>> > > to build them using 'bundle install' versus using RPM based
>> > > installation.
>> >
>> > Vendoring hasn't (to my knowledge) caused many issues for Debian users
>> > (Michael?), and having consistent build processes makes it easier for
>> > anyone to support users on different OSs.
>>
>> From a user's perspective it's not that bad, but IMHO the current
>> approach does have some downsides regarding plugins and I'm unsure if a
>> big plugin like Katello can be handled by the current DEB way of doing
>> things.
>>
>> Regards
>> --
>> Michael Moll
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to foreman-dev+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
>
>
> --
> Have a nice day,
> Tomer Brisker
> Red Hat Engineering
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



-- 
Later,
  Lukas @lzap Zapletal

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [foreman-dev] foreman_tasks V1 endpoint works but V2 returns 404

2017-10-17 Thread Adam Ruzicka
Hello,
as far as I know there is no v1 API for foreman-tasks. So the one you're
reaching with the first curl is actually the v2. I agree it is not
intuitive to have v2 api at the regular /api endpoint, but that's the way
it is.

-- Adam

On Tue, Oct 17, 2017 at 7:15 AM,  wrote:

> Hi,
>
> If I
>
> *#* curl --user admin:changeme -H "Content-Type:application/json" -k
> https://katelloserver/foreman_tasks/api/tasks/3574500b-0394-
> 4a94-9f86-8ff1890ceadb
>
> I get the expected response back. But if I send the request to V2 of the
> api as follows
>
> *#* curl --user sledge:hammer -H "Content-Type:application/json" -k
> https://ind2q00katello01.qa.local/foreman_tasks/api/*V2*/
> tasks/3574500b-0394-4a94-9f86-8ff1890ceadb
> *OR*
> *#* curl --user sledge:hammer -H "Content-Type:application/json" -H
> "Accept:application/json,*version=2*" -k https://ind2q00katello01.qa.
> local/foreman_tasks/api/tasks/3574500b-0394-4a94-9f86-8ff1890ceadb
>
>
> I basically get a 404 resource not found error.
>
>
> Installed foreman_tasks core package is on 0.1.4 and foreman_tasks package
> is on 0.9.4.
>
> Any suggestions, what I am missing here? Thanks in advance.
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [foreman-dev] Vendorizing or Building RPMs

2017-10-17 Thread Tomer Brisker
Thanks for bringing up the discussion!

As one who has been bitten multiple times by the issue of building RPMs for
node modules, a definite +1 (or more like +1000) from me to vendoring the
entire node_modules directory.
This will save a lot of time and effort on the part of maintainers that
will no longer have to wrestle with koji for every module, as well as
reduce the load on the builders since we won't be wasting cycles building
RPMs for the sole purpose of consuming them during our assets compilation.

However, I'm not sure if this approach should also apply to ruby gems. In
general, the number of gems and the frequency in which they are updates are
both lower, and unlike the node RPMs they are actually used during runtime
and not only during build time, making sense for them to be separate RPMs
that can be in some cases even be upgraded without upgrading the core
packages (which again doesn't apply to node modules which will require a
recompile anyways).

On Mon, Oct 16, 2017 at 4:50 PM, Michael Moll  wrote:

> Hi,
>
> On Mon, Oct 16, 2017 at 02:36:27PM +0100, Greg Sutcliffe wrote:
> > > Ruby Gems
> > >
> > > Option 2, to vendorize, is a deviation from our prior practices in
> > > the area of production deployment. Thus, I am reaching out to the
> > > community to get feedback. One interesting consideration for
> > > vendorizing is when containers are considered and having the ability
> > > to build them using 'bundle install' versus using RPM based
> > > installation.
> >
> > Vendoring hasn't (to my knowledge) caused many issues for Debian users
> > (Michael?), and having consistent build processes makes it easier for
> > anyone to support users on different OSs.
>
> From a user's perspective it's not that bad, but IMHO the current
> approach does have some downsides regarding plugins and I'm unsure if a
> big plugin like Katello can be handled by the current DEB way of doing
> things.
>
> Regards
> --
> Michael Moll
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Have a nice day,
Tomer Brisker
Red Hat Engineering

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: [foreman-dev] Current State of Nightlies

2017-10-17 Thread Lukas Zapletal
EPEL is not great place to be for Rails or Node components. You should
not bump versions in EPEL7 (major relase should go into EPEL8).

On Tue, Oct 17, 2017 at 12:01 AM, Eric D Helms  wrote:
>
>
> On Oct 16, 2017 5:17 PM, "Sean O'Keeffe"  wrote:
>
> Why dont we ask the maintainer to pkg a new version or someone offer to
> become a co-maintainer and get a new version into EPEL ?
>
>
> While I think this is the right open source path, I'd weigh:
>
>   1) how long will nighties be broken waiting on a new package?
>   2) 2.0 to 4.1 is a large jump and as a base dependency other EPEL packages
> may not work.
>
>
>
> On Mon, Oct 16, 2017 at 9:31 PM, Ewoud Kohl van Wijngaarden
>  wrote:
>>
>> On Mon, Oct 16, 2017 at 04:18:53PM -0400, Eric D Helms wrote:
>>>
>>> Nightly RPM and tests have been broken for around 2 weeks now. This
>>> morning
>>> a bit of a regression was merged to foreman core to fix the breaking RPM
>>> aspect and I can report that nightly RPMs are now building. However, this
>>> leads to a breakage in plugin asset usage with the newer React
>>> components.
>>> To potentially address this I have opened [1] for testing and input. As
>>> part of the original breakage, I added to test_develop running npm
>>> install
>>> and webpack compile the same way our RPMs do in order to catch these sort
>>> of issues earlier.
>>
>>
>> Thanks for this!
>>
>>> The second half is that after RPMs were built, repoclosure on the test
>>> pipeline is currently failing [2]. The highlight being:
>>>
>>> *20:10:35* package: nodejs-react-dom-15.6.2-1.el7.noarch from
>>> undertest-yum_el7-4203183943-68*20:10:35*   unresolved deps:
>>> *20:10:35*  npm(object-assign) >= 0:4.1.0*20:10:35*
>>> npm(loose-envify) < 0:2*20:10:35*  npm(loose-envify) >= 0:1.1.0
>>>
>>>
>>>
>>> nodejs-object-assign-2.0.0 exists in EPEL leading me to believe we will
>>> need to add both of these as top level packages carried in
>>> foreman-packaging. Can anyone confirm or deny that is how this should be
>>> working? Or should another package we build be providing these?
>>
>>
>> I do believe we need to package this ourselves since we need a newer
>> version than in EPEL.
>>
>>> [1] https://github.com/theforeman/foreman/pull/4924
>>> [2] http://ci.theforeman.org/job/packaging_repoclosure/37110/console
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to foreman-dev+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



-- 
Later,
  Lukas @lzap Zapletal

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


[foreman-dev] foreman_tasks V1 endpoint works but V2 returns 404

2017-10-17 Thread rajesh . erasani
Hi,

If I 

*#* curl --user admin:changeme -H "Content-Type:application/json" -k 
https://katelloserver/foreman_tasks/api/tasks/3574500b-0394-4a94-9f86-8ff1890ceadb
 

I get the expected response back. But if I send the request to V2 of the 
api as follows

*#* curl --user sledge:hammer -H "Content-Type:application/json" -k 
https://ind2q00katello01.qa.local/foreman_tasks/api/*V2*
/tasks/3574500b-0394-4a94-9f86-8ff1890ceadb
*OR*
*#* curl --user sledge:hammer -H "Content-Type:application/json" -H 
"Accept:application/json,*version=2*" -k 
https://ind2q00katello01.qa.local/foreman_tasks/api/tasks/3574500b-0394-4a94-9f86-8ff1890ceadb


I basically get a 404 resource not found error.


Installed foreman_tasks core package is on 0.1.4 and foreman_tasks package 
is on 0.9.4.

Any suggestions, what I am missing here? Thanks in advance.

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.