[foreman-dev] 1.15.4 - 1.16 RC.1 - 1.17 status

[Daniel Lobato Garcia]

Hi all,

Just a heads up on the status of all of the upcoming releases:

  - 1.15.4
- After doing all the cherry-picks, 1.15.stable went red in Jenkins
  https://github.com/theforeman/foreman/pull/4827 fixes it, after
  that is merged we should be able to start the usual build
  tarballs/sign/release dance to release.

  - 1.16.RC1
- Tags and packages have been added to Koji, branching was done.
  Many dependencies need to be built in the foreman-1.16-rhel7 tag.
  As soon as the dependencies finish building, we can start building
  the first RC and figuring out what are the blockers and must-haves
  for this release.
  http://koji.katello.org/koji/tags

  - 1.17.0
- This should be the Rails 5 release. In addition to that, we need
  to find a way to skip parameters parsing, for ActiveJob and
  Katello to be more performant. As soon as this is merged we can
  move testing to be on Rails 5.
  https://github.com/Katello/katello/pull/6875
  If the CentOS team does not migrate rh-ror50 (software collection
  for Rails 5) to http://softwarecollections.org/, it sounds like we
  will have to do our own SCLo again.

Happy to help, give more details about any of these, or accept any
collaborators that want to nanny any of these to completion :)


Best,

--
Daniel Lobato Garcia

@dLobatog
blog.daniellobato.me
daniellobato.me

GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30
Keybase: https://keybase.io/elobato

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


signature.asc
Description: PGP signature

Re: [foreman-dev] Bundler is unable to solve deps for fresh checkout

[Ivan Necas]

The easiest way to get the recent Gemfile.lock is to look at the build
artifacts in jenkins jobs
(also useful when investigating sudden test failures):

http://ci.theforeman.org/job/test_develop/database=postgresql,ruby=2.4,slave=fast/

-- Ivan

On Thu, Sep 14, 2017 at 3:34 PM, Ivan Necas  wrote:
> What errors are you seeing. I've tried with ruby 2.2.2, and 2.3.1
> bundler 1.15.4, everything worked fine.
> What gem --version do you have?
>
> Anything in `bundler.d/Gemfile.local.rb` that could be causing this?
>
> -- Ivan
>
> On Thu, Sep 14, 2017 at 3:13 PM, Lukas Zapletal  wrote:
>> Hey,
>>
>> when I do fresh foreman clone with clean Ruby version (tried 2.0.0 and
>> 2.4.1), bundle install is not able to resolve dependencies and loops
>> forever. I am using latest stable bundler, tried also pre1 version.
>>
>> Can someone confirm and provide a workaround? I think copying
>> Gemfile.lock from someone else should do it. Can someone attach a
>> pastebin me one? I currently have my workstation down (CPU in RMA) and
>> got clean setup.
>>
>> If this is confirmed, it is quite an issue for newcomers. The second
>> command new developer is asked to execute fails hard. Any suggestions?
>>
>> LZ
>>
>> --
>> Later,
>>   Lukas @lzap Zapletal
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to foreman-dev+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Bundler is unable to solve deps for fresh checkout

[Ivan Necas]

What errors are you seeing. I've tried with ruby 2.2.2, and 2.3.1
bundler 1.15.4, everything worked fine.
What gem --version do you have?

Anything in `bundler.d/Gemfile.local.rb` that could be causing this?

-- Ivan

On Thu, Sep 14, 2017 at 3:13 PM, Lukas Zapletal  wrote:
> Hey,
>
> when I do fresh foreman clone with clean Ruby version (tried 2.0.0 and
> 2.4.1), bundle install is not able to resolve dependencies and loops
> forever. I am using latest stable bundler, tried also pre1 version.
>
> Can someone confirm and provide a workaround? I think copying
> Gemfile.lock from someone else should do it. Can someone attach a
> pastebin me one? I currently have my workstation down (CPU in RMA) and
> got clean setup.
>
> If this is confirmed, it is quite an issue for newcomers. The second
> command new developer is asked to execute fails hard. Any suggestions?
>
> LZ
>
> --
> Later,
>   Lukas @lzap Zapletal
>
> --
> You received this message because you are subscribed to the Google Groups 
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to foreman-dev+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Re: [POC] Automatic inspection of user-created provisioning templates

[Shimon Shtein]

Ewoud:

About rack endpoint: It should be created for use case 2, where the check
is running in "online"-ish mode.
I prefer creating it as a microservice for scalability reasons, so I
wouldn't want to tie it too tightly with foreman-templates. Besides that,
tying it into foreman-templates will mean the same life cycle that I would
prefer to avoid.
As I already wrote for case 1 and 3, we could use the underlying ruby
wrapper directly, hence avoid the usage of the API.
As for additional dependencies, I probably would add passenger as an
optional (development) dependency, so you will have the full feature set on
development machines. In production it will check if if passenger is
available only if it runs in standalone mode. It will not ask for passenger
if someone has decided to use something else, like webrick (personally I
would recommend against webrick, it's not parallel at all which is
important here).

About the command line tool, I have nothing against it, it should be
available in the gem.





On Thu, Sep 14, 2017 at 2:06 PM, Ewoud Kohl van Wijngaarden <
ew...@kohlvanwijngaarden.nl> wrote:

> On Wed, Sep 13, 2017 at 12:05:49PM -0700, ssht...@redhat.com wrote:
>
>>
>> First attempt to create a design. It's an open discussion, everyone who
>> wants to chime in, please do.
>>
>> The engine: will be deployed as a separate gem. My name suggestion
>> the-detective 
>> (Sinatra
>> plays a cop).
>>
>> It will wrap the invocation of rubocop with defaults and parameters needed
>> to support our use case:
>> 1. Support for erb
>> 2. Support for completely customized set of cops.
>> 3. Parametrized list of folders containing cops to be added to the list.
>>
>> In addition it will add tooling to expose a rack endpoint for rubocop
>> invocation:
>> 1. List of all available cops (kind of metadata)
>> 2. A POST method that receives a source file, list of cops, and output
>> format that will return the result of rubocop's analysis.
>> 3. Will be mountable to any Rails application
>> 4. Will have an option to run as a standalone process (probably using
>> passenger with sort-lived process retention settings, since its one
>> process
>> per request nature)
>>
>
> Why should it be a rack endpoint? My thinking was much more of a normal
> ruby API with a command line tool around it. There should be no passenger
> dependency to keep its dependencies small. foreman_templates can consume
> the ruby API and expose it to the user as it wants.
>
> Usage for foreman needs:
>>
>> Use case 1 (community templates CI):
>> 1. Reference the detective gem from templates plugin.
>> 2. Deploy foreman-core with templates plugin enabled.
>> 3. Add rake task that will invoke rubocop on specified folder using
>> detective's invocation wrapper.
>>
>
> Ideally we'd have a light command line tool that does:
>
> detective \
>  --cops /path/to/foreman/checkout/cops \
>  --cops /path/to/katello/checkout/cops \
>  --cops /path/to/other/plugin/with/cops \
>  /path/to/some/template/dir \
>  /path/to/another/template/dir
>
> That way we can do a simple git clone foreman in community-templates,
> bundle install and run it within Travis. This can indeed be wrapped in a
> rake task but given the paths can change on a developers workstation it is
> good to have an easy manual option.
>
> Use case 2 (Validate single template from templates UI)
>> 1. Reference detective gem from templates plugin.
>> 2. Add cops declaration ability to plugins in foreman core
>> 3. Templates plugin is responsible for adding/maintaining detective's
>> endpoint.
>> 4. Foreman core exposes an option to add actions to template editing
>> screen.
>> 5. Templates plugin uses extension point from 4 to add its own action that
>> will invoke detective's endpoint and modify template editor to show the
>> result as linting (it's possible with ace and monaco).
>>
>> Use case 3 (upgrade scenario):
>> As a first step, we can try and report broken templates after the upgrade.
>> It will be pretty similar to community templates CI use case, only the
>> templates code will be exported from user's database.
>>
>>
>> I want to start working on the engine gem as soon as possible, so I would
>> really appreciate any inputs on the process before I have started with
>> this
>> implementation.
>>
>> Shim.
>>
>>
>>
>> On Wednesday, August 30, 2017 at 11:48:09 AM UTC+3, ssh...@redhat.com
>> wrote:
>>
>>>
>>>
>>> After a great talk on community demo
>>> , here is a follow up with
>>> the points that were raised during the discussion:
>>>
>>> Use cases:
>>>
>>>1. Run all cops as part of community templates CI against the whole
>>>repository
>>>2. Run all cops against a single template invoked by the user from
>>>template editing screen (foreman core)
>>>3. Upgrade scenario: Preferably run cops for the next foreman version
>>>before the actual upgrade to make sure the templates w

[foreman-dev] Bundler is unable to solve deps for fresh checkout

[Lukas Zapletal]

Hey,

when I do fresh foreman clone with clean Ruby version (tried 2.0.0 and
2.4.1), bundle install is not able to resolve dependencies and loops
forever. I am using latest stable bundler, tried also pre1 version.

Can someone confirm and provide a workaround? I think copying
Gemfile.lock from someone else should do it. Can someone attach a
pastebin me one? I currently have my workstation down (CPU in RMA) and
got clean setup.

If this is confirmed, it is quite an issue for newcomers. The second
command new developer is asked to execute fails hard. Any suggestions?

LZ

-- 
Later,
  Lukas @lzap Zapletal

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] Re: [POC] Automatic inspection of user-created provisioning templates

[Ewoud Kohl van Wijngaarden]

On Wed, Sep 13, 2017 at 12:05:49PM -0700, ssht...@redhat.com wrote:


First attempt to create a design. It's an open discussion, everyone who
wants to chime in, please do.

The engine: will be deployed as a separate gem. My name suggestion
the-detective  (Sinatra
plays a cop).

It will wrap the invocation of rubocop with defaults and parameters needed
to support our use case:
1. Support for erb
2. Support for completely customized set of cops.
3. Parametrized list of folders containing cops to be added to the list.

In addition it will add tooling to expose a rack endpoint for rubocop
invocation:
1. List of all available cops (kind of metadata)
2. A POST method that receives a source file, list of cops, and output
format that will return the result of rubocop's analysis.
3. Will be mountable to any Rails application
4. Will have an option to run as a standalone process (probably using
passenger with sort-lived process retention settings, since its one process
per request nature)


Why should it be a rack endpoint? My thinking was much more of a normal 
ruby API with a command line tool around it. There should be no 
passenger dependency to keep its dependencies small. foreman_templates 
can consume the ruby API and expose it to the user as it wants.



Usage for foreman needs:

Use case 1 (community templates CI):
1. Reference the detective gem from templates plugin.
2. Deploy foreman-core with templates plugin enabled.
3. Add rake task that will invoke rubocop on specified folder using
detective's invocation wrapper.


Ideally we'd have a light command line tool that does:

detective \
 --cops /path/to/foreman/checkout/cops \
 --cops /path/to/katello/checkout/cops \
 --cops /path/to/other/plugin/with/cops \
 /path/to/some/template/dir \
 /path/to/another/template/dir

That way we can do a simple git clone foreman in community-templates, 
bundle install and run it within Travis. This can indeed be wrapped in a 
rake task but given the paths can change on a developers workstation it 
is good to have an easy manual option.



Use case 2 (Validate single template from templates UI)
1. Reference detective gem from templates plugin.
2. Add cops declaration ability to plugins in foreman core
3. Templates plugin is responsible for adding/maintaining detective's
endpoint.
4. Foreman core exposes an option to add actions to template editing screen.
5. Templates plugin uses extension point from 4 to add its own action that
will invoke detective's endpoint and modify template editor to show the
result as linting (it's possible with ace and monaco).

Use case 3 (upgrade scenario):
As a first step, we can try and report broken templates after the upgrade.
It will be pretty similar to community templates CI use case, only the
templates code will be exported from user's database.


I want to start working on the engine gem as soon as possible, so I would
really appreciate any inputs on the process before I have started with this
implementation.

Shim.



On Wednesday, August 30, 2017 at 11:48:09 AM UTC+3, ssh...@redhat.com wrote:



After a great talk on community demo
, here is a follow up with
the points that were raised during the discussion:

Use cases:

   1. Run all cops as part of community templates CI against the whole
   repository
   2. Run all cops against a single template invoked by the user from
   template editing screen (foreman core)
   3. Upgrade scenario: Preferably run cops for the next foreman version
   before the actual upgrade to make sure the templates will remain valid.


Features:

   1. List of rues should be pluggable [Shim]: It looks like it is a
   must-have for the engine.
   2. Deployment options
   1. Engine as a separate gem, cops in a relevant repository - core cops
  in core, plugin cops in plugins.
  2. Engine with all cops in a single gem, versioned per foreman
  version.
  3. Engine as part of templates plugin, cops as part of relevant
  plugins.
  4. Separate gems for everything: foreman-cops-engine,
  foreman-cops-core, foreman-cops-plugin1, foreman-cops-plugin2 e.t.c. 
Engine
  is versioned per foreman release version (for the sake of rubocop 
version),
  cops are versioned per plugin version.

General comments:

   1. Cops writing should be enforced on PR's that are changing the way
   to write templates [mhulan]
   2. Cops are dependent on core/plugin version [gwmngilfen]




On Monday, August 14, 2017 at 2:29:02 PM UTC+3, ssh...@redhat.com wrote:


TL;DR: I have developed a way to scan any template and see if there are
suspicious/incorrect code patterns in them, so the templates will remain
valid even after foreman code changes.

Recently I have started to think about user created templates and foreman
upgrades.

When user upgrades foreman, hist default templates get upgraded by the
installer/migrations, but templates created by the user 

Re: [foreman-dev] Re: [POC] Automatic inspection of user-created provisioning templates

[Ivan Necas]

Yes, I'm have mainly the DOS (I guess it might not even needed
distributed part here) in mind.

-- Ivan

On Thu, Sep 14, 2017 at 12:18 PM, Shimon Shtein  wrote:
>
> First, I don't think that this service has any secret information that
> should be kept under authorization/authentication.
>
> Even if we assume that some level of auth is indeed needed, the easiest way
> to implement it would be through Rack middleware.
> If the service is added to an application with existing auth middleware,
> then you can add it to the relevant endpoints when mounting it.
> In case we are trying to use the standalone version, we can either create an
> extension point to add middleware, or mount it manually, just like in the
> previous case.
>
> Anyway I don't see a reason to limit access to those endpoints, except only
> for cases of DDOS (which should be handled even before the Rack stack
> anyway).
>
> On Thu, Sep 14, 2017 at 10:44 AM, Ivan Necas  wrote:
>>
>> How would be the authentication/authorization work with this approach?
>>
>> - - Ivan
>>
>> On Wed, Sep 13, 2017 at 9:05 PM,   wrote:
>> >
>> > First attempt to create a design. It's an open discussion, everyone who
>> > wants to chime in, please do.
>> >
>> > The engine: will be deployed as a separate gem. My name suggestion
>> > the-detective (Sinatra plays a cop).
>> >
>> > It will wrap the invocation of rubocop with defaults and parameters
>> > needed
>> > to support our use case:
>> > 1. Support for erb
>> > 2. Support for completely customized set of cops.
>> > 3. Parametrized list of folders containing cops to be added to the list.
>> >
>> > In addition it will add tooling to expose a rack endpoint for rubocop
>> > invocation:
>> > 1. List of all available cops (kind of metadata)
>> > 2. A POST method that receives a source file, list of cops, and output
>> > format that will return the result of rubocop's analysis.
>> > 3. Will be mountable to any Rails application
>> > 4. Will have an option to run as a standalone process (probably using
>> > passenger with sort-lived process retention settings, since its one
>> > process
>> > per request nature)
>> >
>> > Usage for foreman needs:
>> >
>> > Use case 1 (community templates CI):
>> > 1. Reference the detective gem from templates plugin.
>> > 2. Deploy foreman-core with templates plugin enabled.
>> > 3. Add rake task that will invoke rubocop on specified folder using
>> > detective's invocation wrapper.
>> >
>> > Use case 2 (Validate single template from templates UI)
>> > 1. Reference detective gem from templates plugin.
>> > 2. Add cops declaration ability to plugins in foreman core
>> > 3. Templates plugin is responsible for adding/maintaining detective's
>> > endpoint.
>> > 4. Foreman core exposes an option to add actions to template editing
>> > screen.
>> > 5. Templates plugin uses extension point from 4 to add its own action
>> > that
>> > will invoke detective's endpoint and modify template editor to show the
>> > result as linting (it's possible with ace and monaco).
>> >
>> > Use case 3 (upgrade scenario):
>> > As a first step, we can try and report broken templates after the
>> > upgrade.
>> > It will be pretty similar to community templates CI use case, only the
>> > templates code will be exported from user's database.
>> >
>> >
>> > I want to start working on the engine gem as soon as possible, so I
>> > would
>> > really appreciate any inputs on the process before I have started with
>> > this
>> > implementation.
>> >
>> > Shim.
>> >
>> >
>> >
>> > On Wednesday, August 30, 2017 at 11:48:09 AM UTC+3, ssh...@redhat.com
>> > wrote:
>> >>
>> >>
>> >> After a great talk on community demo, here is a follow up with the
>> >> points
>> >> that were raised during the discussion:
>> >>
>> >> Use cases:
>> >>
>> >> Run all cops as part of community templates CI against the whole
>> >> repository
>> >> Run all cops against a single template invoked by the user from
>> >> template
>> >> editing screen (foreman core)
>> >> Upgrade scenario: Preferably run cops for the next foreman version
>> >> before
>> >> the actual upgrade to make sure the templates will remain valid.
>> >>
>> >>
>> >> Features:
>> >>
>> >> List of rues should be pluggable [Shim]: It looks like it is a
>> >> must-have
>> >> for the engine.
>> >> Deployment options
>> >>
>> >> Engine as a separate gem, cops in a relevant repository - core cops in
>> >> core, plugin cops in plugins.
>> >> Engine with all cops in a single gem, versioned per foreman version.
>> >> Engine as part of templates plugin, cops as part of relevant plugins.
>> >> Separate gems for everything: foreman-cops-engine, foreman-cops-core,
>> >> foreman-cops-plugin1, foreman-cops-plugin2 e.t.c. Engine is versioned
>> >> per
>> >> foreman release version (for the sake of rubocop version), cops are
>> >> versioned per plugin version.
>> >>
>> >> General comments:
>> >>
>> >> Cops writing should be enforced on PR's that are changing the way to
>> >> write
>> >> templates [mhul

Re: [foreman-dev] Re: [POC] Automatic inspection of user-created provisioning templates

[Shimon Shtein]

First, I don't think that this service has any secret information that
should be kept under authorization/authentication.

Even if we assume that some level of auth is indeed needed, the easiest way
to implement it would be through Rack middleware.
If the service is added to an application with existing auth middleware,
then you can add it to the relevant endpoints when mounting it.
In case we are trying to use the standalone version, we can either create
an extension point to add middleware, or mount it manually, just like in
the previous case.

Anyway I don't see a reason to limit access to those endpoints, except only
for cases of DDOS (which should be handled even before the Rack stack
anyway).

On Thu, Sep 14, 2017 at 10:44 AM, Ivan Necas  wrote:

> How would be the authentication/authorization work with this approach?
>
> - - Ivan
>
> On Wed, Sep 13, 2017 at 9:05 PM,   wrote:
> >
> > First attempt to create a design. It's an open discussion, everyone who
> > wants to chime in, please do.
> >
> > The engine: will be deployed as a separate gem. My name suggestion
> > the-detective (Sinatra plays a cop).
> >
> > It will wrap the invocation of rubocop with defaults and parameters
> needed
> > to support our use case:
> > 1. Support for erb
> > 2. Support for completely customized set of cops.
> > 3. Parametrized list of folders containing cops to be added to the list.
> >
> > In addition it will add tooling to expose a rack endpoint for rubocop
> > invocation:
> > 1. List of all available cops (kind of metadata)
> > 2. A POST method that receives a source file, list of cops, and output
> > format that will return the result of rubocop's analysis.
> > 3. Will be mountable to any Rails application
> > 4. Will have an option to run as a standalone process (probably using
> > passenger with sort-lived process retention settings, since its one
> process
> > per request nature)
> >
> > Usage for foreman needs:
> >
> > Use case 1 (community templates CI):
> > 1. Reference the detective gem from templates plugin.
> > 2. Deploy foreman-core with templates plugin enabled.
> > 3. Add rake task that will invoke rubocop on specified folder using
> > detective's invocation wrapper.
> >
> > Use case 2 (Validate single template from templates UI)
> > 1. Reference detective gem from templates plugin.
> > 2. Add cops declaration ability to plugins in foreman core
> > 3. Templates plugin is responsible for adding/maintaining detective's
> > endpoint.
> > 4. Foreman core exposes an option to add actions to template editing
> screen.
> > 5. Templates plugin uses extension point from 4 to add its own action
> that
> > will invoke detective's endpoint and modify template editor to show the
> > result as linting (it's possible with ace and monaco).
> >
> > Use case 3 (upgrade scenario):
> > As a first step, we can try and report broken templates after the
> upgrade.
> > It will be pretty similar to community templates CI use case, only the
> > templates code will be exported from user's database.
> >
> >
> > I want to start working on the engine gem as soon as possible, so I would
> > really appreciate any inputs on the process before I have started with
> this
> > implementation.
> >
> > Shim.
> >
> >
> >
> > On Wednesday, August 30, 2017 at 11:48:09 AM UTC+3, ssh...@redhat.com
> wrote:
> >>
> >>
> >> After a great talk on community demo, here is a follow up with the
> points
> >> that were raised during the discussion:
> >>
> >> Use cases:
> >>
> >> Run all cops as part of community templates CI against the whole
> >> repository
> >> Run all cops against a single template invoked by the user from template
> >> editing screen (foreman core)
> >> Upgrade scenario: Preferably run cops for the next foreman version
> before
> >> the actual upgrade to make sure the templates will remain valid.
> >>
> >>
> >> Features:
> >>
> >> List of rues should be pluggable [Shim]: It looks like it is a must-have
> >> for the engine.
> >> Deployment options
> >>
> >> Engine as a separate gem, cops in a relevant repository - core cops in
> >> core, plugin cops in plugins.
> >> Engine with all cops in a single gem, versioned per foreman version.
> >> Engine as part of templates plugin, cops as part of relevant plugins.
> >> Separate gems for everything: foreman-cops-engine, foreman-cops-core,
> >> foreman-cops-plugin1, foreman-cops-plugin2 e.t.c. Engine is versioned
> per
> >> foreman release version (for the sake of rubocop version), cops are
> >> versioned per plugin version.
> >>
> >> General comments:
> >>
> >> Cops writing should be enforced on PR's that are changing the way to
> write
> >> templates [mhulan]
> >> Cops are dependent on core/plugin version [gwmngilfen]
> >>
> >>
> >>
> >>
> >> On Monday, August 14, 2017 at 2:29:02 PM UTC+3, ssh...@redhat.com
> wrote:
> >>>
> >>> TL;DR: I have developed a way to scan any template and see if there are
> >>> suspicious/incorrect code patterns in them, so the templates will
> remain
> >>> valid ev

Re: [foreman-dev] Re: [POC] Automatic inspection of user-created provisioning templates

[Ivan Necas]

How would be the authentication/authorization work with this approach?

- - Ivan

On Wed, Sep 13, 2017 at 9:05 PM,   wrote:
>
> First attempt to create a design. It's an open discussion, everyone who
> wants to chime in, please do.
>
> The engine: will be deployed as a separate gem. My name suggestion
> the-detective (Sinatra plays a cop).
>
> It will wrap the invocation of rubocop with defaults and parameters needed
> to support our use case:
> 1. Support for erb
> 2. Support for completely customized set of cops.
> 3. Parametrized list of folders containing cops to be added to the list.
>
> In addition it will add tooling to expose a rack endpoint for rubocop
> invocation:
> 1. List of all available cops (kind of metadata)
> 2. A POST method that receives a source file, list of cops, and output
> format that will return the result of rubocop's analysis.
> 3. Will be mountable to any Rails application
> 4. Will have an option to run as a standalone process (probably using
> passenger with sort-lived process retention settings, since its one process
> per request nature)
>
> Usage for foreman needs:
>
> Use case 1 (community templates CI):
> 1. Reference the detective gem from templates plugin.
> 2. Deploy foreman-core with templates plugin enabled.
> 3. Add rake task that will invoke rubocop on specified folder using
> detective's invocation wrapper.
>
> Use case 2 (Validate single template from templates UI)
> 1. Reference detective gem from templates plugin.
> 2. Add cops declaration ability to plugins in foreman core
> 3. Templates plugin is responsible for adding/maintaining detective's
> endpoint.
> 4. Foreman core exposes an option to add actions to template editing screen.
> 5. Templates plugin uses extension point from 4 to add its own action that
> will invoke detective's endpoint and modify template editor to show the
> result as linting (it's possible with ace and monaco).
>
> Use case 3 (upgrade scenario):
> As a first step, we can try and report broken templates after the upgrade.
> It will be pretty similar to community templates CI use case, only the
> templates code will be exported from user's database.
>
>
> I want to start working on the engine gem as soon as possible, so I would
> really appreciate any inputs on the process before I have started with this
> implementation.
>
> Shim.
>
>
>
> On Wednesday, August 30, 2017 at 11:48:09 AM UTC+3, ssh...@redhat.com wrote:
>>
>>
>> After a great talk on community demo, here is a follow up with the points
>> that were raised during the discussion:
>>
>> Use cases:
>>
>> Run all cops as part of community templates CI against the whole
>> repository
>> Run all cops against a single template invoked by the user from template
>> editing screen (foreman core)
>> Upgrade scenario: Preferably run cops for the next foreman version before
>> the actual upgrade to make sure the templates will remain valid.
>>
>>
>> Features:
>>
>> List of rues should be pluggable [Shim]: It looks like it is a must-have
>> for the engine.
>> Deployment options
>>
>> Engine as a separate gem, cops in a relevant repository - core cops in
>> core, plugin cops in plugins.
>> Engine with all cops in a single gem, versioned per foreman version.
>> Engine as part of templates plugin, cops as part of relevant plugins.
>> Separate gems for everything: foreman-cops-engine, foreman-cops-core,
>> foreman-cops-plugin1, foreman-cops-plugin2 e.t.c. Engine is versioned per
>> foreman release version (for the sake of rubocop version), cops are
>> versioned per plugin version.
>>
>> General comments:
>>
>> Cops writing should be enforced on PR's that are changing the way to write
>> templates [mhulan]
>> Cops are dependent on core/plugin version [gwmngilfen]
>>
>>
>>
>>
>> On Monday, August 14, 2017 at 2:29:02 PM UTC+3, ssh...@redhat.com wrote:
>>>
>>> TL;DR: I have developed a way to scan any template and see if there are
>>> suspicious/incorrect code patterns in them, so the templates will remain
>>> valid even after foreman code changes.
>>>
>>> Recently I have started to think about user created templates and foreman
>>> upgrades.
>>>
>>> When user upgrades foreman, hist default templates get upgraded by the
>>> installer/migrations, but templates created by the user (both cloned and
>>> from scratch) are not touched.
>>> This could lead to invalid templates and broken provisioning
>>> functionality for the user.
>>> Good example for this would be the change from calling to <%= foreman_url
>>> %> to <%= foreman_url('built') %>
>>>
>>> I was looking for a way to inspect any template, in order to identify
>>> problematic code as soon as the system is upgraded.
>>>
>>> I came down to a solution based on rubocop - it's already analyzing
>>> source files for patterns.
>>> I have created a POC that analyzes a template written to a file, and
>>> presents the resulting errors as regular rubocop (clang style).
>>> All source codes are available as gist:
>>> https://gist.github.com/ShimShtein/3