[foreman-dev] Re: [POC] Automatic inspection of user-created provisioning templates

[sshtein]

First attempt to create a design. It's an open discussion, everyone who 
wants to chime in, please do.

The engine: will be deployed as a separate gem. My name suggestion 
the-detective  
(Sinatra 
plays a cop).

It will wrap the invocation of rubocop with defaults and parameters needed 
to support our use case:
1. Support for erb
2. Support for completely customized set of cops.
3. Parametrized list of folders containing cops to be added to the list.

In addition it will add tooling to expose a rack endpoint for rubocop 
invocation:
1. List of all available cops (kind of metadata)
2. A POST method that receives a source file, list of cops, and output 
format that will return the result of rubocop's analysis.
3. Will be mountable to any Rails application
4. Will have an option to run as a standalone process (probably using 
passenger with sort-lived process retention settings, since its one process 
per request nature)

Usage for foreman needs:

Use case 1 (community templates CI): 
1. Reference the detective gem from templates plugin.
2. Deploy foreman-core with templates plugin enabled.
3. Add rake task that will invoke rubocop on specified folder using 
detective's invocation wrapper.

Use case 2 (Validate single template from templates UI)
1. Reference detective gem from templates plugin.
2. Add cops declaration ability to plugins in foreman core
3. Templates plugin is responsible for adding/maintaining detective's 
endpoint.
4. Foreman core exposes an option to add actions to template editing screen.
5. Templates plugin uses extension point from 4 to add its own action that 
will invoke detective's endpoint and modify template editor to show the 
result as linting (it's possible with ace and monaco).

Use case 3 (upgrade scenario):
As a first step, we can try and report broken templates after the upgrade.
It will be pretty similar to community templates CI use case, only the 
templates code will be exported from user's database.


I want to start working on the engine gem as soon as possible, so I would 
really appreciate any inputs on the process before I have started with this 
implementation.

Shim.



On Wednesday, August 30, 2017 at 11:48:09 AM UTC+3, ssh...@redhat.com wrote:
>
>
> After a great talk on community demo 
> , here is a follow up with 
> the points that were raised during the discussion:
>
> Use cases:
>
>1. Run all cops as part of community templates CI against the whole 
>repository
>2. Run all cops against a single template invoked by the user from 
>template editing screen (foreman core)
>3. Upgrade scenario: Preferably run cops for the next foreman version 
>before the actual upgrade to make sure the templates will remain valid.
>
>
> Features:
>
>1. List of rues should be pluggable [Shim]: It looks like it is a 
>must-have for the engine.
>2. Deployment options
>1. Engine as a separate gem, cops in a relevant repository - core cops 
>   in core, plugin cops in plugins.
>   2. Engine with all cops in a single gem, versioned per foreman 
>   version.
>   3. Engine as part of templates plugin, cops as part of relevant 
>   plugins.
>   4. Separate gems for everything: foreman-cops-engine, 
>   foreman-cops-core, foreman-cops-plugin1, foreman-cops-plugin2 e.t.c. 
> Engine 
>   is versioned per foreman release version (for the sake of rubocop 
> version), 
>   cops are versioned per plugin version.
>
> General comments:
>
>1. Cops writing should be enforced on PR's that are changing the way 
>to write templates [mhulan]
>2. Cops are dependent on core/plugin version [gwmngilfen]
>
>
>
>
> On Monday, August 14, 2017 at 2:29:02 PM UTC+3, ssh...@redhat.com wrote:
>>
>> TL;DR: I have developed a way to scan any template and see if there are 
>> suspicious/incorrect code patterns in them, so the templates will remain 
>> valid even after foreman code changes.
>>
>> Recently I have started to think about user created templates and foreman 
>> upgrades.
>>
>> When user upgrades foreman, hist default templates get upgraded by the 
>> installer/migrations, but templates created by the user (both cloned and 
>> from scratch) are not touched.
>> This could lead to invalid templates and broken provisioning 
>> functionality for the user.
>> Good example for this would be the change 
>> 
>>  
>> from calling to <%= foreman_url %> to <%= foreman_url('built') %> 
>>
>> I was looking for a way to inspect any template, in order to identify 
>> problematic code as soon as the system is upgraded.
>>
>> I came down to a solution based on rubocop - it's already analyzing 
>> source files for patterns.
>> I have created a POC that analyzes a template written to a file, and 
>> presents the resulting errors as regular rubocop 

[foreman-dev] Rails 5.0 used by default now

[Michael Moll]

Hi,

today PR #4709 was merged to Foreman core, which sets Rails 5.0.6 as
default when using Ruby >= 2.3.

All plugin authors should now start to make their plugins compatible
with Rails 5.0, as Foreman 1.17 will very likely not support Rails 4.2
anymore.

RPM and DEB builds are not affected at this moment, as RPMs still use
Ruby 2.2 and DEBs are pinned to Rails 4.2 for compatibility reasons
with currently released plugin versions.

Regards
-- 
Michael Moll

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[foreman-dev] Re: Katello CI red due to recent foreman-tasks release

[Ivan Necas]

One observation: it seems you need to rebase the foreman PRs in order
the change to be effective
in the tests.

-- Ivan

On Wed, Sep 13, 2017 at 3:35 PM, Ivan Necas  wrote:
> We are back to green again. Keep calm and carry on merging on green tests 
> again
>
> -- Ivan
>
> On Tue, Sep 12, 2017 at 9:45 PM, Ivan Necas  wrote:
>> Hi,
>>
>> We've pushed new version of foreman-tasks, that included the last
>> piece of transition of Dynfow into the core. Unfortunately (or rather
>> fortunately
>> - better in CI than in production:) the Katello tests discovered one
>> issue in the
>> way the transition of the Dynflow initialization was made.
>>
>> It caused the Katello tests to fail on
>>
>> ActiveRecord::StatementInvalid: PG::ObjectInUse: ERROR:  database
>> "test_develop_pr_katello-2-test" is being accessed by other users
>>
>> For more details, see http://projects.theforeman.org/issues/20908
>>
>> The proposed fix to the foreman repo is here:
>> https://github.com/theforeman/foreman/pull/4826/files
>>
>> Reviews more than welcome
>>
>> -- Ivan

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[foreman-dev] Re: Katello CI red due to recent foreman-tasks release

[Ivan Necas]

We are back to green again. Keep calm and carry on merging on green tests again

-- Ivan

On Tue, Sep 12, 2017 at 9:45 PM, Ivan Necas  wrote:
> Hi,
>
> We've pushed new version of foreman-tasks, that included the last
> piece of transition of Dynfow into the core. Unfortunately (or rather
> fortunately
> - better in CI than in production:) the Katello tests discovered one
> issue in the
> way the transition of the Dynflow initialization was made.
>
> It caused the Katello tests to fail on
>
> ActiveRecord::StatementInvalid: PG::ObjectInUse: ERROR:  database
> "test_develop_pr_katello-2-test" is being accessed by other users
>
> For more details, see http://projects.theforeman.org/issues/20908
>
> The proposed fix to the foreman repo is here:
> https://github.com/theforeman/foreman/pull/4826/files
>
> Reviews more than welcome
>
> -- Ivan

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] From Foreman to Iot Platforme

[Fairouz el ouazi]

With this suggestion do i need to make any changes on my platform . Because 
it's only reached by REST api / 

Le mercredi 13 septembre 2017 12:19:47 UTC+2, Ewoud Kohl van Wijngaarden a 
écrit :
>
> In that case I would start with foreman_hooks and perform the POST 
> request. You can even do this in bash with curl if you want. 
>
> https://github.com/theforeman/foreman_hooks#usage describes how you can 
> create your own hooks. 
>
> On Wed, Sep 13, 2017 at 02:39:58AM -0700, Fairouz el ouazi wrote: 
> >I m a beginner in ruby so that 's why i  felt a little bit a shame to 
>  put 
> >it in public im trying to make at first the plugin functional . For my 
> >case  i know that i want to make a post request to my platform by in the 
> >foreman side how can i do it .. 
> > 
> >Le mercredi 13 septembre 2017 11:31:33 UTC+2, Ewoud Kohl van Wijngaarden 
> a 
> >écrit : 
> >> 
> >> On Wed, Sep 13, 2017 at 02:25:01AM -0700, Fairouz el ouazi wrote: 
> >> > I m trying to develop a plugin in Foreman so i can manage  all 
> the 
> >> >devices that are connected to my IoT  platform . I can communicate 
> with 
> >> my 
> >> >IoT platform via REST API . In first place with GET request i can get 
> all 
> >> >the devices on Foreman with their parameters . My problem now is when 
> i 
> >> >change the parameters in foreman or deleting devices  I need to see 
> the 
> >> >changes on my IoT platform . i really need  help because  i m trying 
> to 
> >> >convince my colleague that Foreman can do  it . 
> >> 
> >> Is the plugin open source and available somewhere? That generally makes 
> >> it much easier to help. 
> >> 
> >> For inspiration it might be good to look at foreman_hooks. That has 
> >> triggers when an object is changed or deleted. You might even find that 
> >> just using foreman_hooks could be enough for your use case. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] From Foreman to Iot Platforme

[Ewoud Kohl van Wijngaarden]

In that case I would start with foreman_hooks and perform the POST 
request. You can even do this in bash with curl if you want.


https://github.com/theforeman/foreman_hooks#usage describes how you can 
create your own hooks.


On Wed, Sep 13, 2017 at 02:39:58AM -0700, Fairouz el ouazi wrote:

I m a beginner in ruby so that 's why i  felt a little bit a shame to  put
it in public im trying to make at first the plugin functional . For my
case  i know that i want to make a post request to my platform by in the
foreman side how can i do it ..

Le mercredi 13 septembre 2017 11:31:33 UTC+2, Ewoud Kohl van Wijngaarden a
écrit :


On Wed, Sep 13, 2017 at 02:25:01AM -0700, Fairouz el ouazi wrote:
> I m trying to develop a plugin in Foreman so i can manage  all the
>devices that are connected to my IoT  platform . I can communicate with
my
>IoT platform via REST API . In first place with GET request i can get all
>the devices on Foreman with their parameters . My problem now is when i
>change the parameters in foreman or deleting devices  I need to see the
>changes on my IoT platform . i really need  help because  i m trying to
>convince my colleague that Foreman can do  it .

Is the plugin open source and available somewhere? That generally makes
it much easier to help.

For inspiration it might be good to look at foreman_hooks. That has
triggers when an object is changed or deleted. You might even find that
just using foreman_hooks could be enough for your use case.


--
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] From Foreman to Iot Platforme

[Fairouz el ouazi]

I m a beginner in ruby so that 's why i  felt a little bit a shame to  put 
it in public im trying to make at first the plugin functional . For my 
case  i know that i want to make a post request to my platform by in the 
foreman side how can i do it ..

Le mercredi 13 septembre 2017 11:31:33 UTC+2, Ewoud Kohl van Wijngaarden a 
écrit :
>
> On Wed, Sep 13, 2017 at 02:25:01AM -0700, Fairouz el ouazi wrote: 
> > I m trying to develop a plugin in Foreman so i can manage  all the 
> >devices that are connected to my IoT  platform . I can communicate with 
> my 
> >IoT platform via REST API . In first place with GET request i can get all 
> >the devices on Foreman with their parameters . My problem now is when i 
> >change the parameters in foreman or deleting devices  I need to see the 
> >changes on my IoT platform . i really need  help because  i m trying to 
> >convince my colleague that Foreman can do  it . 
>
> Is the plugin open source and available somewhere? That generally makes 
> it much easier to help. 
>
> For inspiration it might be good to look at foreman_hooks. That has 
> triggers when an object is changed or deleted. You might even find that 
> just using foreman_hooks could be enough for your use case. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [foreman-dev] From Foreman to Iot Platforme

[Ewoud Kohl van Wijngaarden]

On Wed, Sep 13, 2017 at 02:25:01AM -0700, Fairouz el ouazi wrote:

I m trying to develop a plugin in Foreman so i can manage  all the
devices that are connected to my IoT  platform . I can communicate with my
IoT platform via REST API . In first place with GET request i can get all
the devices on Foreman with their parameters . My problem now is when i
change the parameters in foreman or deleting devices  I need to see the
changes on my IoT platform . i really need  help because  i m trying to
convince my colleague that Foreman can do  it .


Is the plugin open source and available somewhere? That generally makes 
it much easier to help.


For inspiration it might be good to look at foreman_hooks. That has 
triggers when an object is changed or deleted. You might even find that 
just using foreman_hooks could be enough for your use case.


--
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[foreman-dev] From Foreman to Iot Platforme

[Fairouz el ouazi]

Hi everyone , 

 I m trying to develop a plugin in Foreman so i can manage  all the 
devices that are connected to my IoT  platform . I can communicate with my 
IoT platform via REST API . In first place with GET request i can get all 
the devices on Foreman with their parameters . My problem now is when i 
change the parameters in foreman or deleting devices  I need to see the 
changes on my IoT platform . i really need  help because  i m trying to 
convince my colleague that Foreman can do  it .
 THANKS 

-- 
You received this message because you are subscribed to the Google Groups 
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to foreman-dev+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.