[Puppet Users] Re: [Puppet-dev] Converting to Ruby DSL

[Ken Barber]

I've moved the doc and cleaned it up a bit now. Its a very rudimentary 
start:

https://projects.puppetlabs.com/projects/puppet/wiki/Ruby_Dsl

As far as recommended DSL conventions for bits that aren't supported yet, 
here are some suggestions to open up the discussion as I'm sure people have 
their own opinions:

Imports:

import "file.pp"

Resource Defaults:

resource_defaults "foo", :param1 => value1

Resource Collections on exports:

resource_collect "foo", :tag => "tagme"

Resource Realisation on virtuals:

resource_realise "foo", :title => "ken"

Resource override:

resource_override :service, [:title => "apache"], :require => 
"Package[apache2]"

Resource utilisation:

This already exists but I would actually change this to noun_verb to fit the 
rest. The opposite could be true for all accounts of course - it just made 
more sense for 'resource_defaults' instead of 'defaults_resource'. I also 
would not call it 'create' because its not really 'creation' per se:

resource_init :package, "puppet", :ensure => :installed

Variable setting:

var_set "nameofvar", "value"

Variable getting:

var_get "nameofvar"

Variable unsetting:

var_unset "nameofvar"

For relationships I wouldn't use the term 'edge' for semantic reasons. While 
it is an edge internally - it doesn't match the semantics used in the 
documentation. The language tutorial refers to it as 'chaining resources' 
... so I'm not sure what's best. Otherwise the argument list you provided 
looks great. 

What we don't have yet is a way to deal with multiple chains like in Puppet 
DSL. In reality this kind of syntax:

Foo[bar] -> Bar[baz] -> Bob[mary]

Is a kind of Puppet DSL syntactic sugar anyway ... so I'm not sure how 
important it is to replicate.

I guess it might be acceptable to have your edge method accept multiple 
terms:

edge "Foo[bar]" => "Bar[baz]", "Bar[baz]" => "Bob[mary"

Syntax like this:

Foo[bar] -> Bar[baz] <- Bob[mary]

Could be done like this:

edge ["Foo[bar]", "Bob[mary]"] => "Bar[baz]"

Defining collections in chained resources would probably be done with 
similar string based syntax I guess:

edge "Foo <| |>" => "Package <| title='puppet' |>"

ken.

PS. I'm stuck using this terrible new google groups interface. Sorry if my 
post looks terrible.

On Sunday, December 26, 2010 5:44:22 PM UTC, Luke Kanies wrote:
>
> On Dec 19, 2010, at 5:07 AM, Ken Barber wrote:
>
> Thanks Luke.
>
> So I've been making some notes here:
>
> http://wiki.bob.sh/confluence/display/TECH/Ruby+DSL
>
>
> This is a great doc.  Would you be willing to migrate it to the Redmine 
> wiki?
>
> I especially like how you draw out the bits that have pure ruby interfaces 
> and the bits that don't.  Do you have any inspirations for good interfaces 
> for those unsupported bits yet?
>
> While I definitely don't want to get in the habit of directly exposing 
> internal interfaces in a lot of ruby scripts, one of the big benefits of the 
> pure ruby DSL is that people can do exactly this - hack in the support they 
> need until it's available officially.
>
> And writing up some 
> working examples:
>
> https://github.com/bobsh/puppet-rubydsl-examples
>
> ... based on your notes 
> and some digging. You wouldn't be able to take a look and correct me where 
> I've tapped into the wrong part of the API? I realise I'm obviously touching 
> internals that may change ... I just figure if I can figure out how to do it 
> with internals then it shouldn't be hard to extend the DSL.
>
> So obviously some of the ways I've done this should really be moved to 
> convenience methods:
>
> * Defaults: I presume the capitalisation of the first letter of a 
> missing_method would indicate someone trying to set a default
>
>
> This is a bit different in Ruby because capitalized terms are presumed to 
> be constants, and they're actually pretty annoying to deal with 
> automatically.  I'd probably start with that, but I wouldn't hold out hope 
> that it'll work in the long term.
>
> * Vars: not sure how you make this work at the correct scope in ruby for 
> proper 'my_var = "foo"' type sugar ... maybe a wrapper is more appropriate 
> (a bit like create_resource).
>
>
> You should be able to use normal instance variables for all of the variable 
> setting/getting except at the very top scope:
>
> @var = "foo"
>
> This works for dereferencing variables set in Puppet DSL code, too.
>
> * Relationships: I'm not sure how to do this yet, looking at the 
> AST::Relationship bit there is a lot going on so I imagine I tap that, but 
> how I do that without extending the dsl code itself is not obvious (so 
> obviously I'm going to try that :-).
>
>
> I would just add an 'edge' method and pass it a hash; something like:
>
> edge "Package[foo]" => "Service[foo]"
>
> This would then probably directly add edges to the current catalog, rathe

[Puppet Users] Re: [Puppet-dev] Converting to Ruby DSL

[Luke Kanies]

On Dec 19, 2010, at 5:07 AM, Ken Barber wrote:

> Thanks Luke.
> 
> So I've been making some notes here:
> 
> http://wiki.bob.sh/confluence/display/TECH/Ruby+DSL

This is a great doc.  Would you be willing to migrate it to the Redmine wiki?

I especially like how you draw out the bits that have pure ruby interfaces and 
the bits that don't.  Do you have any inspirations for good interfaces for 
those unsupported bits yet?

While I definitely don't want to get in the habit of directly exposing internal 
interfaces in a lot of ruby scripts, one of the big benefits of the pure ruby 
DSL is that people can do exactly this - hack in the support they need until 
it's available officially.

> And writing up some working examples:
> 
> https://github.com/bobsh/puppet-rubydsl-examples
> 
> ... based on your notes and some digging. You wouldn't be able to take a look 
> and correct me where I've tapped into the wrong part of the API? I realise 
> I'm obviously touching internals that may change ... I just figure if I can 
> figure out how to do it with internals then it shouldn't be hard to extend 
> the DSL.
> 
> So obviously some of the ways I've done this should really be moved to 
> convenience methods:
> 
> * Defaults: I presume the capitalisation of the first letter of a 
> missing_method would indicate someone trying to set a default

This is a bit different in Ruby because capitalized terms are presumed to be 
constants, and they're actually pretty annoying to deal with automatically.  
I'd probably start with that, but I wouldn't hold out hope that it'll work in 
the long term.

> * Vars: not sure how you make this work at the correct scope in ruby for 
> proper 'my_var = "foo"' type sugar ... maybe a wrapper is more appropriate (a 
> bit like create_resource).

You should be able to use normal instance variables for all of the variable 
setting/getting except at the very top scope:

@var = "foo"

This works for dereferencing variables set in Puppet DSL code, too.

> * Relationships: I'm not sure how to do this yet, looking at the 
> AST::Relationship bit there is a lot going on so I imagine I tap that, but 
> how I do that without extending the dsl code itself is not obvious (so 
> obviously I'm going to try that :-).

I would just add an 'edge' method and pass it a hash; something like:

edge "Package[foo]" => "Service[foo]"

This would then probably directly add edges to the current catalog, rather than 
relying on AST::Relationship:

def edge(rels)
  rels.each do |source, target|
raise "Could not find resource #{source}" unless source = 
catalog.resource(source)
raise "Could not find resource #{target}" unless target = 
catalog.resource(target)

catalog.add_edge(source, target)
  end
end

Something like that, anyway.

> How would one go about extending it to deal with top scope? I'm guessing 
> somehow you need to be able to populate @__created_ast_objects__ directly 
> right from ResourceTypeAPI?

I hope not...?

Your question here is how to make all of these methods available in the top 
scope, outside of a node/class/definition, right?

I don't really know, and I had largely decided against that as a design 
decision, figuring people didn't want the top scope polluted.  Given how many 
people have asked for this, though, I was clearly wrong about what people 
wanted.

I don't have clarity on how exactly to do it, but it should probably resemble 
how we get those node/hostclass/etc methods in the top scope - put them in a 
module which is then used to extend Object.  The methods themselves would then 
modify the 'main' class (which would need to be autocreated if it were missing).

Does that make sense?

> On Friday, December 10, 2010 6:47:42 AM UTC, Luke Kanies wrote:
> On Dec 8, 2010, at 11:09 AM, Ken Barber wrote:
> 
>> (cross posting here as I originally posted in puppet-users - I figure the 
>> question is more of a dev one then for puppet-users)
>> 
>> Hi everyone,
>> 
>> I'm experimenting with converting some of my live puppet content to
>> Ruby DSL and have found a few gaps I wouldn't mind some advice on. I'm
>> not sure how many of you have already experimented in this arena yet.
>> 
>> I've already read Dan Bode's excellent blog article on the subject:
>> 
>> http://www.puppetlabs.com/blog/ruby-dsl/
>> 
>> I know the Ruby DSL is quite new - but I figured perhaps some of these
>> items deserve to be tickets (or perhaps documented) and I thought I'd
>> ask first :-).
> Thanks for testing this.  As you say, it's still quite new, and definitely 
> incomplete, but as with most things we do, I wanted to get a simple thing out 
> and in the wild rather than wait for perfection before shipping.  This kind 
> of feedback is exactly what we need (although of course patches are even 
> better).
>> 1. Defaults
>> 
>> How do I set defaults in Ruby DSL? For example what is the ruby DSL
>> equivalent to this:
>> 
>>   Service {
>> hasstatus => true
>>   }
> This is one of the things I left 

[Puppet Users] Re: [Puppet-dev] Converting to Ruby DSL

[Ken Barber]

Thanks Luke.

So I've been making some notes here:

http://wiki.bob.sh/confluence/display/TECH/Ruby+DSL

And writing up some 
working examples:

https://github.com/bobsh/puppet-rubydsl-examples

... based on your notes 
and some digging. You wouldn't be able to take a look and correct me where 
I've tapped into the wrong part of the API? I realise I'm obviously touching 
internals that may change ... I just figure if I can figure out how to do it 
with internals then it shouldn't be hard to extend the DSL.

So obviously some of the ways I've done this should really be moved to 
convenience methods:

* Defaults: I presume the capitalisation of the first letter of a 
missing_method would indicate someone trying to set a default
* Vars: not sure how you make this work at the correct scope in ruby for 
proper 'my_var = "foo"' type sugar ... maybe a wrapper is more appropriate 
(a bit like create_resource).
* Relationships: I'm not sure how to do this yet, looking at the 
AST::Relationship bit there is a lot going on so I imagine I tap that, but 
how I do that without extending the dsl code itself is not obvious (so 
obviously I'm going to try that :-).

How would one go about extending it to deal with top scope? I'm guessing 
somehow you need to be able to populate @__created_ast_objects__ directly 
right from ResourceTypeAPI?

ken.

On Friday, December 10, 2010 6:47:42 AM UTC, Luke Kanies wrote:
>
> On Dec 8, 2010, at 11:09 AM, Ken Barber wrote:
>
> (cross posting here as I originally posted in puppet-users - I figure the 
> question is more of a dev one then for puppet-users)
>
>
> Hi everyone,
>
> I'm experimenting with converting some of my live puppet content to
> Ruby DSL and have found a few gaps I wouldn't mind some advice on. I'm
> not sure how many of you have already experimented in this arena yet.
>
> I've already read Dan Bode's excellent blog article on the subject:
> http://www.puppetlabs.com/blog/ruby-dsl/
>
> I know the Ruby DSL is quite new - but I figured perhaps some of these
> items deserve to be tickets (or perhaps documented) and I thought I'd
> ask first :-).
>
> Thanks for testing this.  As you say, it's still quite new, and definitely 
> incomplete, but as with most things we do, I wanted to get a simple thing 
> out and in the wild rather than wait for perfection before shipping.  This 
> kind of feedback is exactly what we need (although of course patches are 
> even better).
>
> 1. Defaults
>
> How do I set defaults in Ruby DSL? For example what is the ruby DSL
> equivalent to this:
>
>   Service {
> hasstatus => true
>   }
>
> This is one of the things I left out of my initial implementation.
>
> You might be able to figure out how to add it by patching one of the files 
> in lib/puppet/dsl.
>
> 2. Top scope
>
> I notice the convenience methodology does not work at 'top scope' for
> example
> this will fail:
>
>   file "/tmp/zzz", :content => "foo"
>
> But this will not:
>
>   node "default" do
> file "/tmp/zzz", :content => "foo"
>   end
>
> Is there a proper way to call resources without using the convenience
> methods? Or
> perhaps is there a way to define elements that reside in top scope?
>
> I made a decision when I did this not to pollute the top namespace - both 
> for cleanliness, and also for simplicity of implementation.  This has come 
> up a few times, though, so it's something that I'm happy to revisit.
>
> 3. Class resource
>
> I think Dan Bode already raised this as a bug:
> https://projects.puppetlabs.com/issues/5236
>
> The following returns an error 'method_missing':
>
>   hostclass :foo do
> notify "bar"
>   end
>   node "default" do
> hostclass "foo", :stage => "baz"
>   end
>
> This is because hostclass has not been defined as a convenience method
> it would seem. Just like issues 2 If someone knows
> a way to call a resource directly without requiring the convenience
> method that would be a decent enough work-around.
>
> Hmm.  It should be, but it looks like this is failing somewhere and has 
> been marked as a high priority bug. :/
>
> 4. exec Resource
>
> The 'exec' resource as a convenience method is overlapping with the
> ruby 'exec'
> method. So if you do this:
>
>   node "default" do
> exec "ls", :command => "/bin/ls"
>   end
>
> You get:
>
>   can't convert Hash into String on node obelisk.usr.bob.sh
>
> Returned from the Kernel.exec call.
>
> You should be able to use 'create_resource :exec, ...' here.
>
> 5. Referencing other resources
>
> I'm not quite clear how to reference other resources that already
> exist. This is obviously a problem for the 'require' attribute:
>
>   node "default" do
> package "foo", :ensure => "installed"
> service "foo", :enable => "true", :ensure => "running", :require
> => ???
>   end
>
> I believe the internal implementation allows you to do 'require => 
> [:package, "foo"]'.  I couldn't f

[Puppet Users] Re: [Puppet-dev] Converting to Ruby DSL

[Luke Kanies]

On Dec 8, 2010, at 11:09 AM, Ken Barber wrote:

> (cross posting here as I originally posted in puppet-users - I figure the 
> question is more of a dev one then for puppet-users)
> 
> Hi everyone,
> 
> I'm experimenting with converting some of my live puppet content to
> Ruby DSL and have found a few gaps I wouldn't mind some advice on. I'm
> not sure how many of you have already experimented in this arena yet.
> 
> I've already read Dan Bode's excellent blog article on the subject:
> 
> http://www.puppetlabs.com/blog/ruby-dsl/
> 
> I know the Ruby DSL is quite new - but I figured perhaps some of these
> items deserve to be tickets (or perhaps documented) and I thought I'd
> ask first :-).
Thanks for testing this.  As you say, it's still quite new, and definitely 
incomplete, but as with most things we do, I wanted to get a simple thing out 
and in the wild rather than wait for perfection before shipping.  This kind of 
feedback is exactly what we need (although of course patches are even better).
> 1. Defaults
> 
> How do I set defaults in Ruby DSL? For example what is the ruby DSL
> equivalent to this:
> 
>   Service {
> hasstatus => true
>   }
This is one of the things I left out of my initial implementation.

You might be able to figure out how to add it by patching one of the files in 
lib/puppet/dsl.
> 2. Top scope
> 
> I notice the convenience methodology does not work at 'top scope' for
> example
> this will fail:
> 
>   file "/tmp/zzz", :content => "foo"
> 
> But this will not:
> 
>   node "default" do
> file "/tmp/zzz", :content => "foo"
>   end
> 
> Is there a proper way to call resources without using the convenience
> methods? Or
> perhaps is there a way to define elements that reside in top scope?
I made a decision when I did this not to pollute the top namespace - both for 
cleanliness, and also for simplicity of implementation.  This has come up a few 
times, though, so it's something that I'm happy to revisit.
> 3. Class resource
> 
> I think Dan Bode already raised this as a bug:
> 
> https://projects.puppetlabs.com/issues/5236
> 
> The following returns an error 'method_missing':
> 
>   hostclass :foo do
> notify "bar"
>   end
>   node "default" do
> hostclass "foo", :stage => "baz"
>   end
> 
> This is because hostclass has not been defined as a convenience method
> it would seem. Just like issues 2 If someone knows
> a way to call a resource directly without requiring the convenience
> method that would be a decent enough work-around.
Hmm.  It should be, but it looks like this is failing somewhere and has been 
marked as a high priority bug. :/
> 4. exec Resource
> 
> The 'exec' resource as a convenience method is overlapping with the
> ruby 'exec'
> method. So if you do this:
> 
>   node "default" do
> exec "ls", :command => "/bin/ls"
>   end
> 
> You get:
> 
>   can't convert Hash into String on node obelisk.usr.bob.sh
> 
> Returned from the Kernel.exec call.
You should be able to use 'create_resource :exec, ...' here.

> 5. Referencing other resources
> 
> I'm not quite clear how to reference other resources that already
> exist. This is obviously a problem for the 'require' attribute:
> 
>   node "default" do
> package "foo", :ensure => "installed"
> service "foo", :enable => "true", :ensure => "running", :require
> => ???
>   end
I believe the internal implementation allows you to do 'require => [:package, 
"foo"]'.  I couldn't find a reasonable syntax for this, without a lot more work 
making, e.g., Package["foo"] work, which was my ideal.
> 6. Chaining Resources
> 
> Is there a syntactical equivalent to this in Ruby DSL yet? For
> example:
> 
>   Stage[pre] -> Stage[main] -> Stage[post]
> 
> Without knowing how to solve 5 I'm kind of stuck when it comes to
> defining dependencies :-).
There isn't.

The DSL is obviously still a subset of the "full" Puppet language (ironic, 
given how small the language is).

-- 
Computers are not intelligent.  They only think they are.
-
Luke Kanies  -|-   http://puppetlabs.com   -|-   +1(615)594-8199



-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-us...@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.