Re: [Puppet Users] File header including timestamp

[Jakov Sosic]

On 11/08/2012 10:49 AM, James Fellows wrote:


For me, the benefit of the header is being able to immediately tell
whether the file has been modified by someone or something since the
last puppet run (by comparing the header to the last modified
timestamp on the file).


You can already do that with puppet noop run:

# puppet agent -t --noop

which won't modify system but only announce what changes will be applied 
on the next "real" agent run. If you find your file in the list of 
resources that puppet wants to (re)appy, you can be certain that someone 
has messed around that resource. When it was done can be seen by 
filesystem timestamps.


If, on the other hand, puppet was the last who modified that file and 
you want to know when, again you can use the filesystem timestamps.


Now, the only corner case in which I can see potential use of that kind 
of header is if someone changed manifests on your master, and puppet now 
wants to modify the file - which was last modified also by puppet. But 
if you use some kind of VCS for keeping track of your manifests, you 
could know even that.




Would others find it useful?  If so I could raise a feature request
for it to open a formal discussion, and (one day, maybe!) fork and
contribute to the code.


That kind of header IMHO only brings one thing to the table - and that 
is fancy look :) Other than that I find it pretty much useless.


But then again I don't mind useless as long as it doesn't do performance 
penalties. But this does impose additional strain on both the master 
(catalog compilation time increase) and the agents (run time increase). 
And management software should be as non-intrusive as possible. I want 
puppet, zabbix, ossec and other management related agents to consume as 
little resources as possible...



--
Jakov Sosic
www.srce.unizg.hr

--
You received this message because you are subscribed to the Google Groups "Puppet 
Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] File header including timestamp

[jcbollinger]

On Thursday, November 8, 2012 3:49:12 AM UTC-6, j4m3s wrote:
>
> Thanks John.  Yes I realised that from going through the crontab.rb 
> and parsedfile.rb scripts. 
>
> You are absolutely correct that there is nothing that could easily be 
> lifted and used on top of puppet. 
>
> I believe that, in order to make this work, a change would be required 
> to one or more of the core template funtions in puppet - possibly to 
> add a "flag" to say to include the last-updated header (or call a 
> separate templateWithHeader function) , and to later ignore it when 
> parseing the files to check for changes.



Nope.  There is no way the issue could be solved by modifying the template 
functions, because they have nothing to do with checking file content for 
changes.  I outlined already what would be needed: a file with a 
content-timestamp header would need to be treated as at least two parts (a 
composite *resource* -- header and body).  It might be possible to build 
something like that on top of the Concat module, which is all about 
building files from pieces.  Even if not, it's at that level (resource 
type) where the problem would need to be addressed.

 

>  The fact that the comment 
> marker would be different in some cases does complicate matters - but 
> not insurmountably.  It could default to being determined based on the 
> file extension (the part before the .erb) and the function could take 
> a parameter that overwrode it if required.  It's a common process when 
> auto-generating source file headers for copyright in maven projects 
> for example (where there are some java files, some sql files, some xml 
> files etc). 
>


The problem is not with generating or formatting the header, it's with 
ignoring the header when the file content is checked to see whether it's in 
sync.  The template functions have nothing whatever to do with that.

 

>
> I'm actually a Java guy by background so am loving the power of puppet 
> but struggling someone to make anything more than the most basic 
> puppet functions.  That said, if I get the chance I would love to code 
> or contribute to an enhancement like this, if it would be valued by 
> the community. 
>


I wouldn't personally use it.  For me,

   1. It only matters that the content is in sync, not when it was 
   generated.
   2. If I wanted to know when the content was generated, then I would be 
   content to rely on the file's modification timestamp as recorded by the 
   file system.
   3. The extra complexity involved in managing a content timestamp would 
   produce extra work for the master and extra space for bugs.
   
Indeed, if I were looking for something along these lines, it would more 
likely be a *template* version or timestamp than a content timestamp, and a 
template timestamp doesn't need any special support.


> For me, the benefit of the header is being able to immediately tell 
> whether the file has been modified by someone or something since the 
> last puppet run (by comparing the header to the last modified 
> timestamp on the file).  It's very helpful when fault-finding.  It 
> also gives a nice straightforward view (from within the file itself) 
> of the last time it changed - again helping to narrow down the source 
> of problems. 
>


I understand.  I just don't have concerns along those lines with the files 
I am currently managing on any of my hosts.  I also think you may be 
overlooking some of the implications of the Puppet agent keeping managed 
files in sync -- unlike once- or rarely-generated files, files with 
Puppet-managed content can be re-generated up to once every Puppet run 
(with new content timestamps, if any).  Manual changes will be clobbered on 
that schedule, which is once every 30 minutes by default.  As a result, 
admins will quickly learn not to make manual changes to managed files, and 
if they do make such changes then you can always tease out the information 
you want (with more difficulty) from the Puppet log.


John

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/puppet-users/-/dR-yGuVifmkJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] File header including timestamp

[James Fellows]

Thanks John.  Yes I realised that from going through the crontab.rb
and parsedfile.rb scripts.

You are absolutely correct that there is nothing that could easily be
lifted and used on top of puppet.

I believe that, in order to make this work, a change would be required
to one or more of the core template funtions in puppet - possibly to
add a "flag" to say to include the last-updated header (or call a
separate templateWithHeader function) , and to later ignore it when
parseing the files to check for changes.  The fact that the comment
marker would be different in some cases does complicate matters - but
not insurmountably.  It could default to being determined based on the
file extension (the part before the .erb) and the function could take
a parameter that overwrode it if required.  It's a common process when
auto-generating source file headers for copyright in maven projects
for example (where there are some java files, some sql files, some xml
files etc).

I'm actually a Java guy by background so am loving the power of puppet
but struggling someone to make anything more than the most basic
puppet functions.  That said, if I get the chance I would love to code
or contribute to an enhancement like this, if it would be valued by
the community.

For me, the benefit of the header is being able to immediately tell
whether the file has been modified by someone or something since the
last puppet run (by comparing the header to the last modified
timestamp on the file).  It's very helpful when fault-finding.  It
also gives a nice straightforward view (from within the file itself)
of the last time it changed - again helping to narrow down the source
of problems.

Would others find it useful?  If so I could raise a feature request
for it to open a formal discussion, and (one day, maybe!) fork and
contribute to the code.

James

On 7 November 2012 23:04, jcbollinger  wrote:
>
>
> On Wednesday, November 7, 2012 8:17:50 AM UTC-6, j4m3s wrote:
>>
>> Thank you very much Matt.  My searches hadn't turned that up - I was
>> searching on keywords relating to templates I think.
>>
>> Thanks again, James.
>>
>
> Do note that the reason that approach can work for cron jobs and hosts
> entries is that the relevant resources model individual parts of the file
> (one job or one host) rather than the file as a whole.  I imagine you could
> patch something together that would treat ordinary files as made up of
> parts, but it will probably need to be somewhat specific to the target file
> format.  I personally can't see doing that unless it were particularly
> important to have a content timestamp in the file.  I certainly don't see
> doing it routinely for every managed file.
>
>
> John
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/puppet-users/-/XN7EX_wBe2EJ.
> To post to this group, send email to puppet-users@googlegroups.com.
> To unsubscribe from this group, send email to
> puppet-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/puppet-users?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] File header including timestamp

[jcbollinger]

On Wednesday, November 7, 2012 8:17:50 AM UTC-6, j4m3s wrote:
>
> Thank you very much Matt.  My searches hadn't turned that up - I was 
> searching on keywords relating to templates I think.
>
> Thanks again, James.
>
>
Do note that the reason that approach can work for cron jobs and hosts 
entries is that the relevant resources model individual parts of the file 
(one job or one host) rather than the file as a whole.  I imagine you could 
patch something together that would treat ordinary files as made up of 
parts, but it will probably need to be somewhat specific to the target file 
format.  I personally can't see doing that unless it were particularly 
important to have a content timestamp in the file.  I certainly don't see 
doing it routinely for every managed file.


John

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/puppet-users/-/XN7EX_wBe2EJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] File header including timestamp

[j4m3s]

Thank you very much Matt.  My searches hadn't turned that up - I was 
searching on keywords relating to templates I think.

Thanks again, James.

On Wednesday, November 7, 2012 2:12:27 PM UTC, Matthew Burgess wrote:
>
> On Wed, Nov 7, 2012 at 2:00 PM, j4m3s > 
> wrote: 
> > I have noticed that if I create a hosts file entry using the host {} 
> type, a 
> > header is added to teh top of the file showing when it was last updated. 
> > I'm wondering if it is possible something similar in a template? 
>  Calling 
> > <%= Time.now().to_s() %> inside the template (obviously) causes the file 
> to 
> > be updated each time puppet runs - which isn't what I'm looking for. 
> > 
> > I have tried to find the host type on github (in the puppet source) but 
> > couldn't track it down.  Is it possible to reproduce the functionality 
> in my 
> > own template headers? 
>
> There's a thread on this topic at 
>
> https://groups.google.com/forum/?fromgroups#!search/puppet$20cron$20job$20class/puppet-users/PMTVEpXw8I0/Wsckx5euwRgJ
>  
> - in short, copy and paste code from Puppet's crontab.rb. 
>
> Hope this helps, 
>
> Matt. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/puppet-users/-/LQ3z59qD1eoJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

Re: [Puppet Users] File header including timestamp

[Matthew Burgess]

On Wed, Nov 7, 2012 at 2:00 PM, j4m3s  wrote:
> I have noticed that if I create a hosts file entry using the host {} type, a
> header is added to teh top of the file showing when it was last updated.
> I'm wondering if it is possible something similar in a template?  Calling
> <%= Time.now().to_s() %> inside the template (obviously) causes the file to
> be updated each time puppet runs - which isn't what I'm looking for.
>
> I have tried to find the host type on github (in the puppet source) but
> couldn't track it down.  Is it possible to reproduce the functionality in my
> own template headers?

There's a thread on this topic at
https://groups.google.com/forum/?fromgroups#!search/puppet$20cron$20job$20class/puppet-users/PMTVEpXw8I0/Wsckx5euwRgJ
- in short, copy and paste code from Puppet's crontab.rb.

Hope this helps,

Matt.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.

[Puppet Users] File header including timestamp

[j4m3s]

I have noticed that if I create a hosts file entry using the host {} type, 
a header is added to teh top of the file showing when it was last updated.  
I'm wondering if it is possible something similar in a template?  Calling 
<%= Time.now().to_s() %> inside the template (obviously) causes the file to 
be updated each time puppet runs - which isn't what I'm looking for.  

I have tried to find the host type on github (in the puppet source) but 
couldn't track it down.  Is it possible to reproduce the functionality in 
my own template headers?

Thanks, James.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Users" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/puppet-users/-/72U23qmrzYsJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.