On Oct 31, 2008, at 5:40 PM, Brice Figureau wrote:
>
> The lexer maintains a stack of last seen comments.
> Each AST nodes has a doc property that is filled with the
> last seen comments on node creation (in fact only on important node
> creation representing statements).
>
> Signed-off-by: Brice Figureau <[EMAIL PROTECTED]>
> ---
> lib/puppet/parser/ast.rb | 2 +
> lib/puppet/parser/grammar.ra | 50 ++++---
> lib/puppet/parser/lexer.rb | 42 ++++++-
> lib/puppet/parser/parser.rb | 256 +++++++++++++++++
> +-----------------
> lib/puppet/parser/parser_support.rb | 16 ++-
> 5 files changed, 212 insertions(+), 154 deletions(-)
>
> diff --git a/lib/puppet/parser/ast.rb b/lib/puppet/parser/ast.rb
> index ddf8852..ede3842 100644
> --- a/lib/puppet/parser/ast.rb
> +++ b/lib/puppet/parser/ast.rb
> @@ -12,6 +12,8 @@ class Puppet::Parser::AST
>
> include Puppet::Util::Errors
> include Puppet::Util::MethodHelper
> + include Puppet::Util::Docs
> +
> attr_accessor :line, :file, :parent, :scope
>
> # Does this ast object set something? If so, it gets evaluated
> first.
> diff --git a/lib/puppet/parser/grammar.ra b/lib/puppet/parser/
> grammar.ra
> index 23c2934..d01ab61 100644
> --- a/lib/puppet/parser/grammar.ra
> +++ b/lib/puppet/parser/grammar.ra
> @@ -77,26 +77,28 @@ fstatement: NAME LPAREN funcvalues RPAREN {
> result = ast AST::Function,
> :name => val[0],
> :arguments => args,
> - :ftype => :statement
> + :ftype => :statement,
> + :doc => @lexer.getcomment
> }
Wouldn't it be easier to have the 'ast' method add the 'doc' attribute?
If you're not adding docs to every ast class, then you could have a
boolean on the AST classes that should add docs.
That would make this patch *much* smaller.
[...]
> diff --git a/lib/puppet/parser/lexer.rb b/lib/puppet/parser/lexer.rb
> index 9226434..4ff2d25 100644
> --- a/lib/puppet/parser/lexer.rb
> +++ b/lib/puppet/parser/lexer.rb
> @@ -17,7 +17,7 @@ class Puppet::Parser::Lexer
>
> # Our base token class.
> class Token
> -
> attr_accessor :regex, :name, :string, :skip, :incr_line, :skip_text
> +
> attr_accessor
> :regex, :name, :string, :skip, :incr_line, :skip_text, :accumulate
>
> def initialize(regex, name)
> if regex.is_a?(String)
> @@ -32,6 +32,10 @@ class Puppet::Parser::Lexer
> self.skip
> end
>
> + def accumulate?
> + self.accumulate
> + end
> +
> def to_s
> if self.string
> @string
> @@ -155,7 +159,10 @@ class Puppet::Parser::Lexer
> [string_token, value]
> end
>
> - TOKENS.add_token :COMMENT, %r{#.*}, :skip => true
> + TOKENS.add_token :COMMENT, %r{#.*}, :accumulate => true, :skip
> => true do |lexer,value|
> + value.sub!('#','')
> + [self, value]
> + end
>
> TOKENS.add_token :RETURN, "\n", :skip => true, :incr_line =>
> true, :skip_text => true
>
> @@ -320,6 +327,7 @@ class Puppet::Parser::Lexer
> @namestack = []
> @indefine = false
> @expected = []
> + @commentstack = ['']
> end
>
> # Make any necessary changes to the token and/or value.
> @@ -328,12 +336,18 @@ class Puppet::Parser::Lexer
>
> skip() if token.skip_text
>
> - return if token.skip
> + return if token.skip and not token.accumulate?
>
> token, value = token.convert(self, value) if
> token.respond_to?(:convert)
>
> return unless token
>
> + if token.accumulate?
> + @commentstack.last << value+"\n"
> + end
> +
> + return if token.skip
> +
> return token, value
> end
>
> @@ -388,6 +402,13 @@ class Puppet::Parser::Lexer
>
> next unless final_token
>
> + # push comments on block change
> + if final_token.name == :LBRACE
> + commentpush('')
> + elsif final_token.name == :RBRACE
> + commentpop
> + end
> +
> if match = @@pairs[value] and final_token.name !
> = :DQUOTE and final_token.name != :SQUOTE
> @expected << match
> elsif exp = @expected[-1] and exp == value and
> final_token.name != :DQUOTE and final_token.name != :SQUOTE
> @@ -448,4 +469,19 @@ class Puppet::Parser::Lexer
> def string=(string)
> @scanner = StringScanner.new(string)
> end
> +
> + # returns the content of the currently accumulated content cache
> + def commentpop
> + return @commentstack.pop
> + end
> +
> + def getcomment
> + comment = @commentstack.pop
> + @commentstack.push('')
> + return comment
> + end
Why add the empty string?
>
> + def commentpush(value)
> + @commentstack.push(value)
> + end
> end
Looks good.
--
I do not feel obliged to believe that the same God who has endowed us
with sense, reason, and intellect has intended us to forgo their use.
-- Galileo Galilei
---------------------------------------------------------------------
Luke Kanies | http://reductivelabs.com | http://madstop.com
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/puppet-dev?hl=en
-~----------~----~----~----~------~----~------~--~---
