Re: [Gogo] Grammar and commands

[Peter Kriens]

One of the key reasons for the grammar is that the parser could be  
trivial. You basically only need a token and string parser and  
recursively parse blocks enclosed by [<({[], etc. A trivial grammar  
that is very similar to TCL. Though it is a nice exercise to turn it  
into an ANTLR grammar, the increased size would void one of my primary  
design goals. That is, if you look at beanshell it is around 130k, and  
that is largely the zillion parse classes.

I think keeping this simplicity has some value ...
Kind regards,

        Peter Kriens




On 21 aug 2009, at 21:34, David Savage wrote:

On Fri, Aug 21, 2009 at 4:50 PM, Guillaume Nodet<gno...@gmail.com>  
wrote:
Right, I think it makes sense to define the grammar.
But the grammar just defines ... well, the syntax, not the semantics.
I think most of the discussion is about what semantic to apply on the
grammar.

Hmmm but having agreed on a syntax doesn't that mean we then have a
set of tokens to which we can apply semantics? At the risk of
reopening the a,b,c debate...

ab
a b
a ; b
a = b
a | b
a {b}
a <b>
"a b"
a = $b
a = {b}
a = <b>
$ab
$a b

Generate a variety of different tokens and it is the tokens to which
we apply semantics? Also the expected result of each of these varies
depending on whether a or b is assigned to be a command or an object
and also whether it does IO, is a value or returns a value.

I think if we focus on one part of the problem and tweek the parsing
code to fix this we are highly likely to generate some other
unexpected behaviour. Where as if we define a parser that can generate
a set of tokens via a grammar the runtime only has to define the
actions it applies to the tokens?

Though it still remains to be seen if a grammar is possible so perhaps
this is a moot point till one is available - hopefully the above
illustrates the concern though?


For commands, imnsho, I think we should discuss and encourage the  
use of
what I committed in

https://svn.apache.org/repos/asf/felix/trunk/gogo/commands/src/main/java/org/apache/felix/gogo/commands/

Those are much more powerfull than a simple method on an object,  
because
they allow:
  * help on the command
  * automatic parsing / conversion of arguments
  * variable number of arguments
  * command line switches


Certainly I think those classes look very useful and wrapper classes
and utilities are great in real world scenarios to avoid code
duplication. But I don't think we should ignore the OO case. Being
able to call things like String.indexOf and startsWith type methods in
a shell without having to generate minutae wrapper commands to achieve
things the vm already provides us with helps in the DRY aspect of
software engineering.

We have a bunch of commands defined in Karaf that are not specific  
to Karaf,
so it would make sense to move them in a command ground.
You can find some of them at:

https://svn.apache.org/repos/asf/felix/trunk/karaf/gshell/gshell-osgi/src/main/java/org/apache/felix/karaf/gshell/osgi/
The only problem is that those are slightly tied to blueprint atm.

In terms of this level of the debate I'm not sure we need "real"
commands only to agree on a set of representative commands - so we
know exactly what echo does, for example. If those representative
commands are also useful that's obviously a bonus, but at least if
we're agreed on what the various types of commands and tokens are,
then we should be able to agree (or not) how to treat them in a shell
environment - avoids the potential for circular debates. Finally we
should certainly implement these representative commands, to allow us
to create meaningful tests.

Again just some thoughts...

Regards,

Dave


On Fri, Aug 21, 2009 at 16:54, David Savage  
<dave.sav...@paremus.com> wrote:

Hi there,

Seems like a good time to wade into the discussion ;) it seems  
like it
could be beneficial to start looking at the TSL syntax from the  
point
of view of a formal grammar specification such as ANTLR? Currently
most of the tests are via unit tests and the parsing is done by  
custom
java code. The danger is that without a formal grammar specification
we may "fix" one part of the parser to handle one use case only to
expose a secondary problem. Unit tests are certainly one way to  
catch
this but they can only be as good as the number of tests we define,
where as I believe a logically consistent grammar is a test in  
itself?

Of course this may also be a fools errand (firstly as I have no
experience with antlr grammars) and looking at this link (which I  
came
at by searching for "bash antlr grammar" in google) it may not  
even be
possible - though the post is very old:

http://www.antlr.org/pipermail/antlr-interest/2006-May/016235.html

Wondering if there are any language experts following this list  
who can
comment?

I'm also wondering if it may make sense to extract the shell that
interprets the commands and the runtime that provides the  
commands. I
really like the TSL impl but in the end it's just one possible way  
of
running commands? One of the really nice things about RFC 132 is it
provides a common way for developers to add commands in osgi, but  
how
we refer to those commands is a secondary issue, here thinking of  
the
sh,csh,bash type debate...

Finally though the echo echo type debate is good for simplifying the
problem down in unit tests in email it does depend on how we think
echo is defined. Does echo return a value or does it write the  
result
to the stream or both (?!). It seems like it would be useful to  
define
a set of commands for debate that are unambiguous? I guess these  
could
be abstract commands which could go on a wiki or some such? Some
examples:

nsc -> no such command
void -> command that does nothing and returns nothing
echo -s hello -> echo "hello" to stream
echo -v -> echo "hello" to value
echo -sv hello -> echo "hello" to stream and value
array a b c -> returns [a,b,c] as an array

Others?

Just my 2 pence anyway...

Regards,

Dave

On Thu, Aug 20, 2009 at 5:45 PM, Guillaume Nodet<gno...@gmail.com>  
wrote:
On Thu, Aug 20, 2009 at 15:40, Derek Baum <derek.b...@paremus.com>
wrote:


I disagree with having eval as a command.  The reason is that  
is has
two
sides effects:
 * all parameters are evaluated once more, so that $xxx expansion
will be
done once again, and it could lead to unwanted results

this is offset by not implicitly evaluating the args - re- 
evaluation
only occurs when explicitly invoking eval.

 * all parameters are converted to strings, which i think is  
not what
is
expected.

I'm not sure this is a problem. The 3.patch eval is like eval in  
bash,
and can be used to re-evaluate a string as a script.

Derek



Well, I think this really lead to undesirable effects:
x = <bundle 0>
...
$x toString
org.apache.felix.framework [0]
eval $x toString
 Command not found *:org.apache.felix.framework

I think both should be identical.
If you want to evaluate the arguments as a fully new command  
line, you
could
use quoting
eval "$x toString"
But the opposite can't be done.
So I still think we should come back to my earlier proposal about  
making
it
a real keyword instead of a command.

I think this is independant of wether arguments are re-parsed,  
though
they
are related.

I've also spotted another problem, but this looks like a different
problem:
echo "$x"
java.lang.Exception: Unable to convert from  
[org.apache.felix.framework
[0]]
to java.util.List<java.lang.String>(error converting collection  
entry)

I would have thought it would behave the same as
echo <$x toString>


--
Cheers,
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
------------------------
Open Source SOA
http://fusesource.com




--
Cheers,
Guillaume Nodet
------------------------
Blog: http://gnodet.blogspot.com/
------------------------
Open Source SOA
http://fusesource.com