Re: code is data
I missed this reply earlier. Fredrik Lundh wrote: > there might be cognitive theories that argue that the length of the > symbols used to describe something is more important than the symbols > you use and how they can be "chunked" by the brain Expert communication is known to work differently. For example, take a doctor's note. It's minimalist representation and domain specific notational standardizations are absolute gibberish for a reader without the knowledge of medicine (or more specifically, the sub-specialty) and the context of the patient. But it is very efficient for the audience it is intended for. Now, the hand writing - that's a wholly different story :-). BTW, I am studying clinical information representations. This discussion was interesting to me because I am trying to see programming language representation as a better discussed and more structured microcosm of the relatively fuzzy domain I am studying. Of course, there are many important differences and I may be drawing the parallels liberally at this point but it gives me an interesting perspective. -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Anton Vredegoor wrote: > Paul Boddie wrote: > > > > I was going to write a long reply to one of your previous messages, but > > the above link references a project which may intersect with some of > > your expectations. Meanwhile, it should be noted that the availability > > Somehow I get the impression of getting away with my posts luckily, > while you now are duping other interested readers into not reading your > innermost feelings about this subject. Let's get it in the open, don't > spare me :-) I was just going to say that your Web application could have used XSLT more or less exclusively if the data arrives in XML (and obviously leaves in some kind of XML), provided you aren't doing exotic processing on the data. [...] > Well maybe this trick of vehemently denying the existence of something > on Usenet worked again, by materializing the thing as a reaction. My intention was to correct the perception that before a bunch of work was done on exposing the Python AST (the so-called "AST branch") there weren't any tools to get nice ASTs which could be used to generate code. Of course there are such tools, with the compiler module being in the standard library itself - not to be confused with the parser module which everyone seems to mention and about whose representation everyone seems to complain, but that's also obviously in the standard library too. I'm not sure about the ready availability of Python-to-Python transformers, though. I have made HTML "reports" of Python programs - annotated source code with type information in CSS-based pop-ups - and a plain text serialiser for the compiler module's AST wouldn't be as difficult as that. The hardest part of working with the compiler API would be in the modification of the AST, and perhaps it is in that area that the "AST branch" shows promise. [...] > Maybe a Python webserver could also emit pieces of javascript code by > getting them from a *Python* code library after translating Python > code on the fly? http://subway.python-hosting.com/file/crackajax/trunk/crackajax.py I personally don't go for this kind of thing, but people have given it some thought. Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Anton Vredegoor wrote:
(snip)
> However, I knew of the existence of such languages but I am mostly
> interested in standardized code interchange, like for example with JSONP
> which fetches some external javascriptcode from another server using
> JSON and places the translated javascript into a webpage at the request
> of the clients browser or so it seems.
This is AJAX with JSON instead of XML (should we call this AJAJ ?-).
It's quite handy, since it saves both the extra bits to be transfered
(XML is way much verbose than JSON) and the XML to javascript parsing.
> Maybe a Python webserver could
> also emit pieces of javascript code by getting them from a *Python* code
> library after translating Python code on the fly?
If you restrict this on data, it's already done (Turbogears uses this
for AJAX).
> That would open up the web to Python programmers without browsers
> needing to understand Python. Like Jython, but now as separately
> distributed functions from different servers.
>
> Anton
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in '[EMAIL PROTECTED]'.split('@')])"
--
http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Anton Vredegoor wrote:
> Paul Boddie wrote:
>
> > Anton Vredegoor wrote:
>
> >> Yes, but also what some other posters mentioned, making Pythons internal
> >> parsing tree available to other programs (and to Python itself) by using
> >> a widely used standard like XML as its datatype.
> >
> > http://pysch.sourceforge.net/ast.html
>
> Very interesting, it led me to some sxml describing pages and it also
> tricked me into reading some Python documentation that I had always
> considered to be hiding some arcane deep Python magic. I guess now that
> Python is officially entering tree territory (as opposed to providing
> third party functionality) it seems unavoidable that Python's officially
> endorsed tree datatype will also be used for some of its internal
> structures, thereby making it more accessible to programmers like me and
> to outside programmers.
I had the same initial impression with the "deep" aspects of Python
parser API but I learned soon that I just stumbled about an awkward
kind parse-tree representation. The structures are as simple as they
could be - due to the nice LL(1) Python grammar which are reflected.
I think node manipulation is just a first basic step for defining
source transformers i.e. they provide a sound basis. For usability
purposes a template language is a far better solution.
Here is my own attempt. Defining a repeat statement ( for demonstration
purposes ) using EasyExtend requires following translation:
repeat:
until:
==>
while 1:
if :
break
The placeholders and correspond to nodes in the
parse-tree ( and constants in symbol.py ). The repeat-stmt is defined
by an added grammar rule. The corresponding node in the parse-tree is
dispatched against Transformer.handle_repeat_stmt() during parse-tree
traversal. Matching against and in the repeat_stmt node
is done by applying:
suite_node = find_node(repeat_node, symbol.suite)
test_node = find_node(repeat_node, symbol.test, level=1)
What becomes tedious is the creation of the corresponding while_stmt
node which is the expected result of the transformation. I actually
want to use the declaration as it is written above and insert the
suite_node and the test_node where the placeholders and
are defined. Using EasyExtend one can define an extension language
where the placeholders ( and some enhanced versions of those ) become
language elements! Given this one can put everything together:
import macro# module that corresonds to the macro extension
language
class FastTransformer(Transformer):
def handle_repeat_stmt(self, repeat_node):
suite_node = find_node(repeat_node, symbol.suite)
test_node = find_node(repeat_node, symbol.test, level=1)
target_stmt = """
while 1:
if :
break
"""
while_node = macro.expand(target_stmt,
{'suite': suite_node,
'test': test_node},
goal =
symbol.while_stmt)
return any_stmt(while_node)
This is pretty safe and works recursively: if the node contains
another node it will be replaced as well.
--
http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Anton Vredegoor <[EMAIL PROTECTED]> wrote: > > However, I knew of the existence of such languages but I am > mostly interested in standardized code interchange, like for > example with JSONP which fetches some external javascriptcode > from another server using JSON and places the translated > javascript into a webpage at the request of the clients browser > or so it seems. Maybe a Python webserver could also emit pieces > of javascript code by getting them from a *Python* code library > after translating Python code on the fly? > > Anton I have no idea how close it is to what you are talking about, but pypy does have some sort of python->javascript support: http://codespeak.net/pypy/dist/pypy/doc/getting- started.html#translating-the-flow-graph-to-javascript-code About two thirds down, if the above link is broken: http://codespeak.net/pypy/dist/pypy/doc/getting-started.html max -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Paul Boddie wrote: > Anton Vredegoor wrote: >> Yes, but also what some other posters mentioned, making Pythons internal >> parsing tree available to other programs (and to Python itself) by using >> a widely used standard like XML as its datatype. > > http://pysch.sourceforge.net/ast.html Very interesting, it led me to some sxml describing pages and it also tricked me into reading some Python documentation that I had always considered to be hiding some arcane deep Python magic. I guess now that Python is officially entering tree territory (as opposed to providing third party functionality) it seems unavoidable that Python's officially endorsed tree datatype will also be used for some of its internal structures, thereby making it more accessible to programmers like me and to outside programmers. > I was going to write a long reply to one of your previous messages, but > the above link references a project which may intersect with some of > your expectations. Meanwhile, it should be noted that the availability Somehow I get the impression of getting away with my posts luckily, while you now are duping other interested readers into not reading your innermost feelings about this subject. Let's get it in the open, don't spare me :-) > of Python AST processing tools is not a recent thing: the compiler > module has been around for a long time, and it is possible to modify > the AST and to generate bytecode from it; my own experiments have > centred on producing other representations from the AST, and other more > successful projects (eg. ShedSkin) produce other languages (eg. C++) > from the AST. Well maybe this trick of vehemently denying the existence of something on Usenet worked again, by materializing the thing as a reaction. However, I knew of the existence of such languages but I am mostly interested in standardized code interchange, like for example with JSONP which fetches some external javascriptcode from another server using JSON and places the translated javascript into a webpage at the request of the clients browser or so it seems. Maybe a Python webserver could also emit pieces of javascript code by getting them from a *Python* code library after translating Python code on the fly? That would open up the web to Python programmers without browsers needing to understand Python. Like Jython, but now as separately distributed functions from different servers. Anton -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Ravi Teja wrote: > You blogged on Django. Let's use that. Don't you think model creation > in Django can be represented better, given that it is done often > enough? nope, because 1) it's not done very often, and 2) the existing syntax is already very minimal, and defined in terms of a language that I already understand. there might be cognitive theories that argue that the length of the symbols used to describe something is more important than the symbols you use, and how they can be "chunked" by the brain, but sturgeon's law applies to cognitive scientists too ;-) > Since you are on thread and are a prominent and involved member of the > Python community, I would like it if you (or any such other) can > provide feedback on the rest of my previous post rather than be > dismissive by just a small portion of it. Perhaps, that will give me > some insight how these language design decisions are rationally made (I > am not strictly a programmer by profession, much less a language > designer). see Ian's posts for some excellent discussion. -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
> I don't think that distinction is very meaningful. As a programmer I
> have to understand both.
> I understand the Python compiler well, and it gives me reasonably good
> feedback when I
> get things wrong, and it has a lot of flexibility along several
> orthogonal lines.
> We're talking about programming languages, so it's useless to consider
> a cognitive perspective without considering the compiler perspective.
As some one who has followed your excellent articles, posts and tools
over the last many years, I don't doubt your expertise at all.
I admit I am not be entirely looking at this as a programmer. I am
working in informatics for the last few years. At least in this domain,
it is useful to seperate the perspectives. Sort of like designing a
functional language based primarily on mathematical abstractions rather
than imperative realities. Yes! Once the implementation is underway,
the practical realities are unavoidable but they interfere in thinking,
otherwise.
> I understand you, but it's very unclear to me how you can make a thin
> layer that's not horribly leaky or stunted. In my experience one or
> both are likely in DSLs, and the result is horrible and only useful as
> a toy.
I am not making this up out of thin air. There is already an
implementation, Logix, which allows you make thin layer DSLs on top of
itself (which in turn sits on top of Python). It did not fly but I
could not technically fault it (except maybe that it is not very
optimized at the moment). Maybe with your better understanding of
language implementations, you can.
> I understood the distinction you were making. But you were also
> speaking generally about generally programmable syntax, and I don't
> think that's a good idea, and it's very unclear how that mixes with
> Python. You can always do:
>
> model = make_model("""
> my funny syntax
> """)
>
> And you actually get something that can be a close peer to normal
> Python code. But if you are talking about Python and some novel syntax
> interleaved, then the details seem to matter a lot, because the
> implementation is substantial and potentially invasive.
NO! I have a feeling that you are interpreting me as a language
radical. I do not think that Python syntax should have too many
novelties. Currently Python feels a cohesive whole. A well designed
entity rather than something that evolved by accretion of features like
Perl. With active attempts to prune inconsistencies for that end
(Python 3000). And I value all that.
But Logix does not add ANY new syntactic constructs to Python. There is
a special import function that imports a macro supporting Python
language variant module where one can create special syntactic
constructs. I am quite satisfied with the macro capabilities of Logix.
But I am just grumbling that the Python community does not find macros
interesting. But without enough users, it simply dies away.
--
http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Anton Vredegoor wrote: > Bruno Desthuilliers wrote: > > > You mean like 'converting' javascript to python or python to ruby (or > > converting any home-grown DSL to Python, etc) ? > > Yes, but also what some other posters mentioned, making Pythons internal > parsing tree available to other programs (and to Python itself) by using > a widely used standard like XML as its datatype. http://pysch.sourceforge.net/ast.html I was going to write a long reply to one of your previous messages, but the above link references a project which may intersect with some of your expectations. Meanwhile, it should be noted that the availability of Python AST processing tools is not a recent thing: the compiler module has been around for a long time, and it is possible to modify the AST and to generate bytecode from it; my own experiments have centred on producing other representations from the AST, and other more successful projects (eg. ShedSkin) produce other languages (eg. C++) from the AST. Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
Bruno Desthuilliers wrote: > You mean like 'converting' javascript to python or python to ruby (or > converting any home-grown DSL to Python, etc) ? Yes, but also what some other posters mentioned, making Pythons internal parsing tree available to other programs (and to Python itself) by using a widely used standard like XML as its datatype. >> Then there are some people who keep insisting they don't understand what >> I'm talking about until I simplify things enough to get them on-board, > > count me in then :( Sorry about that. >> but then simply dismiss my ideas with 'you can already do that easily >> with this standard python construct'. This strategy was also eloquently >> refuted by some other poster, so I don't need to repeat it :-) >> >> I've gotten a lot of things to think about, so thanks all for your >> thoughts, but since this is getting way above my head I'll just wimp out >> and leave the rest of the thread to the experts! > > No way you will escape from your responsabilities so easily !-) Ok, count me back in then too :-) Of course I will be available for further discussion. If more than ten people demand a PEP and no better champion is available (very unlikely) I'll even write a proposal. Anton -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Ravi Teja wrote:
> > Or... maybe to be more specific, the hard work later on goes into
> > *code*. If you are enhancing your model, you do so with methods on the
> > model classes, and those methods don't effect the DSL, they are just
> > "code". You create some raw XML in the beginning, but quickly it's
> > just a matter of gluing those pieces together, using functions instead
> > of DSLs, and that's just "code".
>
> > That doesn't look that much better. How do you create it
> > programmatically? I know how to pass a variable to
> > CharField(maxlength=200); can I pass a variable to "char length 200"
> > just as easily? Can I use **kw? Can I read it from a CSV file and
> > construct the class that way? Maybe, but only by recreating all the
> > native patterns that I can infer easily looking at the Django class.
>
> I am looking at it from the cognitive perspective. You are looking at
> it from the compiler perspective.
I don't think that distinction is very meaningful. As a programmer I
have to understand both. I have to be able to synthesize correct
expressions. I need to understand the compiler. I understand the
Python compiler well, and it gives me reasonably good feedback when I
get things wrong, and it has a lot of flexibility along several
orthogonal lines.
We're talking about programming languages, so it's useless to consider
a cognitive perspective without considering the compiler perspective.
> I think you are talking about full blown DSLs like SQL which try to be
> self contained for a given domain. The ones I am referring are only
> thin layers on Python.
I understand you, but it's very unclear to me how you can make a thin
layer that's not horribly leaky or stunted. In my experience one or
both are likely in DSLs, and the result is horrible and only useful as
a toy.
If you are really serious about the implementation, sure. But DSL
almost screams out that it lacks seriousness, and a long-term
commitment to debuggability, generality, and documentation.
> However I think that you are seeing my sample differently than I meant
> it. I did not mean to create a special syntax file that would be parsed
> as a text file such that it would loose all the benefits of Python. It
> is just a thin layer over Python code for specific representational
> benefits. Kay Schluehr does a good job of identifying it as such in his
> reply.
I understood the distinction you were making. But you were also
speaking generally about generally programmable syntax, and I don't
think that's a good idea, and it's very unclear how that mixes with
Python. You can always do:
model = make_model("""
my funny syntax
""")
And you actually get something that can be a close peer to normal
Python code. But if you are talking about Python and some novel syntax
interleaved, then the details seem to matter a lot, because the
implementation is substantial and potentially invasive.
There are also very *specific* things that can be discussed that are
more conventional, and driven as much by the ease of implementation as
expressiveness. I think that is much more productive, because I think
the general case is a bad idea. The 'make' syntax is an example of
this.
Ian
--
http://mail.python.org/mailman/listinfo/python-list
Re: code is data
> Or... maybe to be more specific, the hard work later on goes into > *code*. If you are enhancing your model, you do so with methods on the > model classes, and those methods don't effect the DSL, they are just > "code". You create some raw XML in the beginning, but quickly it's > just a matter of gluing those pieces together, using functions instead > of DSLs, and that's just "code". > That doesn't look that much better. How do you create it > programmatically? I know how to pass a variable to > CharField(maxlength=200); can I pass a variable to "char length 200" > just as easily? Can I use **kw? Can I read it from a CSV file and > construct the class that way? Maybe, but only by recreating all the > native patterns that I can infer easily looking at the Django class. I am looking at it from the cognitive perspective. You are looking at it from the compiler perspective. I think you are talking about full blown DSLs like SQL which try to be self contained for a given domain. The ones I am referring are only thin layers on Python. > Words are great. Python is light on symbols, and that is good. Agreed. When I came to Python from Perl, I loved the clean syntax. Scalars, arrays, hashes occur too frequently in Perl code that using symbols to denote them causes more noise than cognitive assistance. On the other hand, using symbols to denote an occational special construct is helpful (as in decorators). > Even the Lisps stick to an incredibly homogenous syntax (far more > homogeneous than Python) to make macros feel familiar. Yes! The parser friendly, "everything is a list" syntax does help. I did consider that to be an essential feature to enable dynamic macros. However I changed my mind when I saw Logix macro syntax. > Constrained context is a step backward! How do you add methods? How > do you do looping? How do you write *code*? If you aren't going to > allow those things, then just make a parser and build the structure > from the file, and make it a DSL implemented entirely external to > Python. That's completely okay, though in my experience it's not very > satisfying for something like a model definition (see MiddleKit for an > example of an ORM that doesn't use Python code). I agree that constrained context is a step back in terms flexibility. But it is a strategic step backwards, in this case to trade for representational benefits. The extent of constraints is a judgement call. And proof of utility can only be emperical. However I think that you are seeing my sample differently than I meant it. I did not mean to create a special syntax file that would be parsed as a text file such that it would loose all the benefits of Python. It is just a thin layer over Python code for specific representational benefits. Kay Schluehr does a good job of identifying it as such in his reply. -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
Anton Vredegoor wrote:
> Diez B. Roggisch wrote:
>
> <...>
>
>>> The whole point of a code transformation mechanism like the one Anton is
>>> talking about is to be dynamic. Else one just needs a preprocessor...
>>
>>
>> No, it is not the whole point. The point is
>> ""
>> The idea is that we now have a fast parser (ElementTree) with a
>> reasonable 'API' and a data type (XML or JSON) that can be used as an
>> intermediate form to store parsing trees. Especially statically typed
>> little languages seem to be very swallow-able. Maybe I will be able to
>> reimplement GFABasic (my first love computer language, although not my
>> first relationship) someday, just for fun.
>> """
>>
>> No on-the-fly code generation here. He essentially wants
>> lisp-style-macros
>> with better parsing. Still a programming language. Not a data-monger.
>
>
> The 'problem' is that a lot of incredibly smart people are reading and
> replying here who are seeing a lot more into my post than I was prepared
> for :-)
no comment...
(snip various transformations examples)
>
> So if we can transform documents, images and XML, why not sourcecode?
>
(snip preliminary precautions)
> it would be easy to convert all datatypes
> into the datatypes of another language, thereby making it possible to
> exchange code between languages.
You mean like 'converting' javascript to python or python to ruby (or
converting any home-grown DSL to Python, etc) ?
> Algorithms just being things that
> convert sets of data-objects into other sets of data-objects.
>
> Now if one would equate standardized code exchange between languages and
> within a language with macros then I guess there is nothing left for me
> to do but wait till a certain google bot comes knocking at my ip-address
> port 80 and transfers me to the google equivalent of Guantanamo.
Lol !-)
Well, given this quote from another of your posts:
"""
The idea is to have a way to transform a Python (.py) module into XML
and then do source code manipulations in XML-space using ElementTree.
"""
I effectively understood something like a python to python
transformation, which of course led me to something very very like macros.
> But the whole point of distinguishing macros from official language
> structures *is* standardization, as some other clever poster already
> pointed out, so it would be extremely unfair to equate trans-language
> standardized code exchange with the guerrilla type macro activities that
> are plaguing the Lisp community.
>
> Then there are some people who keep insisting they don't understand what
> I'm talking about until I simplify things enough to get them on-board,
count me in then :(
> but then simply dismiss my ideas with 'you can already do that easily
> with this standard python construct'. This strategy was also eloquently
> refuted by some other poster, so I don't need to repeat it :-)
>
> I've gotten a lot of things to think about, so thanks all for your
> thoughts, but since this is getting way above my head I'll just wimp out
> and leave the rest of the thread to the experts!
No way you will escape from your responsabilities so easily !-)
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in '[EMAIL PROTECTED]'.split('@')])"
--
http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
Diez B. Roggisch wrote: <...> >> The whole point of a code transformation mechanism like the one Anton is >> talking about is to be dynamic. Else one just needs a preprocessor... > > No, it is not the whole point. The point is > > "" > The idea is that we now have a fast parser (ElementTree) with a > reasonable 'API' and a data type (XML or JSON) that can be used as an > intermediate form to store parsing trees. Especially statically typed > little languages seem to be very swallow-able. Maybe I will be able to > reimplement GFABasic (my first love computer language, although not my > first relationship) someday, just for fun. > """ > > No on-the-fly code generation here. He essentially wants lisp-style-macros > with better parsing. Still a programming language. Not a data-monger. The 'problem' is that a lot of incredibly smart people are reading and replying here who are seeing a lot more into my post than I was prepared for :-) Anyway, the last few weeks I have been busy transforming MsWord documents into XML using Open Office, and next parsing this XML and transforming it into a special subset of HTML using ElementTree's XMLWriter class. Then the output of the XMLWriter was put into a Zope/Plone page but I added special markup for footnotes, making them plone objects that could be separately edited, and I added image tags for images that were retrieved from a separate server using an XSLT script. To accomplish that a special zope parser was written to recognize my nonstandard footnote and image tags, and to create the necessary objects, and to insert them into the page. After that I came across some turbogears code (which is stacking code at different levels like it were those things you put under your beer glass) and still later I saw some JSON equivalents of XML. JSON looks a lot like Python dicts which makes it seem likely that javascript will be able to interface with Python more efficiently. Remember that ElementTree comes from the same place that brought us PIL which is a package that can transform images into different types. So if we can transform documents, images and XML, why not sourcecode? Especially if it's not a conversion into a 'lossy' file format, (I consider dynamically typed code versus statically typed code the analog thing to JPEG versus bitmaps) it would be easy to convert all datatypes into the datatypes of another language, thereby making it possible to exchange code between languages. Algorithms just being things that convert sets of data-objects into other sets of data-objects. Now if one would equate standardized code exchange between languages and within a language with macros then I guess there is nothing left for me to do but wait till a certain google bot comes knocking at my ip-address port 80 and transfers me to the google equivalent of Guantanamo. But the whole point of distinguishing macros from official language structures *is* standardization, as some other clever poster already pointed out, so it would be extremely unfair to equate trans-language standardized code exchange with the guerrilla type macro activities that are plaguing the Lisp community. Then there are some people who keep insisting they don't understand what I'm talking about until I simplify things enough to get them on-board, but then simply dismiss my ideas with 'you can already do that easily with this standard python construct'. This strategy was also eloquently refuted by some other poster, so I don't need to repeat it :-) I've gotten a lot of things to think about, so thanks all for your thoughts, but since this is getting way above my head I'll just wimp out and leave the rest of the thread to the experts! Regards, Anton -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
bruno at modulix wrote: > Anton Vredegoor wrote: >> bruno at modulix wrote: >> >>> I still don't get the point. >> >> Well, I've got to be careful here, lest I'd be associated with the >> terr.., eh, the childp..., eh the macro-enablers. >> >> The idea is to have a way to transform a Python (.py) module into XML >> and then do source code manipulations in XML-space using ElementTree. > > My my my... I'm not against the idea of dynamic source code > transformation, but for heaven's sake, *why* would one put XML in the > mix ??? If a good transform could "reveal" xml as "python's s-expression syntax", not only would source2source transform using xml t2t transform tools be facilitated, but generally speaking it would be easier for "xml retro-coded" python source to find its way through xml-enabled tools&chains. And incite developpers of such tools to consider python a better candidate (eg than it currently is) whenever the matter of scripting the tool comes up. -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
>> While the _result_ of a transformation might be a less efficient piece of >> code (e.g. introducing a lock around each call to enable concurrent >> access), the transformation itself is very - if not totally - static - > > really ? See below. > Nope, it's runned each time the module is loaded (with 'loaded' distinct > from 'imported') - which can make a real difference in some execution > models... I already mentioned that latency. If it for whatever reason really becomes important, it would be the best to cache the result of the transformation. Which would BTW eliminate any complexity driven runtime penalty - regardless of the tool used. So - loading time is _not_ an issue. And I spare you the premature optimization babble... :) >> So except from a start up latency, it has no impact. > > Having a high startup latency can be a problem in itself. See above. > But the problem may not be restricted to startup latency. If for example > you use a metaclasse and a function that *dynamically* creates new > classes using this metaclass, then both the class statement and the > metaclass code transformation will be executed on each call to this > function. This is an assumption I don't agree upon. The whole point of the OPs post was about creating DSLs or alter the syntax of python itself. All that to enhance expressiveness. But we are still talking about CODE here - things that get written by programmers. Even if that is piped through so many stages, it won't grow endlessly. Runtime (runtime meaning here not on a startup-phase, but constantly/later) feeding of something that generates new code - I wouldn't say that is unheard of, but I strongly doubt it occurs so often that it rules out tree transformations that don't try and squeeze the latest bit of performance out themselves. Which, BTW, would rule out python in itself as nothing beats runtime assembly generation BY assembly. Don't you think? > The whole point of a code transformation mechanism like the one Anton is > talking about is to be dynamic. Else one just needs a preprocessor... No, it is not the whole point. The point is "" The idea is that we now have a fast parser (ElementTree) with a reasonable 'API' and a data type (XML or JSON) that can be used as an intermediate form to store parsing trees. Especially statically typed little languages seem to be very swallow-able. Maybe I will be able to reimplement GFABasic (my first love computer language, although not my first relationship) someday, just for fun. """ No on-the-fly code generation here. He essentially wants lisp-style-macros with better parsing. Still a programming language. Not a data-monger. Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
Diez B. Roggisch wrote:
> bruno at modulix wrote:
>
>
>>Diez B. Roggisch wrote:
>>
>because lots of people know how to describe XML transformations, and
>there are plenty of tools that implement such transformations
>efficiently ?
Efficiently enough for dynamic (runtime) use ?
>>>
>>>
>>>Using XML-transformation for AST manipulation isn't my first choice
>>>either - yet efficiency concerns aren't really the point here - after
>>>all we're talking about generating code,
>>
>>I thought we were talking about *transforming* code - just like one uses
>>metaclasses to transform a class definition, or @decorators to transform
>>a function definition...
>
>
> Yes we were. So where does the runtime efficiency you mention come in to
> play?
class transformations via metaclasses and function wrapping does happen
at runtime - when the class or (decorated) def statements are eval'd.
This is not the same as having a distinct preprocessing phase that would
write a new .py file.
> While the _result_ of a transformation might be a less efficient piece of
> code (e.g. introducing a lock around each call to enable concurrent
> access), the transformation itself is very - if not totally - static -
really ?
> and
> usually only run once.
Nope, it's runned each time the module is loaded (with 'loaded' distinct
from 'imported') - which can make a real difference in some execution
models...
> So except from a start up latency, it has no impact.
Having a high startup latency can be a problem in itself.
But the problem may not be restricted to startup latency. If for example
you use a metaclasse and a function that *dynamically* creates new
classes using this metaclass, then both the class statement and the
metaclass code transformation will be executed on each call to this
function.
The whole point of a code transformation mechanism like the one Anton is
talking about is to be dynamic. Else one just needs a preprocessor...
> So if for whatever
> reason XSLT is someones favorite method of AST-transformation because it
> fits her mindset - perfect. As I said: it wouldn't be mine either, but I
> can't see your concerns about efficiency.
cf above.
> And XSLT certainly is suited for tree manipulation, so it might be that it
> would be good for e.g. recursivly stripping type annotations of some kind
> (think of e.g. type-verifying decorators that you want to get rid of for
> production.)
>
> Diez
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in '[EMAIL PROTECTED]'.split('@')])"
--
http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
bruno at modulix wrote: > Diez B. Roggisch wrote: because lots of people know how to describe XML transformations, and there are plenty of tools that implement such transformations efficiently ? >>> >>> >>> Efficiently enough for dynamic (runtime) use ? >> >> >> Using XML-transformation for AST manipulation isn't my first choice >> either - yet efficiency concerns aren't really the point here - after >> all we're talking about generating code, > > I thought we were talking about *transforming* code - just like one uses > metaclasses to transform a class definition, or @decorators to transform > a function definition... Yes we were. So where does the runtime efficiency you mention come in to play? While the _result_ of a transformation might be a less efficient piece of code (e.g. introducing a lock around each call to enable concurrent access), the transformation itself is very - if not totally - static - and usually only run once. So except from a start up latency, it has no impact. So if for whatever reason XSLT is someones favorite method of AST-transformation because it fits her mindset - perfect. As I said: it wouldn't be mine either, but I can't see your concerns about efficiency. And XSLT certainly is suited for tree manipulation, so it might be that it would be good for e.g. recursivly stripping type annotations of some kind (think of e.g. type-verifying decorators that you want to get rid of for production.) Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
Diez B. Roggisch wrote:
>>> because lots of people know how to describe XML transformations, and
>>> there are plenty of tools that implement such transformations
>>> efficiently ?
>>
>>
>> Efficiently enough for dynamic (runtime) use ?
>
>
> Using XML-transformation for AST manipulation isn't my first choice
> either - yet efficiency concerns aren't really the point here - after
> all we're talking about generating code,
I thought we were talking about *transforming* code - just like one uses
metaclasses to transform a class definition, or @decorators to transform
a function definition...
(snip)
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in '[EMAIL PROTECTED]'.split('@')])"
--
http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Fredrik Lundh wrote: > Kay Schluehr wrote: > > > If it is just a different kind of representation of common data > > structures > > but how do you know ? > > The semantics is specified by the syntax transformer so it is actually compile-time semantics relative to the base language Python . For any custom statement/expression ( expressed by a production rule / node in the parse-tree ) one or more target statements/expressions in standard Python are created. The specification of the with-statement in PEP 343 can be regarded as a good example of this definition practice. The with-statement is expanded to a "protocol" that can be expressed in Python 2.4. In a more general case this expansion might involve additional libraries e.g. ctypes or elementree. -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
Fredrik Lundh a écrit : > Laurent Pointal wrote: > The idea is to have a way to transform a Python (.py) module into XML and then do source code manipulations in XML-space using ElementTree. >>> >>> My my my... I'm not against the idea of dynamic source code >>> transformation, but for heaven's sake, *why* would one put XML in the >>> mix ??? > > because lots of people know how to describe XML transformations, and > there are plenty of tools that implement such transformations efficiently ? > >> Because its "à la mode", and is better for a commercial point of view, >> even if inefficient for this problem. > > why would XML be inefficient ? As a storage tool, its nice, and as you says, there are many tools to deal with. But as an internal representation for an AST, I certainly prefer an ad-hoc classes definitions with well defined members. Just have a wrapper between both representations. Laurent A+ -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Kay Schluehr wrote: > If it is just a different kind of representation of common data > structures but how do you know ? -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Ian Bicking wrote:
> > I don't use Django and I made this up quickly, so please don't pick on
> > subtleties.
> >
> > @Poll:
> > question: char length 200
> > pub_date('date published'): date
> >
> > @Choice:
> > poll -> Poll
> > choice: char length 200
> > votes: int
>
> That doesn't look that much better. How do you create it
> programmatically? I know how to pass a variable to
> CharField(maxlength=200); can I pass a variable to "char length 200"
> just as easily? Can I use **kw? Can I read it from a CSV file and
> construct the class that way? Maybe, but only by recreating all the
> native patterns that I can infer easily looking at the Django class.
If it is just a different kind of representation of common data
structures as in YAML the answer might be a translation of these
declarative blocks into dicts/lists ( or derivatives of those ) at
compile time. The underlying semantics would be that of an "implicitely
embedded DSL" ( there are quite a lot in Python ). Enabling code
generation would just make them more explicit. For example XML syntax
could be considered as an alternate surface syntax for elementrees. XML
elements in Python code might be translated to aequivalent elementree
annotation syntax at compile time.
Under this considerations "choice: char length 200" and
"CharField(maxlength = 200 )" are essentially the same thing. I guess
@Choice.choice would finally represented by a property.
Regards,
Kay
--
http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
Fredrik Lundh wrote: > because lots of people know how to describe XML transformations, and > there are plenty of tools that implement such transformations efficiently ? > > why would XML be inefficient ? XML Transformations (XSLT) would *certainly* be an overkill here. They've invented a whole new declarative programming language, and we certainly don't need that when we've got Python! :) XML by itself feels completely out of place in this context. What we need is, just a flexible, easy to manipulate, in-memory tree structure (AST) for the Python source. Regards Sreeram signature.asc Description: OpenPGP digital signature -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Ravi Teja wrote:
> Fredrik Lundh wrote:
> > Ravi Teja wrote:
> >
> > > Web frameworks, which seem to be the rage now in Python community could
> > > have benefited tremendously from Macro capabilities since they have a
> > > lot of boiler plate.
> >
> > they do? methinks you haven't done much web programming lately...
> >
> >
>
> You blogged on Django. Let's use that. Don't you think model creation
> in Django can be represented better, given that it is done often
> enough?
Actually, no, it's not done that much. Creating and managing tables
isn't something done lightly. It's essential to building a new
application, but (at least in my experience, in similar systems) the
database models stabalize early and you don't spend that much time with
them. Especially not with the DSL aspects. I add and remove methods
often, but I am loathe to add and remove columns.
Now, this might seem like I'm being pedantic, but in my experience lots
of seemingly obvious DSLs end up not being that obvious. XML
generation, for instance. It's nice to have a good syntax -- and you
can get a pretty good syntax in Python (e.g., HTMLGen, stan, etc). But
efforts that go further are generally misplaced, because it's actually
not a very hard or common thing to do, even when you are slinging
around lots of XML.
Or... maybe to be more specific, the hard work later on goes into
*code*. If you are enhancing your model, you do so with methods on the
model classes, and those methods don't effect the DSL, they are just
"code". You create some raw XML in the beginning, but quickly it's
just a matter of gluing those pieces together, using functions instead
of DSLs, and that's just "code".
> Let's take an example from the official tutorial
> from
> http://www.djangoproject.com/documentation/tutorial1/#creating-models
>
> class Poll(models.Model):
> question = models.CharField(maxlength=200)
> pub_date = models.DateTimeField('date published')
>
> class Choice(models.Model):
> poll = models.ForeignKey(Poll)
> choice = models.CharField(maxlength=200)
> votes = models.IntegerField()
>
> I don't use Django and I made this up quickly, so please don't pick on
> subtleties.
>
> @Poll:
> question: char length 200
> pub_date('date published'): date
>
> @Choice:
> poll -> Poll
> choice: char length 200
> votes: int
That doesn't look that much better. How do you create it
programmatically? I know how to pass a variable to
CharField(maxlength=200); can I pass a variable to "char length 200"
just as easily? Can I use **kw? Can I read it from a CSV file and
construct the class that way? Maybe, but only by recreating all the
native patterns that I can infer easily looking at the Django class.
> The following is my rationale. Annoted variables, symbols and code
> layout visually cue more efficiently to the object nature than do
> explicit text definitions. Of course, this is only sensible when there
> aren't too many of any of those. In that case, the cognitive cost of
> notation outweighs the representational cost of text.
Words are great. Python is light on symbols, and that is good. Python
is not perfect when it comes to expressing data structures (the more I
think about it, the more PEP 359 grows on me), but real DSLs are
questionable to me.
Even the Lisps stick to an incredibly homogenous syntax (far more
homogeneous than Python) to make macros feel familiar.
> Representational minimalism is troublesome in general code (ala Perl),
> but not so in a DSL where the context is constrained.
Constrained context is a step backward! How do you add methods? How
do you do looping? How do you write *code*? If you aren't going to
allow those things, then just make a parser and build the structure
from the file, and make it a DSL implemented entirely external to
Python. That's completely okay, though in my experience it's not very
satisfying for something like a model definition (see MiddleKit for an
example of an ORM that doesn't use Python code).
Ian
--
http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
>> because lots of people know how to describe XML transformations, and >> there are plenty of tools that implement such transformations efficiently ? > > Efficiently enough for dynamic (runtime) use ? Using XML-transformation for AST manipulation isn't my first choice either - yet efficiency concerns aren't really the point here - after all we're talking about generating code, which would be pretty useless if the work was to be done by the transformation instead of that very code generated ... So the question is: do XML/XSL give an advantage here? As I said - I personally don't think so, IMHO a standard reducer using a decent visitor is easy enough and works well. But your (or better Fredrik's) MMV. Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Kay Schluehr wrote: > Ravi Teja wrote: > > > People have however written various language interpreters (Scheme, > > Forth and yes, even Basic) in Python, just for kicks. Still does not > > make it a DSL language anymore than it makes C a DSL language. > > > > At present, the closest thing to writing a DSL in Python is Logix > > http://livelogix.net/logix/ > > Too bad though, the project is defunct and there has never been enough > > interest in it. > > You might be interested in EasyExtend: > > http://www.fiber-space.de/EasyExtend/doc/EE.html Your framework does look very interesting and might just be what I am looking for. Will give it a try. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
BJörn Lindqvist wrote: > > > > community has no interest in it. When I absolutely need macros, I will > > > > go elsewhere. > > I *like* 1..5 (ada, ruby) instead of range(5). If I had macros, I would > > have done it myself for *my* code. > I think this example more is a symptom of a childish need to get > things your way than of a deficiency in Python. I thought I had enough asterisks in there to indicate that it is a preference that I will not be defending on rational grounds. I had a better argument before it in the same post. But you had to choose only the trivial one to dismiss me as childish. Didn't you? :-) > BTW, range(5) = 0..4 in Ada and Ruby. My bad. I usually write range(1, 5 + 1) to get 1..5. I could write range(1, 6). But I would like to see the upper bound explicitly. Of course, I could write a function to wrap that up. > You said "when I absolutely need macros" but none of your examples > demonstrate any "absolute need." I can't see your point. Did you miss the word - *WHEN*? I don't need them absolutely now. And I know, that I won't get them here. And just so you don't misinterpret, I don't call that a "deficiency". Just a mismatch between the personal and the community mindset. BTW, the recent language changes - decorators, conditional expressions and with statements are not absolute either. That did not stop them from being welcome additions. -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
Fredrik Lundh wrote:
> Laurent Pointal wrote:
>> Bruno Desthuilliers wrote:
>>> Anton Vredegoor wrote:
The idea is to have a way to transform a Python (.py) module into XML
and then do source code manipulations in XML-space using ElementTree.
>
>>>
>
>>> My my my... I'm not against the idea of dynamic source code
>>> transformation, but for heaven's sake, *why* would one put XML in the
>>> mix ???
>
>
> because lots of people know how to describe XML transformations, and
> there are plenty of tools that implement such transformations efficiently ?
Efficiently enough for dynamic (runtime) use ?
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in '[EMAIL PROTECTED]'.split('@')])"
--
http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Ravi Teja wrote: > People have however written various language interpreters (Scheme, > Forth and yes, even Basic) in Python, just for kicks. Still does not > make it a DSL language anymore than it makes C a DSL language. > > At present, the closest thing to writing a DSL in Python is Logix > http://livelogix.net/logix/ > Too bad though, the project is defunct and there has never been enough > interest in it. You might be interested in EasyExtend: http://www.fiber-space.de/EasyExtend/doc/EE.html Unlike Logix there are no macros defined in application code and there are no runtime macro expansions. So extension language semantics is always fixed at compile time. > Personally, I would like to see macros in Python (actually Logix > succeeding is good enough). But I am no language designer and the > community has no interest in it. When I absolutely need macros, I will > go elsewhere. Although Logix was written in Python and compiled to CPython bytecodes it was a language on its own right: a Python / Lisp hybrid. Opposed to this EasyExtend is a Python framework for language extensions and not itself a language. A quite typical use case may not involve any new grammar rules or terminals, but just generates code. See the coverage fiber as an example. -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
Laurent Pointal wrote: >>> The idea is to have a way to transform a Python (.py) module into XML >>> and then do source code manipulations in XML-space using ElementTree. >> >> My my my... I'm not against the idea of dynamic source code >> transformation, but for heaven's sake, *why* would one put XML in the >> mix ??? because lots of people know how to describe XML transformations, and there are plenty of tools that implement such transformations efficiently ? > Because its "à la mode", and is better for a commercial point of view, > even if inefficient for this problem. why would XML be inefficient ? -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
bruno at modulix a écrit : > Anton Vredegoor wrote: >> bruno at modulix wrote: >> >>> I still don't get the point. >> >> Well, I've got to be careful here, lest I'd be associated with the >> terr.., eh, the childp..., eh the macro-enablers. >> >> The idea is to have a way to transform a Python (.py) module into XML >> and then do source code manipulations in XML-space using ElementTree. > > My my my... I'm not against the idea of dynamic source code > transformation, but for heaven's sake, *why* would one put XML in the > mix ??? Because its "à la mode", and is better for a commercial point of view, even if inefficient for this problem. -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
Anton Vredegoor wrote:
> bruno at modulix wrote:
>
>> I still don't get the point.
>
>
> Well, I've got to be careful here, lest I'd be associated with the
> terr.., eh, the childp..., eh the macro-enablers.
>
> The idea is to have a way to transform a Python (.py) module into XML
> and then do source code manipulations in XML-space using ElementTree.
My my my... I'm not against the idea of dynamic source code
transformation, but for heaven's sake, *why* would one put XML in the
mix ???
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in '[EMAIL PROTECTED]'.split('@')])"
--
http://mail.python.org/mailman/listinfo/python-list
Re: code is data
John Roth wrote: > I saw the "make" statement as a breath of fresh air. > Then it got shot down for what were, to me, totally > trivial reasons. Which reasons? I as I recall, Guido cut it out without giving any reason. Of course Guido has the right to do so, but it is not respectful of all the work people like Steven Bethard and others did :-( Michele Simionato -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
bruno at modulix wrote: > I still don't get the point. Well, I've got to be careful here, lest I'd be associated with the terr.., eh, the childp..., eh the macro-enablers. The idea is to have a way to transform a Python (.py) module into XML and then do source code manipulations in XML-space using ElementTree. But rest assured, there is no such module, nor will we ever need it for anything. Anton "use cases are for the faint-hearted" -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Ravi Teja wrote:
(snip)
> Annoted variables, symbols and code
> layout visually cue more efficiently to the object nature than do
> explicit text definitions. Of course, this is only sensible when there
> aren't too many of any of those. In that case, the cognitive cost of
> notation outweighs the representational cost of text.
>
> Representational minimalism is troublesome in general code (ala Perl),
> but not so in a DSL where the context is constrained.
This still impose the need to learn a new language.
> I would also like to symbolize field types since they occur so commonly
> in a definition file and only a few of them are commonly used. I admit
> though that I find the code below a bit visually jarring and I might
> use something else. But it serves to illustrate the point. I chose the
> respective symbols based on their colloquial use and association with
> the field types.
>
> @Poll:
> $question: length 200
> %pub_date('date published')
>
> @Choice:
> poll -> Poll
> $choice: length 200
> #votes
Yuck.
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in '[EMAIL PROTECTED]'.split('@')])"
--
http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Ravi Teja wrote:
> BJörn Lindqvist wrote:
>
>>>Personally, I would like to see macros in Python (actually Logix
>>>succeeding is good enough). But I am no language designer and the
>>>community has no interest in it. When I absolutely need macros, I will
>>>go elsewhere.
>>
>>One must wonder, when is that? When do you absolutely need macros?
>
>
> Whenever there is significant boiler plate code that functions and
> classes cannot eliminate alone.
> Whenever there is a more elegant way to express your code.
>
> Python 2.5 introduced conditional expressions and with statement. With
> macros, one would not have to wait for the language team to implement
> them. More so for features which only a small part of the community has
> an interest in.
>
> I *like* 1..5 (ada, ruby) instead of range(5). If I had macros, I would
> have done it myself for *my* code.
And that's the downside with macros - and with anything that's not
officially part of the language or it's standard lib : everybody
implements it it's own way, and you end up with dozens non-standard ways
of doing the same thing.
Not to say this is absolutely bad, but there's a balance to be found
here. One could do function decorators long before we had official
syntactic sugar for it, but it only started to be a common idiom with
the @decorator syntax. Python 2.5 introduces a 'partial' type, that is
quite easy to implement with 2.4 (and probably with older versions too),
but having it in the builtins or standard lib means it will become the
standard way to do it - no need to deal with half a dozen half-backed
implementations of it no more.
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in '[EMAIL PROTECTED]'.split('@')])"
--
http://mail.python.org/mailman/listinfo/python-list
Re: code is data
BJörn Lindqvist wrote:
>> Personally, I would like to see macros in Python (actually Logix
>> succeeding is good enough). But I am no language designer and the
>> community has no interest in it. When I absolutely need macros, I will
>> go elsewhere.
>
>
> One must wonder, when is that? When do you absolutely need macros?
>
One must wonder, when do you absolutely need HOFs, closures, OO,
functions, or even structured programming. All we 'absolutely' need is
tests and gotos... (and love, of course !-).
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in '[EMAIL PROTECTED]'.split('@')])"
--
http://mail.python.org/mailman/listinfo/python-list
Re: [OT] code is data
Anton Vredegoor wrote:
> With the inclusion of ElementTree (an XML-parser) in Python25 and recent
> developments concerning JSON (a very Pythonesque but somewhat limited
> XML notation scheme, let's call it statically typed XML)
JSON stands for JavaScript Object Notation, and has *nothing* to do with
XML - except for the fact that it's more and more used instead of XML
for AJAX stuff.
> Python seems to
> have reached a stage where it now seems to be possible to completely
> swallow lesser languages code, modify it, and spit out new source code
> targeting the original language the code was written in, or even make a
> translation to other languages.
If you mean "parsing source in a given format and outputting another -
modified or not - representation, in the same or another format", Python
as always been able to do so.
> The idea is that we now have a fast parser (ElementTree) with a
> reasonable 'API' and a data type (XML or JSON) that can be used as an
> intermediate form to store parsing trees. Especially statically typed
> little languages seem to be very swallow-able. Maybe I will be able to
> reimplement GFABasic (my first love computer language, although not my
> first relationship) someday, just for fun.
>
> Then there are things like cTypes (calling functions from native DLL's)
> and PyPy (implementing Python in Python).
>
> All this taken together, to me it starts looking like we're now entering
> a territory that traditionally was exclusively in the Lisp domain.
Sorry, but I just don't get the point. Parsing, working with trees and
calling native code are in no way "exclusively in the Lisp domain".
> Yes, Python had eval and exec for a long time already, and metatypes and
> generators are having some strange unexplored possibilities too, but the
> day will come soon (and at last when PyPy is reaching execution speeds
> close to cPython) where Python will be able to swallow smaller
> languages, and finally it will be able to swallow its own tail, like
> Lisp but then more powerful
I'm afraid Python is still very far from Lisp - and will never get there
(FWIW, this seems not to be the goal anyway).
> (because of the widely used standard data
> types and the code exchange between languages that that makes possible).
I still don't get the point.
> Your thoughts please.
>
> Anton
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in '[EMAIL PROTECTED]'.split('@')])"
--
http://mail.python.org/mailman/listinfo/python-list
Re: code is data
> > > community has no interest in it. When I absolutely need macros, I will > > > go elsewhere. > I *like* 1..5 (ada, ruby) instead of range(5). If I had macros, I would > have done it myself for *my* code. I think this example more is a symptom of a childish need to get things your way than of a deficiency in Python. BTW, range(5) = 0..4 in Ada and Ruby. You said "when I absolutely need macros" but none of your examples demonstrate any "absolute need." I can't see your point. -- mvh Björn -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Ravi Teja <[EMAIL PROTECTED]> said: > I *like* 1..5 (ada, ruby) instead of range(5). If I had macros, I would > have done it myself for *my* code. You can write your own preprocessor to handle things like that. -- Roberto Bonvallet -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Fredrik Lundh wrote:
> Ravi Teja wrote:
>
> > Web frameworks, which seem to be the rage now in Python community could
> > have benefited tremendously from Macro capabilities since they have a
> > lot of boiler plate.
>
> they do? methinks you haven't done much web programming lately...
>
>
You blogged on Django. Let's use that. Don't you think model creation
in Django can be represented better, given that it is done often
enough?
Let's take an example from the official tutorial
from
http://www.djangoproject.com/documentation/tutorial1/#creating-models
class Poll(models.Model):
question = models.CharField(maxlength=200)
pub_date = models.DateTimeField('date published')
class Choice(models.Model):
poll = models.ForeignKey(Poll)
choice = models.CharField(maxlength=200)
votes = models.IntegerField()
I don't use Django and I made this up quickly, so please don't pick on
subtleties.
@Poll:
question: char length 200
pub_date('date published'): date
@Choice:
poll -> Poll
choice: char length 200
votes: int
The following is my rationale. Annoted variables, symbols and code
layout visually cue more efficiently to the object nature than do
explicit text definitions. Of course, this is only sensible when there
aren't too many of any of those. In that case, the cognitive cost of
notation outweighs the representational cost of text.
Representational minimalism is troublesome in general code (ala Perl),
but not so in a DSL where the context is constrained.
I would also like to symbolize field types since they occur so commonly
in a definition file and only a few of them are commonly used. I admit
though that I find the code below a bit visually jarring and I might
use something else. But it serves to illustrate the point. I chose the
respective symbols based on their colloquial use and association with
the field types.
@Poll:
$question: length 200
%pub_date('date published')
@Choice:
poll -> Poll
$choice: length 200
#votes
Since you are on thread and are a prominent and involved member of the
Python community, I would like it if you (or any such other) can
provide feedback on the rest of my previous post rather than be
dismissive by just a small portion of it. Perhaps, that will give me
some insight how these language design decisions are rationally made (I
am not strictly a programmer by profession, much less a language
designer).
--
http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Ravi Teja wrote: > Web frameworks, which seem to be the rage now in Python community could > have benefited tremendously from Macro capabilities since they have a > lot of boiler plate. they do? methinks you haven't done much web programming lately... -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Paddy wrote: > Ravi Teja wrote: > > BJörn Lindqvist wrote: > > > > Personally, I would like to see macros in Python (actually Logix > > > > succeeding is good enough). But I am no language designer and the > > > > community has no interest in it. When I absolutely need macros, I will > > > > go elsewhere. > > > > > > One must wonder, when is that? When do you absolutely need macros? > > > > Whenever there is significant boiler plate code that functions and > > classes cannot eliminate alone. > > Whenever there is a more elegant way to express your code. > > > > Me, I am torn. I should now better. I have listened to the arguments > against Macros in Python and the ones that struck home were the > argument for maintainability: > Without macros, Python is Python. Statements do what you expect. Yes! I heard those arguments too. And I am not convinced. Static language programmer: Lack of static typing removes the necessary safeguards. The code is more error prone. Objects have behavior that is not obvious. Dynamic language programmer: Really? I don't seem to have any more bugs than in my statically typed code. And my code is compact and reads better. I don't want to go back. No to macros proponent: Macros introduce a lot of potential for abuse. Code will be worse to read than Perl. Macros proponent: Really? We have been doing macros for decades. We all think our code is better for macros, not worse. We are not going back. I just don't get it. Don't we often invoke the "We are all adults here" argument. Writing a macro is not as simple as writing a function. Sort of like metaclasses. Many will stay off them. Those that really need them will walk that extra mile. Don't we all believe that "Simple should be possible. Complex should be doable" > And the argument against DSLs altogether: > Make Python your DSL! If you design your own DSL before long you start > to embellish it with more statements or data types and before long it > becomes complex. If you used Python from the beginning then you would > have a community for support. Python has a low cognitive overhead. But it not a DSL by definition. No language can be. The idea is that when the domain changes, a DSL should be driven by the new domain as warranted. In other words, driven "by the problem, not the tool". I don't want "a DSL". I want a language that allows me to make "my DSL" based on it. That means I don't loose the community connection. I can still use all the rich libraries in my DSL. I like Python for its indentation syntax, sensible semantics and readability. I invested a lot of time in Python. After much language hopping, I settled with Python. I like the community and the code base available for it. The libraries just seem to be designed at the right level of abstraction for me (as opposed to say, Java). When I need to do something, I know where to go. But all this ties me to the language tightly that I cannot change. > I know the arguments, but every once in a while I think if only I could > craft my own ??? statement or My thoughts exactly. Web frameworks, which seem to be the rage now in Python community could have benefited tremendously from Macro capabilities since they have a lot of boiler plate. -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Ravi Teja wrote: > BJörn Lindqvist wrote: > > > Personally, I would like to see macros in Python (actually Logix > > > succeeding is good enough). But I am no language designer and the > > > community has no interest in it. When I absolutely need macros, I will > > > go elsewhere. > > > > One must wonder, when is that? When do you absolutely need macros? > > Whenever there is significant boiler plate code that functions and > classes cannot eliminate alone. > Whenever there is a more elegant way to express your code. > Me, I am torn. I should now better. I have listened to the arguments against Macros in Python and the ones that struck home were the argument for maintainability: Without macros, Python is Python. Statements do what you expect. And the argument against DSLs altogether: Make Python your DSL! If you design your own DSL before long you start to embellish it with more statements or datatypes and before long it becomes complex. If you used Python from the beginning then you would have a community for support. I know the arguments, but every once in a while I think if only I could craft my own ??? statement or Don't go their Paddy. ;-) -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
BJörn Lindqvist wrote: > > Personally, I would like to see macros in Python (actually Logix > > succeeding is good enough). But I am no language designer and the > > community has no interest in it. When I absolutely need macros, I will > > go elsewhere. > > One must wonder, when is that? When do you absolutely need macros? Whenever there is significant boiler plate code that functions and classes cannot eliminate alone. Whenever there is a more elegant way to express your code. Python 2.5 introduced conditional expressions and with statement. With macros, one would not have to wait for the language team to implement them. More so for features which only a small part of the community has an interest in. I *like* 1..5 (ada, ruby) instead of range(5). If I had macros, I would have done it myself for *my* code. I would like special behaviour code blocks in my programs, for say DBC (I am aware of the work arounds). -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
> Personally, I would like to see macros in Python (actually Logix > succeeding is good enough). But I am no language designer and the > community has no interest in it. When I absolutely need macros, I will > go elsewhere. One must wonder, when is that? When do you absolutely need macros? -- mvh Björn -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Anton Vredegoor wrote: > With the inclusion of ElementTree (an XML-parser) in Python25 and recent > developments concerning JSON (a very Pythonesque but somewhat limited > XML notation scheme, let's call it statically typed XML) Python seems to > have reached a stage where it now seems to be possible to completely > swallow lesser languages code, modify it, and spit out new source code > targeting the original language the code was written in, or even make a > translation to other languages. > > The idea is that we now have a fast parser (ElementTree) with a > reasonable 'API' and a data type (XML or JSON) that can be used as an > intermediate form to store parsing trees. Especially statically typed > little languages seem to be very swallow-able. Maybe I will be able to > reimplement GFABasic (my first love computer language, although not my > first relationship) someday, just for fun. > > Then there are things like cTypes (calling functions from native DLL's) > and PyPy (implementing Python in Python). > > All this taken together, to me it starts looking like we're now entering > a territory that traditionally was exclusively in the Lisp domain. > > Yes, Python had eval and exec for a long time already, and metatypes and > generators are having some strange unexplored possibilities too, but the > day will come soon (and at last when PyPy is reaching execution speeds > close to cPython) where Python will be able to swallow smaller > languages, and finally it will be able to swallow its own tail, like > Lisp but then more powerful (because of the widely used standard data > types and the code exchange between languages that that makes possible). > > Your thoughts please. I don't share your optimism at all. Most of the things you mentioned have existed for long. Just because some of them are now included in the standard library isn't going to change things drastically. Installing them earlier was never hard at all. People like to call everything with the lightest semblence, a DSL. That gives the feel that the language is more powerful. Ruby people do it all the time. Python cannot be called a DSL language until, creating them is a natural language feature (like Lisp). And that does not seem to be happening anytime soon. Boo for example allows you to write new constructs with it's AST library. It still cannot be called a DSL "language". People have however written various language interpreters (Scheme, Forth and yes, even Basic) in Python, just for kicks. Still does not make it a DSL language anymore than it makes C a DSL language. At present, the closest thing to writing a DSL in Python is Logix http://livelogix.net/logix/ Too bad though, the project is defunct and there has never been enough interest in it. Personally, I would like to see macros in Python (actually Logix succeeding is good enough). But I am no language designer and the community has no interest in it. When I absolutely need macros, I will go elsewhere. -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Paddy wrote: > Anton Vredegoor wrote: > > With the inclusion of ElementTree (an XML-parser) in Python25 and recent > > developments concerning JSON (a very Pythonesque but somewhat limited > > XML notation scheme, let's call it statically typed XML) > > > > Your thoughts please. > > > > Anton > > Hi Anton. > If you mean this JSON: http://www.json.org/example.html > then I'd just point out that JSON isn't XML-like at all. In fact the > examples look like valid Python nested dictionaries. It is the same JSON. JSON is typically seen as a human friendly replacement for some of the functions that XML is otherwise used for, where the full blown XML spec is an overkill and JSON does not need complicated parsers in some common languages because it can express hierarchical data just like XML. -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Anton Vredegoor wrote: > With the inclusion of ElementTree (an XML-parser) in Python25 and recent > developments concerning JSON (a very Pythonesque but somewhat limited > XML notation scheme, let's call it statically typed XML) > Your thoughts please. > > Anton Hi Anton. If you mean this JSON: http://www.json.org/example.html then I'd just point out that JSON isn't XML-like at all. In fact the examples look like valid Python nested dictionaries. - Pad. P.S. This is good too: http://en.wikipedia.org/wiki/JSON -- http://mail.python.org/mailman/listinfo/python-list
Re: code is data
Anton Vredegoor wrote: > With the inclusion of ElementTree (an XML-parser) in Python25 and recent > developments concerning JSON (a very Pythonesque but somewhat limited > XML notation scheme, let's call it statically typed XML) Python seems to > have reached a stage where it now seems to be possible to completely > swallow lesser languages code, modify it, and spit out new source code > targeting the original language the code was written in, or even make a > translation to other languages. When I heard of the new AST based compiler, I thought it would finally be possible to extend things cleanly. Then I learned that they weren't going to allow modification of the AST. There are a lot of things you can't do in source if the language doesn't allow it. Python is pretty good compared to other languages, but it's still not possible to create new control structures with short circuit semantics. And that's one example. I saw the "make" statement as a breath of fresh air. Then it got shot down for what were, to me, totally trivial reasons. That's a second one. Sigh. John Roth > > > Your thoughts please. > > Anton -- http://mail.python.org/mailman/listinfo/python-list
[OT] code is data
With the inclusion of ElementTree (an XML-parser) in Python25 and recent developments concerning JSON (a very Pythonesque but somewhat limited XML notation scheme, let's call it statically typed XML) Python seems to have reached a stage where it now seems to be possible to completely swallow lesser languages code, modify it, and spit out new source code targeting the original language the code was written in, or even make a translation to other languages. The idea is that we now have a fast parser (ElementTree) with a reasonable 'API' and a data type (XML or JSON) that can be used as an intermediate form to store parsing trees. Especially statically typed little languages seem to be very swallow-able. Maybe I will be able to reimplement GFABasic (my first love computer language, although not my first relationship) someday, just for fun. Then there are things like cTypes (calling functions from native DLL's) and PyPy (implementing Python in Python). All this taken together, to me it starts looking like we're now entering a territory that traditionally was exclusively in the Lisp domain. Yes, Python had eval and exec for a long time already, and metatypes and generators are having some strange unexplored possibilities too, but the day will come soon (and at last when PyPy is reaching execution speeds close to cPython) where Python will be able to swallow smaller languages, and finally it will be able to swallow its own tail, like Lisp but then more powerful (because of the widely used standard data types and the code exchange between languages that that makes possible). Your thoughts please. Anton -- http://mail.python.org/mailman/listinfo/python-list
