Concat, Cows Basil

2004-08-24 Thread Sam Phillips
/lurk
Hello,
I am currently in the early stages of writing a (parrot targeting) 
compiler for an interpreted language I have been working on/with for the 
past 4 years.

Background:
The language (Basil) is basically a functional (ish) language with a 
C-like syntax that has an implicit concatenation (or list join for 
efficiency) between each statement. It kinda looks like slightly funny 
perl with tags in.

Thus a large tree of strings is efficiently filtered and a big chunk of 
text is thrown out at the end. It is used for creating very flexible web 
templates.

The original Basil was created for a company I worked for who wrote 
everything in C despite doing only web-based work. Perl and PHP were 
banned. So I wrote a scripting layer that would be handed a symbol table 
and return a huge chunk of formatted HTML. I guess these days most folks 
would use XSLT for this.

However, I still find XSLT a closed-book and would love to see Basil 
running on Parrot, being called from a Perl Web Application to spit out 
formatted HTML. Especially as you could link the two lots of code as one 
and use the usual calling convention to call the Basil as if it were a 
standard Perl sub (or Python, etc). Plus Basil has of course grown over 
the years to be an object-oriented, stand-alone CGI environment that is 
very good at one thing - making web pages - and I find it hard to work 
without it.

Problem:
1) To create an efficient implementation of Basil I'll need a big 
PerlArray that is formed by expressions that create PerlStrings or more 
PerlArrays. At the end the tree of PerlStrings and PerlArrays is 
iterated over and concatenated together and returned (or just printed 
out). The PerlArray is just a way of avoiding even more concats plus it 
allows you to build list stuctures to iterate over.. By using the 
standard Perl ones I'm hoping to be easy-to-call plus I get the added 
bonuses of stretchy arrays, int-to-string converstion etc.

So my main question is this: Are PerlArrays and PerlStrings an efficient 
way to go with this? - how much copying goes on within a string 
concatenation? Iterator? etc ( ie does using COW prevent a deep copy 
every time I make a concatenation).

I tried to make sense of PerlString concatenation and they seem to use 
string_concat which in turn has two mem_sys_memcopys doing the actual 
concatenation.

2) If concat isn't the way forward then push probably is (I could just 
hand back a tree and iterator (as a closure err do I mean continuation?) 
to the calling function). I take it there are no hidden caveats in using 
push on a PerlArray?

3) I take it a proper iterator is far more efficient than a loop with a 
shift in it? eg:-

.sub _printall
  .param PerlArray xs
  .local PerlString s
  .local int t
 
  loop:

 unless xs, endloop
 shift s, xs
 typeof t, s
 if t == .PerlArray goto recurse
 print s
 branch loop
  recurse:
 _printall(s)
 branch loop
  endloop:
  .pcc_begin_return
  .pcc_end_return
.end
4) why doesn't '.' work on a PerlString in IMC? :-)
5) I guess the way-forward for the future is a custom class of very-lazy 
string-lists that can be concatenated, split, iterated, searched, sorted 
etc by always doing the least amount of copying they can get away with. 
But I'll have to get my head round Basil OOP first (and to be honest, 
PerlArrays  PerlStrings have 90% of what I need if they too are 
very-lazy). Would you agree?

I may of course be just plain getting things wrong (I still can't get 
iterators to work properly).. just trying to get my head round it all.. 
so feel free to humiliate me (or indeed ignore me)..

Anyway - I'm having fun. Thanks for all your hard work so far It's most 
appreciated.

Any flames, advice  encouragement would be gratefully accepted..
Cheers,
Sambeau
[EMAIL PROTECTED]
lurk


Re: Concat, Cows Basil

2004-08-24 Thread Leopold Toetsch
Sam Phillips [EMAIL PROTECTED] wrote:

 Problem:
 1) To create an efficient implementation of Basil I'll need a big
 PerlArray that is formed by expressions that create PerlStrings or more
 PerlArrays.

How big is big?

 So my main question is this: Are PerlArrays and PerlStrings an efficient
 way to go with this? - how much copying goes on within a string
 concatenation?

There ar basically 2 forms of concat, one inplace and the 2nd creating a
new string. The former is much more efficient the more you are appending
small items to the first string.

 ... Iterator?

Iteration doesn't involve any copying.

 ... etc ( ie does using COW prevent a deep copy
 every time I make a concatenation).

COW doesn't help in the case of concat. COW is mainly useful for
extracting substrings.

 I tried to make sense of PerlString concatenation and they seem to use
 string_concat which in turn has two mem_sys_memcopys doing the actual
 concatenation.

Well, that depends. Inplace needs one only.

 2) If concat isn't the way forward then push probably is (I could just
 hand back a tree and iterator (as a closure err do I mean continuation?)
 to the calling function). I take it there are no hidden caveats in using
 push on a PerlArray?

Push is fine.

 3) I take it a proper iterator is far more efficient than a loop with a
 shift in it?

Yep.

 4) why doesn't '.' work on a PerlString in IMC? :-)

It works - at least partially - inplcace is missing ;)
$ cat c.pir
  .sub main
$P0 = new PerlString
$P1 = new PerlString
$P2 = new PerlString
$P0 = a
$P1 = b
$P2 = $P0 . $P1 # note the space before the dot
print_item $P2
# $P0 .=  $P1
# print_item $P0
print_newline
end
  .end
$ parrot c.pir
ab

 5) I guess the way-forward for the future is a custom class of very-lazy
 string-lists that can be concatenated, split, iterated, searched, sorted
 etc

You might have a look at cord which can be found e.g. in the gcc sources
under the boehm-gc subdirectory.

 Sambeau

leo