[julia-users] Re: Julia and Object-Oriented Programming
Swift has an interesting take on composition via protocols (note, it also
has the concept of single inheritance):
https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swift_Programming_Language/Protocols.html
it avoids copying anything into the child class, but rather sets up both
the variables and methods that a child class should support. How the
variable is actually implemented in the class is up to the class (which I
think is the nice part). Thus you can do:
protocol Foo {
var x:Int { get set } // x must evaluate to an Int, and can be both 'get'
and 'set'
}
class Bar: Foo {
var x:Int // treat it as a simple Int
}
class Baz: Foo {
var x_latent:Int
// x is a 'functional' variable where we explicitly define how it is
get'd and set'd
var x:Int {
get {
return x_latent/2
}
set {
x_latent = 2*x
}
}
While it does involve rewriting code (which I don't love), it importantly
protects the developer from making mistakes (i.e. it will complain that `x`
is not defined, and will force `x` to be of type Int).
Swift also has the concept of generics. You can make Protocols generic, so
in theory you could make X of any type. Unfortunately, there appears to be
a gap in the language (like Julia, it is still an evolving language), and
making variables in the protocol appears to create issues down the line.
Re: [julia-users] Julia and Object-Oriented Programming
An OO approach is really just specifying an interface in a formal manner.
The second you write any type of interface, you always risk making a choice
that will haunt you down the road. I don't see the difference between:
class Foo {
float getX() { ... }
float getY() { ... }
}
and:
type Foo { ... }
function getX(f::Foo) -> float { ... }
function getY(f::Foo) -> float { ... }
except that an instance of `foo` is being passed implicitly in the first
case. To that extent, I would say Julia OO (i.e. on the dispatching). Where
it is not OO is in the layout of the data.
It is true that with a more OO language, because of its formalism, you run
the risk of declaring variable types in a parent class that might not be
correct in the child classes, so some planning is required. However, what
you gain is better code in the long run. For example, I might have:
abstract Baz
type Foo <: Baz {
x::Float
}
type Bar <: Baz {
x::Float
}
and an interface:
getX{T <: Baz}(b::T) -> Float = b.x
which works fine, except maybe someone else comes along and writes:
type Blob <: Baz {
myX::Float
}
Now to fix your interface, you have to write a separate `getX` for `Blob`.
This might not seem like a big deal, except you might not catch this issue
until run time (I don't think there is a way a static-checker could
identify the problem). Imagine a large library or base of code, with many
people working on the code, and you suddenly are exposed to a large number
of issues.
This is why people advocate OOP. In the first definition of `Foo`, I know
it will always have an `x`, and I can easily see the interface necessary to
access it.
So, yes, an OO usually requires more work up-front. If Julia is meant to be
purely scientific language which requires a code rewrite for real-world
use, then I won't argue with not having a way to compose the data. However,
in any project of medium to large complexity, it will make life much more
difficult not having that capability.
On Thursday, October 22, 2015 at 4:00:36 PM UTC-4, vav...@uwaterloo.ca
wrote:
>
> One way to build a code-base that everyone can share is to specify
> interfaces to datatypes as in methods in C++ base classes. Another way is
> for each individual to write his/her own code, and then read everyone
> else's code, and then meet at coffee shops and talk on the phone to get
> things to work together. I am claiming that the second way is better for
> scientific software than the first way because scientific software is more
> open-ended than, say, systems software. I am claiming that specifying
> interfaces from the outset can lead to prematurely locking in design
> decisions, and that I have seen examples of this in the real world. Keep
> in mind that Julia is being touted as a suitable language for both the
> initial and the later phases of scientific software development.
>
> In the event that Julia adds new OO or other interface-specifying
> features, obviously it would be possible for a team to ignore them, at
> least initially. But I have also seen in previous projects that there is a
> tendency for people (including me) to jump onto the fanciest possible
> solution offered by the programming language to solve a software design
> problem.
>
> -- Steve Vavasis
>
>
> On Thursday, October 22, 2015 at 12:02:21 PM UTC-4, g wrote:
>>
>> What is the other option here? It seemed like with the OO/Julia way you
>> are complaining about you at least have working (but slow) code handling
>> your new polynomial type. In a case where your new type doesn't work with
>> "obtainCoefficient", it won't
>> work with any of your other code either. You would just have no working
>> code with your new polynomial type. How is that better?
>>
>> But the next week, someone asks whether you can handle polynomials
>>> specified as p(x)=det(A-x*B), where A and B are n-by-n matrices. For
>>> polynomials in this format, "obtainCoefficient" is expensive and would not
>>> be regarded as a simple "getter" operation. If many people had already
>>> written functions invoking the 'obtainCoefficient' method, then you would
>>> be stuck. You would retrospectively realize that the obtainCoefficient
>>> member function was not a good idea. This example is typical of open-ended
>>> scientific software projects.
>>>
Re: [julia-users] Julia and Object-Oriented Programming
Ah, okay, that is different then. What's the advantage of creating a new
method versus copying the fields? I would imagine there is a penalty with
each deference you have to follow in order to make that work.
I'm not familiar with Scala, so sorry if I'm telling you something you
> know, but in go the member variables and methods are not copied over. The
> Unicycle struct in go does not itself have those fields, the fields of the
> struct do. In other words, defining
>
> type Unicycle struct {
> Frame
> Wheel
> Seat
> }
>
> Is exactly like declaring
>
> type Unicycle struct {
>Frame Frame
>Wheel Wheel
>SeatSeat
> }
>
> except that in the former case one can call unicycle.Cushiness() , and in
> the latter one must call unicycle.Seat.Cushiness(). In particular, if you
> swap out one Frame in the unicycle for a different Frame, the field values
> change (the methods won't, since methods are with a type).
>
Re: [julia-users] Julia and Object-Oriented Programming
I think this is generally in-line with what I've been advocating for, and
not that different from Scala's mix-ins:
trait Wheel {
// ...
}
trait Frame {
// ...
}
class Unicycle extends trait Wheel, Frame {
// ...
}
is essentially doing the same thing (i.e. copying the member variables and
methods over).
As Stefan mentioned, Go is not traditional OO. This "composition-based"
> approach is part of Go's OO approach. In go, a type can have named fields
> of different types (like most OO languages), but also has "embedding". For
> example, if one declared
>
> type Unicycle struct {
> Wheel
> Frame
> Seat
> }
>
> then you could access the individual fields directly. So, if "u' is of
> type Unicycle, then you could access/set u.Cushiness directly. Similarly,
> if the embedded types had methods, i.e. frame.Color(), they can be called
> directly through unicycle, u.Color(). This is similar to, but distinct
> from, inheritance (the difference is that the Unicycle type doesn't
> actually have these fields, just the ability to call them easily, which is
> significant because of interfaces). In the case of clashes, the program
> must specify which is meant, so if both Seat and Frame had a Color()
> method, the user would have to specify u.Frame.Color vs. u.Seat.Color. In
> Go, this is handled by the compiler, and would need to be handled somewhere
> in the Julia runtime/REPL. It's a very nice paradigm, but would have to be
> made to fit within the broader Julia context.
>
>
>
>
[julia-users] Re: Julia and Object-Oriented Programming
As for class composition, this is how Scala does it:
http://www.scala-lang.org/old/node/117
I've previously suggested introducing a `trait` label as to keep type and
dispatch information separate. Thus you could do:
trait A
x::Int64
y::Int64
end
trait B
z::Int64
end
type foo(A, B) <: bar
# ...
end
It would then be natural to hang interfaces on traits (thus, instead of
assuming a variable exists, you could specify an interface applies to a
trait)
# only classes that mix-in trait B can call this function
get_x{C <: B}(b::C) = b.x
get_y{C <: B}(b::C) = b.y
That's my $0.02.
On Tuesday, October 20, 2015 at 2:42:37 PM UTC-4, ssarkaray...@gmail.com
wrote:
>
> It is a suggestion. There is no project. If some of the experts agree on
> such a combined language in this group, a combined grammar
> can be created and ScalaJulia functional language can be implemented
> through a project.
>
> We need some exchange of ideas. Such a combined language has implication
> for Spark, Hadoop, Mlib and many other things to be integrated
> faster with Julia.
>
> SS
>
>
> On Tuesday, October 20, 2015 at 4:40:23 AM UTC-7, Sisyphuss wrote:
>>
>> I'm sorry, but I didn't find the ScalaJulia project as you mentioned.
>>
>>
>> On Monday, October 19, 2015 at 7:39:04 PM UTC+2, ssarkaray...@gmail.com
>> wrote:
>>>
>>> I am confident that a new OOP language *ScalaJulia *is possible
>>> integrating syntax of both functional languages Scala and Julia.
>>> Thanks,
>>> SS
>>>
>>> On Sunday, October 18, 2015 at 5:41:58 AM UTC-7, Sisyphuss wrote:
When I'm learning Julia, I am always thinking what is the correct way
to do OOP in this language. It seems to me that what I learned in C++ does
not apply in Julia.
It took me long to realize that the equivalent of Class of C++ in Julia
is not Type, but Module. Module is the basic function unit in Julia.
Thus, a Class in Julia is like
module ClassName # class Name {
using# include<> // should be outside
import # include<>
export function # public function;
var = 1 # private static var;
end # }
This provides the same structure as C++.
However, this design has two issues:
1) The visit control is not as fine-grained as in C++, the
encapsulation is not strict;
2) Variables at the top level of a module are global variables.
These two points are closely correlated. If we let module have private
variables, then they are not too different from local variables, ans thus
can be type inferred.
I think this is a natural way to do OOP with Julia.
Re: [julia-users] Re: Julia and Object-Oriented Programming
I think I agree with 90% of what you wrote. I also don't know if anyone is arguing about single or multiple dispatch. Also, while Java and Borland's Object Pascal claimed a strong OO paradigm, C++ does does not (if you read Bjarne's design guide, he felt strongly that both OOP and functional were important). I'll note too, Java has slowly been moving towards a multi-paradigm approach by including Lambdas in the latest version. Two of the languages you list below, Swift and Scala, also mix OO and functional programming. While Scala has a huge push for functional programming, it also provides the necessary constructs to talk about inheritance of data (while it doesn't provide multi-inheritance, it does have mix-ins). I really like Julia's multi-dispatch approach, and have made great use of it in my own libraries. I don't think there should be any reason that classes should have to "own" the methods. However, what it currently doesn't do, is provide a method to specify the relationship of data within types between each other. You stated in the past this is because you want to keep dispatch-type separate from representation type. However, I think that's actually one of the most important aspects of OOP. Yes, you can always write an interface that assumes that each type has variable X, but I would claim ultimately that creates less maintainable code (partially because of DRY and partially because it turns what should be a compile-time error into a run-time error). On Sunday, October 18, 2015 at 11:28:20 PM UTC-4, Stefan Karpinski wrote: > > It's a highly debatable claim that single dispatch class-based OO leads to > better or more maintainable software designs. I grew up on these kinds of > languages – Object Pascal, C++, Ruby, Java. Despite the ongoing rhetoric > that this paradigm solves all sorts of program design problems, that's just > not what I've seen. I think this style of programming has failed to be the > panacea that it was promised to be. If it was really such a success, then > we wouldn't be seeing a fresh crop of promising new programming languages – > Julia, Rust, Go, Clojure, Scala, Elixir, Elm, Swift, etc. – not a single > one of which is a traditional class-based OO language. The message is > clear: we need more effective solutions. So far, Julia's multiple dispatch > approach seems to me far better at addressing program maintainability and > extensibility than classes ever were. Julia does still need a way of > talking about interfaces and protocols in the language itself, but that's > ongoing design problem, not a fundamental issue. > > On Mon, Oct 19, 2015 at 8:18 AM, Abe Schneider <abe.sc...@gmail.com > > wrote: > >> >> >> On Sunday, October 18, 2015 at 9:44:21 PM UTC-4, vav...@uwaterloo.ca >> wrote: >>> >>> Going a bit on a tangent, I claim that object-oriented programming, at >>> least as embodied in C++, is not really suitable for scientific computing >>> and in fact has led certain codes to go astray. >>> >> >> C++ is purposefully a multi-paradigm language for this reason. C++11 >> added a lot of features specifically for functional aspects of the language. >> >> >>> >>> To give a simple example, suppose you are told to write a 'matrix' class >>> in C++ that contains all the usual operations. You finish writing your >>> class, and a week later, you are told to derive 'symmetric matrices' as a >>> subclass of 'matrices'. This is trouble! Unless you wrote your original >>> matrix class in a very particular way, you will have a hard time writing a >>> symmetric matrix class that inherits from a matrix base class. This is not >>> just a theoretical example; I saw someone try to do this (unsuccessfully) >>> in a code he had written. >>> >> >> >> Sure, you always have the option to write bad code, but that doesn't make >> a particular paradigm good or bad. I think Torch7 has a good example of >> using object oriented programming for scientific programming. >> >> >>> >>> The problem with object-oriented programming is that you have make >>> design decisions when you write the base classes that lock you into a >>> certain path. This path may not be compatible with the flexibility needed >>> for scientific software projects. >>> >> >> I think that's true with any language. In Julia you still have to define >> an interface to access the variables. Yes, there are fewer mechanisms in >> place that restrict access, but that isn't always a good thing. You end up >> trading compile-time errors for run-time errors. I much rather deal with
Re: [julia-users] Re: Julia and Object-Oriented Programming
My main argument in this thread is that I wouldn't consider Julia "true OO" in the terms many other languages would define it, but trying not to pass any real judgement of whether that is a good thing or not (I agree with Stefan that it's an interesting experiment, and the language is still a work in progress). Julia supports the OO concept in hierarchy in dispatch-type, but not in data-type. Clearly there is benefit to providing some type of organizational framework, and a hierarchical has helped on the dispatch-front. I've heard two arguments on data-type hierarchy: (1) It's not needed, and doesn't help in any way, and (2) it's not clear how to implement it, so it hasn't been attempted yet. I disagree with (1). and am strongly empathetic to (2). I don't think that hierarchy is the only way to organize code, but that it is a good way to organize some of the code some of the time. You will find other cases where functional or aspect programming is a better fit. Thus, I think instead of prescribing the best methodology for developers, it's better to provide the tools that they can employ however they see fit. On Monday, October 19, 2015 at 9:42:11 AM UTC-4, Tim Holy wrote: > > On Sunday, October 18, 2015 07:38:24 PM Abe Schneider wrote: > > > Julia isn’t object-oriented, and will probably never be. But that > doesn’t > > > make it less powerful :) > > > > Of course it doesn't make it less powerful, but it also doesn't alway > lead > > to nice code. The point of OOP isn't that it gets you more power, but it > > gives you more maintainable code. > > From my personal experience, I'd say it's abundantly clear that "true OO" > (whatever that really means) is not more maintainable than julia's type > system > + multiple dispatch. OO is great, and something you study in school > because > it's been around a long time, but that doesn't mean that in 2015 it's > still > the best organizing principle for code. > > --Tim > >
Re: [julia-users] Re: Julia and Object-Oriented Programming
On Monday, October 19, 2015 at 12:17:16 PM UTC-4, Tom Breloff wrote: > > Here's an example where there would normally be a strict hierarchy in > classic OOP: Square <: Rectangle <: Shape > > But it actually makes much more sense in Julia (why have 2 fields for > Square??): > Unfortunately most of the classic examples are simplified to the point of not being informative. A previous example I gave was the NN design of Torch7, where each module has an `output` and a `grad_input` parameter. The forward/backward ops of each module is different, but it's important to have those common variables. > > Now I completely agree that ensuring a compile-time agreement of interface > is a valuable part of the language (but should ideally be optional). > People do see value in this, and it's an active design discussion. (search > for "traits") > I did take a look at what the Traits people were doing. Unfortunately, at least last time I looked at it, it still seemed to require a lot of code copying. > > For composition vs inheritance, I completely agree that sometimes it's > nice to save code/typing when you have structures with a bunch of fields > that are mostly the same. I'd argue you don't actually care about > inheritance, you just want composition but without the extra layer of > structure. > I strongly agree with this. If you hide all direct access to variables and only allow access via an interface, you can define a type as a composition of other types and a list of supported interfaces: @compose C(protocol1, protocol2) <| (A, B) other::AbstractString end Above, a relatively simple macro can replace all the "Type..." with the > fields of the composed types, applying multiple inheritance of the > structures without the baggage required in classic OOP. Then you can > compose your type from other types, but without having to write > "unicycle.wheel.radius"... you can just write "unicycle.radius". > I suggested such a macro awhile ago (maybe a year?), and it was met with disapproval. Not that that should stop anyone from writing it, but it was deemed un-Julian (at least my take on people's responses). I think this can be seen as a type of mix-in similar to what Ruby and Scala do. > > On Mon, Oct 19, 2015 at 11:32 AM, Abe Schneider <abe.sc...@gmail.com > > wrote: > >> I think I agree with 90% of what you wrote. I also don't know if anyone >> is arguing about single or multiple dispatch. Also, while Java and >> Borland's Object Pascal claimed a strong OO paradigm, C++ does does not (if >> you read Bjarne's design guide, he felt strongly that both OOP and >> functional were important). I'll note too, Java has slowly been moving >> towards a multi-paradigm approach by including Lambdas in the latest >> version. >> >> Two of the languages you list below, Swift and Scala, also mix OO and >> functional programming. While Scala has a huge push for functional >> programming, it also provides the necessary constructs to talk about >> inheritance of data (while it doesn't provide multi-inheritance, it does >> have mix-ins). >> >> I really like Julia's multi-dispatch approach, and have made great use of >> it in my own libraries. I don't think there should be any reason that >> classes should have to "own" the methods. However, what it currently >> doesn't do, is provide a method to specify the relationship of data within >> types between each other. You stated in the past this is because you want >> to keep dispatch-type separate from representation type. However, I think >> that's actually one of the most important aspects of OOP. Yes, you can >> always write an interface that assumes that each type has variable X, but I >> would claim ultimately that creates less maintainable code (partially >> because of DRY and partially because it turns what should be a compile-time >> error into a run-time error). >> >> On Sunday, October 18, 2015 at 11:28:20 PM UTC-4, Stefan Karpinski wrote: >>> >>> It's a highly debatable claim that single dispatch class-based OO leads >>> to better or more maintainable software designs. I grew up on these kinds >>> of languages – Object Pascal, C++, Ruby, Java. Despite the ongoing rhetoric >>> that this paradigm solves all sorts of program design problems, that's just >>> not what I've seen. I think this style of programming has failed to be the >>> panacea that it was promised to be. If it was really such a success, then >>> we wouldn't be seeing a fresh crop of promising new programming languages – >>> Julia, Rust, Go, Clojure, Sca
[julia-users] Re: Julia and Object-Oriented Programming
>
>
> But I agree with Tobias and Simon - there is very little reason to try to
> make Julia more object-oriented. All the power you get from OO is still
> there, but you might need to think a little differently to harness it. If
> you have a concrete problem, along the lines of “this is how I’d do it in
> OO, but I don’t know how to do it in Julia”, the community is usually
> pretty good att figuring something out. A recent example was that someone
> wanted man.eat(food) rather than eat(man, food). A little thinking
> outside the box turned out the IMO superior solution - instead of changing
> the language to make it read better, just change the verb: feed(man, food)
> .
>
> Julia isn’t object-oriented, and will probably never be. But that doesn’t
> make it less powerful :)
>
>
Of course it doesn't make it less powerful, but it also doesn't alway lead
to nice code. The point of OOP isn't that it gets you more power, but it
gives you more maintainable code.
> On Sunday, October 18, 2015 at 8:32:06 PM UTC+2, Tobias Knopp wrote:
>
> Julia is fully object oriented. The core of OO is not to write a dot
>> between an object and an associated function but that one has virtual
>> functions. Multiple dispatch provides just that.
>>
>> Cheers
>>
>> Tobi
>>
>> Am Sonntag, 18. Oktober 2015 15:15:50 UTC+2 schrieb Simon Danisch:
>>>
>>> This design has at least 3 issues, since you can't have instances of a
>>> module.
>>> In general:
>>> I personally would value the discussion of why Julia needs more OOP
>>> constructs much more, If one can show that there are terrible flaws in
>>> Julia's model which are swiftly solvable with OOP.
>>> My guess is, that the argument "I learned X and can't transfer it to
>>> Julia, so Julia needs to change", can't really convince anyone to start
>>> implementing something rather involved.
>>>
>>> There are a lot of comments on people trying to bring OOP to Julia,
>>> which are best summarized by: give Julia's model a chance and don't stick
>>> to OOP - it will pay off.
>>> And this kind of comment is pretty understandable in my opinion, since
>>> Julia is quite a sophistaced language with plenty of constructs to avoid
>>> OOP.
>>> I don't see much value in OOP and Julia's multiple dispatch is actually
>>> one of the things that brought me here ( I exclusively worked with OOP
>>> languages before).
>>> Besides the current flaws, I think it's a better model for numerical
>>> computing, it yields concise code, easier maintenance and it's easier to
>>> add feature to an existing code base than it is in OOP.
>>>
>>> I think Julia should move forward by not mimicking OOP further, but
>>> instead improve the current model - there's lots to do already.
>>> Wasting time by chasing after features from other languages will make it
>>> much harder to turn Julia into a well rounded, sound language. (this only
>>> applies to feature that get chased without a concrete problem)
>>>
>>> Best,
>>> Simon
>>>
>>>
>>> Am Sonntag, 18. Oktober 2015 14:41:58 UTC+2 schrieb Sisyphuss:
When I'm learning Julia, I am always thinking what is the correct way
to do OOP in this language. It seems to me that what I learned in C++ does
not apply in Julia.
It took me long to realize that the equivalent of Class of C++ in Julia
is not Type, but Module. Module is the basic function unit in Julia.
Thus, a Class in Julia is like
module ClassName # class Name {
using# include<> // should be outside
import # include<>
export function # public function;
var = 1 # private static var;
end # }
This provides the same structure as C++.
However, this design has two issues:
1) The visit control is not as fine-grained as in C++, the
encapsulation is not strict;
2) Variables at the top level of a module are global variables.
These two points are closely correlated. If we let module have private
variables, then they are not too different from local variables, ans thus
can be type inferred.
I think this is a natural way to do OOP with Julia.
>
[julia-users] Re: Julia and Object-Oriented Programming
On Sunday, October 18, 2015 at 9:44:21 PM UTC-4, vav...@uwaterloo.ca wrote:
>
> Going a bit on a tangent, I claim that object-oriented programming, at
> least as embodied in C++, is not really suitable for scientific computing
> and in fact has led certain codes to go astray.
>
C++ is purposefully a multi-paradigm language for this reason. C++11 added
a lot of features specifically for functional aspects of the language.
>
> To give a simple example, suppose you are told to write a 'matrix' class
> in C++ that contains all the usual operations. You finish writing your
> class, and a week later, you are told to derive 'symmetric matrices' as a
> subclass of 'matrices'. This is trouble! Unless you wrote your original
> matrix class in a very particular way, you will have a hard time writing a
> symmetric matrix class that inherits from a matrix base class. This is not
> just a theoretical example; I saw someone try to do this (unsuccessfully)
> in a code he had written.
>
Sure, you always have the option to write bad code, but that doesn't make a
particular paradigm good or bad. I think Torch7 has a good example of using
object oriented programming for scientific programming.
>
> The problem with object-oriented programming is that you have make design
> decisions when you write the base classes that lock you into a certain
> path. This path may not be compatible with the flexibility needed for
> scientific software projects.
>
I think that's true with any language. In Julia you still have to define an
interface to access the variables. Yes, there are fewer mechanisms in place
that restrict access, but that isn't always a good thing. You end up
trading compile-time errors for run-time errors. I much rather deal with
compiler-time errors.
>
> I for one am glad that the designers of Julia decided not to make Julia an
> object-oriented language, at least not in the C++ sense of the term.
>
>
I think there is a lot to be gained from a multi-paradigm approach. Having
a language that is scientific-code friendly is important, but so is
integrating it into larger systems.
>
>
>
> On Sunday, October 18, 2015 at 8:41:58 AM UTC-4, Sisyphuss wrote:
>>
>> When I'm learning Julia, I am always thinking what is the correct way to
>> do OOP in this language. It seems to me that what I learned in C++ does not
>> apply in Julia.
>>
>> It took me long to realize that the equivalent of Class of C++ in Julia
>> is not Type, but Module. Module is the basic function unit in Julia.
>>
>> Thus, a Class in Julia is like
>> module ClassName # class Name {
>> using# include<> // should be outside
>> import # include<>
>> export function # public function;
>> var = 1 # private static var;
>> end # }
>> This provides the same structure as C++.
>>
>> However, this design has two issues:
>> 1) The visit control is not as fine-grained as in C++, the encapsulation
>> is not strict;
>> 2) Variables at the top level of a module are global variables.
>>
>> These two points are closely correlated. If we let module have private
>> variables, then they are not too different from local variables, ans thus
>> can be type inferred.
>> I think this is a natural way to do OOP with Julia.
>>
>>
>>
>>
[julia-users] Re: Julia and Object-Oriented Programming
I would say that the module is closest to a c++ namespace, which itself can
be thought of as a static class. Note, however, that in both cases you
can't really have separate instances.
Because of the desire to separate dispatch from type information, Julia
doesn't really follow the OOP paradigm. You can have a hierarchy of
abstract types, but abstract types contain no information besides their
name and place in the hierarchy. Because of multi-dispatch, functions live
outside of types (i.e., because you can dispatch on all parameters, usually
no one type owns that function). Finally, you can build an interface to
mimic an OOP design, but each concrete type still has to re-define the
variables (which, is one of the biggest advantages of an OOP design).
One of the really nice things about Julia is that you could in theory
create a macro to simulate OOP in Julia:
@class foo <: a, b, c
# ...
end
where `@class` creates a new type `foo`, which pulls in the member
variables of `a`, `b`, and `c` to simulate a mix-in design.
On Sunday, October 18, 2015 at 8:41:58 AM UTC-4, Sisyphuss wrote:
>
> When I'm learning Julia, I am always thinking what is the correct way to
> do OOP in this language. It seems to me that what I learned in C++ does not
> apply in Julia.
>
> It took me long to realize that the equivalent of Class of C++ in Julia is
> not Type, but Module. Module is the basic function unit in Julia.
>
> Thus, a Class in Julia is like
> module ClassName # class Name {
> using# include<> // should be outside
> import # include<>
> export function # public function;
> var = 1 # private static var;
> end # }
> This provides the same structure as C++.
>
> However, this design has two issues:
> 1) The visit control is not as fine-grained as in C++, the encapsulation
> is not strict;
> 2) Variables at the top level of a module are global variables.
>
> These two points are closely correlated. If we let module have private
> variables, then they are not too different from local variables, ans thus
> can be type inferred.
> I think this is a natural way to do OOP with Julia.
>
>
>
>
Re: [julia-users] Re: macros generating macros in modules
That's definitely an option, though it seems sub-optimal to me if it's possible to automate it from a user's perspective. It may be that it just isn't possible, but if so, is it because it's something the language doesn't currently support or does it go against the philosophy of the language? Specifically, several possible methods could provide the necessary mechanism: (1) Ability to get a function based on a signature and a namespace. Yes, you can get the names of functions in a namespace, but I don't see a way to convert that name to an actual function without an eval. (2) Ability to create an call-expression with a namespace indicate to use the function from that namespace. For example: Expr(:call, :sym, :foobar_namespace, args). This means no additional lookup is required on the user's part. (3) Allow macros to generate macros. This is already partially supported, though it's not clear if this is intentional or not. c++ has shown that template programming can be both useful and powerful (e.g. offloading computation to compile time rather than runtime). All of these methods are general and not specific to the problem. On Sunday, April 5, 2015 at 10:18:03 AM UTC-4, Toivo Henningsson wrote: If users should supply additional parsers (which parse sub-ASTs, right?) then it's not really possible to hide all the metaprogramming machinery from them anyway. So why not let the user create the macro, and just supply functions that make writing the macro implementation as easy as possible?
Re: [julia-users] Re: macros generating macros in modules
I think if I just copy bits of the current code in it might be the easiest. I have the following macro: macro grammar(name, definitions) parsers = [:+, :*, :?, :|, :-, :^, :list, :integer, :float] mapped_parsers = map_symbol_to_function(parsers) return parseGrammar(name, definitions, ParserData(mapped_parsers)) end where parseGrammar goes through the definitions and converts them into actual code. Its output is an Expr tree. The key part here, is that per line, another function `parseDefinition` is recursively called. The simplified version of it is: function parseDefinition(name::String, ex::Expr, pdata::ParserData) # ... # look up the symbol in the expression tree to get the function parser = get(pdata.parsers, ex.args[1], nothing) if parser !== nothing # call function on the rest of the expression rule = parser(name, pdata, ex.args[2:end]) end end Thus, all the parsers listed above will be called appropriately if their symbol is encountered. This used to be a monolithic function, but it seemed a much better design to keep each parser as its own thing. The parsers themselves are not complex. Here's the definition of the or-rule: function|(name::String, pdata::ParserData, args::Array) left = parseDefinition($(name)_1, args[1], pdata) right = parseDefinition($(name)_2, args[2], pdata) return OrRule(name, value) end The `map_symbol_to_function` thus traverses the original `parsers` list created and maps the symbol to this function. So, all of this currently works (and is viewable to anyone who is interested to see the full versions on PEGParser's github). The issue is that currently `parsers` is a fixed list. I'm trying to figure out how to let the users of the library add their own. For example, suppose and `integer` parser wasn't yet made. Someone could create their own: function integer(name::String, pdata::ParserData, args::Array) return IntegerRule(name) end and then pass it into the macro somehow in so when `parseDefinition` is called, it will be used. Originally, the macro was written to allow custom parsers to be passed in with the grammar definition: @grammar foo [integer] begin start = list(integer, r,[ \t\n]*) end Thus, the `grammar` macro would just append whatever you listed to the `parsers` list. However, the mapping of symbols to functions will fail because of the namespace issue. Maybe, as others have argued, that's okay, because I'm trying to do too much in a macro. That's fine, so the next logical conclusion is that I should just generate code that does the mapping. Also fine. BUT, don't forget that this mapping is there to support the function of macro. Specifically, the `parsers` list is used in the `parseDefinition` function for transforming a grammar definition into code. Thus, that means I would need a macro that generates code for another macro, bringing us to the original question posed. One possible way around this might be that the macro doesn't actually have to generate another macro (which I think is probably the more elegant solution), but rather you could generate variables which the second macro can use. For example: @set_parsers(integer) # creates a parsers = [...] list @grammar foo begin # checks if `parsers` list exists, if not creates its own end However, this approach seems dangerous, as it introduces variables that the user is unaware of. On Sunday, April 5, 2015 at 12:28:21 AM UTC-4, Patrick O'Leary wrote: Let me ask the question slightly differently. Without constraining yourself to legal Julia syntax, what would you like to go in, and what do you want to come out? I get the feeling there's a design here that doesn't have this complexity--my intuition is that a generic function can be used to drive this just fine (called within a macro to facilitate the final splice.) But that's still hard to concretely suggest. On Saturday, April 4, 2015 at 9:39:57 PM UTC-5, Abe Schneider wrote: I don't understand how what I'm trying to do falls outside the domain of transforming code. My main goal is to take a user-defined grammar and create relevant code. More to the point, I think the general ability to generate expressions based on functions/namespaces is also solidly in the domain of transforms. I can imagine plenty of use-cases for language-bindings, communication, etc. Yes, there are many simple macros one can create, which are useful, but there exists many useful things one can do with macros that can get more complex. A On Saturday, April 4, 2015 at 10:11:25 PM UTC-4, Jameson wrote: I think the underlying issue may be that you are trying to have the macro do too much, when you should instead be just doing source-code transforms and preprocessing. One such example of a great use of a macro is simply: macro special_foo(syntax_tree) return quote special_foo( $(QuoteNode(syntax_tree) ) end end On Sat, Apr 4, 2015 at 8:05 PM
[julia-users] macros generating macros in modules
I should start off, not entirely sure this is an okay thing to do with Julia. Suppose I want to create a macro that generates another macro, I can write: macro meta_macro(x) quote macro foo(y) $x+y end end end and then call it and the generated macro: @meta_macro(5) println(@foo(3)) # 8 Okay, wasn't entirely sure that was going to work, but it appears to. However, if I try to move the exact same macro code into a module, it no longer works: module meta_macro_mod export @meta_macro, @foo macro meta_macro(x) quote macro foo(y) $x+y end end end end Even without trying to invoke `@foo`, when I execute `@meta_macro`, I get: ERROR: syntax: invalid macro definition It seems like how the macro gets evaluated changes when it's in the module. If I change it to a module to just doing an 'include', everything works again.
[julia-users] Re: macros generating macros in modules
The issue I'm dealing with is that I have a macro that I want to pass a list of functions that the macro can employ for parsing grammars. The macro itself exists already, and it currently has its own static list of functions. I'm trying to figure out a way to allow the user of the library to customize it. For example: @grammar foo begin start = list(integer, r, ) end where `list` is a user supplied function. The issue that I keep running into is that there doesn't seem to be any easy way of just passing in a function to a macro. Turning an expression into a function is problematic, because `eval`s are: (a) only work in the modules namespace, and (b) are frowned on. I've thought about manually looking up the symbol name in the various namespaces, but while I've found methods to list the functions, I haven't found a way to do the reverse. Ideally, it would be nice to generate an expression with `call` that took a symbol and a namespace, that way no lookups would have to be done. Having hit a dead end in this direction, it occurred to me that it might be possible to do something like this: @set_parsers(list, etc) @grammar foo begin ... end where '@set_parsers` would generate the `@grammar` macro with the static list it already has plus whatever else the user adds in. Which works, except for the whole module issue. Thanks, A On Saturday, April 4, 2015 at 12:41:27 PM UTC-4, Patrick O'Leary wrote: On Saturday, April 4, 2015 at 9:04:10 AM UTC-5, Abe Schneider wrote: I should start off, not entirely sure this is an okay thing to do with Julia. Suppose I want to create a macro that generates another macro... I'm not sure whether this should work or not, but either way I'm not sure how it's any more expressive than macros alone? Can you describe what it is you want to do at a high level in a little more detail? There might be another way.
Re: [julia-users] Re: macros generating macros in modules
I don't understand how what I'm trying to do falls outside the domain of transforming code. My main goal is to take a user-defined grammar and create relevant code. More to the point, I think the general ability to generate expressions based on functions/namespaces is also solidly in the domain of transforms. I can imagine plenty of use-cases for language-bindings, communication, etc. Yes, there are many simple macros one can create, which are useful, but there exists many useful things one can do with macros that can get more complex. A On Saturday, April 4, 2015 at 10:11:25 PM UTC-4, Jameson wrote: I think the underlying issue may be that you are trying to have the macro do too much, when you should instead be just doing source-code transforms and preprocessing. One such example of a great use of a macro is simply: macro special_foo(syntax_tree) return quote special_foo( $(QuoteNode(syntax_tree) ) end end On Sat, Apr 4, 2015 at 8:05 PM Abe Schneider abe.sc...@gmail.com javascript: wrote: The issue I'm dealing with is that I have a macro that I want to pass a list of functions that the macro can employ for parsing grammars. The macro itself exists already, and it currently has its own static list of functions. I'm trying to figure out a way to allow the user of the library to customize it. For example: @grammar foo begin start = list(integer, r, ) end where `list` is a user supplied function. The issue that I keep running into is that there doesn't seem to be any easy way of just passing in a function to a macro. Turning an expression into a function is problematic, because `eval`s are: (a) only work in the modules namespace, and (b) are frowned on. I've thought about manually looking up the symbol name in the various namespaces, but while I've found methods to list the functions, I haven't found a way to do the reverse. Ideally, it would be nice to generate an expression with `call` that took a symbol and a namespace, that way no lookups would have to be done. Having hit a dead end in this direction, it occurred to me that it might be possible to do something like this: @set_parsers(list, etc) @grammar foo begin ... end where '@set_parsers` would generate the `@grammar` macro with the static list it already has plus whatever else the user adds in. Which works, except for the whole module issue. Thanks, A On Saturday, April 4, 2015 at 12:41:27 PM UTC-4, Patrick O'Leary wrote: On Saturday, April 4, 2015 at 9:04:10 AM UTC-5, Abe Schneider wrote: I should start off, not entirely sure this is an okay thing to do with Julia. Suppose I want to create a macro that generates another macro... I'm not sure whether this should work or not, but either way I'm not sure how it's any more expressive than macros alone? Can you describe what it is you want to do at a high level in a little more detail? There might be another way.
Re: [julia-users] Re: inserting an Expr into AST via macro
As a (hopeful) final note, I figured out the correct way to escape the
expression:
for rule in all_rules
if typeof(rule.action) != Function
dot = Expr(:(.), rule, QuoteNode(:action))
fn = Expr(:(-),
Expr(:tuple, :rule, :value, :first, :last, :children),
Expr(:block, rule.action))
push!(code, Expr(:escape, Expr(:(=), dot, fn)))
end
end
which seems to do the trick.
On Monday, March 23, 2015 at 8:00:23 AM UTC-4, Abe Schneider wrote:
Okay, I think I almost have a solution. I now collect all the actions from
the rules with the function:
function collect_rules(rule::Rule, lst::Array)
push!(lst, rule)
for child in get_children(rule)
collect_rules(child, lst)
end
return lst
end
which is called from the loop:
all_rules = Rule[]
code = {}
for definition in expr.args[2:2:end]
name = string(definition.args[1])
ref_by_name = Expr(:ref, :rules, name)
rule = parseDefinition(name, definition.args[2], pdata)
push!(code, :(rules[$name] = $rule))
all_rules = collect_rules(rule, all_rules)
end
Finally, I add the transform to all the actions in the loop:
for rule in all_rules
if typeof(rule.action) != Function
dot = Expr(:(.), rule, QuoteNode(:action))
fn = Expr(:(-),
Expr(:tuple, :rule, :value, :first, :last, :children),
Expr(:block, rule.action))
push!(code, Expr(:(=), dot, fn))
end
end
Unfortunately, I'm still getting the namespace problem. I assume this is
because I'm not escaping properly in the macro. However, after playing
around with different variations, I can't figure out the proper place to
put the escape.
Thanks for all the help!
On Sunday, March 22, 2015 at 10:02:28 AM UTC-4, Jameson wrote:
the trick with macros is that everything should be generating code, not
actually evaluating that code. think of it as a series of functions that do
a text transformation to a new structured document, not a [partial]
execution of that document. for example:
function parseDefinition(name::String, expr::Expr, pdata::ParserData)
#...
elseif expr.head === :curly
action_expr = expand_names(expr.args[2])
rule_expr = parseDefinition(name, expr.args[1], action_expr, pdata)
else
#...
return rule_expr
end
macro doAction(expr)
return parseDefinition(..., expr, ...)
end
On Sun, Mar 22, 2015 at 8:15 AM Abe Schneider abe.schnei...@gmail.com
http://mailto:abe.schnei...@gmail.com wrote:
I'm in the process of trying to figure out how to do something similar to
that, but it's unfortunately not easy. I was hoping that there might be a
method to insert an expression into the AST without an eval required, as it
would greatly simplify the code Jameson suggested using the `insert` macro,
which does as advertised, expect for the namespace issue.
Re: [julia-users] Re: inserting an Expr into AST via macro
Okay, I think I almost have a solution. I now collect all the actions from
the rules with the function:
function collect_rules(rule::Rule, lst::Array)
push!(lst, rule)
for child in get_children(rule)
collect_rules(child, lst)
end
return lst
end
which is called from the loop:
all_rules = Rule[]
code = {}
for definition in expr.args[2:2:end]
name = string(definition.args[1])
ref_by_name = Expr(:ref, :rules, name)
rule = parseDefinition(name, definition.args[2], pdata)
push!(code, :(rules[$name] = $rule))
all_rules = collect_rules(rule, all_rules)
end
Finally, I add the transform to all the actions in the loop:
for rule in all_rules
if typeof(rule.action) != Function
dot = Expr(:(.), rule, QuoteNode(:action))
fn = Expr(:(-),
Expr(:tuple, :rule, :value, :first, :last, :children),
Expr(:block, rule.action))
push!(code, Expr(:(=), dot, fn))
end
end
Unfortunately, I'm still getting the namespace problem. I assume this is
because I'm not escaping properly in the macro. However, after playing
around with different variations, I can't figure out the proper place to
put the escape.
Thanks for all the help!
On Sunday, March 22, 2015 at 10:02:28 AM UTC-4, Jameson wrote:
the trick with macros is that everything should be generating code, not
actually evaluating that code. think of it as a series of functions that do
a text transformation to a new structured document, not a [partial]
execution of that document. for example:
function parseDefinition(name::String, expr::Expr, pdata::ParserData)
#...
elseif expr.head === :curly
action_expr = expand_names(expr.args[2])
rule_expr = parseDefinition(name, expr.args[1], action_expr, pdata)
else
#...
return rule_expr
end
macro doAction(expr)
return parseDefinition(..., expr, ...)
end
On Sun, Mar 22, 2015 at 8:15 AM Abe Schneider abe.schnei...@gmail.com
http://mailto:abe.schnei...@gmail.com wrote:
I'm in the process of trying to figure out how to do something similar to
that, but it's unfortunately not easy. I was hoping that there might be a
method to insert an expression into the AST without an eval required, as it
would greatly simplify the code Jameson suggested using the `insert` macro,
which does as advertised, expect for the namespace issue.
Re: [julia-users] Re: inserting an Expr into AST via macro
Sorry, I'm not being as precise I should with language, but I did mean 'generate code to create a dictionary'. The macro exists already at the bottom of: https://github.com/abeschneider/PEGParser.jl/blob/master/src/rules.jl On Sunday, March 22, 2015 at 11:27:54 PM UTC-4, Jameson wrote: My macro’s primary role is to create a dictionary mapping symbols to Nodes. Each Node has an associated action (passed in with the curly braces). The actions themselves are Exprs, which need to be encapsulated as functions. that’s not something a macro can do (directly). and it's also why a macro is different from a function. specifically, a function’s purpose is to change one object into another thing. on the other hand, a macro is for transforming code into more code. so the return value from the macro isn’t a new object, but the code that describes how to to make that new object. so for the example above, rather than directly assigning expr.args[2] into rule.action, you would return the equivalent expression (quote; rule.action = $(expr.args[2]); end) that do that operation at runtime On Sun, Mar 22, 2015 at 11:01 PM Abe Schneider abe.schnei...@gmail.com http://mailto:abe.schnei...@gmail.com wrote: Sorry, when I said AST, I meant the code that gets executed. A macro, at least the way I think about it, is inserting the resulting Expr into the main AST. My macro's primary role is to create a dictionary mapping symbols to `Node`s. Each Node has an associated action (passed in with the curly braces). The actions themselves are `Expr`s, which need to be encapsulated as functions. I was hoping initially that the `Expr` for the action could be converted inline with the creation of the `Node`, as it would save the need of a lot of book keeping. Thus, the: `rule.action = insert_action_expr(expr.args[2])`. Another solution might to do two passes, where in the second pass the rule's `Expr`s are converted. Unfortunately, that seems less elegant than being able to convert an `Expr` to a function. On Sunday, March 22, 2015 at 9:39:30 AM UTC-4, Toivo Henningsson wrote: Maybe there is a confusion of terms here? An Expr is the way to represent an AST in Julia (some very simple ASTs are instead represented by ints/floats/strings/symbols). So when you talk about inserting an Expr into an AST it seems to me that all it means is to make an Expr where some subtree is given by another Expr. Macros are really only a way to save typing. Consider a few cases: what is the input AST, and what is the output AST that you want to generate in each case. From there, you should be able to figure out how to write the code that does the transformation.
Re: [julia-users] Re: inserting an Expr into AST via macro
I'm in the process of trying to figure out how to do something similar to that, but it's unfortunately not easy. I was hoping that there might be a method to insert an expression into the AST without an eval required, as it would greatly simplify the code Jameson suggested using the `insert` macro, which does as advertised, expect for the namespace issue. On Sunday, March 22, 2015 at 2:15:17 AM UTC-4, Toivo Henningsson wrote: Do you really need to call eval from within a macro? That will indeed give these kinds of troubles. Cannot you arrange for the macro to emit code that sets the appropriate rule.action when run instead? Then you will be in the invoker's context, and you can use esc.
Re: [julia-users] Re: inserting an Expr into AST via macro
Sorry, when I said AST, I meant the code that gets executed. A macro, at least the way I think about it, is inserting the resulting Expr into the main AST. My macro's primary role is to create a dictionary mapping symbols to `Node`s. Each Node has an associated action (passed in with the curly braces). The actions themselves are `Expr`s, which need to be encapsulated as functions. I was hoping initially that the `Expr` for the action could be converted inline with the creation of the `Node`, as it would save the need of a lot of book keeping. Thus, the: `rule.action = insert_action_expr(expr.args[2])`. Another solution might to do two passes, where in the second pass the rule's `Expr`s are converted. Unfortunately, that seems less elegant than being able to convert an `Expr` to a function. On Sunday, March 22, 2015 at 9:39:30 AM UTC-4, Toivo Henningsson wrote: Maybe there is a confusion of terms here? An Expr is the way to represent an AST in Julia (some very simple ASTs are instead represented by ints/floats/strings/symbols). So when you talk about inserting an Expr into an AST it seems to me that all it means is to make an Expr where some subtree is given by another Expr. Macros are really only a way to save typing. Consider a few cases: what is the input AST, and what is the output AST that you want to generate in each case. From there, you should be able to figure out how to write the code that does the transformation.
Re: [julia-users] Re: inserting an Expr into AST via macro
Sadly it looks like this doesn't entirely solve the problem. The issue is
that while it does indeed insert the function into the Expr tree, the
namespace for the variables is for the module that the macro is defined in,
and not the user's file (the standard issue with eval'ing).
For example, suppose I define a custom functions `foo` and `bar` in my file
and define a rule like:
value = integer { foo(_1) } | float { bar(_1) }
the execution will fail since `foo` and `bar` are not defined.
On Thursday, March 19, 2015 at 9:59:19 AM UTC-4, Jameson wrote:
yes and no. the definition of your insert_expr function below is:
insert_expr(fn_expr) = @eval (rule, first, last, children) - $fn_expr
so it’s not hard to define. but you could also just have
parse_action_definition return the necessary function in the first place.
@eval is relatively expensive, since it will analyze the expression. this
is good, under the assumption that the resulting function will be reused
many times. but really bad if the user then tries to write a giant for loop
that repeated calls eval on the same thing. whereas wrapping the same
expression in an function, evaluating that once, then calling the function
repeatedly in the for loop lets the compiler optimize the loop much better.
eval is intentionally somewhat restricted in its abilities (such as not
being able to see local scope), specifically to make such abuses harder /
more obvious.
On Thu, Mar 19, 2015 at 7:31 AM Abe Schneider abe.schnei...@gmail.com
http://mailto:abe.schnei...@gmail.com wrote:
Thank you Jameson, that makes a lot of sense. Generally I try to organize
my code to build up a single Expr, but sometimes it makes for less than
ideal code.
Out of curiosity, would it make sense for Julia to have an 'insert'
function to convert Expr's to ASTs? This would help alleviate the need for
macros in certain cases as well 'eval'. For example, in my case, if I had:
fn_expr = parse_action_definition(rule) # fn(rule, first, last, children)
rule.action = insert_expr(fn_expr)
would cause `rule.action` be set to the actual function, rather than the
Expr. If you tried to use `eval`, not only is it frowned on, but it would
also fail because of it's global scope.
While ultimately you can by, as you suggest, with building up the Exprs, I
think something like this would allow for easier to read/maintain code.
On Thursday, March 19, 2015 at 12:43:29 AM UTC-4, Jameson wrote:
writing a code snippet like
val = :(parsefloat(3.1459))
@test2(val)
is perhaps a great way to confuse yourself, although it is a common
mistake.
the problem is that it makes it seem like the two arguments execute in
sequence, whereas the macro expansion pass actually runs prior to the code
execution. it should, therefore, be viewed in complete isolation from the
runtime expressions. in pseudo-code, that might look something like:
fullexpression = macroexpand(:(@test2(val)))
///
val = :(parsefloat(3.1459))
$fullexpression
in summary, macros are functions whose arguments are syntax, not values.
when a macro is executed, val does not get passed as reference to a value
stored in a variable, it is the expression tree that describes that
variable. that is the fundamental distinction between a function and a
macro.
but, it doesn't sound like you need a macro. macros that call other macros
can get really confusing really quickly. but a normal function will perhaps
do exactly what you wanted anyways? it's perfectly valid for you to write a
function that takes `val` in as an argument and returns some new expression
(which you can then combine with other expressions before returning the
whole thing from the main macro)
On Wed, Mar 18, 2015 at 9:51 AM Abe Schneider abe.schnei...@gmail.com
http://mailto:abe.schnei...@gmail.com wrote:
Unfortunately my solution only works if you pass in an expression
directly. If you try something like this:
val = :(parsefloat(3.1459))
@test2(val)
It will fail, since `val` is a symbol. Any other ideas?
The reason I'm trying to solve the problem in the first place is that I
have semantic actions attached to rules formed in a grammar. The actions
take the form of expression, which I want to wrap up in a function like:
(rule,first, last,children) - $ex
where `$ex` is my found expression. If I do this in my main macro,
everything work correctly. However, the logic gets more complicated,
because I want to allow semantic actions to be tied to any subrule, which
means it has to be handled in the definition parsing functions. That should
be fine, but I try to create a function that wraps my function, I get into
this issue. I can't just insert the expression into the AST like I did
before.in the main macro.
On Wednesday, March 18, 2015 at 9:30:32 AM UTC-4, Abe Schneider wrote:
After some thought, I realized that the easiest way to unquote is to reach
inside
Re: [julia-users] Re: inserting an Expr into AST via macro
I have, but it's not entirely clear to me how I would use it in this particular case. The code in question is being called from a function (which while ultimately is being called from a macro) doesn't directly return an expression in this case. Specifically, I have something that looks like: insert_action_expr(fn_expr) = @eval (rule, value, first, last, children) - $fn_expr function parseDefinition(name::String, expr::Expr, pdata::ParserData) #... elseif expr.head === :curly rule = parseDefinition(name, expr.args[1], pdata) action_expr = expand_names(expr.args[2]) rule.action = insert_action_expr(action_expr) else #... end The issue is that `rule.action` eventually needs a function associated with it. The `insert_action_expr` takes the Expr that occurs in the braces and turns it into a function. From my understanding, because this occurs within a function and it's not returning an Expr directly, `esc` isn't applicable. On Saturday, March 21, 2015 at 11:02:28 AM UTC-4, Toivo Henningsson wrote: Have you read the macro hygiene section in the manual chapter about metaprogramming? The esc function is what you want to be able to pass through symbols to be evaluated in the invoker's name space.
Re: [julia-users] Re: inserting an Expr into AST via macro
Thank you Jameson, that makes a lot of sense. Generally I try to organize my code to build up a single Expr, but sometimes it makes for less than ideal code. Out of curiosity, would it make sense for Julia to have an 'insert' function to convert Expr's to ASTs? This would help alleviate the need for macros in certain cases as well 'eval'. For example, in my case, if I had: fn_expr = parse_action_definition(rule) # fn(rule, first, last, children) rule.action = insert_expr(fn_expr) would cause `rule.action` be set to the actual function, rather than the Expr. If you tried to use `eval`, not only is it frowned on, but it would also fail because of it's global scope. While ultimately you can by, as you suggest, with building up the Exprs, I think something like this would allow for easier to read/maintain code. On Thursday, March 19, 2015 at 12:43:29 AM UTC-4, Jameson wrote: writing a code snippet like val = :(parsefloat(3.1459)) @test2(val) is perhaps a great way to confuse yourself, although it is a common mistake. the problem is that it makes it seem like the two arguments execute in sequence, whereas the macro expansion pass actually runs prior to the code execution. it should, therefore, be viewed in complete isolation from the runtime expressions. in pseudo-code, that might look something like: fullexpression = macroexpand(:(@test2(val))) /// val = :(parsefloat(3.1459)) $fullexpression in summary, macros are functions whose arguments are syntax, not values. when a macro is executed, val does not get passed as reference to a value stored in a variable, it is the expression tree that describes that variable. that is the fundamental distinction between a function and a macro. but, it doesn't sound like you need a macro. macros that call other macros can get really confusing really quickly. but a normal function will perhaps do exactly what you wanted anyways? it's perfectly valid for you to write a function that takes `val` in as an argument and returns some new expression (which you can then combine with other expressions before returning the whole thing from the main macro) On Wed, Mar 18, 2015 at 9:51 AM Abe Schneider abe.schnei...@gmail.com http://mailto:abe.schnei...@gmail.com wrote: Unfortunately my solution only works if you pass in an expression directly. If you try something like this: val = :(parsefloat(3.1459)) @test2(val) It will fail, since `val` is a symbol. Any other ideas? The reason I'm trying to solve the problem in the first place is that I have semantic actions attached to rules formed in a grammar. The actions take the form of expression, which I want to wrap up in a function like: (rule,first, last,children) - $ex where `$ex` is my found expression. If I do this in my main macro, everything work correctly. However, the logic gets more complicated, because I want to allow semantic actions to be tied to any subrule, which means it has to be handled in the definition parsing functions. That should be fine, but I try to create a function that wraps my function, I get into this issue. I can't just insert the expression into the AST like I did before.in the main macro. On Wednesday, March 18, 2015 at 9:30:32 AM UTC-4, Abe Schneider wrote: After some thought, I realized that the easiest way to unquote is to reach inside the Expr and pull out what is needed. If I do a dump of `ex` I get: Expr head: Symbol quote args: Array(Any,(1,)) 1: Expr head: Symbol call args: Array(Any,(2,)) 1: Symbol parsefloat 2: ASCIIString 3.1459 typ: Any So I can rewrite the macro as: macro test2(ex) ex.args[1] end which appears to work: @test2(:(parsefloat(3.1459))) # 3.1459 On Wednesday, March 18, 2015 at 8:08:35 AM UTC-4, Abe Schneider wrote: If I do something simple like: macro test() :(parsefloat(3.1459)) end @test() # 3.1459 everything works as expected. However, if I try this instead: macro test2(ex) ex end @test2(:(parsefloat(3.1459))) # :(parsefloat(3.1459)) Not what I expected, but it makes sense what's happening: macroexpand(:(@test(:(parsefloat(3.1459) # :($(Expr(:copyast, :(:(parsefloat(3.1459)) So, anything I pass to the macro is automatically quoted. Obviously if I just do: @test2(parsefloat(3.1459)) it will work. However, the use-case I'm trying to deal with is that I have an Expr that I would like inserted into the AST. Besides being frowned on, I can't eval, due to not wanting things to be evaluated in global space. Is there some way I can unquote the expression in the macro for test2?
Re: [julia-users] inserting an Expr into AST via macro
Thank you for the link. However, I am familiar with interpolation. What I don't understand is how you change `test2` in order to use it. Because there's no quote directly in that macro, Julia will complain about using '$' in a non-quoted expression. On Wednesday, March 18, 2015 at 8:37:25 AM UTC-4, Isaiah wrote: See http://docs.julialang.org/en/release-0.3/manual/metaprogramming/#interpolation On Mar 18, 2015 8:08 AM, Abe Schneider abe.sc...@gmail.com javascript: wrote: If I do something simple like: macro test() :(parsefloat(3.1459)) end @test() # 3.1459 everything works as expected. However, if I try this instead: macro test2(ex) ex end @test2(:(parsefloat(3.1459))) # :(parsefloat(3.1459)) Not what I expected, but it makes sense what's happening: macroexpand(:(@test(:(parsefloat(3.1459) # :($(Expr(:copyast, :(:(parsefloat(3.1459)) So, anything I pass to the macro is automatically quoted. Obviously if I just do: @test2(parsefloat(3.1459)) it will work. However, the use-case I'm trying to deal with is that I have an Expr that I would like inserted into the AST. Besides being frowned on, I can't eval, due to not wanting things to be evaluated in global space. Is there some way I can unquote the expression in the macro for test2?
[julia-users] inserting an Expr into AST via macro
If I do something simple like: macro test() :(parsefloat(3.1459)) end @test() # 3.1459 everything works as expected. However, if I try this instead: macro test2(ex) ex end @test2(:(parsefloat(3.1459))) # :(parsefloat(3.1459)) Not what I expected, but it makes sense what's happening: macroexpand(:(@test(:(parsefloat(3.1459) # :($(Expr(:copyast, :(:(parsefloat(3.1459)) So, anything I pass to the macro is automatically quoted. Obviously if I just do: @test2(parsefloat(3.1459)) it will work. However, the use-case I'm trying to deal with is that I have an Expr that I would like inserted into the AST. Besides being frowned on, I can't eval, due to not wanting things to be evaluated in global space. Is there some way I can unquote the expression in the macro for test2?
[julia-users] Re: inserting an Expr into AST via macro
After some thought, I realized that the easiest way to unquote is to reach inside the Expr and pull out what is needed. If I do a dump of `ex` I get: Expr head: Symbol quote args: Array(Any,(1,)) 1: Expr head: Symbol call args: Array(Any,(2,)) 1: Symbol parsefloat 2: ASCIIString 3.1459 typ: Any So I can rewrite the macro as: macro test2(ex) ex.args[1] end which appears to work: @test2(:(parsefloat(3.1459))) # 3.1459 On Wednesday, March 18, 2015 at 8:08:35 AM UTC-4, Abe Schneider wrote: If I do something simple like: macro test() :(parsefloat(3.1459)) end @test() # 3.1459 everything works as expected. However, if I try this instead: macro test2(ex) ex end @test2(:(parsefloat(3.1459))) # :(parsefloat(3.1459)) Not what I expected, but it makes sense what's happening: macroexpand(:(@test(:(parsefloat(3.1459) # :($(Expr(:copyast, :(:(parsefloat(3.1459)) So, anything I pass to the macro is automatically quoted. Obviously if I just do: @test2(parsefloat(3.1459)) it will work. However, the use-case I'm trying to deal with is that I have an Expr that I would like inserted into the AST. Besides being frowned on, I can't eval, due to not wanting things to be evaluated in global space. Is there some way I can unquote the expression in the macro for test2?
[julia-users] Re: inserting an Expr into AST via macro
Unfortunately my solution only works if you pass in an expression directly. If you try something like this: val = :(parsefloat(3.1459)) @test2(val) It will fail, since `val` is a symbol. Any other ideas? The reason I'm trying to solve the problem in the first place is that I have semantic actions attached to rules formed in a grammar. The actions take the form of expression, which I want to wrap up in a function like: (rule,first, last,children) - $ex where `$ex` is my found expression. If I do this in my main macro, everything work correctly. However, the logic gets more complicated, because I want to allow semantic actions to be tied to any subrule, which means it has to be handled in the definition parsing functions. That should be fine, but I try to create a function that wraps my function, I get into this issue. I can't just insert the expression into the AST like I did before.in the main macro. On Wednesday, March 18, 2015 at 9:30:32 AM UTC-4, Abe Schneider wrote: After some thought, I realized that the easiest way to unquote is to reach inside the Expr and pull out what is needed. If I do a dump of `ex` I get: Expr head: Symbol quote args: Array(Any,(1,)) 1: Expr head: Symbol call args: Array(Any,(2,)) 1: Symbol parsefloat 2: ASCIIString 3.1459 typ: Any So I can rewrite the macro as: macro test2(ex) ex.args[1] end which appears to work: @test2(:(parsefloat(3.1459))) # 3.1459 On Wednesday, March 18, 2015 at 8:08:35 AM UTC-4, Abe Schneider wrote: If I do something simple like: macro test() :(parsefloat(3.1459)) end @test() # 3.1459 everything works as expected. However, if I try this instead: macro test2(ex) ex end @test2(:(parsefloat(3.1459))) # :(parsefloat(3.1459)) Not what I expected, but it makes sense what's happening: macroexpand(:(@test(:(parsefloat(3.1459) # :($(Expr(:copyast, :(:(parsefloat(3.1459)) So, anything I pass to the macro is automatically quoted. Obviously if I just do: @test2(parsefloat(3.1459)) it will work. However, the use-case I'm trying to deal with is that I have an Expr that I would like inserted into the AST. Besides being frowned on, I can't eval, due to not wanting things to be evaluated in global space. Is there some way I can unquote the expression in the macro for test2?
Re: [julia-users] creation of parametric variables in a macro
Awesome, thanks for the suggestions. I ended up independently implementing
your second solution, which I think is the more elegant way to do it. But
it's good to know about the first approach as well.
For anyone who is interested, here's the code I ended up with:
# general case, just return value
expand_names(value) = value
# if it's a symbol, check that it matches, and if so, convert it
function expand_names(sym::Symbol)
m = match(r_(\d+), string(sym))
if m !== nothing
return :(children[parseint($(m.captures[1]))])
end
return sym
end
# if it's an expression, recursively go through tree and
# transform all symbols that match '_i'
function expand_names(expr::Expr)
new_args = [expand_names(arg) for arg in expr.args]
return Expr(expr.head, new_args...)
end
Then, you can can run expand_names on any expression/symbol passed via the
braces.
On Friday, March 6, 2015 at 10:58:43 PM UTC-5, Jameson wrote:
oh, that’s reasonably easy then:
macro testfn(expr)
children = esc(:children)
syms = [:($(symbol(string(_, i))) = $children[$i]) for i = 1:10]
return Expr(:let, Expr(:block, esc(expr)), :(children=children), syms..)
end
which will expand to:
let children=children, _1 = children[1], _2 = children[2], ...
$expr
end
(I added the first children=children line, so that the user could
theoretically assign to a variable named children, without leaking it to
the enclosing environment. it's not strictly necessary).
-
alternatively, you could recurse through their expr code and replace all
variables of the form `_(\d+)` with :(children[\1])
-
On Fri, Mar 6, 2015 at 9:30 PM Abe Schneider abe.schnei...@gmail.com
http://mailto:abe.schnei...@gmail.com wrote:
Hmmm, good to know. Thank you.
The rationale for doing so is to provide a shortcut for the elements of
a variable `children`. Specifically, for a grammar, I might have a rule
like:
```
@grammar foo begin
number = r[0-9]+ { parseint(children[1]) }
end
```
What I would like to have instead, to make it more succint is:
```
@grammar foo begin
number = r[0-9]+ { parseint(_1) }
end
```
Originally, just to get things working, I used an `eval`, which while
worked, also made the assignment global, which was less than ideal.
I'd be curious if anyone has a suggestion on other methods to accomplish
this, or if this is outside the scope of what's possible to do with Julia.
On Friday, March 6, 2015 at 5:10:07 PM UTC-5, Jameson wrote:
you can't do what you are proposing, by design. a macro cannot do
anything that you cannot express directly, it simply allows you to express
it more succinctly by templating the redundant parts.
if you want a set or numbered list, use a set or number list.
variables are bad at that sort of task. whereas an Array is very good at it.
On Fri, Mar 6, 2015 at 8:05 AM Abe Schneider abe.sc...@gmail.com
wrote:
I'm trying to create a set of variables (_1, _2, ...) from items within
a list in a macro. I have a (much) condensed version of the code:
macro testfn()
quote
i = 1
value = [1, 2, 3]
$(Expr(:(=), Expr(:symbol, Expr(:string, _, :i)), :value))
println(_1)
end
end
which gives me:
ERROR: syntax: invalid assignment location
Any ideas on what might be wrong or the proper way to do this? I would
like to keep this in the quotes, as in the actual version there's a lot
more code surrounding the assignment.
I can get things to work with an eval, but I rather avoid the eval, and
it appears to create a non-local variable.
Thanks!
Re: [julia-users] creation of parametric variables in a macro
Hmmm, good to know. Thank you.
The rationale for doing so is to provide a shortcut for the elements of a
variable `children`. Specifically, for a grammar, I might have a rule like:
```
@grammar foo begin
number = r[0-9]+ { parseint(children[1]) }
end
```
What I would like to have instead, to make it more succint is:
```
@grammar foo begin
number = r[0-9]+ { parseint(_1) }
end
```
Originally, just to get things working, I used an `eval`, which while
worked, also made the assignment global, which was less than ideal.
I'd be curious if anyone has a suggestion on other methods to accomplish
this, or if this is outside the scope of what's possible to do with Julia.
On Friday, March 6, 2015 at 5:10:07 PM UTC-5, Jameson wrote:
you can't do what you are proposing, by design. a macro cannot do anything
that you cannot express directly, it simply allows you to express it more
succinctly by templating the redundant parts.
if you want a set or numbered list, use a set or number list.
variables are bad at that sort of task. whereas an Array is very good at it.
On Fri, Mar 6, 2015 at 8:05 AM Abe Schneider abe.sc...@gmail.com
javascript: wrote:
I'm trying to create a set of variables (_1, _2, ...) from items within a
list in a macro. I have a (much) condensed version of the code:
macro testfn()
quote
i = 1
value = [1, 2, 3]
$(Expr(:(=), Expr(:symbol, Expr(:string, _, :i)), :value))
println(_1)
end
end
which gives me:
ERROR: syntax: invalid assignment location
Any ideas on what might be wrong or the proper way to do this? I would
like to keep this in the quotes, as in the actual version there's a lot
more code surrounding the assignment.
I can get things to work with an eval, but I rather avoid the eval, and
it appears to create a non-local variable.
Thanks!
[julia-users] creation of parametric variables in a macro
I'm trying to create a set of variables (_1, _2, ...) from items within a list in a macro. I have a (much) condensed version of the code: macro testfn() quote i = 1 value = [1, 2, 3] $(Expr(:(=), Expr(:symbol, Expr(:string, _, :i)), :value)) println(_1) end end which gives me: ERROR: syntax: invalid assignment location Any ideas on what might be wrong or the proper way to do this? I would like to keep this in the quotes, as in the actual version there's a lot more code surrounding the assignment. I can get things to work with an eval, but I rather avoid the eval, and it appears to create a non-local variable. Thanks!
[julia-users] assignment operator
I was recently thinking about the nature of `=` of not being an operator.
The reason given is that it sets the binding of the variable, rather than
the value. It seems like it would be nice to have an operator that dealt
with values rather than bindings. The reason being: (a) if it's an operator
you can overload its behavior, which when dealing with values could be very
useful, and (b) it provides a more natural syntax than `copy`.
While I don't have strong feels about any particular syntax, I could
imagine something like:
x := y # copied
x = y # bound
function :=(var, value)
#...
end
One side benefit of doing is this it can help solve other issues. For
example, you could make a `ref` class for cases where you want to pass a
variable bound to a scalar to be altered within a function:
type ref{T}
var::T
end
function :={T}(var::ref{T}, value::T)
ref.var := value
end
[julia-users] Abstract types with fields
Don't worry, I'm not re-opening the currently debated issue . It occurred to me a while ago that it should be possible to support this feature without any changes to the language itself. Instead I've created the macros `@_abstract`, `@_type`, and `@_immutable`. The idea is that `@_abstract` does two things: 1. It adds the fields to a global registry and creates an abstract class with the given name 2. When a @_type or @_immutable is created, it copies the fields from the abstract registry and adds its own The current working code can be found at: https://github.com/abeschneider/AbstractFields.jl A quick example: @_abstract Point begin x::Float64 y::Float64 end @_type 3DPoint Point begin z::Float64 end # create a 3DPoint (abstract's values get filled first) point = 3DPoint(1.0, 1.0, 1.0) One of the main advantages of using these macros is that you know if you have a function that takes a `Point`, it will have `x` and `y` defined. As has been pointed out, this isn't strictly necessary, as you could provide an interface with `getx`, `gety`, `setx`, `sety` for each subclass of `Point`. However, this can result in a lot of extra boilerplate code. Anyways, I'm not sure if this is useful/interesting to anyone else, but it's something I've wanted for my own code.
[julia-users] Re: Function with state calling stateless version, or the other way around?
You can think of a Task like a function that maintains state automatically. However, instead of using returning values with 'return x', you use 'produce(x)' instead. This will cause execution of the Task to pause until it is called again. Tasks are called with the 'consume' function, which will in turn return the value given by the 'produce'. The official Julia documentation gives code examples of how to create and use tasks. In the case of the documentation, a task is returned from within a function. The variables defined in the function get bound to the Task returned, which in turn are modified between each call to consume. Julia allows you to also use Tasks as iterators. You can get similar behavior with returning a function from a function. The inner function will be bound to the outer function's variables and can modify them. If the variables are scalars, they will be copied, if they are complex types a reference is created.
[julia-users] Function with state calling stateless version, or the other way around?
It may be worth having both around. If you wrap the stateless versions with something that keeps state, users get the choice of which one to use. For example, you could use Tasks to create generators that produce the next value. The Task will keep the state for you, which allows you to keep your DSP code simple and stateless.
[julia-users] Grammatical Evolution
For anyone who is interested, I've been working on a Grammatical Evolution (GE) [Conor Ryan, JJ Collins and Michael O'Neill, 1998] http://en.wikipedia.org/wiki/Grammatical_evolution#cite_note-1 library: https://github.com/abeschneider/GrammaticalEvolution It is not quite ready to reach version 0.0.0, though it does have decent test coverage. Included is an example to optimize a mathematical expression to match some arbitrary ground truth. For those who are not familiar with (GE), it is an Evolutionary Technique that is similar to Genetic Programming (GP). However, unlike GP, it doesn't suffer the same problems with fixing damaged trees. Instead, it uses a grammar that is combined with a genome of integers. The genome is used to select which branch to follow for or-rules. Julia is a really nice language to implement GE due to its ability to inject new code into the current program (thus no need for an auxiliary grammar or parsing of strings) as well as macros that allow a grammar to be easily defined. Add to that the ability to easily distribute code that can be compiled in a JIT manner means it's possible to get very good speeds with the library (note: currently no benchmarking has been done and I still haven't written the code to distribute the workload).
[julia-users] Re: Equivalent of c++ functor objects, std::bind, and lambdas
Here's a quick test:
function make_counter(x)
return (delta) - begin
x += delta
x
end
end
c0 = make_counter(0)
c1 = make_counter(0)
c0(1) # 1
c1(1) # 1
c0(1) # 2
c1(1) # 2
So it looks like they don't share the bound 'x' variable.
On Friday, August 15, 2014 11:50:03 AM UTC-4, Noah Brenowitz wrote:
suppose I have a function with prototype f(x, y, z). How can bind one
of the arguments to a new function argument by copying vs reference? In
c++11 I could do
z = ... ;
auto newfun = [=](x,y) { f(x,y, z)};
if I wanted newfun to have its own copy of z. Then I could make some
changes to original object z without affecting the evaluation of newfun. An
alternative way to do this would be using a Functor class or std::bind.
Is it possible to replicate this behavior in julia?
Re: [julia-users] Using a callback from a macro
I still haven't been able to get the code to work with using quotes.
However, with some experimentation I was able to write things out directly
with Exprs. A condensed version of the code is:
macro grammar(grammar_name::Symbol, ex::Expr)
code = {}
push!(code, :(rules = Dict()))
for definition in ex.args[2:2:end]
name = Expr(:call, :string, Expr(:quote, definition.args[1]))
ex = Expr(:quote, definition.args[2])
action = Expr(:escape, definition.args[1].args[2])
# rules[name] = parseDefinition(name, ex, nothing)
push!(code, Expr(:(=), Expr(:ref, :rules, name),
Expr(:call, :parseDefinition, name, ex, action)))
end
# grammar_name = Grammar(rules)
push!(code, :($(esc(grammar_name)) = Grammar(rules)))
return Expr(:block, code...)
end
The tricky part being the definition of `action`. It can't be evaluated in
the macro itself, as it would be in the wrong namespace, and thus had to be
escaped. In a separate thread Tim Holy provided a method to get the
expression passed to the `parseDefinition` method, but I couldn't get it to
work with everything else.
I actually think keeping everything as `Expr`s is easier to understand,
though I'm still curious how to properly express this in quote form.
On Thursday, August 7, 2014 1:51:30 PM UTC-4, Jameson wrote:
A function which has access to the namespace in which it was defined is
also known in Julia as a closure, or simply function. All functions are
first-class objects, which don't need eval or macros. Eval and macros are
really bad at pretending to be functions.
Without a bit more example code, it is hard to understand what problem you
are trying to solve. Macros are for reducing typing (syntax transforms),
not magic (altering the environment), which is why they generally don't
have access to the environment.
On Friday, August 1, 2014, Abe Schneider abe.sc...@gmail.com
javascript: wrote:
I tried a second approach where instead of keeping a function around I
keep just the symbol. I have a `transform` function that applies the
function to the resulting values. However, this only moves the problem to a
new place. In my `transform` method I have:
eval(Expr(:call, action, values))
where `action` is the symbol that I am now carrying around. The `eval`
still works in the module's namespace, so I get the same error.
If Julia had a way to access the namespace from the calling function,
this could be made to work. Or for that matter, if I could get the calling
namespace from the macro, I could then just get the macro that way.
I'm guessing Julia doesn't currently have this functionality, but would
it be something possible to add?
On Friday, August 1, 2014 12:22:46 PM UTC-4, Abe Schneider wrote:
I think the problem is that I'm not accessing it directly through the
macro parameters. The call:
@grammar foo begin ... end
passes just the single expression block to `foo`. Thus, I'm getting the
function from the resulting Expr tree. Even if I quote it, it still ends up
as an expression that contains a symbol that must be evaluated (unless I'm
missing something...)
On Friday, August 1, 2014 11:42:52 AM UTC-4, Simon Kornblith wrote:
That looks like the output of :(esc(fn)), but you don't want to quote
that, you want to evaluate it when generating the code, i.e. use
$(esc(fn)) wherever you were previously using $fn
On Friday, August 1, 2014 11:38:44 AM UTC-4, Abe Schneider wrote:
That's correct, I'm generating code that keeps a pointer to that
function to call later on.
If I do that, I have the type `esc(fn)` in the macro (this is what I
get when printing it out), and taking a dump of that argument gives:
Expr
head: Symbol call
args: Array(Any,(2,))
1: Symbol esc
2: Symbol fn
typ: Any
Does that mean instead of carrying around a function I will need to
carry around the Expr?
Thanks!
On Friday, August 1, 2014 11:17:50 AM UTC-4, Simon Kornblith wrote:
Assuming you're generating code that calls fn, as opposed to trying
to call it when generating code in the macro (usually a bad idea), you
should use esc(fn) in place of fn
On Friday, August 1, 2014 10:50:15 AM UTC-4, Abe Schneider wrote:
I've come across a problem where I have macro to define a grammar
(similar to how PEGParser currently works):
@grammar foo begin
rule[fn] = some + (rule | test)
end
where the `[fn]` next to the rule defines a function to call on the
results (in this case it's an expansion). The issue is that parsing the
Expr tree, `fn` is given as a Symbol (which makes sense).
However, if I try to turn `fn` into a function I run into the
namespace issue I've had previously. If `fn` is defined in my module,
it
works without problem. If it's defined in the code that imports the
module,
it will not work because that function doesn't exist in the namespace
of
the module.
I'm guessing there isn't an easy
[julia-users] Passing an expression from a macro to a function
I found one previous conversation related to this, but unfortunately the answer didn't work for me. Hopefully I'm not asking an obvious question. Suppose I have some macro: macro mytest(fn::Symbol, ex::Expr) quote $(esc(fn))($ex) end end and I have some function that takes in the expression to operate on it: myfun(ex::Expr) = ... the problem (which may be obvious) is that `$ex` gets evaluated with the macro, and I won't pass an `Expr` to `myfun`. A quick demonstration: foo(x, y) = x+y println(expandmacro(:(@mytest foo 1+2))) gives: begin # .../mtestmod.jl, line 38: foo(mtestmod.+(1,2)) end What I can't figure out is how to keep `ex` an expression that is passed to foo (at least without writing out Expr by hand). I've tried many combinations of syntax to try to preserve the Expr-ness of `ex`. Thanks!
Re: [julia-users] Passing an expression from a macro to a function
Awesome, that works. Thank you! Is there some master list of these macro-tricks? An IPython-type worksheet might serve as a nice look-up table for how to get things done with macros. On Sunday, August 3, 2014 3:04:53 PM UTC-4, Tim Holy wrote: It's not obvious. Try this: macro mytest(fn::Symbol, ex::Expr) qex = Expr(:quote, ex) # here's the magic incantation quote $(esc(fn))($qex) end end Works for symbols that you want to keep as symbols, too. But your `foo` method is not defined for an expression, so your test case will throw an error. --Tim On Sunday, August 03, 2014 11:46:43 AM Abe Schneider wrote: I found one previous conversation related to this, but unfortunately the answer didn't work for me. Hopefully I'm not asking an obvious question. Suppose I have some macro: macro mytest(fn::Symbol, ex::Expr) quote $(esc(fn))($ex) end end and I have some function that takes in the expression to operate on it: myfun(ex::Expr) = ... the problem (which may be obvious) is that `$ex` gets evaluated with the macro, and I won't pass an `Expr` to `myfun`. A quick demonstration: foo(x, y) = x+y println(expandmacro(:(@mytest foo 1+2))) gives: begin # .../mtestmod.jl, line 38: foo(mtestmod.+(1,2)) end What I can't figure out is how to keep `ex` an expression that is passed to foo (at least without writing out Expr by hand). I've tried many combinations of syntax to try to preserve the Expr-ness of `ex`. Thanks!
[julia-users] Using a callback from a macro
I've come across a problem where I have macro to define a grammar (similar to how PEGParser currently works): @grammar foo begin rule[fn] = some + (rule | test) end where the `[fn]` next to the rule defines a function to call on the results (in this case it's an expansion). The issue is that parsing the Expr tree, `fn` is given as a Symbol (which makes sense). However, if I try to turn `fn` into a function I run into the namespace issue I've had previously. If `fn` is defined in my module, it works without problem. If it's defined in the code that imports the module, it will not work because that function doesn't exist in the namespace of the module. I'm guessing there isn't an easy solution to fix this problem, but I thought I'd check to see if someone had an idea. Thanks!
[julia-users] Re: Using a callback from a macro
I think the problem is that I'm not accessing it directly through the macro parameters. The call: @grammar foo begin ... end passes just the single expression block to `foo`. Thus, I'm getting the function from the resulting Expr tree. Even if I quote it, it still ends up as an expression that contains a symbol that must be evaluated (unless I'm missing something...) On Friday, August 1, 2014 11:42:52 AM UTC-4, Simon Kornblith wrote: That looks like the output of :(esc(fn)), but you don't want to quote that, you want to evaluate it when generating the code, i.e. use $(esc(fn)) wherever you were previously using $fn On Friday, August 1, 2014 11:38:44 AM UTC-4, Abe Schneider wrote: That's correct, I'm generating code that keeps a pointer to that function to call later on. If I do that, I have the type `esc(fn)` in the macro (this is what I get when printing it out), and taking a dump of that argument gives: Expr head: Symbol call args: Array(Any,(2,)) 1: Symbol esc 2: Symbol fn typ: Any Does that mean instead of carrying around a function I will need to carry around the Expr? Thanks! On Friday, August 1, 2014 11:17:50 AM UTC-4, Simon Kornblith wrote: Assuming you're generating code that calls fn, as opposed to trying to call it when generating code in the macro (usually a bad idea), you should use esc(fn) in place of fn On Friday, August 1, 2014 10:50:15 AM UTC-4, Abe Schneider wrote: I've come across a problem where I have macro to define a grammar (similar to how PEGParser currently works): @grammar foo begin rule[fn] = some + (rule | test) end where the `[fn]` next to the rule defines a function to call on the results (in this case it's an expansion). The issue is that parsing the Expr tree, `fn` is given as a Symbol (which makes sense). However, if I try to turn `fn` into a function I run into the namespace issue I've had previously. If `fn` is defined in my module, it works without problem. If it's defined in the code that imports the module, it will not work because that function doesn't exist in the namespace of the module. I'm guessing there isn't an easy solution to fix this problem, but I thought I'd check to see if someone had an idea. Thanks!
[julia-users] Re: Using a callback from a macro
I tried a second approach where instead of keeping a function around I keep just the symbol. I have a `transform` function that applies the function to the resulting values. However, this only moves the problem to a new place. In my `transform` method I have: eval(Expr(:call, action, values)) where `action` is the symbol that I am now carrying around. The `eval` still works in the module's namespace, so I get the same error. If Julia had a way to access the namespace from the calling function, this could be made to work. Or for that matter, if I could get the calling namespace from the macro, I could then just get the macro that way. I'm guessing Julia doesn't currently have this functionality, but would it be something possible to add? On Friday, August 1, 2014 12:22:46 PM UTC-4, Abe Schneider wrote: I think the problem is that I'm not accessing it directly through the macro parameters. The call: @grammar foo begin ... end passes just the single expression block to `foo`. Thus, I'm getting the function from the resulting Expr tree. Even if I quote it, it still ends up as an expression that contains a symbol that must be evaluated (unless I'm missing something...) On Friday, August 1, 2014 11:42:52 AM UTC-4, Simon Kornblith wrote: That looks like the output of :(esc(fn)), but you don't want to quote that, you want to evaluate it when generating the code, i.e. use $(esc(fn)) wherever you were previously using $fn On Friday, August 1, 2014 11:38:44 AM UTC-4, Abe Schneider wrote: That's correct, I'm generating code that keeps a pointer to that function to call later on. If I do that, I have the type `esc(fn)` in the macro (this is what I get when printing it out), and taking a dump of that argument gives: Expr head: Symbol call args: Array(Any,(2,)) 1: Symbol esc 2: Symbol fn typ: Any Does that mean instead of carrying around a function I will need to carry around the Expr? Thanks! On Friday, August 1, 2014 11:17:50 AM UTC-4, Simon Kornblith wrote: Assuming you're generating code that calls fn, as opposed to trying to call it when generating code in the macro (usually a bad idea), you should use esc(fn) in place of fn On Friday, August 1, 2014 10:50:15 AM UTC-4, Abe Schneider wrote: I've come across a problem where I have macro to define a grammar (similar to how PEGParser currently works): @grammar foo begin rule[fn] = some + (rule | test) end where the `[fn]` next to the rule defines a function to call on the results (in this case it's an expansion). The issue is that parsing the Expr tree, `fn` is given as a Symbol (which makes sense). However, if I try to turn `fn` into a function I run into the namespace issue I've had previously. If `fn` is defined in my module, it works without problem. If it's defined in the code that imports the module, it will not work because that function doesn't exist in the namespace of the module. I'm guessing there isn't an easy solution to fix this problem, but I thought I'd check to see if someone had an idea. Thanks!
[julia-users] overloading an abstract interface
I have a pattern that has come up multiple times where in a module I might have some function `foo` which will call some other function `bar` on a given type. The exact function `bar` isn't defined yet because it is defined by the user-type. The problem I run into is that if `bar` is defined outside of the module, `foo` won't see it. Here is a simple example: module testmod export foo, Baz abstract Baz bar(x::Baz) = nothing function foo(x::Baz) return bar(x) end end with the user-defined code: using testmod type ConcreteBaz : Baz value::Int64 end function bar(baz::ConcreteBaz) return baz.value end baz = ConcreteBaz(42) println(result = $(foo(baz))) The output is `result = nothing`, and thus the wrong function is called. If I move all of the code to the same file it works. My question is: is this expected behavior? If so, what's the proper way of creating an interface for an abstract type which can later be overridden in a user file?
[julia-users] Re: overloading an abstract interface
Thank you. If I do: function testmodule.bar(baz::ConcreteBaz) return baz.value end it works. I'm guessing this means that Julia won't look outside of the namespace for a function. It hadn't occurred to me that writing: function module.fn() # ... end would work. On Saturday, July 26, 2014 6:23:16 PM UTC-4, Ivar Nesje wrote: Yes, this is expected behavior. You will have to define testmodule.baz() to not create a new function, but extend the existing function with a new method.
[julia-users] A julia version of diy-lisp (python project)
If you're interested, PEGParser includes a lisp grammar in the examples section. The transform converts to Julia expressions, but that can easily be changed.
[julia-users] Sorting behavior
It wasn't obvious to me initially why `sort` wasn't working for me (strings
and composite types). On further investigation it looks like that it only
works for single-dimension arrays -- which makes sense. However, if I type:
lst = [a b c]
sort(lst)
I get an error. The answer is that it's of type `Array{ASCIIString, 2}`,
whereas `sort` wants the type to be `Array{ASCIIString, 1}`. The correct
solution is to write this instead:
lst = [a, b, c]
sort(lst)
The problem seems to derive from two design decisions:
1. It is not obvious to someone new to Julia why one form gives a two
dimensional array whereas the other gives a one dimensional array.
2. `sort` doesn't try to determine if the array passed is actually jeust
one dimensional.
I'm not sure there is a simple solution. I assume there's a good reason for
(1) and (2) involves some overhead which might be undesirable.
Re: [julia-users] PEG Parser
I agree, I think PEG grammars are likely not well-suited to parsing large
amounts of data (especially where a hand-crafted parser can be created).
That said, I am curious as to why the speed difference exists. In the case
of CSV, I think the packrat caching mechanism probably doesn't help, but
disabling that is still slow.
The next logical point of slowness is the number of function-calls
required. I know that the multi-dispatch has some added penalty over
single-dispatch, but I'm not sure by how much. It's possible that I could
rewrite some of the CSV grammar to make less calls by either rewriting some
of the rules or by using more regular expressions.
Finally, the last likely place for the slow-down is the fact that PEGParser
has to create the AST as it goes along. This is a huge overhead that the
DataFrames CSV parser does not need to do since it's a single-purpose
parser.
I'm going to continue work on the Dot parser as I think it's probably more
useful (and more difficult to write by hand).
Also, I've created a pull request to add PEGParser to the official package
list. If it's something that interests other people, it would be great to
have other people involved.
On Sunday, July 6, 2014 9:08:40 PM UTC-4, John Myles White wrote:ly,,
Thanks for looking into this, Abe. That’s too bad that the CSV parser is
much slower than the hand-crafted one. PEG seems like a great tool for
tasks where maximum performance isn’t as important.
— John
On Jul 4, 2014, at 11:09 AM, Abe Schneider abe.sc...@gmail.com
javascript: wrote:
I got sidetracked by a couple of other things, but the parser is now
updated with a bunch of bug fixes. I have a preliminary CSV and graphdot
parser (very reduced from the full grammar). I'm starting to put together
some more comprehensive tests together.
As for speed comparison to DataFrames for parsing CSV it's much slower.
I've spent time trying to optimize things, but I suspect a large part of
the speed issue is the overhead of function calls. Also, I suspect it will
be hard to come close to the speed of DataFrames as the code looks like
it's fairly optimized for reading just CSV files.
After a few more tests are written, I'm getting ready to officially call a
version 0.1.
On Thursday, June 5, 2014 6:56:37 AM UTC-4, Abe Schneider wrote:
I also forgot to push the changes last night.
On Wednesday, June 4, 2014 11:01:33 PM UTC-4, Abe Schneider wrote:
After playing around with a bunch of alternatives, I think I've come up
with decent action semantics:
@transform name begin
label = action
end
For example, a simple graph grammar might look like:
@grammar nodetest begin
start = +node_def
node_def = node_label + node_name + lbrace + data + rbrace
node_name = string_value + space
data = *(line + semicolon)
line = string_value + space
string_value = r[_a-zA-Z][_a-zA-Z0-9]*
lbrace = { + space
rbrace = } + space
semicolon = ; + space
node_label = node + space
space = r[ \t\n]*
end
with it's actions to create some data structure:
type MyNode
name
values
function MyNode(name, values)
new(name, values)
end
end
with:
@transform tograph begin
# ignore these
lbrace = nothing
rbrase = nothing
semicolon = nothing
node_label = nothing
space = nothing
# special action so we don't have to define every label
default = children
string_value = node.value
value = node.value
line = children
data = MyNode(, children)
node_def = begin
local name = children[1]
local cnode = children[2]
cnode.name = name
return cnode
end
end
and finally, to apply the transform:
(ast, pos, error) = parse(nodetest, data)
result = apply(tograph, ast)
println(result)# {MyNode(foo,{a,b}),MyNode(bar,{c,d})}
The magic in '@transform' basically just creates the dictionary like
before, but automatically wraps the expression on the RHS as an anonymous
function (node, children) - expr.
I'm currently looking for a better name than 'children', as it's
potentially confusing and misleading. It's actually the values of the child
nodes (as opposed to node.children). Maybe cvalues?
On Sunday, May 25, 2014 10:28:45 PM UTC-4, Abe Schneider wrote:
I wrote a quick PEG Parser for Julia with Packrat capabilities:
https://github.com/abeschneider/PEGParser
It's a first draft and needs a ton of work, testing, etc., but if this
is of interest to anyone else, here is a quick description.
Grammars can be defined using most of the standard EBNF syntax. For
example, a simple math grammar can be defined as:
@grammar mathgrammar begin
start = expr
number = r([0-9]+)
expr = (term + op1 + expr) | term
term = (factor + op2 + term) | factor
factor = number | pfactor
pfactor = ('(' + expr + ')')
op1 = '+' | '-'
op2 = '*' | '/'
end
To parse a string with the grammar:
(node, pos, error) = parse(mathgrammar, 5
[julia-users] Re: PEG Parser
I got sidetracked by a couple of other things, but the parser is now
updated with a bunch of bug fixes. I have a preliminary CSV and graphdot
parser (very reduced from the full grammar). I'm starting to put together
some more comprehensive tests together.
As for speed comparison to DataFrames for parsing CSV it's much slower.
I've spent time trying to optimize things, but I suspect a large part of
the speed issue is the overhead of function calls. Also, I suspect it will
be hard to come close to the speed of DataFrames as the code looks like
it's fairly optimized for reading just CSV files.
After a few more tests are written, I'm getting ready to officially call a
version 0.1.
On Thursday, June 5, 2014 6:56:37 AM UTC-4, Abe Schneider wrote:
I also forgot to push the changes last night.
On Wednesday, June 4, 2014 11:01:33 PM UTC-4, Abe Schneider wrote:
After playing around with a bunch of alternatives, I think I've come up
with decent action semantics:
@transform name begin
label = action
end
For example, a simple graph grammar might look like:
@grammar nodetest begin
start = +node_def
node_def = node_label + node_name + lbrace + data + rbrace
node_name = string_value + space
data = *(line + semicolon)
line = string_value + space
string_value = r[_a-zA-Z][_a-zA-Z0-9]*
lbrace = { + space
rbrace = } + space
semicolon = ; + space
node_label = node + space
space = r[ \t\n]*
end
with it's actions to create some data structure:
type MyNode
name
values
function MyNode(name, values)
new(name, values)
end
end
with:
@transform tograph begin
# ignore these
lbrace = nothing
rbrase = nothing
semicolon = nothing
node_label = nothing
space = nothing
# special action so we don't have to define every label
default = children
string_value = node.value
value = node.value
line = children
data = MyNode(, children)
node_def = begin
local name = children[1]
local cnode = children[2]
cnode.name = name
return cnode
end
end
and finally, to apply the transform:
(ast, pos, error) = parse(nodetest, data)
result = apply(tograph, ast)
println(result)# {MyNode(foo,{a,b}),MyNode(bar,{c,d})}
The magic in '@transform' basically just creates the dictionary like
before, but automatically wraps the expression on the RHS as an anonymous
function (node, children) - expr.
I'm currently looking for a better name than 'children', as it's
potentially confusing and misleading. It's actually the values of the child
nodes (as opposed to node.children). Maybe cvalues?
On Sunday, May 25, 2014 10:28:45 PM UTC-4, Abe Schneider wrote:
I wrote a quick PEG Parser for Julia with Packrat capabilities:
https://github.com/abeschneider/PEGParser
It's a first draft and needs a ton of work, testing, etc., but if this
is of interest to anyone else, here is a quick description.
Grammars can be defined using most of the standard EBNF syntax. For
example, a simple math grammar can be defined as:
@grammar mathgrammar begin
start = expr
number = r([0-9]+)
expr = (term + op1 + expr) | term
term = (factor + op2 + term) | factor
factor = number | pfactor
pfactor = ('(' + expr + ')')
op1 = '+' | '-'
op2 = '*' | '/'
end
To parse a string with the grammar:
(node, pos, error) = parse(mathgrammar, 5*(2-6))
This will create an AST which can then be transformed to a value.
Currently this is accomplished by doing:
math = Dict()
math[number] = (node, children) - float(node.value)
math[expr] = (node, children) -
length(children) == 1 ? children : eval(Expr(:call, children[2],
children[1], children[3]))
math[factor] = (node, children) - children
math[pfactor] = (node, children) - children[2]
math[term] = (node, children) -
length(children) == 1 ? children : eval(Expr(:call, children[2],
children[1], children[3]))
math[op1] = (node, children) - symbol(node.value)
math[op2] = (node, children) - symbol(node.value)
Ideally, I would like to simplify this to using multi-dispatch on
symbols (see previous post), but for now this is the easiest way to define
actions based on node attributes.
Finally, to transform the tree:
result = transform(math, node) # will give the value of 20
Originally I was going to attach the transforms to the rules themselves
(similiar to boost::spirit). However, there were two reasons for not doing
this:
1. To implement the packrat part of the parser, I needed to cache
the results which meant building an AST anyways
2. It's nice to be apply to get different transforms for the same
grammar (e.g. you may want to transform the result into HTML, LaTeX,
etc.)
The downside of the separation is that it adds some more complexity to
the process.
Re: [julia-users] type design question
Out of curiosity, what is wrong with Scala's method to handle this problem (where only the concrete types can have constructors)? From the thread you linked, it seems like the main complaint against doing so would create a stronger coupling between the abstract and concrete types than desired. However, I think the grouping method suggested above creates the same problem. Sure, you can create a layer of abstraction with a getter/setter. However, if you're really worried about fields disappearing, the grouping method creates more work (you not only change to type, but you have to alter the getters/setters). Also, it seems like the separation between structure and behaviour is already broken by having concrete types. Allowing a explicit declaration of a structure hierarchy of what you are already doing implicitly with the grouping doesn't change the structure/behaviour relationship to me. If you couldn't group and if there weren't concrete types, then I would have to agree with you. Thanks for listening! On Saturday, June 21, 2014 11:35:53 AM UTC-4, Stefan Karpinski wrote: In particular, I think the constructor issue cuts to the heart of the matter. If you have any good thoughts on that, it might help revive that discussion. So far the best proposal for subtype construction in the presence of abstract types with fields basically leads you to the parent structure as leading field pattern that's been suggested here. On Jun 21, 2014, at 11:23 AM, Stefan Karpinski stefan.k...@gmail.com javascript: wrote: It's non-orthogonal because subtyping becomes about sharing both behavior and structure, not just behavior. An orthogonal approach would use an independent feature for sharing structure. This is all covered in the discussion in https://github.com/JuliaLang/julia/issues/4935. On Jun 21, 2014, at 9:59 AM, Abe Schneider abe.sc...@gmail.com javascript: wrote: I think that's a fair point (though I may disagree that it's non-othogonal). My main point is that you are already implicitly using a hierarchical structure with grouping. Therefore, I don't see this as adding a new feature to the language, it's already something you can do in Julia, but rather providing the syntax to allow the relationship to be declared in useful way. The code: abstract A x::Int64 end type B : A y::Int64 end type C : A z::Float64 end is much clearer to me than: type A int x::Int64 end abstract AType getx(a::AType) = a.x setx(a::AType, x::Int64) = a.x = x type B : AType parent::A end type C : AType parent::C end or: abstract A type B : A x::Int64 y::Int64 end type C : A x::Int64 z::Float64 end The last two examples violate DRY, make for potentially difficult to maintain code, and are less obvious to someone else looking at the code as to what is happening. On Saturday, June 21, 2014 9:02:52 AM UTC-4, Stefan Karpinski wrote: The harm is adding unnecessary, non-orthogonal language features. We've tended not to add features until it becomes very clear that we need them and why, and that's been a good approach so far. Aside from several discussions similar to this, I don't feel like the problems this feature would alleviate are especially pressing. So far subtyping in Julia is purely about behavior, not structure and changing that even a little bit would require some significant motivation – certainly more than why not? On Jun 21, 2014, at 8:45 AM, Abe Schneider abe.sc...@gmail.com wrote: After some thought, it occurred to me that with the grouping you suggested, you are really implementing a type of inheritance. If you rename your sub-type as 'parent' you have: type ParentClass # vars end abstract ParentClassType type ChildClass : ParentClassType parent::ParentClass # vars end which is exactly what a Mixin paradigm would accomplish, only that the compiler would help with the construction (which at the end of the day is all OOP really is). If this type of construction can be done by hand, I'm not sure what the harm is in having the compiler make things easier. On Saturday, June 21, 2014 3:09:36 AM UTC-4, Tobias Knopp wrote: When I switched from C++ to C# I had a similar opinion about the issue of duplicated fields in all classes implementing an interface. But when getting used to grouping things together and using has-a relations this has changed my thinking about it. I now think that it enforces structuring code in a sane way. When inheriting field members one often observes far to late that one has structural issues and refactoring then becomes really hard. When looking at Julia base code (or several of the packages) one will see that the issue of inheriting field members does not come up so much. One side comment. In Julia the possibility to use duck typing gives a lot of flexibility. And when one reaches limits (e.g. due to the absence
Re: [julia-users] type design question
After some thought, it occurred to me that with the grouping you suggested, you are really implementing a type of inheritance. If you rename your sub-type as 'parent' you have: type ParentClass # vars end abstract ParentClassType type ChildClass : ParentClassType parent::ParentClass # vars end which is exactly what a Mixin paradigm would accomplish, only that the compiler would help with the construction (which at the end of the day is all OOP really is). If this type of construction can be done by hand, I'm not sure what the harm is in having the compiler make things easier. On Saturday, June 21, 2014 3:09:36 AM UTC-4, Tobias Knopp wrote: When I switched from C++ to C# I had a similar opinion about the issue of duplicated fields in all classes implementing an interface. But when getting used to grouping things together and using has-a relations this has changed my thinking about it. I now think that it enforces structuring code in a sane way. When inheriting field members one often observes far to late that one has structural issues and refactoring then becomes really hard. When looking at Julia base code (or several of the packages) one will see that the issue of inheriting field members does not come up so much. One side comment. In Julia the possibility to use duck typing gives a lot of flexibility. And when one reaches limits (e.g. due to the absence of abstract multiple inheritance) duck typing is often a solution. Am Samstag, 21. Juni 2014 03:43:57 UTC+2 schrieb Abe Schneider: I agree, I think it's the best solution given the tools (and what I'm going to use for my code). However, it still feels more like a hack around the design than good programming practice. On Friday, June 20, 2014 5:41:02 PM UTC-4, Spencer Russell wrote: I'd just like to second Jameson's suggestion of aggregating the common fields into a type that all your subclasses contain. I did quite a bit of thinking on this issue when it came up in AudioIO, and was lucky enough to have both Jeff and Stefan around to bounce ideas off of. My main issues with the duplicated data in subclasses were: 1. It's annoying to have to add the same set of fields every time you define a subtype, and violates DRY. It's also error prone. 2. If you want to add a feature to the base type that requires a new field, EVERYONE WHO EVER SUBTYPED your base type now has to add the field to their subtype. It's bad enough when this is within your own codebase, but if there's other code subtyping it then you're really in trouble. Encapsulating the common fields solves both those issues.If you want to add new fields later on you can just add them to the aggregating type. Most importantly, it does it in really easy-to-reason-about way, without adding any tricky edge cases or complicated rules for developers to understand. peace, s On Fri, Jun 20, 2014 at 3:57 PM, Abe Schneider abe.sc...@gmail.com wrote: I was thinking something along those lines, but as was pointed out, you would have to also create the constructors. Unfortunately, I'm on my phone right now, so I can't effectively post code. I was thinking of a 'mixin' macro which would create a new type (with constructor): @mixin Foo : Bar Baz Would create Foo from both Bar and Baz. However, because there is no MI, you could only inherit from Bar. While it does have some magic to it, it might not be awful. Also, you could still make an outer constructor for it. Of course, I don't know the actual technical challenges to making it, since I haven't had time to write any code.
Re: [julia-users] type design question
I think that's a fair point (though I may disagree that it's non-othogonal). My main point is that you are already implicitly using a hierarchical structure with grouping. Therefore, I don't see this as adding a new feature to the language, it's already something you can do in Julia, but rather providing the syntax to allow the relationship to be declared in useful way. The code: abstract A x::Int64 end type B : A y::Int64 end type C : A z::Float64 end is much clearer to me than: type A int x::Int64 end abstract AType getx(a::AType) = a.x setx(a::AType, x::Int64) = a.x = x type B : AType parent::A end type C : AType parent::C end or: abstract A type B : A x::Int64 y::Int64 end type C : A x::Int64 z::Float64 end The last two examples violate DRY, make for potentially difficult to maintain code, and are less obvious to someone else looking at the code as to what is happening. On Saturday, June 21, 2014 9:02:52 AM UTC-4, Stefan Karpinski wrote: The harm is adding unnecessary, non-orthogonal language features. We've tended not to add features until it becomes very clear that we need them and why, and that's been a good approach so far. Aside from several discussions similar to this, I don't feel like the problems this feature would alleviate are especially pressing. So far subtyping in Julia is purely about behavior, not structure and changing that even a little bit would require some significant motivation – certainly more than why not? On Jun 21, 2014, at 8:45 AM, Abe Schneider abe.sc...@gmail.com javascript: wrote: After some thought, it occurred to me that with the grouping you suggested, you are really implementing a type of inheritance. If you rename your sub-type as 'parent' you have: type ParentClass # vars end abstract ParentClassType type ChildClass : ParentClassType parent::ParentClass # vars end which is exactly what a Mixin paradigm would accomplish, only that the compiler would help with the construction (which at the end of the day is all OOP really is). If this type of construction can be done by hand, I'm not sure what the harm is in having the compiler make things easier. On Saturday, June 21, 2014 3:09:36 AM UTC-4, Tobias Knopp wrote: When I switched from C++ to C# I had a similar opinion about the issue of duplicated fields in all classes implementing an interface. But when getting used to grouping things together and using has-a relations this has changed my thinking about it. I now think that it enforces structuring code in a sane way. When inheriting field members one often observes far to late that one has structural issues and refactoring then becomes really hard. When looking at Julia base code (or several of the packages) one will see that the issue of inheriting field members does not come up so much. One side comment. In Julia the possibility to use duck typing gives a lot of flexibility. And when one reaches limits (e.g. due to the absence of abstract multiple inheritance) duck typing is often a solution. Am Samstag, 21. Juni 2014 03:43:57 UTC+2 schrieb Abe Schneider: I agree, I think it's the best solution given the tools (and what I'm going to use for my code). However, it still feels more like a hack around the design than good programming practice. On Friday, June 20, 2014 5:41:02 PM UTC-4, Spencer Russell wrote: I'd just like to second Jameson's suggestion of aggregating the common fields into a type that all your subclasses contain. I did quite a bit of thinking on this issue when it came up in AudioIO, and was lucky enough to have both Jeff and Stefan around to bounce ideas off of. My main issues with the duplicated data in subclasses were: 1. It's annoying to have to add the same set of fields every time you define a subtype, and violates DRY. It's also error prone. 2. If you want to add a feature to the base type that requires a new field, EVERYONE WHO EVER SUBTYPED your base type now has to add the field to their subtype. It's bad enough when this is within your own codebase, but if there's other code subtyping it then you're really in trouble. Encapsulating the common fields solves both those issues.If you want to add new fields later on you can just add them to the aggregating type. Most importantly, it does it in really easy-to-reason-about way, without adding any tricky edge cases or complicated rules for developers to understand. peace, s On Fri, Jun 20, 2014 at 3:57 PM, Abe Schneider abe.sc...@gmail.com wrote: I was thinking something along those lines, but as was pointed out, you would have to also create the constructors. Unfortunately, I'm on my phone right now, so I can't effectively post code. I was thinking of a 'mixin' macro which would create a new type (with constructor): @mixin Foo : Bar Baz Would create Foo from both Bar and Baz. However, because
Re: [julia-users] type design question
Thanks for the links!
My main complaint with declaring the variables rather than defining the
interface. I was never a fan of Java interfaces, which I think is
essentially what the current Julia design already mimics (if not in a
slightly less formal design). Having to redeclare variables for larger
projects can become difficult for the programmer who might have to visit
each of the interfaces defined to see which variables are expected to be
exported. If you only allow for a single interface to be defined, you are
saved some work, but at the expense of design (I can think of many
instances where multiple instances are desirable).
I think allowing abstract types to declare variables solves most of this
problem. I can appreciate people's reserve on implementing
multiple-inheritance. As another data point, most modern languages seem to
using Mixins instead (e.g.: Java 8, Scala, Ruby, Swift). Java 8 is
especially interesting as they started off with the same interface design.
In Scala (if my memory serves me correctly), Mixins are enabled by having a
separate structure-type called traits. What distinguishes classes from
traits is that classes are allowed constructors whereas traits are not.
This eliminates most of the problems people have with implementing multiple
inheritance.
Obviously, it's possible to design the NNModules with the current feature
set of Julia. However, at least to me, it won't feel like a good design if
I have to repeat the same fields for all children of NNModules (there is
more than just output). It also makes me worry as to how well Julia will
scale for larger projects, though the answer may be that is not it's main
use.
On Friday, June 20, 2014 2:30:04 AM UTC-4, Tobias Knopp wrote:
Hi Abe,
There has been various discussions about this in the following threads:
https://github.com/JuliaLang/julia/issues/4935
https://github.com/JuliaLang/julia/issues/1974
You might see that there is not yet a consus if one of these features will
be implemented or not.
I have been myself trying to figure out how to resolve the issue of
duplicating code until Jeff and Tim showed me the solution Jameson has
outlined above.
I don't really see why solution is too verbose. You only have to define
the getter function for public types where it is encouraged anyway to
define functions for data encapsulation. Fields are in the current view
private.
You might be also interested in
https://github.com/JuliaLang/julia/issues/6975 where we discuss how to
define interfaces more formally as it is done now.
Cheers,
Tobi
Am Freitag, 20. Juni 2014 04:15:21 UTC+2 schrieb Abe Schneider:
Okay, so that matches the Java interface style, where you would then make
one function per variable you want to officially sanction as required.
One thing I didn't except from doing this is that if you do:
type OtherModule : NNModule end
mod = OtherModule
getoutput(mod)
I was expecting an error within the method that 'output' doesn't exist
for 'mod'. Instead I get:
ERROR: no method getoutput(Type{OtherModule})
which is a much nicer complaint.
The downside of the design is that now I have to duplicate the variable
for every type, but also make sure there's a function defined at top.
However, of the choices to make, it's probably the cleanest (though a macro
might save a little on typing).
Thanks for the help!
On Thursday, June 19, 2014 9:14:34 AM UTC-4, Jameson wrote:
here is my recommended solution:
abstract NNModule
getoutput(nnm::NNModule) = nnm.output # -- add this line
type LinearModule : NNModule
weight::Array{Float64}
output::Array{Float64}
end
type SigmoidModule : NNModule
output::Array{Float64}
end
On Thu, Jun 19, 2014 at 8:18 AM, Abe Schneider abe.sc...@gmail.com
wrote:
I'm trying to figure out the best way to factor code that consists of
repeated modules which all share some common attributes.
Specifically, if I have (very abbreviated from actual code):
abstract NNModule
type LinearModule : NNModule
weight::Array{Float64}
output::Array{Float64}
end
type SigmoidModule : NNModule
output::Array{Float64}
end
For both modules I need an 'output' variable. I know the idea of
putting variables in an abstract type has come up before. However, while
it
would solve this problem, it is also not yet resolved (and may or may not
actually happy).
Given that, two alternatives occur to me:
1. Keep the current design and be okay with repeated code (difficult)
2. Use a macro to create the repeated code for me
Option (1) is certainly the easiest, and probably the one I'll stick
with for now. However, it seems error prone and (IMHO) produces ugly code.
Option (2) has the problem that it can make what's actually happening
opaque and could potentially produce bugs that are difficult to track down.
So my questions are:
(1) Have other people come across design issues like this before
Re: [julia-users] type design question
It's good to see other people want multiple inheritance too :). One thing I haven't seen discussion yet is using Scala's method of constructors to make deal with the fields and MI. It seems like abstract types could be equivalent to Scala traits (if abstract types allowed fields), and regular types to classes. Abstract types wouldn't be allowed constructors (like traits), and thus the regular types would have complete control as how to initialize the variables. Thus, in Julia the main difference between the two would be whether a constructor exists (in keeping with their name). I generally agree with not wanting deep hierarchies. However, as it stands, it seems like even shallow hierarchies are difficult. I also think it's languages like Java that tend to create the large hierarchies because it doesn't have multiple inheritance of mixins. Generally in c++ things tend to be fairly shallow.
Re: [julia-users] type design question
I was thinking something along those lines, but as was pointed out, you would have to also create the constructors. Unfortunately, I'm on my phone right now, so I can't effectively post code. I was thinking of a 'mixin' macro which would create a new type (with constructor): @mixin Foo : Bar Baz Would create Foo from both Bar and Baz. However, because there is no MI, you could only inherit from Bar. While it does have some magic to it, it might not be awful. Also, you could still make an outer constructor for it. Of course, I don't know the actual technical challenges to making it, since I haven't had time to write any code.
Re: [julia-users] type design question
I was thinking something along those lines, but as was pointed out, you would have to also create the constructors. Unfortunately, I'm on my phone right now, so I can't effectively post code. I was thinking of a 'mixin' macro which would create a new type (with constructor): @mixin Foo : Bar Baz Would create Foo from both Bar and Baz. However, because there is no MI, you could only inherit from Bar. While it does have some magic to it, it might not be awful. Also, you could still make an outer constructor for it. Of course, I don't know the actual technical challenges to making it, since I haven't had time to write any code.
Re: [julia-users] type design question
I agree, I think it's the best solution given the tools (and what I'm going to use for my code). However, it still feels more like a hack around the design than good programming practice. On Friday, June 20, 2014 5:41:02 PM UTC-4, Spencer Russell wrote: I'd just like to second Jameson's suggestion of aggregating the common fields into a type that all your subclasses contain. I did quite a bit of thinking on this issue when it came up in AudioIO, and was lucky enough to have both Jeff and Stefan around to bounce ideas off of. My main issues with the duplicated data in subclasses were: 1. It's annoying to have to add the same set of fields every time you define a subtype, and violates DRY. It's also error prone. 2. If you want to add a feature to the base type that requires a new field, EVERYONE WHO EVER SUBTYPED your base type now has to add the field to their subtype. It's bad enough when this is within your own codebase, but if there's other code subtyping it then you're really in trouble. Encapsulating the common fields solves both those issues.If you want to add new fields later on you can just add them to the aggregating type. Most importantly, it does it in really easy-to-reason-about way, without adding any tricky edge cases or complicated rules for developers to understand. peace, s On Fri, Jun 20, 2014 at 3:57 PM, Abe Schneider abe.sc...@gmail.com javascript: wrote: I was thinking something along those lines, but as was pointed out, you would have to also create the constructors. Unfortunately, I'm on my phone right now, so I can't effectively post code. I was thinking of a 'mixin' macro which would create a new type (with constructor): @mixin Foo : Bar Baz Would create Foo from both Bar and Baz. However, because there is no MI, you could only inherit from Bar. While it does have some magic to it, it might not be awful. Also, you could still make an outer constructor for it. Of course, I don't know the actual technical challenges to making it, since I haven't had time to write any code.
[julia-users] type design question
I'm trying to figure out the best way to factor code that consists of
repeated modules which all share some common attributes.
Specifically, if I have (very abbreviated from actual code):
abstract NNModule
type LinearModule : NNModule
weight::Array{Float64}
output::Array{Float64}
end
type SigmoidModule : NNModule
output::Array{Float64}
end
For both modules I need an 'output' variable. I know the idea of putting
variables in an abstract type has come up before. However, while it would
solve this problem, it is also not yet resolved (and may or may not
actually happy).
Given that, two alternatives occur to me:
1. Keep the current design and be okay with repeated code (difficult)
2. Use a macro to create the repeated code for me
Option (1) is certainly the easiest, and probably the one I'll stick with
for now. However, it seems error prone and (IMHO) produces ugly code.
Option (2) has the problem that it can make what's actually happening
opaque and could potentially produce bugs that are difficult to track down.
So my questions are:
(1) Have other people come across design issues like this before, and how
did they deal with it?
(2) Is there a macro out there already to accomplish this? After a quick
search, I found '@delegate' proposed as a solution for something like this,
but it was never officially merged. If not, any suggestions on how such a
macro should act/look?
As a side note, it would be great if Julia added properties to abstract
types. The current design gives something equivalent to Java's interfaces,
where most languages opt for mixins instead.
Thanks!
Re: [julia-users] type design question
Okay, so that matches the Java interface style, where you would then make
one function per variable you want to officially sanction as required.
One thing I didn't except from doing this is that if you do:
type OtherModule : NNModule end
mod = OtherModule
getoutput(mod)
I was expecting an error within the method that 'output' doesn't exist for
'mod'. Instead I get:
ERROR: no method getoutput(Type{OtherModule})
which is a much nicer complaint.
The downside of the design is that now I have to duplicate the variable for
every type, but also make sure there's a function defined at top. However,
of the choices to make, it's probably the cleanest (though a macro might
save a little on typing).
Thanks for the help!
On Thursday, June 19, 2014 9:14:34 AM UTC-4, Jameson wrote:
here is my recommended solution:
abstract NNModule
getoutput(nnm::NNModule) = nnm.output # -- add this line
type LinearModule : NNModule
weight::Array{Float64}
output::Array{Float64}
end
type SigmoidModule : NNModule
output::Array{Float64}
end
On Thu, Jun 19, 2014 at 8:18 AM, Abe Schneider abe.sc...@gmail.com
javascript: wrote:
I'm trying to figure out the best way to factor code that consists of
repeated modules which all share some common attributes.
Specifically, if I have (very abbreviated from actual code):
abstract NNModule
type LinearModule : NNModule
weight::Array{Float64}
output::Array{Float64}
end
type SigmoidModule : NNModule
output::Array{Float64}
end
For both modules I need an 'output' variable. I know the idea of putting
variables in an abstract type has come up before. However, while it would
solve this problem, it is also not yet resolved (and may or may not
actually happy).
Given that, two alternatives occur to me:
1. Keep the current design and be okay with repeated code (difficult)
2. Use a macro to create the repeated code for me
Option (1) is certainly the easiest, and probably the one I'll stick with
for now. However, it seems error prone and (IMHO) produces ugly code.
Option (2) has the problem that it can make what's actually happening
opaque and could potentially produce bugs that are difficult to track down.
So my questions are:
(1) Have other people come across design issues like this before, and how
did they deal with it?
(2) Is there a macro out there already to accomplish this? After a quick
search, I found '@delegate' proposed as a solution for something like this,
but it was never officially merged. If not, any suggestions on how such a
macro should act/look?
As a side note, it would be great if Julia added properties to abstract
types. The current design gives something equivalent to Java's interfaces,
where most languages opt for mixins instead.
Thanks!
Re: [julia-users] animation using Gtk+/Cairo
Thank you everyone for the fast replies! After looking at ImageView and the sources, here's the solution I came up with: w = Gtk.@Window() | (body=Gtk.@Box(:v) | (canvas=Gtk.@Canvas(600, 600)) | showall function redraw_canvas(canvas) ctx = getgc(canvas) h = height(canvas) w = width(canvas) # draw background rectangle(ctx, 0, 0, w, h) set_source_rgb(ctx, 1, 1, 1) fill(ctx) # draw objects # ... # tell Gtk+ to redisplay draw(canvas) end function init(canvas, delay::Float64, interval::Float64) update_timer = Timer(timer - redraw_canvas(canvas)) start_timer(update_timer, delay, interval) end update_timer = init(canvas, 2, 1) if !isinteractive() wait(Condition()) end stop_timer(update_timer) I haven't looked yet into what is required to do double-buffering (or if it's enabled by default). I also copied the 'wait(Condition())' from the docs, though it's not clear to me what the condition is (if I close the window, the program is still running -- I'm assuming that means I need to connect the signal for window destruction to said condition). A On Monday, June 16, 2014 9:33:42 PM UTC-4, Jameson wrote: I would definately use Julia's timers. See `Gtk.jl/src/cairo.jl` for an example interface to the Cairo backing to a Gtk window (used in `Winston.jl/src/gtk.jl`). If you are using this wrapper, call `draw(w)` to force a redraw immediately, or `draw(w,false)` to queue a redraw request for when Gtk is idle. On Mon, Jun 16, 2014 at 9:12 PM, Tim Holy tim@gmail.com javascript: wrote: ImageView's navigation.jl contains an example. The default branch is Tk (because as far as binary distribution goes, Tk is solved and Gtk isn't yet), but it has a gtk branch you can look at. --Tim On Monday, June 16, 2014 04:01:46 PM Abe Schneider wrote: I was looking for a way to display a simulation in Julia. Originally I was going to just use PyPlot, but it occurred to me it would be better to just use Gtk+ + Cairo to do the drawing rather than something whose main purpose is drawing graphs. So far, following the examples on the Github page, I have no problem creating a window with a Cairo canvas. I can also display content on the canvas fairly easily (which speaks volumes on the awesomeness of Julia and the Gtk+ library). However, after looking through the code and samples, it's not obvious to me how to redraw the canvas every fraction of a second to display new content. I did find an example of animating with Cairo and Gtk+ in C (http://cairographics.org/threaded_animation_with_cairo/). However, I assume one would want to use Julia's timers instead of of GLibs? Secondly, there in their function 'timer_exe', call is made directly to Gtk+ to send a redraw queue to the window. Is there a cleaner way to do it with the Gtk+ library? Thanks! A
Re: [julia-users] animation using Gtk+/Cairo
@Tim: Awesome, exactly what I was looking for. Thank you. @Jameson: Just to check, do you mean something like: function redraw_canvas(canvas) draw(canvas) end draw(canvas) do widget # ... end If so, I'll re-post my code with the update. It may be useful to someone else to see the entire code as an example. Thanks! A On Tuesday, June 17, 2014 10:44:16 AM UTC-4, Jameson wrote: This code is not valid, since getgc does not always have a valid drawing context to return. Instead you need to provide Canvas with a callback function via a call to redraw in which you do all the work, then just call draw(canvas) in your timer callback to force an update to the view. double-buffering is enabled by default. wait(Condition()) is the same wait(), and means sleep until this task is signaled, and thereby prevents the program from exiting early On Tue, Jun 17, 2014 at 7:46 AM, Abe Schneider abe.sc...@gmail.com javascript: wrote: Thank you everyone for the fast replies! After looking at ImageView and the sources, here's the solution I came up with: w = Gtk.@Window() | (body=Gtk.@Box(:v) | (canvas=Gtk.@Canvas(600, 600)) | showall function redraw_canvas(canvas) ctx = getgc(canvas) h = height(canvas) w = width(canvas) # draw background rectangle(ctx, 0, 0, w, h) set_source_rgb(ctx, 1, 1, 1) fill(ctx) # draw objects # ... # tell Gtk+ to redisplay draw(canvas) end function init(canvas, delay::Float64, interval::Float64) update_timer = Timer(timer - redraw_canvas(canvas)) start_timer(update_timer, delay, interval) end update_timer = init(canvas, 2, 1) if !isinteractive() wait(Condition()) end stop_timer(update_timer) I haven't looked yet into what is required to do double-buffering (or if it's enabled by default). I also copied the 'wait(Condition())' from the docs, though it's not clear to me what the condition is (if I close the window, the program is still running -- I'm assuming that means I need to connect the signal for window destruction to said condition). A On Monday, June 16, 2014 9:33:42 PM UTC-4, Jameson wrote: I would definately use Julia's timers. See `Gtk.jl/src/cairo.jl` for an example interface to the Cairo backing to a Gtk window (used in `Winston.jl/src/gtk.jl`). If you are using this wrapper, call `draw(w)` to force a redraw immediately, or `draw(w,false)` to queue a redraw request for when Gtk is idle. On Mon, Jun 16, 2014 at 9:12 PM, Tim Holy tim@gmail.com wrote: ImageView's navigation.jl contains an example. The default branch is Tk (because as far as binary distribution goes, Tk is solved and Gtk isn't yet), but it has a gtk branch you can look at. --Tim On Monday, June 16, 2014 04:01:46 PM Abe Schneider wrote: I was looking for a way to display a simulation in Julia. Originally I was going to just use PyPlot, but it occurred to me it would be better to just use Gtk+ + Cairo to do the drawing rather than something whose main purpose is drawing graphs. So far, following the examples on the Github page, I have no problem creating a window with a Cairo canvas. I can also display content on the canvas fairly easily (which speaks volumes on the awesomeness of Julia and the Gtk+ library). However, after looking through the code and samples, it's not obvious to me how to redraw the canvas every fraction of a second to display new content. I did find an example of animating with Cairo and Gtk+ in C (http://cairographics.org/threaded_animation_with_cairo/). However, I assume one would want to use Julia's timers instead of of GLibs? Secondly, there in their function 'timer_exe', call is made directly to Gtk+ to send a redraw queue to the window. Is there a cleaner way to do it with the Gtk+ library? Thanks! A
Re: [julia-users] animation using Gtk+/Cairo
Okay, what works for me using your suggestion (except for me [with a 1 day old Julia and package] it's 'draw' and not 'redraw'), I have: function update(canvas, scene::Scene) # update scene # ... # redraw draw(canvas) end function init(canvas, scene::Scene) update_timer = Timer(timer - update(canvas, scene)) start_timer(update_timer, 1, 0.5) return update_timer end function draw_scene(canvas, scene::Scene) ctx = getgc(canvas) h = height(canvas) w = width(canvas) rectangle(ctx, 0, 0, w, h) set_source_rgb(ctx, 1, 1, 1) fill(ctx) # use scene to draw other elements # ... end scene = Scene(...) update_timer = init(canvas, scene) draw(canvas - draw_scene(canvas, scene), canvas) if !isinteractive() cond = Condition() signal_connect(win, :destroy) do widget notify(cond) end wait(cond) end stop_timer(update_timer) I wasn't sure if the way I bound the scene to the redraw is the nicest approach to take. If the function took additional parameters, that seems like it might be the most straight forward. E.g.: draw(canvas, scene) do widget # ... end # here 'myscene' would be passed as the second parameter to the other draw draw(canvas, myscene) A On Tuesday, June 17, 2014 1:16:11 PM UTC-4, Jameson wrote: Yes. Although I think the draw...do function is actually redraw...do (this is actually a shared interface with Tk.jl, although I recommend Gtk :) Sent from my phone. On Tuesday, June 17, 2014, Abe Schneider abe.sc...@gmail.com javascript: wrote: @Tim: Awesome, exactly what I was looking for. Thank you. @Jameson: Just to check, do you mean something like: function redraw_canvas(canvas) draw(canvas) end draw(canvas) do widget # ... end If so, I'll re-post my code with the update. It may be useful to someone else to see the entire code as an example. Thanks! A On Tuesday, June 17, 2014 10:44:16 AM UTC-4, Jameson wrote: This code is not valid, since getgc does not always have a valid drawing context to return. Instead you need to provide Canvas with a callback function via a call to redraw in which you do all the work, then just call draw(canvas) in your timer callback to force an update to the view. double-buffering is enabled by default. wait(Condition()) is the same wait(), and means sleep until this task is signaled, and thereby prevents the program from exiting early On Tue, Jun 17, 2014 at 7:46 AM, Abe Schneider abe.sc...@gmail.com wrote: Thank you everyone for the fast replies! After looking at ImageView and the sources, here's the solution I came up with: w = Gtk.@Window() | (body=Gtk.@Box(:v) | (canvas=Gtk.@Canvas(600, 600)) | showall function redraw_canvas(canvas) ctx = getgc(canvas) h = height(canvas) w = width(canvas) # draw background rectangle(ctx, 0, 0, w, h) set_source_rgb(ctx, 1, 1, 1) fill(ctx) # draw objects # ... # tell Gtk+ to redisplay draw(canvas) end function init(canvas, delay::Float64, interval::Float64) update_timer = Timer(timer - redraw_canvas(canvas)) start_timer(update_timer, delay, interval) end update_timer = init(canvas, 2, 1) if !isinteractive() wait(Condition()) end stop_timer(update_timer) I haven't looked yet into what is required to do double-buffering (or if it's enabled by default). I also copied the 'wait(Condition())' from the docs, though it's not clear to me what the condition is (if I close the window, the program is still running -- I'm assuming that means I need to connect the signal for window destruction to said condition). A On Monday, June 16, 2014 9:33:42 PM UTC-4, Jameson wrote: I would definately use Julia's timers. See `Gtk.jl/src/cairo.jl` for an example interface to the Cairo backing to a Gtk window (used in `Winston.jl/src/gtk.jl`). If you are using this wrapper, call `draw(w)` to force a redraw immediately, or `draw(w,false)` to queue a redraw request for when Gtk is idle. On Mon, Jun 16, 2014 at 9:12 PM, Tim Holy tim@gmail.com wrote: ImageView's navigation.jl contains an example. The default branch is Tk (because as far as binary distribution goes, Tk is solved and Gtk isn't yet), but it has a gtk branch you can look at. --Tim On Monday, June 16, 2014 04:01:46 PM Abe Schneider wrote: I was looking for a way to display a simulation in Julia. Originally I was going to just use PyPlot, but it occurred to me it would be better to just use Gtk+ + Cairo to do the drawing rather than something whose main purpose is drawing graphs. So far, following the examples on the Github page, I have no problem creating a window with a Cairo canvas. I can also display content on the canvas fairly easily (which speaks volumes on the awesomeness of Julia and the Gtk+ library). However, after looking through the code and samples, it's not obvious to me how to redraw the canvas every fraction of a second to display
[julia-users] animation using Gtk+/Cairo
I was looking for a way to display a simulation in Julia. Originally I was going to just use PyPlot, but it occurred to me it would be better to just use Gtk+ + Cairo to do the drawing rather than something whose main purpose is drawing graphs. So far, following the examples on the Github page, I have no problem creating a window with a Cairo canvas. I can also display content on the canvas fairly easily (which speaks volumes on the awesomeness of Julia and the Gtk+ library). However, after looking through the code and samples, it's not obvious to me how to redraw the canvas every fraction of a second to display new content. I did find an example of animating with Cairo and Gtk+ in C (http://cairographics.org/threaded_animation_with_cairo/). However, I assume one would want to use Julia's timers instead of of GLibs? Secondly, there in their function 'timer_exe', call is made directly to Gtk+ to send a redraw queue to the window. Is there a cleaner way to do it with the Gtk+ library? Thanks! A
Re: [julia-users] Faster sub-strings
Awesome, thanks for the pointer. I'll open up an issue for this.
One thought, it might be nice to have something a little more automatic.
Specifically, I was thinking if there was an 'ImmutableString' class, you
would always know that you should make a view rather than a copy. It would
also be in keeping with the general design where specifying a type allows
the compiler to do nice optimizations.
On Wednesday, June 11, 2014 9:14:27 AM UTC-4, Kevin Squire wrote:
Hi Abe,
Looks like you just reimplemented SubString.
julia x = Hi there!
Hi there!
julia SubString(x, 2, 4)
i t
julia typeof(ans)
SubString{ASCIIString} (constructor with 1 method)
Which is totally understandable, as there seems to be almost zero
documentation about them. Would you mind opening an issue about that?
Cheers,
Kevin
On Wed, Jun 11, 2014 at 5:08 AM, Abe Schneider abe.sc...@gmail.com
javascript: wrote:
I'm in the middle of profiling my code, and I noticed that I'm paying a
penalty for doing a lot of sub-string copies. For what I'm doing, I don't
actually need copies of the string, but rather just want to keep a pointer
to the string with the range the view occupies. I thought I'd write up a
quick test to see if I could speed things up (please ignore the horrible
names):
import Base: getindex, endof, substring
immutable StringView
value::String
first::Int64
last::Int64
end
immutable FastString : String
value::String
end
getindex(s::FastString, r::UnitRange{Int64}) = StringView(s.value,
r.start, r.stop)
endof(s::FastString) = endof(s.value)
const size = 1000
function teststring()
s = randstring(size)
for i=1:size-10
value = s[i:(i+10)]
end
end
function teststringview()
local s::FastString = FastString(randstring(size))
for i=1:size-10
value = s[i:(i+10)]
end
end
teststring()
@time teststring()
teststringview()
@time teststringview()
The results I get are:
elapsed time: 0.910467582 seconds (890006256 bytes allocated)
elapsed time: 0.392893967 seconds (40912 bytes allocated)
The speed-up isn't incredible, but if you're doing a lot of text
processing, it might help.
I'd be curious if anyone had thoughts or better ways of doing this. Or
for that matter, reasons why this may be a bad idea.
Re: [julia-users] Faster sub-strings
I think that could also work. I prefer keeping that same syntax for regular
strings (the square-brackets make things easier to read). I'm assuming
strings don't normally return views because their contents could change,
thus copies are safer. That's why I'm suggesting creating an
'ImmutableString' type. That way you know the contents won't change, and
returning a view should always be the safe thing to do.
On Wednesday, June 11, 2014 9:45:35 AM UTC-4, gael@gmail.com wrote:
Hi Abe,
Looks like you just reimplemented SubString.
julia x = Hi there!
Hi there!
julia SubString(x, 2, 4)
i t
julia typeof(ans)
SubString{ASCIIString} (constructor with 1 method)
Which is totally understandable, as there seems to be almost zero
documentation about them. Would you mind opening an issue about that?
Hi both of you.
Wouldn't it be logical also to have sub handle Strings as it does for
arrays?
julia sub([1,2,3,4,5,6], 1:2)
2-element SubArray{Int64,1,Array{Int64,1},(UnitRange{Int64},)}:
1
2
and maybe call it view instead of or in addition to sub?
[julia-users] Re: PEG Parser
I also forgot to push the changes last night.
On Wednesday, June 4, 2014 11:01:33 PM UTC-4, Abe Schneider wrote:
After playing around with a bunch of alternatives, I think I've come up
with decent action semantics:
@transform name begin
label = action
end
For example, a simple graph grammar might look like:
@grammar nodetest begin
start = +node_def
node_def = node_label + node_name + lbrace + data + rbrace
node_name = string_value + space
data = *(line + semicolon)
line = string_value + space
string_value = r[_a-zA-Z][_a-zA-Z0-9]*
lbrace = { + space
rbrace = } + space
semicolon = ; + space
node_label = node + space
space = r[ \t\n]*
end
with it's actions to create some data structure:
type MyNode
name
values
function MyNode(name, values)
new(name, values)
end
end
with:
@transform tograph begin
# ignore these
lbrace = nothing
rbrase = nothing
semicolon = nothing
node_label = nothing
space = nothing
# special action so we don't have to define every label
default = children
string_value = node.value
value = node.value
line = children
data = MyNode(, children)
node_def = begin
local name = children[1]
local cnode = children[2]
cnode.name = name
return cnode
end
end
and finally, to apply the transform:
(ast, pos, error) = parse(nodetest, data)
result = apply(tograph, ast)
println(result)# {MyNode(foo,{a,b}),MyNode(bar,{c,d})}
The magic in '@transform' basically just creates the dictionary like
before, but automatically wraps the expression on the RHS as an anonymous
function (node, children) - expr.
I'm currently looking for a better name than 'children', as it's
potentially confusing and misleading. It's actually the values of the child
nodes (as opposed to node.children). Maybe cvalues?
On Sunday, May 25, 2014 10:28:45 PM UTC-4, Abe Schneider wrote:
I wrote a quick PEG Parser for Julia with Packrat capabilities:
https://github.com/abeschneider/PEGParser
It's a first draft and needs a ton of work, testing, etc., but if this is
of interest to anyone else, here is a quick description.
Grammars can be defined using most of the standard EBNF syntax. For
example, a simple math grammar can be defined as:
@grammar mathgrammar begin
start = expr
number = r([0-9]+)
expr = (term + op1 + expr) | term
term = (factor + op2 + term) | factor
factor = number | pfactor
pfactor = ('(' + expr + ')')
op1 = '+' | '-'
op2 = '*' | '/'
end
To parse a string with the grammar:
(node, pos, error) = parse(mathgrammar, 5*(2-6))
This will create an AST which can then be transformed to a value.
Currently this is accomplished by doing:
math = Dict()
math[number] = (node, children) - float(node.value)
math[expr] = (node, children) -
length(children) == 1 ? children : eval(Expr(:call, children[2],
children[1], children[3]))
math[factor] = (node, children) - children
math[pfactor] = (node, children) - children[2]
math[term] = (node, children) -
length(children) == 1 ? children : eval(Expr(:call, children[2],
children[1], children[3]))
math[op1] = (node, children) - symbol(node.value)
math[op2] = (node, children) - symbol(node.value)
Ideally, I would like to simplify this to using multi-dispatch on symbols
(see previous post), but for now this is the easiest way to define actions
based on node attributes.
Finally, to transform the tree:
result = transform(math, node) # will give the value of 20
Originally I was going to attach the transforms to the rules themselves
(similiar to boost::spirit). However, there were two reasons for not doing
this:
1. To implement the packrat part of the parser, I needed to cache the
results which meant building an AST anyways
2. It's nice to be apply to get different transforms for the same
grammar (e.g. you may want to transform the result into HTML, LaTeX, etc.)
The downside of the separation is that it adds some more complexity to
the process.
[julia-users] Re: Strange macro behavior?
Here's a simple example of what I'm observing: testmodule.jl module testmodule export @wrapexpr # if I put foo here, everything works # foo(x, y) = x*y macro wrapexpr(expr) quote (x, y) - $(expr) end end end and testscript.jl: using testmodule foo(x, y) = x*y fn1 = @wrapexpr x+y fn2 = @wrapexpr begin foo(x, y) end println(fn1 = , fn1(1, 2)) println(fn2 = , fn2(1, 2)) with the output: fn1 = 3 ERROR: foo not defined in anonymous at /home/abraham/tmp/testmodule.jl:6 in include at boot.jl:244 in include_from_node1 at loading.jl:128 while loading /home/abraham/tmp/testscript.jl, in expression starting on line 9 but if I define foo in the module (the commented out line), I get: fn1 = 3 fn2 = 2 On Tuesday, June 3, 2014 9:59:25 PM UTC-4, Abe Schneider wrote: I have a macro that is defined in a module that creates a dictionary of anonymous functions. These anonymous functions wrap arbitrary code which I pass in. If I pass in simple pieces of code (e.g. just returning a simple value), my test script everything works as expected. However, if I make a function in my test script, which then gets wrapped in an anonymous function (e.g. I do a begin/end block), I get an error that the function is not defined. But, if I move the function out of my test script to my module, everything works. I assume what is happening is that when the anonymous function is being made, it sees only the namespace of the module. This almost makes sense to me, except that the macro is being called from my test script, so the namespace should also be in the script. Is this expected behavior? If so, are there any suggestions on how to get around this? I can provide code if it will be helpful (I thought in this case it might just make things more complicated). Also, I'm using Julia version 0.3.0-prerelease+3337. Thanks!
[julia-users] Re: Strange macro behavior?
I should add, I suspect that 'esc' can only be used in a quote block, so I tried this instead: macro wrapexpr(expr) quote ($(esc(x)), $(esc(y))) - $(esc(expr)) end end But get: ERROR: x not defined On Wednesday, June 4, 2014 11:44:43 AM UTC-4, David Moon wrote: I have a pending pull request #6910 to fix macros to behave as you expect, but until that gets integrated you will need to use the esc function in your macro to tell the macro system that x, y, and expr are in the macro caller's context, not the macro definition's context. There is no guarantee that my changes will ever be integrated, but with those changes you would write: macro wrapexpr(expr) quote ($:x, $:y) - $(expr) end end Without my changes I think you would write: macro wrapexpr(expr) x = esc(:x) y = esc(:y) quote ($x, $y) - $(esc(expr)) end end but I have not tested it.
[julia-users] Re: Strange macro behavior?
I got it to work with: macro wrapexpr(expr) x = :x y = :y quote $(esc(quote ($x, $y) - $expr end)) end end I think the problem before was that the entire anonymous function has to be escaped. I also don't claim to understand hygiene well enough to really understand what that means. Hopefully your pull goes through! On Wednesday, June 4, 2014 11:44:43 AM UTC-4, David Moon wrote: I have a pending pull request #6910 to fix macros to behave as you expect, but until that gets integrated you will need to use the esc function in your macro to tell the macro system that x, y, and expr are in the macro caller's context, not the macro definition's context. There is no guarantee that my changes will ever be integrated, but with those changes you would write: macro wrapexpr(expr) quote ($:x, $:y) - $(expr) end end Without my changes I think you would write: macro wrapexpr(expr) x = esc(:x) y = esc(:y) quote ($x, $y) - $(esc(expr)) end end but I have not tested it.
Re: [julia-users] Re: PEG Parser
I'll try to get around to comparing against the DataFrames version and
profiling this week. I got stuck trying to figure out the action semantics.
On Tuesday, May 27, 2014 6:58:42 PM UTC-4, John Myles White wrote:
I'd be really interested to see how this parser compares with DataFrames.
There's a bunch of test files in the DataFrames.jl/test directory.
-- John
On May 27, 2014, at 3:49 PM, Abe Schneider abe.sc...@gmail.com
javascript: wrote:
I don't know how the speed of the parser will be compared to DataFrames --
I've done absolutely no work to date on profiling the code, but I thought
writing a CSV parser was a good way to test out code (and helped find a
bunch of bugs).
I've also committed (under examples/) the CSV parser. The grammar (from
the RFC) is:
@grammar csv begin
start = data
data = record + *(crlf + record)
record = field + *(comma + field)
field = escaped_field | unescaped_field
escaped_field = dquote + *(textdata | comma | cr | lf | dqoute2) +
dquote
unescaped_field = textdata
textdata = r[ !#$%'()*+\-./0-~]+
cr = '\r'
lf = '\n'
crlf = cr + lf
dquote = ''
dqoute2 = \\
comma = ','
end
and the actions are:
tr[crlf] = (node, children) - nothing
tr[comma] = (node, children) - nothing
tr[escaped_field] = (node, children) - node.children[2].value
tr[unescaped_field] = (node, children) - node.children[1].value
tr[field] = (node, children) - children
tr[record] = (node, children) - unroll(children)
tr[data] = (node, children) - unroll(children)
tr[textdata] = (node, children) - node.value
give the data:
parse_data = 1,2,3\r\nthis is,a test,of csv\r\nthese,are,quotes (
)
and running the parser:
(node, pos, error) = parse(csv, parse_data)
result = transform(tr, node)
I get:
{{1,2,3},{this is,a test,of csv},{these,are,quotes
(\\)}}
On Monday, May 26, 2014 3:41:26 AM UTC-4, harven wrote:
Nice!
If you are interested by testing your library on a concrete problem, you
may want to parse comma separated value (csv) files. The bnf is in the
specification RFC4180. http://tools.ietf.org/html/rfc4180
AFAIK, the readcsv function provided in Base does not handle quotations
well whereas the csv parser in DataFrames is slow, so that julia does not
have yet a native efficient way to parse csv files.
Re: [julia-users] parsers?
Currently it only handles strings. The reason is that PEG theoretically have infinite look-ahead. I think this can be made a little better by having a stream that loads data on-demand. In general, I think PEG's choice of memory over speed is good for many things, but you'll probably find some data where an infinite look-ahead isn't a good idea. I have a very simple error handling in place (I based it on how Parsimonous works), but it definitely needs a lot of work. One big question I'm trying to figure out is whether it's better to return an error as a value or raise an exception. I've gone with returning a value so (at a later date) the errors can be collected. Also, errors are currently only emitted for non-matches, but it should be possible for the transforms to also emit errors. I think everything but the rules are typed. The only reason the rules aren't typed was that when I originally wrote the code I wasn't sure their exact value at the time. I originally wrote EBNF for an entirely different reason and got side-tracked when I realized I could write a parser with it. A On Monday, June 2, 2014 6:41:18 PM UTC-4, andrew cooke wrote: random, possibly clueless thoughts as i look at this: yes, transform rules by type would be nice! not sure what that means about having to generate a module within a macro, though (for namespacing). do you parse strings or streams? (or both?) i know nothing about julia streams, yet, but i imagine streams would make it easier to abstract away nasty book-keeping for error reporting (line number etc). do you have any support for error handling? why aren't any of your type contents typed? can julia infer that from use? if not, it will have a big impact on speed and memory use, i would guess. even better if they can be immutable. thanks, andrew On Saturday, 31 May 2014 21:10:37 UTC-4, Abe Schneider wrote: I should add that PEGParser's code is fairly new and untested (besides having an uninspired name). I'm also hoping to have better action semantics soon. On Saturday, May 31, 2014 2:17:27 PM UTC-4, andrew cooke wrote: https://groups.google.com/d/msg/julia-users/t56VxOX1vvk/nszQYWP_pm4J https://groups.google.com/d/msg/julia-users/6jz3Ow5SAAE/TgKHQ48gUG4J thanks! On Saturday, 31 May 2014 14:04:28 UTC-4, Isaiah wrote: There was a nice looking PEG system previewed a few days ago if you search the users list (and I think there was another one several months back by Michael Fox). On Sat, May 31, 2014 at 1:22 PM, andrew cooke and...@acooke.org wrote: are there any libraries for parsing in julia? either parser combinator or something more traditional (maybe a wrapper for something like antlr)? all i can find is an old discussion started by leah h in which jeff b suggests doing everything in julia. that included a pointer to https://github.com/astrieanna/juliaparsec/blob/master/juliaparsec.jl from dan l which is, well, as he says, rather basic. i'm not sure i agree, but i don't want to write my own combinator lib either. i guess i'm looking for things like a clean separation between grammar and implementation, support for errors with line numbers, speed, easy debugging... andrew
Re: [julia-users] parsers?
Yeah, that's a good point. I think instead of using a string an iterator should be used. It should be fairly easy to replace the string-implementation with an iterator-implementation. On Tuesday, June 3, 2014 9:27:05 PM UTC-4, andrew cooke wrote: On Tuesday, 3 June 2014 08:29:31 UTC-4, andrew cooke wrote: On Tuesday, 3 June 2014 08:16:49 UTC-4, Abe Schneider wrote: Currently it only handles strings. The reason is that PEG theoretically have infinite look-ahead. I think this can be made a little better by having a stream that loads data on-demand. In general, I think PEG's choice of memory over speed is good for many things, but you'll probably find some data where an infinite look-ahead isn't a good idea. yeah, i'm sitting here now thinking about this. i just realised that what i have used in the past (i wrote lepl which was an alternative to pyparsing) is effectively an immutable stream. reading from one returns a tuple of (char, newstream) so you can cache the streams and re-use them on backtracking. but that doesn't fit well with julia's IOStream abstraction (from first glance at least). so i am stuck and came here to waste time reading newsgroup posts.. duh. this is equivalent to a julia iterator. I have a very simple error handling in place (I based it on how Parsimonous works), but it definitely needs a lot of work. One big question I'm trying to figure out is whether it's better to return an error as a value or raise an exception. I've gone with returning a value so (at a later date) the errors can be collected. Also, errors are currently only emitted for non-matches, but it should be possible for the transforms to also emit errors. ok, i will look at parsimonous, thanks. i never thought much about multiple errors. I think everything but the rules are typed. The only reason the rules aren't typed was that when I originally wrote the code I wasn't sure their exact value at the time. I originally wrote EBNF for an entirely different reason and got side-tracked when I realized I could write a parser with it. A On Monday, June 2, 2014 6:41:18 PM UTC-4, andrew cooke wrote: random, possibly clueless thoughts as i look at this: yes, transform rules by type would be nice! not sure what that means about having to generate a module within a macro, though (for namespacing). do you parse strings or streams? (or both?) i know nothing about julia streams, yet, but i imagine streams would make it easier to abstract away nasty book-keeping for error reporting (line number etc). do you have any support for error handling? why aren't any of your type contents typed? can julia infer that from use? if not, it will have a big impact on speed and memory use, i would guess. even better if they can be immutable. thanks, andrew On Saturday, 31 May 2014 21:10:37 UTC-4, Abe Schneider wrote: I should add that PEGParser's code is fairly new and untested (besides having an uninspired name). I'm also hoping to have better action semantics soon. On Saturday, May 31, 2014 2:17:27 PM UTC-4, andrew cooke wrote: https://groups.google.com/d/msg/julia-users/t56VxOX1vvk/nszQYWP_pm4J https://groups.google.com/d/msg/julia-users/6jz3Ow5SAAE/TgKHQ48gUG4J thanks! On Saturday, 31 May 2014 14:04:28 UTC-4, Isaiah wrote: There was a nice looking PEG system previewed a few days ago if you search the users list (and I think there was another one several months back by Michael Fox). On Sat, May 31, 2014 at 1:22 PM, andrew cooke and...@acooke.org wrote: are there any libraries for parsing in julia? either parser combinator or something more traditional (maybe a wrapper for something like antlr)? all i can find is an old discussion started by leah h in which jeff b suggests doing everything in julia. that included a pointer to https://github.com/astrieanna/juliaparsec/blob/master/juliaparsec.jl from dan l which is, well, as he says, rather basic. i'm not sure i agree, but i don't want to write my own combinator lib either. i guess i'm looking for things like a clean separation between grammar and implementation, support for errors with line numbers, speed, easy debugging... andrew
[julia-users] Strange macro behavior?
I have a macro that is defined in a module that creates a dictionary of anonymous functions. These anonymous functions wrap arbitrary code which I pass in. If I pass in simple pieces of code (e.g. just returning a simple value), my test script everything works as expected. However, if I make a function in my test script, which then gets wrapped in an anonymous function (e.g. I do a begin/end block), I get an error that the function is not defined. But, if I move the function out of my test script to my module, everything works. I assume what is happening is that when the anonymous function is being made, it sees only the namespace of the module. This almost makes sense to me, except that the macro is being called from my test script, so the namespace should also be in the script. Is this expected behavior? If so, are there any suggestions on how to get around this? I can provide code if it will be helpful (I thought in this case it might just make things more complicated). Also, I'm using Julia version 0.3.0-prerelease+3337. Thanks!
Re: [julia-users] parsers?
I should add that PEGParser's code is fairly new and untested (besides having an uninspired name). I'm also hoping to have better action semantics soon. On Saturday, May 31, 2014 2:17:27 PM UTC-4, andrew cooke wrote: https://groups.google.com/d/msg/julia-users/t56VxOX1vvk/nszQYWP_pm4J https://groups.google.com/d/msg/julia-users/6jz3Ow5SAAE/TgKHQ48gUG4J thanks! On Saturday, 31 May 2014 14:04:28 UTC-4, Isaiah wrote: There was a nice looking PEG system previewed a few days ago if you search the users list (and I think there was another one several months back by Michael Fox). On Sat, May 31, 2014 at 1:22 PM, andrew cooke and...@acooke.org wrote: are there any libraries for parsing in julia? either parser combinator or something more traditional (maybe a wrapper for something like antlr)? all i can find is an old discussion started by leah h in which jeff b suggests doing everything in julia. that included a pointer to https://github.com/astrieanna/juliaparsec/blob/master/juliaparsec.jl from dan l which is, well, as he says, rather basic. i'm not sure i agree, but i don't want to write my own combinator lib either. i guess i'm looking for things like a clean separation between grammar and implementation, support for errors with line numbers, speed, easy debugging... andrew
[julia-users] Re: PEG Parser
I don't know how the speed of the parser will be compared to DataFrames --
I've done absolutely no work to date on profiling the code, but I thought
writing a CSV parser was a good way to test out code (and helped find a
bunch of bugs).
I've also committed (under examples/) the CSV parser. The grammar (from the
RFC) is:
@grammar csv begin
start = data
data = record + *(crlf + record)
record = field + *(comma + field)
field = escaped_field | unescaped_field
escaped_field = dquote + *(textdata | comma | cr | lf | dqoute2) + dquote
unescaped_field = textdata
textdata = r[ !#$%'()*+\-./0-~]+
cr = '\r'
lf = '\n'
crlf = cr + lf
dquote = ''
dqoute2 = \\
comma = ','
end
and the actions are:
tr[crlf] = (node, children) - nothing
tr[comma] = (node, children) - nothing
tr[escaped_field] = (node, children) - node.children[2].value
tr[unescaped_field] = (node, children) - node.children[1].value
tr[field] = (node, children) - children
tr[record] = (node, children) - unroll(children)
tr[data] = (node, children) - unroll(children)
tr[textdata] = (node, children) - node.value
give the data:
parse_data = 1,2,3\r\nthis is,a test,of csv\r\nthese,are,quotes ()
and running the parser:
(node, pos, error) = parse(csv, parse_data)
result = transform(tr, node)
I get:
{{1,2,3},{this is,a test,of csv},{these,are,quotes (\\)
}}
On Monday, May 26, 2014 3:41:26 AM UTC-4, harven wrote:
Nice!
If you are interested by testing your library on a concrete problem, you
may want to parse comma separated value (csv) files. The bnf is in the
specification RFC4180. http://tools.ietf.org/html/rfc4180
AFAIK, the readcsv function provided in Base does not handle quotations
well whereas the csv parser in DataFrames is slow, so that julia does not
have yet a native efficient way to parse csv files.
[julia-users] Re: PEG Parser
I'll take a look. I found the full grammar here: http://www.graphviz.org/doc/info/lang.html Unfortunately I have very little free time (most of the work was done during vacation), but it shouldn't be too difficult to implement. On Monday, May 26, 2014 4:35:52 PM UTC-4, Miles Gould wrote: Nice indeed! We'd also welcome a parser for the dot format in Graphs.jl :-) https://en.wikipedia.org/wiki/DOT_%28graph_description_language%29 Miles On Monday, 26 May 2014 08:41:26 UTC+1, harven wrote: Nice! If you are interested by testing your library on a concrete problem, you may want to parse comma separated value (csv) files. The bnf is in the specification RFC4180. http://tools.ietf.org/html/rfc4180 AFAIK, the readcsv function provided in Base does not handle quotations well whereas the csv parser in DataFrames is slow, so that julia does not have yet a native efficient way to parse csv files.
[julia-users] PEG Parser
I wrote a quick PEG Parser for Julia with Packrat capabilities:
https://github.com/abeschneider/PEGParser
It's a first draft and needs a ton of work, testing, etc., but if this is
of interest to anyone else, here is a quick description.
Grammars can be defined using most of the standard EBNF syntax. For
example, a simple math grammar can be defined as:
@grammar mathgrammar begin
start = expr
number = r([0-9]+)
expr = (term + op1 + expr) | term
term = (factor + op2 + term) | factor
factor = number | pfactor
pfactor = ('(' + expr + ')')
op1 = '+' | '-'
op2 = '*' | '/'
end
To parse a string with the grammar:
(node, pos, error) = parse(mathgrammar, 5*(2-6))
This will create an AST which can then be transformed to a value. Currently
this is accomplished by doing:
math = Dict()
math[number] = (node, children) - float(node.value)
math[expr] = (node, children) -
length(children) == 1 ? children : eval(Expr(:call, children[2],
children[1], children[3]))
math[factor] = (node, children) - children
math[pfactor] = (node, children) - children[2]
math[term] = (node, children) -
length(children) == 1 ? children : eval(Expr(:call, children[2],
children[1], children[3]))
math[op1] = (node, children) - symbol(node.value)
math[op2] = (node, children) - symbol(node.value)
Ideally, I would like to simplify this to using multi-dispatch on symbols
(see previous post), but for now this is the easiest way to define actions
based on node attributes.
Finally, to transform the tree:
result = transform(math, node) # will give the value of 20
Originally I was going to attach the transforms to the rules themselves
(similiar to boost::spirit). However, there were two reasons for not doing
this:
1. To implement the packrat part of the parser, I needed to cache the
results which meant building an AST anyways
2. It's nice to be apply to get different transforms for the same
grammar (e.g. you may want to transform the result into HTML, LaTeX, etc.)
The downside of the separation is that it adds some more complexity to the
process.
[julia-users] Design question
I'm currently working on a problem where I have a tree structure composed of: type Node value label children end which I want to traverse and have different functions called depending on the value of label. Normally I would just sub-type Node. However, the results are coming from a parser which allows the user to specify a myriad of different types. Thus, 'label' depends on the particular parse rule executed. The traversal code would look something like: function transform(fn, node) cvalues = [transform(fn, child) for child in node.children] return fn(node, cvalues, node.label) end The question is the how to get Julia to select the correct function to call based on 'node.label'. For example, I might have something like: htmlformat(node, children, label_is_x) = tag htmlformat(node, children, label_is_y) = /tag Two methods I've come up with so far to do this are: 1. Use the PatternDispatch package 2. Automatically create types that can be used as labels for selecting the proper function I'm currently exploring the first choice (why re-invent the wheel?). However, it seems like it might still be a more direct approach to generate types as labels. For example, I could generate the types when generating the grammars: macro maketype(label) quote type $(esc(label)) end end end which then could be used directly in the methods. Has anyone else tried this approach before? Are there known potential issues? It seems like this could be a nice general approach for a light-weight version of the PatternDispatch. Thanks!
[julia-users] Re: logical indexing... broadcasting
I might try something like (note the code code is written for clarity, not for compactness or completeness): function encode_name(name::String) if name == setosa return [1 0 2]; elseif name == versicolor return [2 1 0]; else return [0 2 1]; end end output = map(encode_name, iris_data) On Wednesday, May 14, 2014 8:05:50 AM UTC-4, Héctor Fabio Satizábal Mejía wrote: Hello I am trying to conduct some tests on machine learning algorithms with Julia but given that I am new to the language I am kind of stuck in a very simple thing. I am working with the Iris dataset, and I want to translate the species name into a code like this: setosa = [1 0 2] versicolor = [2 1 0] virginica = [0 2 1] I would like to have something like: using RDatasets iris_data = dataset(datasets, iris); iris_output = zeros(length(iris_data.columns[5]), 3); iris_output[iris_data.columns[5].==setosa,:] = [1 0 2]; iris_output[iris_data.columns[5].==versicolor,:] = [2 1 0]; iris_output[iris_data.columns[5].==virginica,:] = [0 2 1]; Unfortunately this is not working. I get this error message: ERROR: DimensionMismatch(tried to assign 1x3 array to 50x3 destination) Which I think is a broadcasting problem. Any help? Is there a compact way of doing that? Thanks in advance Héctor By the way, I manage to do this in R using the following code: iris.data - iris iris.output - matrix(0, 3, 150) iris.output[,iris.data[,5]==setosa] - c(1,0,0) iris.output[,iris.data[,5]==versicolor] - c(0,1,0) iris.output[,iris.data[,5]==virginica] - c(0,0,1) iris.output - t(iris.output)
[julia-users] Re: Macro question
You are correct -- it's weird, because I'm sure I tested it several times before posting, but I now get '5', as you suggest. On Thursday, May 15, 2014 7:51:15 AM UTC-4, Mike Innes wrote: I think your second snippet must have gotten a bit muddled, since `expr` should end up with the value 5. macro createVar(name, value) quote $name = $value; end end expr = @createVar foo 5 # This is equivalent to `expr = (foo = 5)`, *not* `expr = :(foo = 5)` expr == 5 If you do want `createVar` to return an expression, it should be a function instead of a macro. Maybe try running the example again to check it's behaving in the expected way? On Thursday, 15 May 2014 12:29:13 UTC+1, Abe Schneider wrote: As an experiment I wrote a simple macro to set a variable: macro createVar(name, value) eval(quote $name = $value; end) end which works as expected: @createVar foobar 5; println(foobar = $foobar); # foobar = 5 (OK) However, if I instead do: macro createVar(name, value) quote $name = $value; end end expr = @createVar(foobar, 5); println($expr); # foobar = 5 (OK) # now evaluate the expression to do the actual assignment eval(expr); println(foobar = $foobar); I get ERROR: foobar not defined. I would expect that if I do the eval outside of the macro I should get the same result as doing the eval inside the macro. Is this expected behavior? I should add that I'm using a version of 0.3 from the repository.
[julia-users] Re: Macro question
That works, thanks! On Thursday, May 15, 2014 8:00:29 AM UTC-4, Mike Innes wrote: Oh, the other thing I should point out (sorry to post repeatedly) is that the macro hygiene pass will mean that `foo` is not actually defined by this macro. You probably want: macro createVar(name, value) quote $(esc(name)) = $value; end end See the hygienehttp://docs.julialang.org/en/latest/manual/metaprogramming/#hygienesection of the manual. On Thursday, 15 May 2014 12:53:13 UTC+1, Mike Innes wrote: Alternatively macroexpand(:@createVar(foo, 5)) Might have the desired behaviour. On Thursday, 15 May 2014 12:51:15 UTC+1, Mike Innes wrote: I think your second snippet must have gotten a bit muddled, since `expr` should end up with the value 5. macro createVar(name, value) quote $name = $value; end end expr = @createVar foo 5 # This is equivalent to `expr = (foo = 5)`, *not* `expr = :(foo = 5)` expr == 5 If you do want `createVar` to return an expression, it should be a function instead of a macro. Maybe try running the example again to check it's behaving in the expected way? On Thursday, 15 May 2014 12:29:13 UTC+1, Abe Schneider wrote: As an experiment I wrote a simple macro to set a variable: macro createVar(name, value) eval(quote $name = $value; end) end which works as expected: @createVar foobar 5; println(foobar = $foobar); # foobar = 5 (OK) However, if I instead do: macro createVar(name, value) quote $name = $value; end end expr = @createVar(foobar, 5); println($expr); # foobar = 5 (OK) # now evaluate the expression to do the actual assignment eval(expr); println(foobar = $foobar); I get ERROR: foobar not defined. I would expect that if I do the eval outside of the macro I should get the same result as doing the eval inside the macro. Is this expected behavior? I should add that I'm using a version of 0.3 from the repository.
[julia-users] Re: tree style
Just in terms of implementing trees, I would do something like:
abstract Node;
type Leaf : Node
value::Float64;
end
type Branch : Node
left::Node;
right::Node;
end
This should allow you to write methods that target either branch or leafs
accordingly:
function sum(node::Branch)
value::Float64 = 0;
value += !isEmpty(node.left) ? sum(node.left) : 0;
value += !isEmpty(node.right) ? sum(node.right) : 0;
end
function sum(node::Leaf)
return node.value;
end
On Tuesday, May 13, 2014 11:47:14 PM UTC-4, Abram Demski wrote:
Hi all,
I'm implementing BSP trees, and I've run into a style issue which I'm
uncertain about.
For my current purposes, the trees will be holding floats, but I wanted to
write things in a way which allowed other possibilities later. At first, I
made an explicit Leaf type:
*Version 1*
abstract Leaf
type FloatLeaf : Leaf
value::Float64
end
abstract Split # abstract type for space partition
# starting out with axis-aligned splits, to implement arbitrary-angle later.
# Axis-aligned splits make rectangles, hence RectSplit
type RectSplit : Split
var::Int
loc::Float64
end
# internal tree node. Two sides of a split are pos and neg.
type TreeNode
split::Split # split definition
pos::Union(TreeNode, Leaf) # negative side
neg::Union(TreeNode, Leaf) # positive side
end
(This is actually a bit simplified; I also had a special Nothing type to
indicate an empty case. The NA type from dataframes was not appropriate
because I wanted different behavior.)
I implemented functions on the trees mainly by using multi-dispatch in a
pattern-matching style, matching against TreeNode or Leaf types to
differentiate between the recursive case and the base case.
I eventually decided that the Leaf type seemed rather pointless, because
it was just a container for a value. Using a Leaf type created some
annoyance with defining functions to operate on leaves, when the obvious
thing to do was just apply those functions to the leaf values. Also, I was
a little worried that putting stuff inside a Leaf container would be a
small efficiency hit (probably not much?).
End result, I modified to this:
*Version 2*
abstract Split
type RectSplit : Split
var::Int
loc::Float64
end
type TreeNode{a}
split::Split # split definition
pos::Union(TreeNode{a}, a) # negative side
neg::Union(TreeNode{a}, a) # positive side
end
This is shorter, which seems like a good sign. It also manages to indicate
that all leafs should have the same type, which the previous version
didn't. However, I couldn't keep using multiple dispatch to define the
different cases for my functions. The closest thing I could do would be to
make the cases which match to Leaf type match to Float instead. This works
when the tree is only holding floats, but I wanted to handle more general
cases.
Since this didn't seem right, I switched most of my code to use if-else in
one function rather than multiple dispatch on the types. Leaving the leaf
case for the final else allows me to write things in a generic way (no
explicit assumption that I'm dealing with Floats in functions that don't
need to interact with this assumption).
However, this approach still doesn't work if I want to write a method
which applies to trees (be they internal nodes or leaf nodes) but allows
other function definitions for other types. Anything which isn't a TreeNode
is now assumed to be a leaf, the way I've done it. This isn't actually a
problem yet, but it could be at some point.
Does this mean I was wrong to switch to the 2nd version?
Any other suggestions for how this should go?
Best,
Abram
