Re: [julia-users] dispatch based on expression head
Actually, having dispatch based on expression head can really help lint Lint itself. One thing that keeps bugging me is that currently I can never be sure I have covered all possible Expr head types. For example, the other day @pao pointed out "bitstype 8 MyBitsType" doesn't lint. I have never used bitstype so I would not have known this. Having a discoverable way to go through all concrete subtypes of an abstract Expr would help catch lint gaps as the language evolves. On Sunday, September 14, 2014 4:37:34 PM UTC+7, Stefan Karpinski wrote: > > Avoiding allocation during parsing could really be helpful. It would be > possible to make the number of subexpressions a type parameter. > >
Re: [julia-users] dispatch based on expression head
Avoiding allocation during parsing could really be helpful. It would be
possible to make the number of subexpressions a type parameter.
On Sun, Sep 14, 2014 at 12:52 AM, Jake Bolewski
wrote:
> Performance would come from reducing the allocation of unnecessary Any[]
> arrays for a handful of expression nodes, which has some overhead.
> Serialization / deserialization of the AST could potentially be faster
> under a more compact representation as we know that hitting the gc is a
> performance bottleneck when doing large amounts of (de)serialization.
> These two factors have some impact on startup time as well.
>
> One version of a function could handle all parametric expression types, it
> would just have to do runtime lookup similar to what you would have to do
> now,
> so isa(arg, Expr) && (arg::Expr).head === :headB would then just become
> isa(arg, Expr{:headB}).
>
> I wouldn't suspect this optimization to have a large impact on
> performance, but it could make things a some % faster (you hope).
>
> On Saturday, September 13, 2014 6:06:02 PM UTC-4, Tim Holy wrote:
>>
>> I wonder whether it would really have much of a performance advantage.
>> Usually
>> expressions are nested, and so an Expr{:headA} contains among its args an
>> Expr{:headB}. But you don't know until runtime what type it is, so
>> runtime
>> lookup would have to be faster than
>> isa(arg, Expr) && (arg::Expr).head == :headB ...`
>> I don't actually know, but I would be a bit surprised if it would help
>> that
>> much.
>>
>> Before someone thinks about tackling this, it would make sense to mock it
>> up;
>> make a MyExpr{:headsym}, nest a bunch of them based on real expressions,
>> and
>> then see if you get any real benefit from having the head as a type
>> parameter.
>>
>> The other negative is it would add to compilation time---currently one
>> version
>> of a function handles all expressions, but with this change you'd have to
>> compile a version for each parametric expression type. That means yet
>> slower
>> startup of any packages that do expression-parsing, and startup speed is
>> already a pretty big problem.
>>
>> --Tim
>>
>> On Saturday, September 13, 2014 01:17:00 PM Kevin Squire wrote:
>> > While this would greatly affect Match.jl, it would be a very welcome
>> change!
>> >
>> > Cheers,
>> >Kevin
>> >
>> > On Saturday, September 13, 2014, Leah Hanson
>> wrote:
>> > > I would expect the Expr type to be abstract, with different concrete
>> > > subtypes for each current value of head. Each value of head indicates
>> a
>> > > specific structure in args, and this can just be reflected in the
>> > > definition of the subtypes. (Then you can dispatch on Expr type, use
>> > > "subtypes(Expr)" to see all possible kinds of Expr, etc.)
>> > >
>> > > -- Leah
>> > >
>> > > On Sat, Sep 13, 2014 at 10:47 AM, Jake Bolewski > > >
>> > > > wrote:
>> > >> We've actually discussed changing our expression representation to
>> use
>> > >>
>> > types instead of the more lisp-like symbols for distinguishing
>> > expression
>> > types. That would allow dispatch on expression types and be more
>> > compact.
>> > It would, however, break almost all macros that do any kind of
>> > expression
>> > inspection.
>> > >>
>> > >> Hmm, interesting. I guess the Expr type would then be Expr{:head}
>> with
>> > >> getindex / setindex overloaded to manipulate the arguments? This
>> would
>> > >> be
>> > >> a nice change as for many nodes you would not have to allocate an
>> args
>> > >> array which could be a performance win (i guess the serialized ast's
>> > >> would
>> > >> be more compact as well). Can't comment on whether it would be
>> enough of
>> > >> a
>> > >> win to justify such a massively breaking change.
>> > >>
>> > >>> On Sat, Sep 13, 2014 at 2:48 AM, Gray Calhoun
>>
>> > >>>
>> > wrote:
>> > > On Wednesday, September 10, 2014 11:50:44 AM UTC-5, Steven G.
>> Johnson
>> > >
>> > > wrote:
>> > >> On Wednesday, September 10, 2014 12:20:59 PM UTC-4, Gray Calhoun
>> > >>
>> > >> wrote:
>> > >>> Are there better ways to do this in general?
>> > >>
>> > >> For this kind of expression-matching code, you may find the
>> Match.jl
>> > >> package handy (https://github.com/kmsquire/Match.jl), to get
>> ML- or
>> > >> Scala-like symbolic pattern-matching.
>> > >
>> > > Thanks, that's pretty cool. For simple cases like I'm using, do
>> you
>> > > know if there are advantages (or disadvantages) to using
>> Match.jl, or
>> > > should I just view it as a nicer syntax? (Obviously, when things
>> get
>> > > more
>> > > complicated Match.jl looks very appealing).
>>
>>
Re: [julia-users] dispatch based on expression head
On Saturday, September 13, 2014 5:52:11 PM UTC-5, Jake Bolewski wrote:
>
> One version of a function could handle all parametric expression types, it
> would just have to do runtime lookup similar to what you would have to do
> now,
> so isa(arg, Expr) && (arg::Expr).head === :headB would then just become
> isa(arg, Expr{:headB}).
>
I think not enough people know about Base.Meta.isexpr(). This can already
be written isexpr(arg, :headB)--and if the implementation of Expr changes,
you're still covered.
Re: [julia-users] dispatch based on expression head
On Saturday, September 13, 2014 03:52:11 PM Jake Bolewski wrote:
> One version of a function could handle all parametric expression types, it
> would just have to do runtime lookup similar to what you would have to do
> now,
> so isa(arg, Expr) && (arg::Expr).head === :headB would then just become
> isa(arg, Expr{:headB}).
But wouldn't a function like
function find_ref_expr(ex::Expr)
if ex.head == :ref
# found it!
else
for i = 1:length(ex.args)
find_ref_expr(ex.args[i])
end
end
end
end up being specialized for each different type of expression head? I'm not
saying that specialization would not ever be useful, but I do wonder about the
cost of compiling 20 different variants when all you care about might be a
single head type.
Anyway, just thought I'd raise one potential reason to perhaps contemplate
_not_ making a breaking change. In any case, it would probably good to try to
get a sense for what the performance tradeoff would be before tackling such a
project.
Best,
--Tim
Re: [julia-users] dispatch based on expression head
Performance would come from reducing the allocation of unnecessary Any[]
arrays for a handful of expression nodes, which has some overhead.
Serialization / deserialization of the AST could potentially be faster
under a more compact representation as we know that hitting the gc is a
performance bottleneck when doing large amounts of (de)serialization.
These two factors have some impact on startup time as well.
One version of a function could handle all parametric expression types, it
would just have to do runtime lookup similar to what you would have to do
now,
so isa(arg, Expr) && (arg::Expr).head === :headB would then just become
isa(arg, Expr{:headB}).
I wouldn't suspect this optimization to have a large impact on performance,
but it could make things a some % faster (you hope).
On Saturday, September 13, 2014 6:06:02 PM UTC-4, Tim Holy wrote:
>
> I wonder whether it would really have much of a performance advantage.
> Usually
> expressions are nested, and so an Expr{:headA} contains among its args an
> Expr{:headB}. But you don't know until runtime what type it is, so runtime
> lookup would have to be faster than
> isa(arg, Expr) && (arg::Expr).head == :headB ...`
> I don't actually know, but I would be a bit surprised if it would help
> that
> much.
>
> Before someone thinks about tackling this, it would make sense to mock it
> up;
> make a MyExpr{:headsym}, nest a bunch of them based on real expressions,
> and
> then see if you get any real benefit from having the head as a type
> parameter.
>
> The other negative is it would add to compilation time---currently one
> version
> of a function handles all expressions, but with this change you'd have to
> compile a version for each parametric expression type. That means yet
> slower
> startup of any packages that do expression-parsing, and startup speed is
> already a pretty big problem.
>
> --Tim
>
> On Saturday, September 13, 2014 01:17:00 PM Kevin Squire wrote:
> > While this would greatly affect Match.jl, it would be a very welcome
> change!
> >
> > Cheers,
> >Kevin
> >
> > On Saturday, September 13, 2014, Leah Hanson > wrote:
> > > I would expect the Expr type to be abstract, with different concrete
> > > subtypes for each current value of head. Each value of head indicates
> a
> > > specific structure in args, and this can just be reflected in the
> > > definition of the subtypes. (Then you can dispatch on Expr type, use
> > > "subtypes(Expr)" to see all possible kinds of Expr, etc.)
> > >
> > > -- Leah
> > >
> > > On Sat, Sep 13, 2014 at 10:47 AM, Jake Bolewski
> > >
> > > ');>>
> wrote:
> > >> We've actually discussed changing our expression representation to
> use
> > >>
> > types instead of the more lisp-like symbols for distinguishing
> > expression
> > types. That would allow dispatch on expression types and be more
> > compact.
> > It would, however, break almost all macros that do any kind of
> > expression
> > inspection.
> > >>
> > >> Hmm, interesting. I guess the Expr type would then be Expr{:head}
> with
> > >> getindex / setindex overloaded to manipulate the arguments? This
> would
> > >> be
> > >> a nice change as for many nodes you would not have to allocate an
> args
> > >> array which could be a performance win (i guess the serialized ast's
> > >> would
> > >> be more compact as well). Can't comment on whether it would be
> enough of
> > >> a
> > >> win to justify such a massively breaking change.
> > >>
> > >>> On Sat, Sep 13, 2014 at 2:48 AM, Gray Calhoun
> > >>>
> > wrote:
> > > On Wednesday, September 10, 2014 11:50:44 AM UTC-5, Steven G.
> Johnson
> > >
> > > wrote:
> > >> On Wednesday, September 10, 2014 12:20:59 PM UTC-4, Gray Calhoun
> > >>
> > >> wrote:
> > >>> Are there better ways to do this in general?
> > >>
> > >> For this kind of expression-matching code, you may find the
> Match.jl
> > >> package handy (https://github.com/kmsquire/Match.jl), to get ML-
> or
> > >> Scala-like symbolic pattern-matching.
> > >
> > > Thanks, that's pretty cool. For simple cases like I'm using, do
> you
> > > know if there are advantages (or disadvantages) to using Match.jl,
> or
> > > should I just view it as a nicer syntax? (Obviously, when things
> get
> > > more
> > > complicated Match.jl looks very appealing).
>
>
Re: [julia-users] dispatch based on expression head
I wonder whether it would really have much of a performance advantage. Usually
expressions are nested, and so an Expr{:headA} contains among its args an
Expr{:headB}. But you don't know until runtime what type it is, so runtime
lookup would have to be faster than
isa(arg, Expr) && (arg::Expr).head == :headB ...`
I don't actually know, but I would be a bit surprised if it would help that
much.
Before someone thinks about tackling this, it would make sense to mock it up;
make a MyExpr{:headsym}, nest a bunch of them based on real expressions, and
then see if you get any real benefit from having the head as a type parameter.
The other negative is it would add to compilation time---currently one version
of a function handles all expressions, but with this change you'd have to
compile a version for each parametric expression type. That means yet slower
startup of any packages that do expression-parsing, and startup speed is
already a pretty big problem.
--Tim
On Saturday, September 13, 2014 01:17:00 PM Kevin Squire wrote:
> While this would greatly affect Match.jl, it would be a very welcome change!
>
> Cheers,
>Kevin
>
> On Saturday, September 13, 2014, Leah Hanson wrote:
> > I would expect the Expr type to be abstract, with different concrete
> > subtypes for each current value of head. Each value of head indicates a
> > specific structure in args, and this can just be reflected in the
> > definition of the subtypes. (Then you can dispatch on Expr type, use
> > "subtypes(Expr)" to see all possible kinds of Expr, etc.)
> >
> > -- Leah
> >
> > On Sat, Sep 13, 2014 at 10:47 AM, Jake Bolewski >
> > > wrote:
> >> We've actually discussed changing our expression representation to use
> >>
> types instead of the more lisp-like symbols for distinguishing
> expression
> types. That would allow dispatch on expression types and be more
> compact.
> It would, however, break almost all macros that do any kind of
> expression
> inspection.
> >>
> >> Hmm, interesting. I guess the Expr type would then be Expr{:head} with
> >> getindex / setindex overloaded to manipulate the arguments? This would
> >> be
> >> a nice change as for many nodes you would not have to allocate an args
> >> array which could be a performance win (i guess the serialized ast's
> >> would
> >> be more compact as well). Can't comment on whether it would be enough of
> >> a
> >> win to justify such a massively breaking change.
> >>
> >>> On Sat, Sep 13, 2014 at 2:48 AM, Gray Calhoun
> >>>
> wrote:
> > On Wednesday, September 10, 2014 11:50:44 AM UTC-5, Steven G. Johnson
> >
> > wrote:
> >> On Wednesday, September 10, 2014 12:20:59 PM UTC-4, Gray Calhoun
> >>
> >> wrote:
> >>> Are there better ways to do this in general?
> >>
> >> For this kind of expression-matching code, you may find the Match.jl
> >> package handy (https://github.com/kmsquire/Match.jl), to get ML- or
> >> Scala-like symbolic pattern-matching.
> >
> > Thanks, that's pretty cool. For simple cases like I'm using, do you
> > know if there are advantages (or disadvantages) to using Match.jl, or
> > should I just view it as a nicer syntax? (Obviously, when things get
> > more
> > complicated Match.jl looks very appealing).
Re: [julia-users] dispatch based on expression head
I'm not sure how that makes linting impossible.
On Sat, Sep 13, 2014 at 11:56 PM, Jameson Nash wrote:
> It would be probably be lasting, because most Exprs would no longer want
> to have an args field with a Vector{Any}
>
>
> On Saturday, September 13, 2014, Stefan Karpinski <
> stefan.karpin...@gmail.com> wrote:
>
>> Surely not in any lasting sense? With a.b syntax overloading we could
>> actually make the transition pretty smooth.
>>
>> On Sep 13, 2014, at 4:47 PM, Tony Fong wrote:
>>
>> That'd be bad news for Lint...
>>
>> On Saturday, September 13, 2014 3:22:04 PM UTC+7, Stefan Karpinski wrote:
>>>
>>> We've actually discussed changing our expression representation to use
>>> types instead of the more lisp-like symbols for distinguishing expression
>>> types. That would allow dispatch on expression types and be more compact.
>>> It would, however, break almost all macros that do any kind of expression
>>> inspection.
>>>
>>> On Sat, Sep 13, 2014 at 2:48 AM, Gray Calhoun
>>> wrote:
>>>
On Wednesday, September 10, 2014 11:50:44 AM UTC-5, Steven G. Johnson
wrote:
>
> On Wednesday, September 10, 2014 12:20:59 PM UTC-4, Gray Calhoun
> wrote:
>>
>> Are there better ways to do this in general?
>>
>
> For this kind of expression-matching code, you may find the Match.jl
> package handy (https://github.com/kmsquire/Match.jl), to get ML- or
> Scala-like symbolic pattern-matching.
>
Thanks, that's pretty cool. For simple cases like I'm using, do you
know if there are advantages (or disadvantages) to using Match.jl, or
should I just view it as a nicer syntax? (Obviously, when things get more
complicated Match.jl looks very appealing).
>>>
>>>
Re: [julia-users] dispatch based on expression head
It would be probably be lasting, because most Exprs would no longer want to
have an args field with a Vector{Any}
On Saturday, September 13, 2014, Stefan Karpinski <
stefan.karpin...@gmail.com> wrote:
> Surely not in any lasting sense? With a.b syntax overloading we could
> actually make the transition pretty smooth.
>
> On Sep 13, 2014, at 4:47 PM, Tony Fong > wrote:
>
> That'd be bad news for Lint...
>
> On Saturday, September 13, 2014 3:22:04 PM UTC+7, Stefan Karpinski wrote:
>>
>> We've actually discussed changing our expression representation to use
>> types instead of the more lisp-like symbols for distinguishing expression
>> types. That would allow dispatch on expression types and be more compact.
>> It would, however, break almost all macros that do any kind of expression
>> inspection.
>>
>> On Sat, Sep 13, 2014 at 2:48 AM, Gray Calhoun
>> wrote:
>>
>>> On Wednesday, September 10, 2014 11:50:44 AM UTC-5, Steven G. Johnson
>>> wrote:
On Wednesday, September 10, 2014 12:20:59 PM UTC-4, Gray Calhoun wrote:
>
> Are there better ways to do this in general?
>
For this kind of expression-matching code, you may find the Match.jl
package handy (https://github.com/kmsquire/Match.jl), to get ML- or
Scala-like symbolic pattern-matching.
>>>
>>> Thanks, that's pretty cool. For simple cases like I'm using, do you know
>>> if there are advantages (or disadvantages) to using Match.jl, or should I
>>> just view it as a nicer syntax? (Obviously, when things get more
>>> complicated Match.jl looks very appealing).
>>>
>>
>>
Re: [julia-users] dispatch based on expression head
While this would greatly affect Match.jl, it would be a very welcome change!
Cheers,
Kevin
On Saturday, September 13, 2014, Leah Hanson wrote:
> I would expect the Expr type to be abstract, with different concrete
> subtypes for each current value of head. Each value of head indicates a
> specific structure in args, and this can just be reflected in the
> definition of the subtypes. (Then you can dispatch on Expr type, use
> "subtypes(Expr)" to see all possible kinds of Expr, etc.)
>
> -- Leah
>
> On Sat, Sep 13, 2014 at 10:47 AM, Jake Bolewski > wrote:
>
>> We've actually discussed changing our expression representation to use
types instead of the more lisp-like symbols for distinguishing expression
types. That would allow dispatch on expression types and be more compact.
It would, however, break almost all macros that do any kind of expression
inspection.
>>>
>> Hmm, interesting. I guess the Expr type would then be Expr{:head} with
>> getindex / setindex overloaded to manipulate the arguments? This would be
>> a nice change as for many nodes you would not have to allocate an args
>> array which could be a performance win (i guess the serialized ast's would
>> be more compact as well). Can't comment on whether it would be enough of a
>> win to justify such a massively breaking change.
>>
>>
>>> On Sat, Sep 13, 2014 at 2:48 AM, Gray Calhoun
wrote:
> On Wednesday, September 10, 2014 11:50:44 AM UTC-5, Steven G. Johnson
> wrote:
>>
>> On Wednesday, September 10, 2014 12:20:59 PM UTC-4, Gray Calhoun
>> wrote:
>>>
>>> Are there better ways to do this in general?
>>>
>>
>> For this kind of expression-matching code, you may find the Match.jl
>> package handy (https://github.com/kmsquire/Match.jl), to get ML- or
>> Scala-like symbolic pattern-matching.
>>
>
> Thanks, that's pretty cool. For simple cases like I'm using, do you
> know if there are advantages (or disadvantages) to using Match.jl, or
> should I just view it as a nicer syntax? (Obviously, when things get more
> complicated Match.jl looks very appealing).
>
>
Re: [julia-users] dispatch based on expression head
Oh, cool. I'll have to read it carefully, but that looks very helpful. Thanks! On Wednesday, September 10, 2014 11:24:58 AM UTC-5, John Myles White wrote: > > You can make parametric types that depend on symbols. Read the code in > Calculus.jl for examples: > https://github.com/johnmyleswhite/Calculus.jl/blob/master/src/symbolic.jl >
Re: [julia-users] dispatch based on expression head
Gray,
You can make parametric types that depend on symbols. Read the code in
Calculus.jl for examples:
https://github.com/johnmyleswhite/Calculus.jl/blob/master/src/symbolic.jl
-- John
On Sep 10, 2014, at 9:20 AM, Gray Calhoun wrote:
> Hi everyone, I'm writing code using expressions fairly heavily (mainly as a
> learning exercise), and am using lots of constructions like:
>
> function haskey(df::MyDataType, key::Expr)
> key.head == :call && return(all([haskey(df, x) for x in
> key.args[2:end]]))
> key.head == :(=) && return(haskey(df, key.args[2]))
> key.head == :block && return(all([haskey(df, x) for x in key.args]))
> key.head == :line && return(true)
> warn("Didn't expect to get here...")
> false
> end
>
> Some quick questions:
>
> 1) Is there a way of mimicking Expr subtypes that I'm not aware of that would
> let me rewrite this as several different methods? Are there better ways to do
> this in general?
>
> 2) Is there a list anywhere of the different possible values for Expr.head?
> (I don't see it in the metaprogramming section of the manual, but may have
> missed it.)
>
> 3) How much of Julia would it break or slow down to make Expr an abstract
> type with the contents of "head" implemented as different subtypes? (This
> last one is mostly just asked out of curiosity.)
>
> Thanks!
[julia-users] dispatch based on expression head
Hi everyone, I'm writing code using expressions fairly heavily (mainly as a
learning exercise), and am using lots of constructions like:
function haskey(df::MyDataType, key::Expr)
key.head == :call && return(all([haskey(df, x) for x in key.args[2:end
]]))
key.head == :(=) && return(haskey(df, key.args[2]))
key.head == :block && return(all([haskey(df, x) for x in key.args]))
key.head == :line && return(true)
warn("Didn't expect to get here...")
false
end
Some quick questions:
1) Is there a way of mimicking Expr subtypes that I'm not aware of that
would let me rewrite this as several different methods? Are there better
ways to do this in general?
2) Is there a list anywhere of the different possible values for Expr.head?
(I don't see it in the metaprogramming section of the manual, but may have
missed it.)
3) How much of Julia would it break or slow down to make Expr an abstract
type with the contents of "head" implemented as different subtypes? (This
last one is mostly just asked out of curiosity.)
Thanks!
