[julia-users] Re: How does promote_op work?
Does the promote_op mechanism in v0.5 play nicely with generated functions?
In Unitful.jl, I use a generated function to determine result units after
computations involving quantities with units. I seem to get errors
(@inferred tests fail) if I remove my promote_op specialization. Perhaps my
problems are all a consequence
of https://github.com/JuliaLang/julia/issues/18465 and they will go away
soon...?
On Friday, September 23, 2016 at 5:54:03 AM UTC-7, Pablo Zubieta wrote:
>
> In julia 0.5 the following should work without needing doing anything to
> promote_op
>
> import Base.+
> immutable Foo end
> +(a::Foo, b::Foo) =1.0
> Array{Foo}(0) + Array{Foo}(0))
>
> promote_op is supposed to be an internal method that you wouldn't need to
> override. If it is not working i because the operation you are doing is
> most likely not type stable. So instead of specializing it you could try to
> remove any type instabilities in the method definitions over your types.
>
> On Friday, September 23, 2016 at 5:35:05 AM UTC+2, Sheehan Olver wrote:
>>
>>
>> The subject says it all: it looks like one can override promote_op to
>> support the following behaviour:
>>
>> *julia> **import Base.+*
>>
>>
>> *julia> **immutable Foo end*
>>
>> WARNING: Method definition (::Type{Main.Foo})() in module Main at
>> REPL[5]:1 overwritten at REPL[10]:1.
>>
>>
>> *julia> **+(a::Foo,b::Foo) = 1.0*
>>
>> *+ (generic function with 164 methods)*
>>
>>
>> *julia> **Base.promote_op(::typeof(+),::Type{Foo},::Type{Foo}) = Float64*
>>
>>
>> *julia> **Array(Foo,0) + Array(Foo,0)*
>>
>> *0-element Array{Float64,1}*
>>
>>
>> Is this documented somewhere? What if we want to override /, -, etc., is
>> the solution to write a promote_op for each case?
>>
>
[julia-users] Re: Unitful.jl for physical units
Unitful.jl has recently undergone a major update with enhanced functionality. If the previous version was an alpha release, this is something like a beta release before the package is registered and tagged. I intend to do that soon so that others can think about using the package more seriously, probably shortly after Julia 0.5.0 is released. Now is a good time to provide feedback if you find bugs or think there are some trouble spots that would prevent the package from being useful to you. I am sorry to inconvenience existing users with syntax changes, but please raise an issue or post here if you have any trouble. I hope the revised documentation explains any questions you may have, and as usual, test/runtests.jl can be useful to figure out how things are working. Once the package is registered and tagged, I will be more conscientious regarding breaking changes. Enjoy, Andrew Keller On Friday, February 12, 2016 at 12:23:22 PM UTC-8, Andrew Keller wrote: > > I'm happy to share a package I wrote for using physical units in Julia, > Unitful.jl <https://www.github.com/ajkeller34/Unitful.jl>. Much credit > and gratitude is due to Keno Fischer for the SIUnits.jl > <https://www.github.com/keno/SIUnits.jl> package which served as my > inspiration. This is a work in progress, but I think perhaps a serviceable > one depending on what you're doing. > > Like SIUnits.jl, this package encodes units in the type signature to avoid > run-time performance penalties. From there, the implementations diverge. > The package is targeted to Julia 0.5 / master, as there are some > limitations with how promote_op is used in Julia 0.4 (#13803) > <https://github.com/JuliaLang/julia/pull/13803>. I decided it wasn't > worth targeting 0.4 if the behavior would be inconsistent. > > Some highlights include: > >- Non-SI units are treated on the same footing as SI units, with only >a few exceptions (unit conversion method). Use whatever weird units >you want. >- Support for units like micron / (meter Kelvin), where some of the >units could cancel out but you don't necessarily want them to. >- Support for LinSpace and other Range types. Probably there are still >some glitches to be found, though. >- Support for rational exponents of units. >- Some tests (see these for usage examples). > > Please see the documentation for a comprehensive discussion, including > issues / to do list, as well as how to add your own units, etc. > Comments and feedback are welcome. > > Best, > Andrew Keller >
[julia-users] Re: Can Julia handle a large number of methods for a function (and a lot of types)?
Hi Oliver, Regarding your first question posed in this thread, I think you might be interested in this documentation <http://docs.julialang.org/en/latest/devdocs/functions/> of how functions will work in Julia 0.5 if you haven't read it already. There is some discussion of how methods are dispatched, as well as compiler efficiency issues. I hope you don't mind that I've tried out your setindex and getindex approach. It is very pleasant to use but I have not benchmarked it in any serious way, mainly because I'm not sure what a sensible benchmark would be. If you'd like me to try out something I'll see what I can do. It sounds like you have probably been thinking deeply about instrument control for a much longer period of time than I have. I'll write you again once I've gotten our codebase more stable and documented, and I welcome criticism. I haven't given much thought to coordinated parallel access yet but I agree that it will be important. A short summary is that right now we have one package containing a module with submodules for each instrument. Each instrument has an explicit type and most methods are generated using metaprogramming based off some template for each instrument type. Most instrument types are subtypes of `InstrumentVISA`, and there are a few methods that are assumed to work on all instruments supporting VISA. I must say, it is not obvious what the best type hierarchy is, and I could easily believe that traits are a better way to go when describing the functionality of instruments. You can find any number of discussion threads, GitHub issues, etc. on traits in Julia but I don't know what current consensus is. Unitful.jl and SIUnits.jl globally have the same approach: encode the units in the type signature of a value. Accordingly, Unitful.jl should have great performance at "run-time," and is a design goal. Anytime some operation with units is happening in a tight loop, it should be fast. I have only had time to look at some of the generated machine code but what I've looked at is the same or very close to what is generated for operations without units. I have not optimized the "compile-time" performance at all. Probably the first time a new unit is encountered, some modest performance penalty results. I'd like to relook at how I'm handling dimensions (length, time, etc.) because in retrospect I think I'm doing some odd things. An open question is how one could dispatch on the dimensions (e.g. x::Length). I tried this once and it sort of worked but the type-gymnastics became very ugly, so maybe something like traits would be better. Last I checked, the two packages are different in that Unitful.jl supports: rational powers of the units (you can do Hz^(1/2), useful for noise spectra); non-SI units like imperial units; the ability to add new units on the fly with macros; preservation of exact conversions. My package only supports Julia 0.5 though. I think the differences between the packages arise mainly because SIUnits.jl was written when Julia was less mature. SIUnits.jl is still sufficient for many needs and is remarkably clever. Thanks for linking to your code. I have no experience with Scala but I will take a look at it. Best, Andrew On Saturday, April 2, 2016 at 4:45:04 AM UTC-7, Oliver Schulz wrote: > > Hi Andrew, > > sorry, I overlooked your post, somehow ... > > On Monday, March 28, 2016 at 3:47:42 AM UTC+2, Andrew Keller wrote: >> >> Just a heads up that I've been working on something similar for the past >> six months or so. >> > > Very nice - I'm open collaborate on this topic, of course. At the moment, > we're still actively using my Scala/Akka based system > https://github.com/daqcore/daqcore-scala) in our group, but I'm starting > to map out it's Julia replacement. daqcore-scala currently implements some > VME flash-ADCs and some slow-control devices like a Labjack, some HV > Supplies, vaccuum pressure gauges, etc., but the architecture is not device > specific. The Julia version is intended to be equally generic, but will > initially target the same/similar devices. I'd like to make it more > modular, though, it doesn't all have to end up in one package. > > One requirement will be coordinated parallel access to the instruments > from different places of the code (can't have one Task sending a command to > an instrument while another task isn't done with his query - which may span > multiple I/O operations - yet). Also multi-host operation for > high-throughput applications might become necessary at some point. The > basis for both came for free with Scala/Akka actors, and I started > Actors.jl (https://github.com/daqcore/Actors.jl) with this in mind. > Another things that was easy in Scala was representing (potentially > overlapping) devices classes using traits. I'm thinking how d
[julia-users] Re: Can Julia handle a large number of methods for a function (and a lot of types)?
Hi Oliver,
Just a heads up that I've been working on something similar for the past
six months or so. I'm fortunate enough to be starting a new project where I
have no legacy code and the freedom to play around a bit. I think we have
similar ideas: properties are abstract / singleton types in my code as
well. I really like your idea of using `getindex` and `setindex!` but I
didn't think of that initially (sigh...) and at present have defined
methods like:
configure(ins::Instrument,::Type{DeviceFeature}, value, channel::Integer=1)
inspect(ins::Instrument, ::Type{DeviceFeature})
My approach would have the same scaling issues you worry about. I hope it
won't be such an issue and it would be kind of a shame to resort to
separate functions for each instrument property in Julia, so I'm just going
full speed ahead.
My code is not yet ready for public use and is not stable, but the source
is in my repo <https://github.com/ajkeller34/PainterQB.jl> if you're
curious. I should caution that its a bit of a mess right now and the
documentation is not up-to-date, so some of it reflects things that are not
be true anymore, or ideas I have abandoned / improved upon. Seeing as how I
need to do some real science soon I hope the source code will stabilize
within a few weeks. I would welcome discussion if you want to combine
efforts on this. I'm not sure if I want to invite PRs yet until I have the
project in a more stable state but discussion is always good.
As I've been working on this since I started learning Julia, there are a
number of things I did early on that I regret and am now fixing. One thing
you'll find becomes annoying quickly is namespace issues: if you want to
put different instruments in different modules, shared properties or
functions may require a lot of careful imports/exports that tend to break
easily. I'd suggest that rather than hard coding properties in source
files, you store what you need in some standard external format. I'm now
storing my SCPI codes, Julia property names, argument types, etc. in a JSON
file that Julia parses to generate code with metaprogramming. With this
approach I believe I'll be able to write code to scrape instrument manuals,
and thereby generate code for new instruments semiautomatically. You'd
still need to decide on property names and such, and write regular
expressions to scrape the manuals, but that's not so bad.
Finally, I think some really great things could be done with this sort of
code if Julia had better native support for physical units. For instance,
on a vector network analyzer you could write something like:
vna[Frequency] = 4GHz:1MHz:10GHz
and thereby set frequency start, stop, and step at once. I have a units
package for Julia v0.5 that I'm also developing when I have the time (
Unitful.jl <https://github.com/ajkeller34/Unitful.jl>), but there are some
tweaks to Base that would help the package play nicely. If that sounds
appealing, it'd be great if you could voice your support for that, e.g.
here: https://github.com/JuliaLang/julia/pull/15394
Best wishes,
Andrew Keller
On Sunday, March 27, 2016 at 5:34:31 AM UTC-7, Oliver Schulz wrote:
>
> Hello,
>
> I'm designing a scheme to control and read out hardware devices (mostly
> lab instruments) via Julia. The basic idea is that each device type is
> characterized by a set of features/properties that have a value type and
> dimensionality and that can be read-only or read/write (a bit like SCPI or
> VISA, conceptually).
>
> Features/properties would be things like "InputVoltage" for a multimeter,
> "Waveform", "TriggerLevel", etc. for an oscilloscope, and so on. They would
> be represented by (abstract or singleton) types. Obviously, certain
> features would be supported by many different device classes.
>
> For the API, I was thinking of an interface like this:
>
> current_feature_value = mydevice[SomeDeviceFeature]
> mydev[SomeDeviceFeature] = new_feature_value
>
> resp. things like
>
> mydev[SomeMultiChannelFeature, channel]
>
> for feature with multiple instances (e.g. in muli-channel devices). The
> implementation would, of course, be
>
> getindex(dev::SomeDeviceType, ::Type{SomeDeviceFeature}) = ...
> setindex!(dev::SomeDeviceType, value, ::Type{SomeDeviceFeature}) = ...
>
> etc. This would mean having a large number of classes for all features
> (let's say around 200), and an even larger number of methods (
> O(number_of_device_classes * typical_number_of_features_per_device), let's
> say around 5000) for getindex and setindex!. Can Julia handle this, or
> would this result in long compile times and/or bad performance?
>
> An alternative would be, of course, to represent each feature by a
> specific set of methods like "trigger_level_set, "trigger_level_get", etc.
> However,
[julia-users] Re: Unitful.jl for physical units
Hi Jake, I agree with Jeffrey's response mostly, but want to clarify that Unitful is not strictly focused on the SI system. You'll see that you can use units of acre-feet, very much not an SI unit, which could be useful if you happen to manage a water reservoir in the United States, I guess. If you were instead asking whether or not you could write methods that dispatch on the dimensions of a unit, the current answer is no, although maybe that could change eventually. Best, Andrew On Wednesday, February 17, 2016 at 3:25:37 AM UTC-8, Jeffrey Sarnoff wrote: > > Jake, > > Julia's type system is well suited to do just that. Unitful is focused on > units in SI system, things like Meters, Kilograms and Joules. > > One approach to abstract units like size, where size may be relative to > the number of pixels on a screen or the width of a page, > is define your own type, a kind of size relative to something. In your > example, s is not a unit of measure (strictly speaking); > s is a quantity interpreted in terms of some absolute or relative unit of > measure -- 5 pixels, 1/4 page. Because pixels and pages > are not always the same number of, say, millimeters, using SI units for > that abstraction likely is not what you want. > > If you want more guidance, please give some more context. > > On Wednesday, February 17, 2016 at 4:43:13 AM UTC-5, Jake Rosoman wrote: >> >> Is it possible to talk about abstract types of units like size. e.g >> `drawline(s::Size) = ...`? >> >> On Saturday, February 13, 2016 at 9:23:22 AM UTC+13, Andrew Keller wrote: >>> >>> I'm happy to share a package I wrote for using physical units in Julia, >>> Unitful.jl <https://www.github.com/ajkeller34/Unitful.jl>. Much credit >>> and gratitude is due to Keno Fischer for the SIUnits.jl >>> <https://www.github.com/keno/SIUnits.jl> package which served as my >>> inspiration. This is a work in progress, but I think perhaps a serviceable >>> one depending on what you're doing. >>> >>> Like SIUnits.jl, this package encodes units in the type signature to >>> avoid run-time performance penalties. From there, the implementations >>> diverge. The package is targeted to Julia 0.5 / master, as there are some >>> limitations with how promote_op is used in Julia 0.4 (#13803) >>> <https://github.com/JuliaLang/julia/pull/13803>. I decided it wasn't >>> worth targeting 0.4 if the behavior would be inconsistent. >>> >>> Some highlights include: >>> >>>- Non-SI units are treated on the same footing as SI units, with >>>only a few exceptions (unit conversion method). Use whatever weird >>>units you want. >>>- Support for units like micron / (meter Kelvin), where some of the >>>units could cancel out but you don't necessarily want them to. >>>- Support for LinSpace and other Range types. Probably there are >>>still some glitches to be found, though. >>>- Support for rational exponents of units. >>>- Some tests (see these for usage examples). >>> >>> Please see the documentation for a comprehensive discussion, including >>> issues / to do list, as well as how to add your own units, etc. >>> Comments and feedback are welcome. >>> >>> Best, >>> Andrew Keller >>> >>
[julia-users] Re: Unitful.jl for physical units
Hi Yousef, Yes, I anticipate that there will be a simpler way to add units sometime, probably via a macro. However, my concern first and foremost is to make sure that operations involving units do not fail unexpectedly, particularly where ranges, arrays, and matrices are involved. This is really key for widespread adoption, I think, as well as for my own application. I'd also like to make it so that exact conversions are preserved where possible (see issue #3 on the Github page). In the meantime, you're welcome to submit a pull request for the units you need, or just modify your local copy. Thanks to everyone for the positive feedback! It looks like there was a recent commit to Julia master that may introduce some breakage—I'll prioritize fixing that once the Mac OS X binary for Julia is updated. I may fix it sooner since I was able to build master. Best Andrew On Tuesday, February 16, 2016 at 6:11:52 AM UTC-8, yousef.k...@gmail.com wrote: > > Hello Andrew, > > Your work is very promising. If you have ever dealt with oil field > equations, then you'll know that units are a headache to deal with. > Your package should make life much easier. > > I'm curious, however, as if you'll implement an easier means of defining > new units. > Something like a function that takes a unit, its base factor, and its > conversion to SI units, then gets it working in Unitful. > > Thanks again for your work, > Yousef >
[julia-users] Unitful.jl for physical units
I'm happy to share a package I wrote for using physical units in Julia, Unitful.jl <https://www.github.com/ajkeller34/Unitful.jl>. Much credit and gratitude is due to Keno Fischer for the SIUnits.jl <https://www.github.com/keno/SIUnits.jl> package which served as my inspiration. This is a work in progress, but I think perhaps a serviceable one depending on what you're doing. Like SIUnits.jl, this package encodes units in the type signature to avoid run-time performance penalties. From there, the implementations diverge. The package is targeted to Julia 0.5 / master, as there are some limitations with how promote_op is used in Julia 0.4 (#13803) <https://github.com/JuliaLang/julia/pull/13803>. I decided it wasn't worth targeting 0.4 if the behavior would be inconsistent. Some highlights include: - Non-SI units are treated on the same footing as SI units, with only a few exceptions (unit conversion method). Use whatever weird units you want. - Support for units like micron / (meter Kelvin), where some of the units could cancel out but you don't necessarily want them to. - Support for LinSpace and other Range types. Probably there are still some glitches to be found, though. - Support for rational exponents of units. - Some tests (see these for usage examples). Please see the documentation for a comprehensive discussion, including issues / to do list, as well as how to add your own units, etc. Comments and feedback are welcome. Best, Andrew Keller
Re: [julia-users] Memory leak for animated plots
I think this is exactly what is happening. Some findings: 1) run the code: using Interact, Gadfly > @manipulate for φ=0:π/16:4π, f=[:sin => sin, :cos => cos] > plot((θ)->f(θ+φ),0,25) > end 2) Chrome dev tools--> Profiles--> heap snapshot. 3) Click sin, cos, sin, cos, sin, cos, ... sin in the notebook 4) Take another heap snapshot and look in comparison view. It looks like among other things there are a lot of SVGPathElements and SVGTextElements that belong to detached DOM trees, suggesting the old plots never get properly disposed. If I instead capture a JS profile in the Chrome dev tools-->Timeline panel, it appears like the number of nodes and listeners increases without bound. Now suppose I use Winston instead of Gadfly. The memory still appears to leak, although the plots are a little more lightweight and the leak is slower. Anyway, I'll submit an issue to the jupyter/notebook repo later today although I wish I could pin down better where exactly the leak is coming from. On Thursday, November 19, 2015 at 10:35:31 AM UTC-8, Keno Fischer wrote: > > Sounds like the memory leak is on the browser side? Maybe something is > keeping a javascript reference to the plot? Potentially a Jupyter/IJulia > bug? > > On Thu, Nov 19, 2015 at 12:01 PM, Stefan Karpinski <ste...@karpinski.org > > wrote: > >> This should work – if there's a memory leak that's never reclaimed by gc, >> that's a bug. >> >> On Thu, Nov 19, 2015 at 11:55 AM, Andrew Keller <andrew.k...@gmail.com >> > wrote: >> >>> Maybe generating a new plot every time is not great practice, on account >>> of the performance hit. That being said, I think it's perfectly legitimate >>> to do what I'm doing for prototyping purposes. I can achieve the frame rate >>> I want and the main example shown on >>> https://github.com/JuliaLang/Interact.jl does the same thing I do, >>> generating a new plot each time. >>> >>> In fact, I'd encourage anyone reading this to just try that example, and >>> repeatedly click between sin and cos. I'm able to make the memory >>> consumption of my browser grow without bound. Surely someone besides myself >>> has noticed this before! I don't think loading another package is a serious >>> solution to the problem I'm describing, although your package certainly >>> looks useful for other purposes. >>> >>> Just to reiterate, this is not a small memory leak; this is like a >>> memory dam breach. I'm happy to help debug this but some assistance would >>> be appreciated. >>> >>> >>> On Thursday, November 19, 2015 at 7:11:55 AM UTC-8, Tom Breloff wrote: >>>> >>>> You're creating a new Gadfly.Plot object every update, which is a bad >>>> idea even if Gadfly's memory management was perfect. Plots.jl gives you >>>> the >>>> ability to add to or replace the underlying data like this: >>>> >>>> using Plots >>>> gadfly() >>>> getxy() = (1:10, rand(10)) >>>> plt = plot(x, y) >>>> >>>> # overwrite underlying plot data without building a new plot >>>> plt[1] = getxy() >>>> >>>> >>>> You can also use familiar push! and append! calls. >>>> >>>> Let me know if this helps, and please post issues if you find bugs. Of >>>> course the memory issue could be while redisplaying in IJulia, in which >>>> case this method won't help. >>>> >>>> On Thursday, November 19, 2015, Andrew Keller <andrew.k...@gmail.com> >>>> wrote: >>>> >>>>> I'd like to use Interact to have a plot that updates frequently in a >>>>> Jupyter notebook, but it seems like there is a large memory leak >>>>> somewhere >>>>> and I am having some trouble tracking down what package is responsible. >>>>> Within a few minutes of running, the following code will cause the memory >>>>> used by the web browser to balloon to well over 1 GB with no sign of >>>>> slowing down. It is almost like the memory allocated for displaying a >>>>> particular plot is never deallocated: >>>>> >>>>> using Reactive, Interact, Gadfly >>>>> >>>>> @manipulate for >>>>>> paused=false, >>>>>> dt = fpswhen(lift(!, paused), 10) >>>>>> plot(x=collect(1:10),y=rand(10)) >>>>>> end >>>>> >>>>> >>>>> I can observe this problem using Julia 0.4.1, together with the most >>>>> recent releases of all relevant packages, in either Safari on OS X or >>>>> Chrome on Windows 10. >>>>> >>>>> Here's hoping someone has an idea of what's going on or advice for how >>>>> to track down this problem. It seems like something that many others >>>>> should >>>>> be experiencing. >>>>> >>>>> Thanks, >>>>> Andrew >>>>> >>>> >> >
Re: [julia-users] Memory leak for animated plots
Thanks for your response. Patchwork.jl is impressive! This fix almost works, but there is some new issue. When I first execute, I see the following: Javascript error adding output! ReferenceError: Patchwork is not defined See your browser Javascript console for more details. If I move the slider, the plot appears, but the Javascript error remains. I can however confirm that the memory leak is gone. I've tried removing several .ji files, rebuilding several things, etc. and the error seems to persist. Thoughts? I'm running Chrome on Windows 10. On Friday, November 20, 2015 at 9:48:51 AM UTC-8, Shashi Gowda wrote: > > If you install Patchwork.jl <http://github.com/shashi/Patchwork.jl>, and > re-run your notebook, it should fix this issue. (you might also need to > pre-compile Compose again - can be done by removing the .ji file in > ~/.julia/lib/v0.4/) > > Compose (which Gadfly uses for rendering to SVG) doesn't depend on > Patchwork, but if you have it installed, it will use the Patchwork backend > when you render a @manipulate of plots - Patchwork will then try to > reconcile previously rendered DOM nodes so that there are no performance > penalties of this sort. > > On Fri, Nov 20, 2015 at 10:48 PM, Andrew Keller <andrew.k...@gmail.com > > wrote: > >> I think this is exactly what is happening. Some findings: >> >> 1) run the code: >> >> using Interact, Gadfly >>> @manipulate for φ=0:π/16:4π, f=[:sin => sin, :cos => cos] >>> plot((θ)->f(θ+φ),0,25) >>> end >> >> >> 2) Chrome dev tools--> Profiles--> heap snapshot. >> 3) Click sin, cos, sin, cos, sin, cos, ... sin in the notebook >> 4) Take another heap snapshot and look in comparison view. >> >> It looks like among other things there are a lot of SVGPathElements and >> SVGTextElements that belong to detached DOM trees, suggesting the old plots >> never get properly disposed. If I instead capture a JS profile in the >> Chrome dev tools-->Timeline panel, it appears like the number of nodes and >> listeners increases without bound. >> >> Now suppose I use Winston instead of Gadfly. The memory still appears to >> leak, although the plots are a little more lightweight and the leak is >> slower. >> >> Anyway, I'll submit an issue to the jupyter/notebook repo later today >> although I wish I could pin down better where exactly the leak is coming >> from. >> >> On Thursday, November 19, 2015 at 10:35:31 AM UTC-8, Keno Fischer wrote: >>> >>> Sounds like the memory leak is on the browser side? Maybe something is >>> keeping a javascript reference to the plot? Potentially a Jupyter/IJulia >>> bug? >>> >>> On Thu, Nov 19, 2015 at 12:01 PM, Stefan Karpinski <ste...@karpinski.org >>> > wrote: >>> >>>> This should work – if there's a memory leak that's never reclaimed by >>>> gc, that's a bug. >>>> >>>> On Thu, Nov 19, 2015 at 11:55 AM, Andrew Keller <andrew.k...@gmail.com> >>>> wrote: >>>> >>>>> Maybe generating a new plot every time is not great practice, on >>>>> account of the performance hit. That being said, I think it's perfectly >>>>> legitimate to do what I'm doing for prototyping purposes. I can achieve >>>>> the >>>>> frame rate I want and the main example shown on >>>>> https://github.com/JuliaLang/Interact.jl does the same thing I do, >>>>> generating a new plot each time. >>>>> >>>>> In fact, I'd encourage anyone reading this to just try that example, >>>>> and repeatedly click between sin and cos. I'm able to make the memory >>>>> consumption of my browser grow without bound. Surely someone besides >>>>> myself >>>>> has noticed this before! I don't think loading another package is a >>>>> serious >>>>> solution to the problem I'm describing, although your package certainly >>>>> looks useful for other purposes. >>>>> >>>>> Just to reiterate, this is not a small memory leak; this is like a >>>>> memory dam breach. I'm happy to help debug this but some assistance would >>>>> be appreciated. >>>>> >>>>> >>>>> On Thursday, November 19, 2015 at 7:11:55 AM UTC-8, Tom Breloff wrote: >>>>>> >>>>>> You're creating a new Gadfly.Plot object every update, which is a bad >>>>>> idea even if Gadfly's memory management was
[julia-users] Memory leak for animated plots
I'd like to use Interact to have a plot that updates frequently in a Jupyter notebook, but it seems like there is a large memory leak somewhere and I am having some trouble tracking down what package is responsible. Within a few minutes of running, the following code will cause the memory used by the web browser to balloon to well over 1 GB with no sign of slowing down. It is almost like the memory allocated for displaying a particular plot is never deallocated: using Reactive, Interact, Gadfly @manipulate for > paused=false, > dt = fpswhen(lift(!, paused), 10) > plot(x=collect(1:10),y=rand(10)) > end I can observe this problem using Julia 0.4.1, together with the most recent releases of all relevant packages, in either Safari on OS X or Chrome on Windows 10. Here's hoping someone has an idea of what's going on or advice for how to track down this problem. It seems like something that many others should be experiencing. Thanks, Andrew
Re: [julia-users] Memory leak for animated plots
Maybe generating a new plot every time is not great practice, on account of the performance hit. That being said, I think it's perfectly legitimate to do what I'm doing for prototyping purposes. I can achieve the frame rate I want and the main example shown on https://github.com/JuliaLang/Interact.jl does the same thing I do, generating a new plot each time. In fact, I'd encourage anyone reading this to just try that example, and repeatedly click between sin and cos. I'm able to make the memory consumption of my browser grow without bound. Surely someone besides myself has noticed this before! I don't think loading another package is a serious solution to the problem I'm describing, although your package certainly looks useful for other purposes. Just to reiterate, this is not a small memory leak; this is like a memory dam breach. I'm happy to help debug this but some assistance would be appreciated. On Thursday, November 19, 2015 at 7:11:55 AM UTC-8, Tom Breloff wrote: > > You're creating a new Gadfly.Plot object every update, which is a bad idea > even if Gadfly's memory management was perfect. Plots.jl gives you the > ability to add to or replace the underlying data like this: > > using Plots > gadfly() > getxy() = (1:10, rand(10)) > plt = plot(x, y) > > # overwrite underlying plot data without building a new plot > plt[1] = getxy() > > > You can also use familiar push! and append! calls. > > Let me know if this helps, and please post issues if you find bugs. Of > course the memory issue could be while redisplaying in IJulia, in which > case this method won't help. > > On Thursday, November 19, 2015, Andrew Keller <andrew.k...@gmail.com > > wrote: > >> I'd like to use Interact to have a plot that updates frequently in a >> Jupyter notebook, but it seems like there is a large memory leak somewhere >> and I am having some trouble tracking down what package is responsible. >> Within a few minutes of running, the following code will cause the memory >> used by the web browser to balloon to well over 1 GB with no sign of >> slowing down. It is almost like the memory allocated for displaying a >> particular plot is never deallocated: >> >> using Reactive, Interact, Gadfly >> >> @manipulate for >>> paused=false, >>> dt = fpswhen(lift(!, paused), 10) >>> plot(x=collect(1:10),y=rand(10)) >>> end >> >> >> I can observe this problem using Julia 0.4.1, together with the most >> recent releases of all relevant packages, in either Safari on OS X or >> Chrome on Windows 10. >> >> Here's hoping someone has an idea of what's going on or advice for how to >> track down this problem. It seems like something that many others should be >> experiencing. >> >> Thanks, >> Andrew >> >
[julia-users] Modules and namespaces
I'm using Julia 0.4.0 on Mac OS X 10.10.5. I'd like to put some code into a
module, but I'm having some trouble with namespaces. The following fails
(`UndefVarError: test.a not defined`) when enclosed inside `module test`.
When outside the module, e.g. pasted into the REPL, the code works fine.
Could someone point me to relevant reading material or explain what is
going on? It seems I can avoid the problem by putting the string "a" in
the dictionary instead of the abstract type, but I want to know why I am
unable to do things as written. Thank you for your patience as I am new to
the language.
module test
abstract a
dict = Dict("key" => a)
function createType(typeName::ASCIIString,supertype::DataType)
typeSymb = symbol(typeName)
superSymb = symbol(supertype)
@eval immutable ($typeSymb){T} <: $superSymb
num::Float64
end
end
createType("b",dict["key"])
end
Re: [julia-users] Modules and namespaces
Thank you for the helpful advice. In this particular case, I can indeed
just do what you suggest and call @eval at the top level in my module in a
for loop. It would be useful to know explicitly why it is considered poor
form to define types inside a function; I don't think it is clear from the
follow-up link, though I found it helpful for other reasons. I'm using the
function for my own internal purposes and not exporting it.
Elsewhere in my code, I define a function to create other *functions*—not
types—based on a few parameters. My module has several includes, and in
each included file it makes sense to create functions based on some
parameters in a Dict. So in this case, I like using the function to create
other functions, because I can just call it in each included file. Is this
considered bad style? If so, what is an alternative that is comparably
concise?
On Sunday, October 11, 2015 at 3:04:25 PM UTC-7, Isaiah wrote:
>
> You are calling `symbol` on an object, which results in a fully-qualified
> name when called inside a module:
>
> julia> module Foo
>abstract a
>f() = symbol(a)
>end
>
> julia> Foo.f()
>
> symbol("Foo.a")
>
>
> (or try adding `@show superSymb` inside your function)
>
> Creating a symbol from a type instance here isn't really necessary because
> you can splice in `$supertype` directly. (see the "Metaprogramming" section
> of the manual)
>
> Having said that: calling a function to create a type is not
> recommended/idiomatic. Instead, you could call `@eval` at the top level in
> your module (possibly in a for loop). There are a handful of examples of
> this in base, for example in "linalg/triangular.jl".
>
>
>
>
> On Sun, Oct 11, 2015 at 12:01 PM, Andrew Keller <andrew.k...@gmail.com
> > wrote:
>
>> I'm using Julia 0.4.0 on Mac OS X 10.10.5. I'd like to put some code into
>> a module, but I'm having some trouble with namespaces. The following fails
>> (`UndefVarError: test.a not defined`) when enclosed inside `module test`.
>> When outside the module, e.g. pasted into the REPL, the code works fine.
>> Could someone point me to relevant reading material or explain what is
>> going on? It seems I can avoid the problem by putting the string "a" in
>> the dictionary instead of the abstract type, but I want to know why I am
>> unable to do things as written. Thank you for your patience as I am new to
>> the language.
>>
>> module test
>>
>> abstract a
>>
>> dict = Dict("key" => a)
>>
>> function createType(typeName::ASCIIString,supertype::DataType)
>>
>> typeSymb = symbol(typeName)
>> superSymb = symbol(supertype)
>> @eval immutable ($typeSymb){T} <: $superSymb
>>
>> num::Float64
>>
>> end
>>
>> end
>>
>> createType("b",dict["key"])
>>
>> end
>>
>
>
