[julia-users] Re: What reason is different result, when I change R code to Julia code? Thank you!
Perfect. Thank you for your help. 在 2015年7月29日星期三 UTC-5上午12:30:59，Tony Kelman写道： The reason is and | operators are bitwise in Julia, and have a higher precedence than == and !=. julia 1 == 1 3 == 3 false julia 1 == 1 3 == 3 true julia (1 == 1) (3 == 3) true julia 1 3 1 There are a few issues on this, combining the scalar bitwise and array-elementwise meanings into the same and | operators is not ideal and can lead to confusion here. When in doubt use extra parentheses around conditionals, and use or || for scalar comparisons.
Re: [julia-users] John L. Gustafson's UNUMs
If anyone wants to follow, comment, or contribute: https://github.com/tbreloff/Unums.jl On Wed, Jul 29, 2015 at 12:01 PM, Job van der Zwan j.l.vanderz...@gmail.com wrote: Steven, have you read the book or are you basing your judgment on the available presentations linked so far? Because you seem to be saying the same things Gustafson says, except with opposite conclusions about unums, ubounds and uboxes.
Re: [julia-users] John L. Gustafson's UNUMs
On Wednesday, July 29, 2015 at 11:33:07 AM UTC-4, Steven G. Johnson wrote: On Wednesday, July 29, 2015 at 11:23:34 AM UTC-4, Scott Jones wrote: For me, the nice thing (if I understand this correctly) is that UNUMs let me know that there *was* roundoff error, whereas with currently IEEE binary *and* decimal standards, you have no way of telling. IEEE floating-point has the inexact exception flag to signal that a roundoff error occurred. The unum proposal would store an inexact bit in each number, and I'm skeptical that this adds much value. (In practice, almost all nontrivial computations with a fixed precision will incur a rounding error, and if one output is inexact usually all of them are.) Right, if you *do* set the exception flag, but I don't think I've ever seen anybody run with that flag set. About the nontrivial computation part, I think that really depends on your use cases. A large part of the calculations done in Caché were exact (think of simple stuff like adding up a bunch of numbers). (I know this, since I wrote the math package myself, and did a lot of testing to see just what sorts of operations were being done [so I could optimize the more frequent ones more], and to make sure I did handle the rounding correctly).
Re: [julia-users] In what version is Julia supposed to mature?
Doing more frequent major releases than has been traditional for programming languages strikes me as not a terrible idea, honestly. On Tue, Jul 28, 2015 at 11:26 PM, Tony Kelman t...@kelman.net wrote: I'm with Kevin, having followed development (too) closely for the last year and a half I find the prospect of 1.0 any time during 2016 totally ridiculous and unrelealistic. Unless you fully anticipate releasing 2.0 some time in 2017. On Tuesday, July 28, 2015 at 6:52:36 PM UTC-7, Stefan Karpinski wrote: That's literally the only part of that post that I would change :-) But no, I'm not trolling, 1.0 should be out next year. Predicting down to the month – or even quarter – is hard, but that's what I think we're looking at. I'll post a 1.0 roadmap issue soon. On Tue, Jul 28, 2015 at 6:56 PM, Kevin Squire kevin@gmail.com wrote: Stefan, are you trolling again? ;-P http://julialang.org/blog/2012/02/why-we-created-julia/ On Tue, Jul 28, 2015 at 12:41 PM, Stefan Karpinski ste...@karpinski.org wrote: Version 1.0 will be released around this time next year. On Tue, Jul 28, 2015 at 3:27 PM, Pileas phoebus@gmail.com wrote: Greetings, I have been following the development of Julia for sometime now and I am really thrilled to know that you guys have reached version 0.3.11. To my understanding sometime in the near future you will release the new version 0.4.0., a version that it is supposed to bring many changes. My question is simple: when is Julia expected to mature, so that a universal (more or less) documentation (or maybe more thorough books than those that exist by now) will follow and less bug fixed will be needed? I wish you the best!
Re: [julia-users] In what version is Julia supposed to mature?
Actually considering that we've been more strict about version discipline than absolutely required by semver for 0.x.y, maybe we could pull a GCC and just start treating major number the way we've been treating minor. It might not be all that different, except we'd be able to do the backporting features thing and have a very good way of dealing with it. So I retract my incredulity. On Wednesday, July 29, 2015 at 9:10:46 AM UTC-7, Stefan Karpinski wrote: Doing more frequent major releases than has been traditional for programming languages strikes me as not a terrible idea, honestly. On Tue, Jul 28, 2015 at 11:26 PM, Tony Kelman to...@kelman.net javascript: wrote: I'm with Kevin, having followed development (too) closely for the last year and a half I find the prospect of 1.0 any time during 2016 totally ridiculous and unrelealistic. Unless you fully anticipate releasing 2.0 some time in 2017. On Tuesday, July 28, 2015 at 6:52:36 PM UTC-7, Stefan Karpinski wrote: That's literally the only part of that post that I would change :-) But no, I'm not trolling, 1.0 should be out next year. Predicting down to the month – or even quarter – is hard, but that's what I think we're looking at. I'll post a 1.0 roadmap issue soon. On Tue, Jul 28, 2015 at 6:56 PM, Kevin Squire kevin@gmail.com wrote: Stefan, are you trolling again? ;-P http://julialang.org/blog/2012/02/why-we-created-julia/ On Tue, Jul 28, 2015 at 12:41 PM, Stefan Karpinski ste...@karpinski.org wrote: Version 1.0 will be released around this time next year. On Tue, Jul 28, 2015 at 3:27 PM, Pileas phoebus@gmail.com wrote: Greetings, I have been following the development of Julia for sometime now and I am really thrilled to know that you guys have reached version 0.3.11. To my understanding sometime in the near future you will release the new version 0.4.0., a version that it is supposed to bring many changes. My question is simple: when is Julia expected to mature, so that a universal (more or less) documentation (or maybe more thorough books than those that exist by now) will follow and less bug fixed will be needed? I wish you the best!
Re: [julia-users] John L. Gustafson's UNUMs
On Wednesday, July 29, 2015 at 10:30:41 AM UTC-4, Tom Breloff wrote: Correct me if I'm wrong, but (fixed-size) decimal floating-point has most of the same issues as floating point in terms of accumulation of errors, right? What issues are you referring to? There are a lot of crazy myths out there about floating-point arithmetic. For any operation that you could perform exactly in fixed-point arithmetic with a given number of bits, the same operation will also be performed exactly in decimal floating-point with the same number of bits for the signficand. However, for the same total width (e.g. 64 bits), decimal floating point sacrifices a few bits of precision in exchange for dynamical scaling (i.e. the exponent), which gives exact representations for a vastly expanded dynamic range. Furthermore, for operations that *do* involve roundoff error in either fixed- or decimal floating-point arithmetic with a fixed number of bits, the error accumulation is usually vastly better in floating point than fixed-point. (e.g. there is no equivalent of pairwise summation, with logarithmic error growth, in fixed-point arithmetic.) If you want no roundoff errors, ever, then you have no choice but to use some kind of (slow) arbitrary-precision type, and even then there are plenty of operations you can't allow (e.g. division, unless you are willing to use arbitrary-precision rationals with exponential complexity) or square roots.
Re: [julia-users] John L. Gustafson's UNUMs
For me, the nice thing (if I understand this correctly) is that UNUMs let me know that there *was* roundoff error, whereas with currently IEEE binary *and* decimal standards, you have no way of telling. On Wednesday, July 29, 2015 at 11:14:43 AM UTC-4, Steven G. Johnson wrote: On Wednesday, July 29, 2015 at 10:30:41 AM UTC-4, Tom Breloff wrote: Correct me if I'm wrong, but (fixed-size) decimal floating-point has most of the same issues as floating point in terms of accumulation of errors, right? What issues are you referring to? There are a lot of crazy myths out there about floating-point arithmetic. For any operation that you could perform exactly in fixed-point arithmetic with a given number of bits, the same operation will also be performed exactly in decimal floating-point with the same number of bits for the signficand. However, for the same total width (e.g. 64 bits), decimal floating point sacrifices a few bits of precision in exchange for dynamical scaling (i.e. the exponent), which gives exact representations for a vastly expanded dynamic range. Furthermore, for operations that *do* involve roundoff error in either fixed- or decimal floating-point arithmetic with a fixed number of bits, the error accumulation is usually vastly better in floating point than fixed-point. (e.g. there is no equivalent of pairwise summation, with logarithmic error growth, in fixed-point arithmetic.) If you want no roundoff errors, ever, then you have no choice but to use some kind of (slow) arbitrary-precision type, and even then there are plenty of operations you can't allow (e.g. division, unless you are willing to use arbitrary-precision rationals with exponential complexity) or square roots.
Re: [julia-users] In what version is Julia supposed to mature?
I guess the waters are a little muddied here lately with Rust having recently put such a big emphasis on stability and reaching 1.0, actively telling people not to use the language prior to that point, and seemingly having really high expectations about how long 1.x will last for. They have a much smaller standard library than we do, but I would think trimming ours down to the bare minimum would be necessary before calling the language 1.0. Maybe that could just as well be a 2.0 or 3.0 target instead. On Wednesday, July 29, 2015 at 9:28:56 AM UTC-7, Stefan Karpinski wrote: I do believe that other languages have not really followed the semantic versioning specification (can't blame them really since it didn't exist) and have introduced backwards incompatible changes in minor versions. If we're going to follow semver, then we will very likely want to make major releases more often since we will probably have some backwards incompatible changes we want to introduce periodically, even if they're not huge. On Wed, Jul 29, 2015 at 12:20 PM, Tony Kelman to...@kelman.net javascript: wrote: Actually considering that we've been more strict about version discipline than absolutely required by semver for 0.x.y, maybe we could pull a GCC and just start treating major number the way we've been treating minor. It might not be all that different, except we'd be able to do the backporting features thing and have a very good way of dealing with it. So I retract my incredulity. On Wednesday, July 29, 2015 at 9:10:46 AM UTC-7, Stefan Karpinski wrote: Doing more frequent major releases than has been traditional for programming languages strikes me as not a terrible idea, honestly. On Tue, Jul 28, 2015 at 11:26 PM, Tony Kelman to...@kelman.net wrote: I'm with Kevin, having followed development (too) closely for the last year and a half I find the prospect of 1.0 any time during 2016 totally ridiculous and unrelealistic. Unless you fully anticipate releasing 2.0 some time in 2017. On Tuesday, July 28, 2015 at 6:52:36 PM UTC-7, Stefan Karpinski wrote: That's literally the only part of that post that I would change :-) But no, I'm not trolling, 1.0 should be out next year. Predicting down to the month – or even quarter – is hard, but that's what I think we're looking at. I'll post a 1.0 roadmap issue soon. On Tue, Jul 28, 2015 at 6:56 PM, Kevin Squire kevin@gmail.com wrote: Stefan, are you trolling again? ;-P http://julialang.org/blog/2012/02/why-we-created-julia/ On Tue, Jul 28, 2015 at 12:41 PM, Stefan Karpinski ste...@karpinski.org wrote: Version 1.0 will be released around this time next year. On Tue, Jul 28, 2015 at 3:27 PM, Pileas phoebus@gmail.com wrote: Greetings, I have been following the development of Julia for sometime now and I am really thrilled to know that you guys have reached version 0.3.11. To my understanding sometime in the near future you will release the new version 0.4.0., a version that it is supposed to bring many changes. My question is simple: when is Julia expected to mature, so that a universal (more or less) documentation (or maybe more thorough books than those that exist by now) will follow and less bug fixed will be needed? I wish you the best!
Re: [julia-users] John L. Gustafson's UNUMs
If you really want me to, I will ;-) (but as you already know, I'm known as being a big PITA!) On Wednesday, July 29, 2015 at 12:47:07 PM UTC-4, Tom Breloff wrote: If anyone wants to follow, comment, or contribute: https://github.com/tbreloff/Unums.jl On Wed, Jul 29, 2015 at 12:01 PM, Job van der Zwan j.l.van...@gmail.com javascript: wrote: Steven, have you read the book or are you basing your judgment on the available presentations linked so far? Because you seem to be saying the same things Gustafson says, except with opposite conclusions about unums, ubounds and uboxes.
Re: [julia-users] John L. Gustafson's UNUMs
On Wednesday, July 29, 2015 at 11:23:34 AM UTC-4, Scott Jones wrote: For me, the nice thing (if I understand this correctly) is that UNUMs let me know that there *was* roundoff error, whereas with currently IEEE binary *and* decimal standards, you have no way of telling. IEEE floating-point has the inexact exception flag to signal that a roundoff error occurred. The unum proposal would store an inexact bit in each number, and I'm skeptical that this adds much value. (In practice, almost all nontrivial computations with a fixed precision will incur a rounding error, and if one output is inexact usually all of them are.)
Re: [julia-users] John L. Gustafson's UNUMs
The inexact exception flag isn't quite the same. That flag just signifies that at some point in the calculation there was rounding. It doesn't give any indication of how wrong the final number may be. The ubit coupled with variable-sized unums allows for potentially very precise interval arithmetic, but without requiring the full storage. For example, if we want to represent the value 1e-100, the unum would effectively be the range (0, smallnum), where smallnum is the smallest positive number we can represent exactly. This actually requires very little actual storage for a packed unum, although it can be an extremely precise range. On Wed, Jul 29, 2015 at 11:33 AM, Steven G. Johnson stevenj@gmail.com wrote: On Wednesday, July 29, 2015 at 11:23:34 AM UTC-4, Scott Jones wrote: For me, the nice thing (if I understand this correctly) is that UNUMs let me know that there *was* roundoff error, whereas with currently IEEE binary *and* decimal standards, you have no way of telling. IEEE floating-point has the inexact exception flag to signal that a roundoff error occurred. The unum proposal would store an inexact bit in each number, and I'm skeptical that this adds much value. (In practice, almost all nontrivial computations with a fixed precision will incur a rounding error, and if one output is inexact usually all of them are.)
Re: [julia-users] In what version is Julia supposed to mature?
I do believe that other languages have not really followed the semantic versioning specification (can't blame them really since it didn't exist) and have introduced backwards incompatible changes in minor versions. If we're going to follow semver, then we will very likely want to make major releases more often since we will probably have some backwards incompatible changes we want to introduce periodically, even if they're not huge. On Wed, Jul 29, 2015 at 12:20 PM, Tony Kelman t...@kelman.net wrote: Actually considering that we've been more strict about version discipline than absolutely required by semver for 0.x.y, maybe we could pull a GCC and just start treating major number the way we've been treating minor. It might not be all that different, except we'd be able to do the backporting features thing and have a very good way of dealing with it. So I retract my incredulity. On Wednesday, July 29, 2015 at 9:10:46 AM UTC-7, Stefan Karpinski wrote: Doing more frequent major releases than has been traditional for programming languages strikes me as not a terrible idea, honestly. On Tue, Jul 28, 2015 at 11:26 PM, Tony Kelman to...@kelman.net wrote: I'm with Kevin, having followed development (too) closely for the last year and a half I find the prospect of 1.0 any time during 2016 totally ridiculous and unrelealistic. Unless you fully anticipate releasing 2.0 some time in 2017. On Tuesday, July 28, 2015 at 6:52:36 PM UTC-7, Stefan Karpinski wrote: That's literally the only part of that post that I would change :-) But no, I'm not trolling, 1.0 should be out next year. Predicting down to the month – or even quarter – is hard, but that's what I think we're looking at. I'll post a 1.0 roadmap issue soon. On Tue, Jul 28, 2015 at 6:56 PM, Kevin Squire kevin@gmail.com wrote: Stefan, are you trolling again? ;-P http://julialang.org/blog/2012/02/why-we-created-julia/ On Tue, Jul 28, 2015 at 12:41 PM, Stefan Karpinski ste...@karpinski.org wrote: Version 1.0 will be released around this time next year. On Tue, Jul 28, 2015 at 3:27 PM, Pileas phoebus@gmail.com wrote: Greetings, I have been following the development of Julia for sometime now and I am really thrilled to know that you guys have reached version 0.3.11. To my understanding sometime in the near future you will release the new version 0.4.0., a version that it is supposed to bring many changes. My question is simple: when is Julia expected to mature, so that a universal (more or less) documentation (or maybe more thorough books than those that exist by now) will follow and less bug fixed will be needed? I wish you the best!
Re: [julia-users] John L. Gustafson's UNUMs
On Wednesday, July 29, 2015 at 11:47:42 AM UTC-4, Tom Breloff wrote: The inexact exception flag isn't quite the same. That flag just signifies that at some point in the calculation there was rounding. It doesn't give any indication of how wrong the final number may be. The ubit coupled with variable-sized unums allows for potentially very precise interval arithmetic, but without requiring the full storage. Interval arithmetic is usually considered to be useless for error analysis in most cases unless it is coupled with sophisticated additional analysis (e.g. Taylor arithmetic) because of the dependency problem; that's why they are proposing uboxes but with vastly greater memory and computation requirements than interval arithmetic. (Interval arithmetic has other applications outside error analysis, though.) And the slight memory gain of using fewer than 64 bits will be completely swamped for any large computation (i.e., any computation where the memory matters) by the cost of software unums. If someone makes an efficient hardware implementation, matters would be different, but I'm not going to hold my breath for that.
Re: [julia-users] John L. Gustafson's UNUMs
Steven, have you read the book or are you basing your judgment on the available presentations linked so far? Because you seem to be saying the same things Gustafson says, except with opposite conclusions about unums, ubounds and uboxes.
[julia-users] VoronoiCells.jl
Here is some code to manipulate Voronoi cells in 2D. https://github.com/jlapeyre/VoronoiCells.jl I am developing these recent packages more or less just enough to get some research done. When things cool down (i.e. baby is grown and I get an endowed chair of free creativity) I plan to make them proper packages. I'm putting them out in case someone happens to have the same need I did, (or wants to contribute or comment). If you do use something and have a question, or want something specific documented, let me know. Of course, this should eventually be integrated into the Julia geometry packages. This code uses VoronoiDelaunay.jl to generate a tesselation. Then the objects representing cells are built from the triangles. There are a lot of choices about what to store and what to compute. The current choice is driven by my current need. The cell containing a random (as in access) point can be retrieved quickly using a standard bucket scheme. The cell stores the generator and the vertices. A simpler (undocumented) structure is provided if you don't need to lookup by random point. A few features such as area of cells are provided. John
Re: [julia-users] John L. Gustafson's UNUMs
Job, I'm basing my judgement on the presentation.
[julia-users] parallel threads broken, replacing module
Parallel threads has stopped working for me. Any ideas ? Code using addprocs and @everywhere include has stopped working (is broken) on one machine. Code that used to work now causes a number of varying errors to be printed, crashes, runaway processes, etc. when it is loaded. Both a recent v0.4 and a 2 month old version that did run the code in the past cause the failure. The same code on a different machine continues to run normally. Maybe my user Julia environment has a problem ? Or maybe I upgraded a system library ? If I use just 2 processes, the code will sometimes both load and run correctly. A minimal example test.jl -- using Distributions -- julia -p 4 @everywhere include(test.jl) WARNING: replacing module Distributions ( 4 times ) signal (11): Segmentation fault unknown function (ip: 0x7f235c7ffd98) jl_module_import at /usr/local/julia-0.4a/bin/../lib/julia/libjulia.so (unknown line) etc. I am running unstable debian linux. Thanks, John
[julia-users] Re: John L. Gustafson's UNUMs
On Wednesday, July 29, 2015 at 4:59:02 PM UTC-4, Zenna Tavares wrote: I read the book (well, somewhere between a skim and a proper read). It's not formal but it is clear and the ideas are concise. I actually think it's a pretty good example of how to explain an idea without unnecessary jargon. As for unums themselves, I am mostly convinced of his arguments on unums. I am less convinced on ubounds. My main takeaways are that a unum may represent a set of values, and it's most salient properties are: - Explicit (via ubit) handling of exact and inexact (i.e. set of) real number - Correct handling of open/closed bounds for sets of reals - Variable size with size of number stored in number - With hardware they could be as or more efficient in energy and speed as floating arithmetic but hardware would be more complex As a result of these properties, unums can be closed under arithmetic operations and won't hide your errors due to approximation. I don't buy the advantages he suggests over interval arithmetic; unums don't solve the dependency problem. Without hardware, the advantages of variable size ( Float64) are overwhelmed by the software overhead. Without solving the dependency problem, representing a set of reals (as opposed to an individual real as in standard FP) is useless for error estimation. And I don't see a clear practical use-case for an inexact bit per value, as opposed to a single inexact flag for a whole set of computations (as in IEEE).
Re: [julia-users] Re: John L. Gustafson's UNUMs
On Wed, Jul 29, 2015 at 4:59 PM, Zenna Tavares zennatava...@gmail.com wrote: As a result of these properties, unums can be closed under arithmetic operations and won't hide your errors due to approximation. I'm glad to hear this because this was that part that made me question the whole thing. I guess the fact that you can represent some numbers exactly helps a bit, but not very much. The most compelling part of the proposal to me was the claim of associativity, which I suppose comes along with the variable precision since you can actually drop trailing bits that you can't get right.
Re: [julia-users] John L. Gustafson's UNUMs
Job: I think you're on the right track. Given current hardware optimizations, you might as well create a mostly fixed-length specification, but with the advantages of truth that the ubit provides. I'm thinking through an implementation that uses generated functions and parameterized Unum types, which provide as much precision/accuracy that you need, filling up a standard bit size (8/16/32/64/128), very similar to the unpacked unums that Gustafson describes. On Wed, Jul 29, 2015 at 5:47 PM, Job van der Zwan j.l.vanderz...@gmail.com wrote: On Thursday, 30 July 2015 00:00:56 UTC+3, Steven G. Johnson wrote: Job, I'm basing my judgement on the presentation. Ah ok, I was wondering I feel like those presentations give a general impression, but don't really explain the details enough. And like I said, your critique overlaps with Gustafson's own critique of traditional interval arithmetic, so I wasn't sure if you meant that you don't buy his suggested alternative ubox method after reading the book, or indicated scepticism based on earlier experience, but without full knowledge of what his suggested alternative is. To be clear, it wasn't a you should read the book putdown - I hate comments like that, they destroy every meaningful discussion. The more I think about it, the more I think the ubit is actually the big breakthrough. As a thought experiment, let's ignore the whole flexible-bitsize bit and just take an existing float, but replace the last bit of the fraction with a ubit. What happens? Well.. we give up one bit of *precision* in the fraction, but *our set of representations is still the same size*. We still have the same number of floats as before! It's just that half of them is now exact (with one bit less precision), and the other half represents open intervals between these exact numbers. Which lets you represent the entire real number line accurately (but with limited precision, unless they happen to be equal to an exact float). Compare that to traditional floating point numbers, which are all exact, but unless your calculation is exactly that floating point representation (which is very rare) guaranteed to be off. Think about it this way: regular floats represent a finite subset of rational numbers. More bits increase that finite subset. But add a ubit, and you got the entire real number line. With limited precision, and using the same finite number of representations as a float that has the same number of bits, but still. I'd like to hear some feedback on the example I used earlier from people more versed in this topic than me: ubound arithmetic tells you that 1 / (0, 1] = [1, inf), and that [1, inf) / inf = 0 Thanks to the ubit, the arithmetic is as simple as divide by both endpoints to get the new interval endpoints. It's so simple that I have a hard time believing this was not possible with interval arithmetic before, and again I don't know that much about the topic. So is it really true, like Gustafson claims, that interval arithmetic so far always used *closed* intervals that accepted *rounded* answers? Because if that is true, and even is that is the only thing unums solve... well then I'm sold :P
Re: [julia-users] John L. Gustafson's UNUMs
BTW, Tom, I was already working on a summary of the book (on an IJulia notebook). I'm on mobile right now so don't have access to it, but I can share it later. I think something like that might be useful to attract more collaborators - we can't expect everyone to read it.
Re: [julia-users] In what version is Julia supposed to mature?
There was an interesting discussion on the dev list not long ago https://groups.google.com/d/msg/julia-dev/CYhPCQ4rCLY/ByyIW_Ka0ooJ about the release schedule of Julia. Although it focused more on timeline and less on version numbering, it touched on several of the topics discussed in this thread as well. We do have some “expectation problems” of our own in the Julia community, for example as Tobi put it in the other thread: the main issue is in my point that during a dev period feature come in and people start to use the dev branch for regular development. The Compat module is in my opinion made the wrong way around. What currently happens is that unstable features are bypassed into a “released” package landscape. Julia is an awesome project, and it’s going to be an awesome product when it’s “done”. But currently, we’re in the middle of a phase of development where you cannot expect to accurately predict where we’re going to be in three or six months, much less predict a date for a 1.0 release (after all, it was once said that *“we’ve been much more on top of the release process this time” https://groups.google.com/d/msg/julia-dev/CYhPCQ4rCLY/ByyIW_Ka0ooJ* about 0.4, but we’re still falling behind schedule on this one too…). Don’t get me wrong - I think this is totally fine for a project like Julia. We *should* be experimenting, trying things out and taking the time to make sure the decisions we make are the right ones. But maybe we should stop expecting to have a production-quality product to play around with. (On the other hand, Julia programmers will probably always have high expectations, considering Julia’s origins http://julialang.org/blog/2012/02/why-we-created-julia/…) // T On Wednesday, July 29, 2015 at 7:00:38 PM UTC+2, Tony Kelman wrote: I guess the waters are a little muddied here lately with Rust having recently put such a big emphasis on stability and reaching 1.0, actively telling people not to use the language prior to that point, and seemingly having really high expectations about how long 1.x will last for. They have a much smaller standard library than we do, but I would think trimming ours down to the bare minimum would be necessary before calling the language 1.0. Maybe that could just as well be a 2.0 or 3.0 target instead. On Wednesday, July 29, 2015 at 9:28:56 AM UTC-7, Stefan Karpinski wrote: I do believe that other languages have not really followed the semantic versioning specification (can't blame them really since it didn't exist) and have introduced backwards incompatible changes in minor versions. If we're going to follow semver, then we will very likely want to make major releases more often since we will probably have some backwards incompatible changes we want to introduce periodically, even if they're not huge. On Wed, Jul 29, 2015 at 12:20 PM, Tony Kelman to...@kelman.net wrote: Actually considering that we've been more strict about version discipline than absolutely required by semver for 0.x.y, maybe we could pull a GCC and just start treating major number the way we've been treating minor. It might not be all that different, except we'd be able to do the backporting features thing and have a very good way of dealing with it. So I retract my incredulity. On Wednesday, July 29, 2015 at 9:10:46 AM UTC-7, Stefan Karpinski wrote: Doing more frequent major releases than has been traditional for programming languages strikes me as not a terrible idea, honestly. On Tue, Jul 28, 2015 at 11:26 PM, Tony Kelman to...@kelman.net wrote: I'm with Kevin, having followed development (too) closely for the last year and a half I find the prospect of 1.0 any time during 2016 totally ridiculous and unrelealistic. Unless you fully anticipate releasing 2.0 some time in 2017. On Tuesday, July 28, 2015 at 6:52:36 PM UTC-7, Stefan Karpinski wrote: That's literally the only part of that post that I would change :-) But no, I'm not trolling, 1.0 should be out next year. Predicting down to the month – or even quarter – is hard, but that's what I think we're looking at. I'll post a 1.0 roadmap issue soon. On Tue, Jul 28, 2015 at 6:56 PM, Kevin Squire kevin@gmail.com wrote: Stefan, are you trolling again? ;-P http://julialang.org/blog/2012/02/why-we-created-julia/ On Tue, Jul 28, 2015 at 12:41 PM, Stefan Karpinski ste...@karpinski.org wrote: Version 1.0 will be released around this time next year. On Tue, Jul 28, 2015 at 3:27 PM, Pileas phoebus@gmail.com wrote: Greetings, I have been following the development of Julia for sometime now and I am really thrilled to know that you guys have reached version 0.3.11. To my understanding sometime in the near future you will release the new version 0.4.0., a version that it is supposed to bring many changes. My question is simple: when is Julia expected to mature, so that a universal (more or less) documentation
[julia-users] Re: John L. Gustafson's UNUMs
I read the book (well, somewhere between a skim and a proper read). It's not formal but it is clear and the ideas are concise. I actually think it's a pretty good example of how to explain an idea without unnecessary jargon. As for unums themselves, I am mostly convinced of his arguments on unums. I am less convinced on ubounds. My main takeaways are that a unum may represent a set of values, and it's most salient properties are: - Explicit (via ubit) handling of exact and inexact (i.e. set of) real number - Correct handling of open/closed bounds for sets of reals - Variable size with size of number stored in number - With hardware they could be as or more efficient in energy and speed as floating arithmetic but hardware would be more complex As a result of these properties, unums can be closed under arithmetic operations and won't hide your errors due to approximation. I don't buy the advantages he suggests over interval arithmetic; unums don't solve the dependency problem. I have been meaning to start a Unum.jl package myself. Zenna On Saturday, July 25, 2015 at 9:11:54 AM UTC-4, Job van der Zwan wrote: So I came across the concept of UNUMs on the Pony language mailing list http://lists.ponylang.org/pipermail/ponydev/2015-July/71.html this morning. I hadn't heard of them before, and a quick search doesn't show up anything on this mailing list, so I guess most people here haven't either. They're a proposed alternate encoding for numbers by John L. Gustafson. This presentation by him sums it up nicely: http://sites.ieee.org/scv-cs/files/2013/03/Right-SizingPrecision1.pdf “Unums”(universal numbers) are to floating point what floating point is to fixed point. Floating-point values self-describe their scale factor, but fix the exponent and fraction size. Unums self-describe the exponent size, fraction size, and inexact state, and include fixed point and IEEE floats as special cases. The presentation can be seen here, provided you have the Silverlight plugin: http://sites.ieee.org/scv-cs/archives/right-sizing-precision-to-save-energy-power-and-storage Now, I don't know enough about this topic to say if they're a good or bad idea, but I figured the idea is interesting/relevant enough to share with the Julia crowd. I'm also wondering if they could be implemented (relatively) easily within Julia, given its flexible type system. If so, they might provide an interesting advanced example, no?
Re: [julia-users] John L. Gustafson's UNUMs
On Thursday, 30 July 2015 00:00:56 UTC+3, Steven G. Johnson wrote: Job, I'm basing my judgement on the presentation. Ah ok, I was wondering I feel like those presentations give a general impression, but don't really explain the details enough. And like I said, your critique overlaps with Gustafson's own critique of traditional interval arithmetic, so I wasn't sure if you meant that you don't buy his suggested alternative ubox method after reading the book, or indicated scepticism based on earlier experience, but without full knowledge of what his suggested alternative is. To be clear, it wasn't a you should read the book putdown - I hate comments like that, they destroy every meaningful discussion. The more I think about it, the more I think the ubit is actually the big breakthrough. As a thought experiment, let's ignore the whole flexible-bitsize bit and just take an existing float, but replace the last bit of the fraction with a ubit. What happens? Well.. we give up one bit of *precision* in the fraction, but *our set of representations is still the same size*. We still have the same number of floats as before! It's just that half of them is now exact (with one bit less precision), and the other half represents open intervals between these exact numbers. Which lets you represent the entire real number line accurately (but with limited precision, unless they happen to be equal to an exact float). Compare that to traditional floating point numbers, which are all exact, but unless your calculation is exactly that floating point representation (which is very rare) guaranteed to be off. Think about it this way: regular floats represent a finite subset of rational numbers. More bits increase that finite subset. But add a ubit, and you got the entire real number line. With limited precision, and using the same finite number of representations as a float that has the same number of bits, but still. I'd like to hear some feedback on the example I used earlier from people more versed in this topic than me: ubound arithmetic tells you that 1 / (0, 1] = [1, inf), and that [1, inf) / inf = 0 Thanks to the ubit, the arithmetic is as simple as divide by both endpoints to get the new interval endpoints. It's so simple that I have a hard time believing this was not possible with interval arithmetic before, and again I don't know that much about the topic. So is it really true, like Gustafson claims, that interval arithmetic so far always used *closed* intervals that accepted *rounded* answers? Because if that is true, and even is that is the only thing unums solve... well then I'm sold :P
Re: [julia-users] Re: MongoDB and Julia
Hi Kevin, I'm sorry to hear (and see) that you haven't gotten much help on the Julia mailing list. It's probably just that your request hasn't fallen on the ears of anyone who has interest in both Mongo and Julia and the time to help. I'm almost in that category, in that I have a small interest in Mongo (and a lot of interest in Julia), but only so much time. I will say that once you get something working, the mailing list will usually be a good resource, but it works best if you have specific questions or problems (e.g., I'm trying to do this with the following code, but it's not working--what am I doing wrong) vs. general requests (e.g., asking for help wrapping mongo with a mostly empty repo). I'm assuming this is your first major Julia project? If so, my first suggestion is to ignore everything written below (for now), and try to get the Mongo.jl library running on a modern Julia first, and make sure you understand everything that it's doing. Only after that would I try anything below. In truth, I don't know the best way forward, but I can give you a little more information, and maybe it will help you decide. Clang.jl is a general framework for wrapping C libraries. It's never necessary--any C library can be called directly using ccall. But ccalls can be kind of clunky, so Clang.jl provides an API which very closely mimics the C API provided in some header file: functions have very similar signatures, and structs become julia types. For a large library with lots of functions, or complicated structs with lots of members, this can make wrapping relatively easy. While the output of Clang.jl is usually reasonably nice, it's still a very low level of abstraction. Actually, very little abstraction, because it matches the C library, which is usually very low level, and still somewhat inconvenient in Julia (but more convenient than ccalls). To be useful, you'll often want to add a higher level API on top of that, which adds functions and/or types that encompass or simplify the lower level calls. It's also often the case that Clang.jl doesn't give you exactly what you need--e.g., it doesn't know how to wrap certain things, such as unions and C macros. In those cases, you'll have to edit the output by hand, or spend some time programmatically filtering/modifying the results. I do this a bit in the wrapper for VideoIO https://github.com/kmsquire/VideoIO.jl/blob/master/util/wrap_libav_split.jl (and I still have to edit some files by hand at the end). The main alternative is to wrap a subset of useful functions by hand (or simply use ccall directly, which amounts to pretty much the same thing). This is probably closer to what the Mongo folks had in mind when they directed you to the Lua driver. If you only need access to a few functions, or if your code is highly specialized in a way that Clang.jl has trouble with, wrapping by hand can be the way to go. For this, you're basically writing Julia functions which ccall out to external library functions (such as those in mongo), and returns the result (or some modification thereof that matches what would normally be done in Julia). Hopefully this was useful. Please feel free to post back here with questions, and I (and maybe others, if the question is right) will try to answer as we have time. Cheers! Kevin On Wed, Jul 29, 2015 at 3:00 PM, Kevin Liu kevinliu2...@gmail.com wrote: Thanks for the valuable advice and modesty, there is no code in the reps. I'm learning how to do this properly. From the Mongo side I have received a little bit of guidance but from the Julia side, very little. Changed it to CMongo.jl. Clang.jl will be of great help (I had bookmarked it, but didn't realize it was a wrapper). On Wed, Jul 29, 2015 at 3:47 AM, Kevin Squire kevin.squ...@gmail.com wrote: I would suggest CMongo.jl (when renaming this, be careful if you're on a case-insensitive filesystem). Of the fully capitalized names in Julia, most of them imply that they are acronyms. Suggestion: at this point, there isn't much code in the repo. Since not many people have (publicly) responded to your posts, IMHO, it would be good to try to get a basic working system in place, and post your progress when something basic is working--this will make it easier for people to contribute. One more thing you should look at is Clang.jl, which makes wrapping C libraries easier. Cheers! Kevin On Tue, Jul 28, 2015 at 5:21 PM, Kevin Liu kevinliu2...@gmail.com wrote: I'll name it CMONGO.jl On Tue, Jul 28, 2015 at 9:13 PM, Kevin Liu kevinliu2...@gmail.com wrote: Any suggestions for the name? I just want to remember this will be a wrapper around C Mongo. On Tue, Jul 28, 2015 at 9:10 PM, Kevin Liu kevinliu2...@gmail.com wrote: Hey Kevin, That's great. Thanks for the advice. On it right now. Cheers! On Tue, Jul 28, 2015 at 7:50 PM, Kevin Squire kevin.squ...@gmail.com wrote: Hi Kevin, If you plan to make this a Julia package
[julia-users] ANN: Jeff Bezanson meetup talk in NYC next Tue
http://www.meetup.com/julia-nyc/events/224261149/ This will have a fair amount of overlap with his keynote at JuliaCon, so if you saw that you may want to skip this, but if you didn't, it's very interesting and has evolved somewhat since June. Also, there's a Julia meetup group in New York. If you live in the area, join the group! Stefan
[julia-users] Re: Irregular Interpolation
I wrote some code that did Polyharmonic (Thin-plate) splines feel free to use them https://github.com/tlycken/Interpolations.jl/issues/6 On Wednesday, July 29, 2015 at 6:16:59 AM UTC-7, Jude wrote: Hi, I have been using the fantastic grid package by Tim Holy for the past while but I really need to allow for a non-equally spaced grids. It is important to have a bunch of points at some parts of the grid but not at others. I was wondering if anyone knows of any package that allows for irregular interpolation. I know it is possible to do this using Tim's package for one dimension but I want to interpolate in 3 dimensions. Has there been any new packages developed lately or does anyone know a fast way to do this? Thank you
Re: [julia-users] John L. Gustafson's UNUMs
I'll keep an eye on it and potentially pitch in! On Wednesday, July 29, 2015 at 12:47:07 PM UTC-4, Tom Breloff wrote: If anyone wants to follow, comment, or contribute: https://github.com/tbreloff/Unums.jl On Wed, Jul 29, 2015 at 12:01 PM, Job van der Zwan j.l.van...@gmail.com javascript: wrote: Steven, have you read the book or are you basing your judgment on the available presentations linked so far? Because you seem to be saying the same things Gustafson says, except with opposite conclusions about unums, ubounds and uboxes.
[julia-users] Re: Irregular Interpolation
It's currently going through a major overhaul to get ready for public consumption, but CompEcon.jl https://github.com/spencerlyon2/CompEcon.jl provides a Julia implementation of the popular (amongst economists) CompEcon matlab toolbox. It does irregular interpolation for an arbitrary number of dimensions using chebyshev, b-spline (of arbitrary order), or piecewise linear basis functions. If you are interested, let me know. I haven't had time to write docs yet, but once I do it should be pretty straightforward. On Wednesday, July 29, 2015 at 10:44:37 AM UTC-4, Nils Gudat wrote: Besided GridInterpolations, you might want to look at ApproXD.jl, which works up to four dimensions. I once wrote a little script that compares different one- and two-dimensional interpolation schemes in Julia, it can be found here https://github.com/nilshg/LearningModels/blob/master/Test_Interpolations.jl (I might have to update it to include GridInterpolations at some point).
[julia-users] Re: parallel threads broken, replacing module
I can file a bug report. But, I'm not entirely sure what to write. Simpler case: julia -p 4 @everywhere using ZChop (simple modules) The following version is from a completely fresh build today. I don't get a crash immediately, but I do get WARNING: replacing module ZChop (4 times) Version 0.4.0-dev+6394 (2015-07-29 21:58 UTC) Commit a2a218b* (0 days old master) Five days ago, i did git pull and built from an older clone Version 0.4.0-dev+6202 (2015-07-24 01:47 UTC) Commit 53a7f2e* (5 days old master) This version prints the same warning and spews many errors including unknown function (ip: 0x7fe94d596ff7) ERROR (unhandled task failure): EOFError: read end of file The following version will load ZChop with no problem, but when loading Distributions, it still prints many errors and fails. It loads some other modules (i.e with @everywhere, as above), such as Roots with no apparent errors. Version 0.4.0-dev+3965 (2015-03-22 12:24 UTC) Commit e1f0310* (129 days old master) A fresh build today on another machine gives errors with @everywhere using Roots. It fails to load Distributions even with no parallel threads, but this appears to be unrelated. On the same (other) machine, the following version loads and runs parallel code using the modules mentioned above, and more. Version 0.4.0-dev+4096 (2015-03-31 08:05 UTC) Commit a3c0743* (120 days old master) I see no error with 0.3.5-pre+121 (2015-01-07 00:19 UTC) On Wednesday, July 29, 2015 at 11:24:55 PM UTC+2, lapeyre@gmail.com wrote: Parallel threads has stopped working for me. Any ideas ? Code using addprocs and @everywhere include has stopped working (is broken) on one machine. Code that used to work now causes a number of varying errors to be printed, crashes, runaway processes, etc. when it is loaded. Both a recent v0.4 and a 2 month old version that did run the code in the past cause the failure. The same code on a different machine continues to run normally. Maybe my user Julia environment has a problem ? Or maybe I upgraded a system library ? If I use just 2 processes, the code will sometimes both load and run correctly. A minimal example test.jl -- using Distributions -- julia -p 4 @everywhere include(test.jl) WARNING: replacing module Distributions ( 4 times ) signal (11): Segmentation fault unknown function (ip: 0x7f235c7ffd98) jl_module_import at /usr/local/julia-0.4a/bin/../lib/julia/libjulia.so (unknown line) etc. I am running unstable debian linux. Thanks, John
Re: [julia-users] Re: MongoDB and Julia
Haha this is my first major project, period! Thanks a lot for putting in the time and effort into guiding. I come from the finance world but became interested in Julia and MongoDB for what they can do with science. It's a hand into understanding so much. Focused on Mongo.jl and will digest your comments after that. I got ahead of myself from Mongo.jl after seeing there was so much more to be done to make the stack fully functioning. Thank you! Will keep you and the community posted. Cheers mate! On Wed, Jul 29, 2015 at 6:50 PM, Kevin Squire kevin.squ...@gmail.com wrote: Hi Kevin, I'm sorry to hear (and see) that you haven't gotten much help on the Julia mailing list. It's probably just that your request hasn't fallen on the ears of anyone who has interest in both Mongo and Julia and the time to help. I'm almost in that category, in that I have a small interest in Mongo (and a lot of interest in Julia), but only so much time. I will say that once you get something working, the mailing list will usually be a good resource, but it works best if you have specific questions or problems (e.g., I'm trying to do this with the following code, but it's not working--what am I doing wrong) vs. general requests (e.g., asking for help wrapping mongo with a mostly empty repo). I'm assuming this is your first major Julia project? If so, my first suggestion is to ignore everything written below (for now), and try to get the Mongo.jl library running on a modern Julia first, and make sure you understand everything that it's doing. Only after that would I try anything below. In truth, I don't know the best way forward, but I can give you a little more information, and maybe it will help you decide. Clang.jl is a general framework for wrapping C libraries. It's never necessary--any C library can be called directly using ccall. But ccalls can be kind of clunky, so Clang.jl provides an API which very closely mimics the C API provided in some header file: functions have very similar signatures, and structs become julia types. For a large library with lots of functions, or complicated structs with lots of members, this can make wrapping relatively easy. While the output of Clang.jl is usually reasonably nice, it's still a very low level of abstraction. Actually, very little abstraction, because it matches the C library, which is usually very low level, and still somewhat inconvenient in Julia (but more convenient than ccalls). To be useful, you'll often want to add a higher level API on top of that, which adds functions and/or types that encompass or simplify the lower level calls. It's also often the case that Clang.jl doesn't give you exactly what you need--e.g., it doesn't know how to wrap certain things, such as unions and C macros. In those cases, you'll have to edit the output by hand, or spend some time programmatically filtering/modifying the results. I do this a bit in the wrapper for VideoIO https://github.com/kmsquire/VideoIO.jl/blob/master/util/wrap_libav_split.jl (and I still have to edit some files by hand at the end). The main alternative is to wrap a subset of useful functions by hand (or simply use ccall directly, which amounts to pretty much the same thing). This is probably closer to what the Mongo folks had in mind when they directed you to the Lua driver. If you only need access to a few functions, or if your code is highly specialized in a way that Clang.jl has trouble with, wrapping by hand can be the way to go. For this, you're basically writing Julia functions which ccall out to external library functions (such as those in mongo), and returns the result (or some modification thereof that matches what would normally be done in Julia). Hopefully this was useful. Please feel free to post back here with questions, and I (and maybe others, if the question is right) will try to answer as we have time. Cheers! Kevin On Wed, Jul 29, 2015 at 3:00 PM, Kevin Liu kevinliu2...@gmail.com wrote: Thanks for the valuable advice and modesty, there is no code in the reps. I'm learning how to do this properly. From the Mongo side I have received a little bit of guidance but from the Julia side, very little. Changed it to CMongo.jl. Clang.jl will be of great help (I had bookmarked it, but didn't realize it was a wrapper). On Wed, Jul 29, 2015 at 3:47 AM, Kevin Squire kevin.squ...@gmail.com wrote: I would suggest CMongo.jl (when renaming this, be careful if you're on a case-insensitive filesystem). Of the fully capitalized names in Julia, most of them imply that they are acronyms. Suggestion: at this point, there isn't much code in the repo. Since not many people have (publicly) responded to your posts, IMHO, it would be good to try to get a basic working system in place, and post your progress when something basic is working--this will make it easier for people to contribute. One more thing you should look at is
Re: [julia-users] Re: MongoDB and Julia
I'm also sharing the stack design in the hopes of speeding the process. Comments are welcome. https://docs.google.com/spreadsheets/d/1rgqtCay8HhnVuYR4UCZi9IGa0bzjbc3VupPxRUT9DS0/edit?usp=sharing On Wednesday, July 29, 2015 at 8:01:11 PM UTC-3, Kevin Liu wrote: Haha this is my first major project, period! Thanks a lot for putting in the time and effort into guiding. I come from the finance world but became interested in Julia and MongoDB for what they can do with science. It's a hand into understanding so much. Focused on Mongo.jl and will digest your comments after that. I got ahead of myself from Mongo.jl after seeing there was so much more to be done to make the stack fully functioning. Thank you! Will keep you and the community posted. Cheers mate! On Wed, Jul 29, 2015 at 6:50 PM, Kevin Squire kevin.squ...@gmail.com wrote: Hi Kevin, I'm sorry to hear (and see) that you haven't gotten much help on the Julia mailing list. It's probably just that your request hasn't fallen on the ears of anyone who has interest in both Mongo and Julia and the time to help. I'm almost in that category, in that I have a small interest in Mongo (and a lot of interest in Julia), but only so much time. I will say that once you get something working, the mailing list will usually be a good resource, but it works best if you have specific questions or problems (e.g., I'm trying to do this with the following code, but it's not working--what am I doing wrong) vs. general requests (e.g., asking for help wrapping mongo with a mostly empty repo). I'm assuming this is your first major Julia project? If so, my first suggestion is to ignore everything written below (for now), and try to get the Mongo.jl library running on a modern Julia first, and make sure you understand everything that it's doing. Only after that would I try anything below. In truth, I don't know the best way forward, but I can give you a little more information, and maybe it will help you decide. Clang.jl is a general framework for wrapping C libraries. It's never necessary--any C library can be called directly using ccall. But ccalls can be kind of clunky, so Clang.jl provides an API which very closely mimics the C API provided in some header file: functions have very similar signatures, and structs become julia types. For a large library with lots of functions, or complicated structs with lots of members, this can make wrapping relatively easy. While the output of Clang.jl is usually reasonably nice, it's still a very low level of abstraction. Actually, very little abstraction, because it matches the C library, which is usually very low level, and still somewhat inconvenient in Julia (but more convenient than ccalls). To be useful, you'll often want to add a higher level API on top of that, which adds functions and/or types that encompass or simplify the lower level calls. It's also often the case that Clang.jl doesn't give you exactly what you need--e.g., it doesn't know how to wrap certain things, such as unions and C macros. In those cases, you'll have to edit the output by hand, or spend some time programmatically filtering/modifying the results. I do this a bit in the wrapper for VideoIO https://github.com/kmsquire/VideoIO.jl/blob/master/util/wrap_libav_split.jl (and I still have to edit some files by hand at the end). The main alternative is to wrap a subset of useful functions by hand (or simply use ccall directly, which amounts to pretty much the same thing). This is probably closer to what the Mongo folks had in mind when they directed you to the Lua driver. If you only need access to a few functions, or if your code is highly specialized in a way that Clang.jl has trouble with, wrapping by hand can be the way to go. For this, you're basically writing Julia functions which ccall out to external library functions (such as those in mongo), and returns the result (or some modification thereof that matches what would normally be done in Julia). Hopefully this was useful. Please feel free to post back here with questions, and I (and maybe others, if the question is right) will try to answer as we have time. Cheers! Kevin On Wed, Jul 29, 2015 at 3:00 PM, Kevin Liu kevinliu2...@gmail.com wrote: Thanks for the valuable advice and modesty, there is no code in the reps. I'm learning how to do this properly. From the Mongo side I have received a little bit of guidance but from the Julia side, very little. Changed it to CMongo.jl. Clang.jl will be of great help (I had bookmarked it, but didn't realize it was a wrapper). On Wed, Jul 29, 2015 at 3:47 AM, Kevin Squire kevin.squ...@gmail.com wrote: I would suggest CMongo.jl (when renaming this, be careful if you're on a case-insensitive filesystem). Of the fully capitalized names in Julia, most of them imply that they are acronyms.
Re: [julia-users] Re: MongoDB and Julia
Thanks! Will need it! On Wed, Jul 29, 2015 at 8:23 PM, Kevin Squire kevin.squ...@gmail.com wrote: Good luck! On Thu, Jul 30, 2015 at 1:01 AM, Kevin Liu kevinliu2...@gmail.com wrote: Haha this is my first major project, period! Thanks a lot for putting in the time and effort into guiding. I come from the finance world but became interested in Julia and MongoDB for what they can do with science. It's a hand into understanding so much. Focused on Mongo.jl and will digest your comments after that. I got ahead of myself from Mongo.jl after seeing there was so much more to be done to make the stack fully functioning. Thank you! Will keep you and the community posted. Cheers mate! On Wed, Jul 29, 2015 at 6:50 PM, Kevin Squire kevin.squ...@gmail.com wrote: Hi Kevin, I'm sorry to hear (and see) that you haven't gotten much help on the Julia mailing list. It's probably just that your request hasn't fallen on the ears of anyone who has interest in both Mongo and Julia and the time to help. I'm almost in that category, in that I have a small interest in Mongo (and a lot of interest in Julia), but only so much time. I will say that once you get something working, the mailing list will usually be a good resource, but it works best if you have specific questions or problems (e.g., I'm trying to do this with the following code, but it's not working--what am I doing wrong) vs. general requests (e.g., asking for help wrapping mongo with a mostly empty repo). I'm assuming this is your first major Julia project? If so, my first suggestion is to ignore everything written below (for now), and try to get the Mongo.jl library running on a modern Julia first, and make sure you understand everything that it's doing. Only after that would I try anything below. In truth, I don't know the best way forward, but I can give you a little more information, and maybe it will help you decide. Clang.jl is a general framework for wrapping C libraries. It's never necessary--any C library can be called directly using ccall. But ccalls can be kind of clunky, so Clang.jl provides an API which very closely mimics the C API provided in some header file: functions have very similar signatures, and structs become julia types. For a large library with lots of functions, or complicated structs with lots of members, this can make wrapping relatively easy. While the output of Clang.jl is usually reasonably nice, it's still a very low level of abstraction. Actually, very little abstraction, because it matches the C library, which is usually very low level, and still somewhat inconvenient in Julia (but more convenient than ccalls). To be useful, you'll often want to add a higher level API on top of that, which adds functions and/or types that encompass or simplify the lower level calls. It's also often the case that Clang.jl doesn't give you exactly what you need--e.g., it doesn't know how to wrap certain things, such as unions and C macros. In those cases, you'll have to edit the output by hand, or spend some time programmatically filtering/modifying the results. I do this a bit in the wrapper for VideoIO https://github.com/kmsquire/VideoIO.jl/blob/master/util/wrap_libav_split.jl (and I still have to edit some files by hand at the end). The main alternative is to wrap a subset of useful functions by hand (or simply use ccall directly, which amounts to pretty much the same thing). This is probably closer to what the Mongo folks had in mind when they directed you to the Lua driver. If you only need access to a few functions, or if your code is highly specialized in a way that Clang.jl has trouble with, wrapping by hand can be the way to go. For this, you're basically writing Julia functions which ccall out to external library functions (such as those in mongo), and returns the result (or some modification thereof that matches what would normally be done in Julia). Hopefully this was useful. Please feel free to post back here with questions, and I (and maybe others, if the question is right) will try to answer as we have time. Cheers! Kevin On Wed, Jul 29, 2015 at 3:00 PM, Kevin Liu kevinliu2...@gmail.com wrote: Thanks for the valuable advice and modesty, there is no code in the reps. I'm learning how to do this properly. From the Mongo side I have received a little bit of guidance but from the Julia side, very little. Changed it to CMongo.jl. Clang.jl will be of great help (I had bookmarked it, but didn't realize it was a wrapper). On Wed, Jul 29, 2015 at 3:47 AM, Kevin Squire kevin.squ...@gmail.com wrote: I would suggest CMongo.jl (when renaming this, be careful if you're on a case-insensitive filesystem). Of the fully capitalized names in Julia, most of them imply that they are acronyms. Suggestion: at this point, there isn't much code in the repo. Since not many people have (publicly) responded to your posts, IMHO, it would be
[julia-users] Re: parallel threads broken, replacing module
Reference: https://github.com/JuliaLang/julia/issues/12381 On Wednesday, July 29, 2015 at 5:35:14 PM UTC-7, Seth wrote: For what it's worth, I'm seeing the same thing: julia @everywhere using LightGraphs WARNING: replacing module LightGraphs WARNING: replacing module LightGraphs WARNING: replacing module LightGraphs WARNING: replacing module LightGraphs exception on 4: ... (Lots of error messages / backtraces removed) ... ERROR: ProcessExitedException() in wait at /usr/local/julia-latest/lib/julia/sys.dylib in sync_end at /usr/local/julia-latest/lib/julia/sys.dylib in anonymous at multi.jl:348 On Wednesday, July 29, 2015 at 4:54:30 PM UTC-7, lapeyre@gmail.com wrote: I can file a bug report. But, I'm not entirely sure what to write. Simpler case: julia -p 4 @everywhere using ZChop (simple modules) The following version is from a completely fresh build today. I don't get a crash immediately, but I do get WARNING: replacing module ZChop (4 times) Version 0.4.0-dev+6394 (2015-07-29 21:58 UTC) Commit a2a218b* (0 days old master) Five days ago, i did git pull and built from an older clone Version 0.4.0-dev+6202 (2015-07-24 01:47 UTC) Commit 53a7f2e* (5 days old master) This version prints the same warning and spews many errors including unknown function (ip: 0x7fe94d596ff7) ERROR (unhandled task failure): EOFError: read end of file The following version will load ZChop with no problem, but when loading Distributions, it still prints many errors and fails. It loads some other modules (i.e with @everywhere, as above), such as Roots with no apparent errors. Version 0.4.0-dev+3965 (2015-03-22 12:24 UTC) Commit e1f0310* (129 days old master) A fresh build today on another machine gives errors with @everywhere using Roots. It fails to load Distributions even with no parallel threads, but this appears to be unrelated. On the same (other) machine, the following version loads and runs parallel code using the modules mentioned above, and more. Version 0.4.0-dev+4096 (2015-03-31 08:05 UTC) Commit a3c0743* (120 days old master) I see no error with 0.3.5-pre+121 (2015-01-07 00:19 UTC) On Wednesday, July 29, 2015 at 11:24:55 PM UTC+2, lapeyre@gmail.com wrote: Parallel threads has stopped working for me. Any ideas ? Code using addprocs and @everywhere include has stopped working (is broken) on one machine. Code that used to work now causes a number of varying errors to be printed, crashes, runaway processes, etc. when it is loaded. Both a recent v0.4 and a 2 month old version that did run the code in the past cause the failure. The same code on a different machine continues to run normally. Maybe my user Julia environment has a problem ? Or maybe I upgraded a system library ? If I use just 2 processes, the code will sometimes both load and run correctly. A minimal example test.jl -- using Distributions -- julia -p 4 @everywhere include(test.jl) WARNING: replacing module Distributions ( 4 times ) signal (11): Segmentation fault unknown function (ip: 0x7f235c7ffd98) jl_module_import at /usr/local/julia-0.4a/bin/../lib/julia/libjulia.so (unknown line) etc. I am running unstable debian linux. Thanks, John
Re: [julia-users] John L. Gustafson's UNUMs
Just thinking out loud... I think it would be feasible to implement a decimal unum, in which everything is essentially the same except it's base 10. This may give a lot of benefits while still maintaining exact numbers in many situations (But decimal intervals when inexact). Since we're already talking about doing operations in software, it may net out to be better/faster? On Wednesday, July 29, 2015, Spencer Lyon spencerly...@gmail.com wrote: I would love to read a summary. I don't think I have time to read the whole book AND contribute, but I might be able to find time to read a summary and contribute. // Spencer On July 29, 2015 at 6:33:52 PM EDT, Job van der Zwan j.l.vanderz...@gmail.com javascript:_e(%7B%7D,'cvml','j.l.vanderz...@gmail.com'); wrote: BTW, Tom, I was already working on a summary of the book (on an IJulia notebook). I'm on mobile right now so don't have access to it, but I can share it later. I think something like that might be useful to attract more collaborators - we can't expect everyone to read it.
Re: [julia-users] John L. Gustafson's UNUMs
That would be great for my purposes. On Wednesday, July 29, 2015 at 7:57:59 PM UTC-4, Tom Breloff wrote: Just thinking out loud... I think it would be feasible to implement a decimal unum, in which everything is essentially the same except it's base 10. This may give a lot of benefits while still maintaining exact numbers in many situations (But decimal intervals when inexact). Since we're already talking about doing operations in software, it may net out to be better/faster? On Wednesday, July 29, 2015, Spencer Lyon spence...@gmail.com javascript: wrote: I would love to read a summary. I don't think I have time to read the whole book AND contribute, but I might be able to find time to read a summary and contribute. // Spencer On July 29, 2015 at 6:33:52 PM EDT, Job van der Zwan j.l.vanderz...@gmail.com wrote: BTW, Tom, I was already working on a summary of the book (on an IJulia notebook). I'm on mobile right now so don't have access to it, but I can share it later. I think something like that might be useful to attract more collaborators - we can't expect everyone to read it.
Re: [julia-users] [ANN] SparseVectors
This was discussed pretty extensively in the github issue: https://github.com/JuliaLang/julia/issues/11324 On Wednesday, July 29, 2015 at 8:32:13 PM UTC-6, Christian Peel wrote: Nice! I saw somewhere recently that the plan was to move more out of base and into packages. I'm curious as to why this needs to go into Julia base. cheers On Wed, Jul 29, 2015 at 2:42 AM, Dahua Lin lind...@gmail.com javascript: wrote: I searched over the forum and it seems that I did not announce this package when it is initially created. So here it is: https://github.com/JuliaSparse/SparseVectors.jl Simply put, it provides sparse vectors (not sparse matrices with one-column) and a series of methods operating thereon. Recently, I did a major upgrade of the package and many more functions are added. The package has been systematically tested and it is ready for general use. A majority of the functionalities will be migrated to Julia base, after Julia v0.4 is released. If your applications involve processing sparse vectors (e.g. text analysis), you may check it out and play with it. Cheers, Dahua -- chris...@ieee.org javascript:
[julia-users] Recommended way of implementing getters and setters
Hi, This question is about programming style. I would like to know what's the recommended way of writing getters and setters in Julia. I have a TimeSignal class written in Matlab that I'm trying to port over to Julia. In the Matlab implementation, I have some getter methods for computing the values of the dependent properties (e.g., the number of samples n_samples, the associated time vector time_vector, etc.). My question is: what's the recommended way of implementing these getter functions in Julia? Maybe implementing a getindex() function so that I can do, for example, time_signal[:n_samples]? Or is it better to explicitly write getter functions like get_n_samples(time_signal), get_time_vector(time_signal), etc? I read the thread about dot overloading at https://github.com/JuliaLang/julia/issues/1974 but this still seems to be an open issue at the moment. Thanks a lot, Adriano
Re: [julia-users] [ANN] SparseVectors
Nice! I saw somewhere recently that the plan was to move more out of base and into packages. I'm curious as to why this needs to go into Julia base. cheers On Wed, Jul 29, 2015 at 2:42 AM, Dahua Lin linda...@gmail.com wrote: I searched over the forum and it seems that I did not announce this package when it is initially created. So here it is: https://github.com/JuliaSparse/SparseVectors.jl Simply put, it provides sparse vectors (not sparse matrices with one-column) and a series of methods operating thereon. Recently, I did a major upgrade of the package and many more functions are added. The package has been systematically tested and it is ready for general use. A majority of the functionalities will be migrated to Julia base, after Julia v0.4 is released. If your applications involve processing sparse vectors (e.g. text analysis), you may check it out and play with it. Cheers, Dahua -- chris.p...@ieee.org
[julia-users] [ANN] SparseVectors
I searched over the forum and it seems that I did not announce this package when it is initially created. So here it is: https://github.com/JuliaSparse/SparseVectors.jl Simply put, it provides sparse vectors (not sparse matrices with one-column) and a series of methods operating thereon. Recently, I did a major upgrade of the package and many more functions are added. The package has been systematically tested and it is ready for general use. A majority of the functionalities will be migrated to Julia base, after Julia v0.4 is released. If your applications involve processing sparse vectors (e.g. text analysis), you may check it out and play with it. Cheers, Dahua
Re: [julia-users] Re: MongoDB and Julia
I would suggest CMongo.jl (when renaming this, be careful if you're on a case-insensitive filesystem). Of the fully capitalized names in Julia, most of them imply that they are acronyms. Suggestion: at this point, there isn't much code in the repo. Since not many people have (publicly) responded to your posts, IMHO, it would be good to try to get a basic working system in place, and post your progress when something basic is working--this will make it easier for people to contribute. One more thing you should look at is Clang.jl, which makes wrapping C libraries easier. Cheers! Kevin On Tue, Jul 28, 2015 at 5:21 PM, Kevin Liu kevinliu2...@gmail.com wrote: I'll name it CMONGO.jl On Tue, Jul 28, 2015 at 9:13 PM, Kevin Liu kevinliu2...@gmail.com wrote: Any suggestions for the name? I just want to remember this will be a wrapper around C Mongo. On Tue, Jul 28, 2015 at 9:10 PM, Kevin Liu kevinliu2...@gmail.com wrote: Hey Kevin, That's great. Thanks for the advice. On it right now. Cheers! On Tue, Jul 28, 2015 at 7:50 PM, Kevin Squire kevin.squ...@gmail.com wrote: Hi Kevin, If you plan to make this a Julia package (and I encourage you to do so), it would be good to look at the Julia package naming conventions http://julia.readthedocs.org/en/latest/manual/packages/#guidelines-for-naming-a-package. You might consider choosing a different name, generating a package skeleton, and moving the files in this repo there. Alternatively, renaming that repo shouldn't be hard. (This isn't mentioned explicitly there, but dashes also won't work for Julia package names.) Cheers! Kevin On Mon, Jul 27, 2015 at 8:28 PM, Kevin Liu kevinliu2...@gmail.com wrote: Hi Julia Users, feel free to contribute to the Julia wrapper of the C Mongo Driver, maintained by Mongo https://github.com/tenthdimension/Julia-C-Mongo This Julia wrapper is based on the Lua wrapper of the C Mongo Driver. Jesse Davis from MongoDB recommended I used it as a reference. On Thursday, July 23, 2015 at 8:26:14 PM UTC-3, Kevin Liu wrote: Thanks On Thursday, July 23, 2015 at 8:24:12 PM UTC-3, tim@multiscalehn.com wrote: https://github.com/pzion/LibBSON.jl/pull/4 https://github.com/pzion/Mongo.jl/pull/6 On Thursday, July 23, 2015 at 3:28:17 PM UTC-7, Kevin Liu wrote: I'm sorry Tim, check this out https://github.com/10gen-labs/mongorover/issues/16 Could you share how you made it work properly? On Wednesday, July 22, 2015 at 5:20:42 PM UTC-3, Kevin Liu wrote: Hi Tim, did it pass specs 1, 2, and part of 3? 4 hasn't been started yet. These new MongoDB drivers conform to published specifications. 1. Server Selection - Deciding which server to send database operations to in a MongoDB deployment. 2. Server Discovery and Monitoring - All the logic required to make a MongoDB application highly available. 3. CRUD API - The API for how we Create, Read, Update and Delete data from MongoDB. 4. Authentication - The rules for how to authenticate to MongoDB servers. https://www.mongodb.com/blog/post/announcing-next-generation-drivers-mongodb On Wednesday, July 22, 2015 at 4:30:00 PM UTC-3, tim@multiscalehn.com wrote: I have just made pull requests to pzion/LibBSON.jl and pzion/Mongo.jl to fix the driver in v0.4. Works fine for me, after adding @compats On Sunday, July 12, 2015 at 12:17:44 AM UTC-7, Kevin Liu wrote: Hi, I have Julia 0.3, Mongodb-osx-x86_64-3.0.4, and Mongo-c-driver-1.1.9 installed, but can't get Julia to access the Mongo Client through this 'untestable' package https://github.com/pzion/Mongo.jl, according to http://pkg.julialang.org/. I have tried Lytol/Mongo.jl and the command require(Mongo.jl) can't open file Mongo.jl, or the auto-generated deps.jl. Is anyone having similar problems trying to make Julia work with Mongo? Thank you Kevin
[julia-users] How do you use using inside a function now that require is deprecated?
The below code used to work to lazy load a module, but now is deprecated, with the warning saying to use `using` or `import`. But `using` doesn't work inside a function. What should be done? ```julia function foo2() require(StatsBase) end ```
Re: [julia-users] Method dispatch shootout
Tim, this is really helpful. Thanks for taking your time!
Em quarta-feira, 29 de julho de 2015 07:54:48 UTC-3, Tim Holy escreveu:
So there are 3 that are identical: Const Val Type, Type, and Const
Functors. To veterans, this is not surprising.
What might be more surprising is that Val Type isn't fast. This is
because
the type isn't inferable: the whole point of Val is to wrap things that
aren't
types and turn them into a type, so the compiler doesn't know the type of
Val{EContinuous}. You can use function barrier techniques to work around
this kind of instability
(
http://docs.julialang.org/en/latest/manual/performance-tips/#separate-kernel-functions),
or (as you did) define consts.
Likewise, the Functors approach would work out mostly fine if instead of
timing at global scope you wrote your timings like this:
function time_functors(n, fc, fs, fe)
s = 0.0
for i = 1:n
s += fc(0.15, 2.0)
s += fs(0.15, 2.0)
s += fe(0.15, 2.0)
end
s
end
time_functors(1, fc, fs, fe)
@time time_functors(1000, fc, fs, fe)
Accumulating into s prevents the compiler from noticing that you don't
actually use these values and elide the entire loop. (This doesn't always
happen without the s, but it's good defensive practice when timing
things.)
(There's actually a small penalty for the non-const functors that I don't
immediately understand, but I lack time to follow it up right now.)
--Tim
On Tuesday, July 28, 2015 07:13:25 PM felip...@gmail.com javascript:
wrote:
Maybe this is worth sharing. When considering using enums instead of
types,
I came up with 7 different ways to dispatch to methods, and these are
the
results of the performance tests.
The code is at: https://gist.github.com/felipenoris/660537161af33db34dcb
*Results:*
julia include(enum-vs-dispatch.jl)
enums with switch code
323.515 milliseconds
Dispatch via Val Type
1.068 seconds (3 k allocations: 458 MB, 1.96% gc time)
Dispatch via Const Val Type
145.355 milliseconds
*Dispatch via Type*
* 148.047 milliseconds*
Vector of functions dispatch
833.969 milliseconds (3 k allocations: 458 MB, 2.14% gc time)
Functors
1.011 seconds (3 k allocations: 458 MB, 1.62% gc time)
Const Functors
147.741 milliseconds
Re: [julia-users] Method dispatch shootout
So there are 3 that are identical: Const Val Type, Type, and Const
Functors. To veterans, this is not surprising.
What might be more surprising is that Val Type isn't fast. This is because
the type isn't inferable: the whole point of Val is to wrap things that aren't
types and turn them into a type, so the compiler doesn't know the type of
Val{EContinuous}. You can use function barrier techniques to work around
this kind of instability
(http://docs.julialang.org/en/latest/manual/performance-tips/#separate-kernel-functions),
or (as you did) define consts.
Likewise, the Functors approach would work out mostly fine if instead of
timing at global scope you wrote your timings like this:
function time_functors(n, fc, fs, fe)
s = 0.0
for i = 1:n
s += fc(0.15, 2.0)
s += fs(0.15, 2.0)
s += fe(0.15, 2.0)
end
s
end
time_functors(1, fc, fs, fe)
@time time_functors(1000, fc, fs, fe)
Accumulating into s prevents the compiler from noticing that you don't
actually use these values and elide the entire loop. (This doesn't always
happen without the s, but it's good defensive practice when timing things.)
(There's actually a small penalty for the non-const functors that I don't
immediately understand, but I lack time to follow it up right now.)
--Tim
On Tuesday, July 28, 2015 07:13:25 PM felipeno...@gmail.com wrote:
Maybe this is worth sharing. When considering using enums instead of types,
I came up with 7 different ways to dispatch to methods, and these are the
results of the performance tests.
The code is at: https://gist.github.com/felipenoris/660537161af33db34dcb
*Results:*
julia include(enum-vs-dispatch.jl)
enums with switch code
323.515 milliseconds
Dispatch via Val Type
1.068 seconds (3 k allocations: 458 MB, 1.96% gc time)
Dispatch via Const Val Type
145.355 milliseconds
*Dispatch via Type*
* 148.047 milliseconds*
Vector of functions dispatch
833.969 milliseconds (3 k allocations: 458 MB, 2.14% gc time)
Functors
1.011 seconds (3 k allocations: 458 MB, 1.62% gc time)
Const Functors
147.741 milliseconds
Re: [julia-users] How do you use using inside a function now that require is deprecated?
On Wed, Jul 29, 2015 at 7:00 AM, Sheehan Olver dlfivefi...@gmail.com wrote: The below code used to work to lazy load a module, but now is deprecated, with the warning saying to use `using` or `import`. But `using` doesn't work inside a function. What should be done? ```julia function foo2() require(StatsBase) @eval using StatsBase end ```
Re: [julia-users] How do you use using inside a function now that require is deprecated?
Hi I am having the same problem, but while loading a module for parallel processing. Before this deprecation message I was using require(file.jl) since both using and include do not work (the other processes do not know about such file). I wonder why this change is happening, but I cannot find any documentation about it. On Wednesday, July 29, 2015 at 8:27:47 AM UTC-3, Yichao Yu wrote: On Wed, Jul 29, 2015 at 7:00 AM, Sheehan Olver dlfiv...@gmail.com javascript: wrote: The below code used to work to lazy load a module, but now is deprecated, with the warning saying to use `using` or `import`. But `using` doesn't work inside a function. What should be done? ```julia function foo2() require(StatsBase) @eval using StatsBase end ```
Re: [julia-users] Re: MongoDB and Julia
Thanks for the valuable advice and modesty, there is no code in the reps. I'm learning how to do this properly. From the Mongo side I have received a little bit of guidance but from the Julia side, very little. Changed it to CMongo.jl. Clang.jl will be of great help (I had bookmarked it, but didn't realize it was a wrapper). On Wed, Jul 29, 2015 at 3:47 AM, Kevin Squire kevin.squ...@gmail.com wrote: I would suggest CMongo.jl (when renaming this, be careful if you're on a case-insensitive filesystem). Of the fully capitalized names in Julia, most of them imply that they are acronyms. Suggestion: at this point, there isn't much code in the repo. Since not many people have (publicly) responded to your posts, IMHO, it would be good to try to get a basic working system in place, and post your progress when something basic is working--this will make it easier for people to contribute. One more thing you should look at is Clang.jl, which makes wrapping C libraries easier. Cheers! Kevin On Tue, Jul 28, 2015 at 5:21 PM, Kevin Liu kevinliu2...@gmail.com wrote: I'll name it CMONGO.jl On Tue, Jul 28, 2015 at 9:13 PM, Kevin Liu kevinliu2...@gmail.com wrote: Any suggestions for the name? I just want to remember this will be a wrapper around C Mongo. On Tue, Jul 28, 2015 at 9:10 PM, Kevin Liu kevinliu2...@gmail.com wrote: Hey Kevin, That's great. Thanks for the advice. On it right now. Cheers! On Tue, Jul 28, 2015 at 7:50 PM, Kevin Squire kevin.squ...@gmail.com wrote: Hi Kevin, If you plan to make this a Julia package (and I encourage you to do so), it would be good to look at the Julia package naming conventions http://julia.readthedocs.org/en/latest/manual/packages/#guidelines-for-naming-a-package. You might consider choosing a different name, generating a package skeleton, and moving the files in this repo there. Alternatively, renaming that repo shouldn't be hard. (This isn't mentioned explicitly there, but dashes also won't work for Julia package names.) Cheers! Kevin On Mon, Jul 27, 2015 at 8:28 PM, Kevin Liu kevinliu2...@gmail.com wrote: Hi Julia Users, feel free to contribute to the Julia wrapper of the C Mongo Driver, maintained by Mongo https://github.com/tenthdimension/Julia-C-Mongo This Julia wrapper is based on the Lua wrapper of the C Mongo Driver. Jesse Davis from MongoDB recommended I used it as a reference. On Thursday, July 23, 2015 at 8:26:14 PM UTC-3, Kevin Liu wrote: Thanks On Thursday, July 23, 2015 at 8:24:12 PM UTC-3, tim@multiscalehn.com wrote: https://github.com/pzion/LibBSON.jl/pull/4 https://github.com/pzion/Mongo.jl/pull/6 On Thursday, July 23, 2015 at 3:28:17 PM UTC-7, Kevin Liu wrote: I'm sorry Tim, check this out https://github.com/10gen-labs/mongorover/issues/16 Could you share how you made it work properly? On Wednesday, July 22, 2015 at 5:20:42 PM UTC-3, Kevin Liu wrote: Hi Tim, did it pass specs 1, 2, and part of 3? 4 hasn't been started yet. These new MongoDB drivers conform to published specifications. 1. Server Selection - Deciding which server to send database operations to in a MongoDB deployment. 2. Server Discovery and Monitoring - All the logic required to make a MongoDB application highly available. 3. CRUD API - The API for how we Create, Read, Update and Delete data from MongoDB. 4. Authentication - The rules for how to authenticate to MongoDB servers. https://www.mongodb.com/blog/post/announcing-next-generation-drivers-mongodb On Wednesday, July 22, 2015 at 4:30:00 PM UTC-3, tim@multiscalehn.com wrote: I have just made pull requests to pzion/LibBSON.jl and pzion/Mongo.jl to fix the driver in v0.4. Works fine for me, after adding @compats On Sunday, July 12, 2015 at 12:17:44 AM UTC-7, Kevin Liu wrote: Hi, I have Julia 0.3, Mongodb-osx-x86_64-3.0.4, and Mongo-c-driver-1.1.9 installed, but can't get Julia to access the Mongo Client through this 'untestable' package https://github.com/pzion/Mongo.jl, according to http://pkg.julialang.org/. I have tried Lytol/Mongo.jl and the command require(Mongo.jl) can't open file Mongo.jl, or the auto-generated deps.jl. Is anyone having similar problems trying to make Julia work with Mongo? Thank you Kevin
[julia-users] Re: ANN: Jeff Bezanson meetup talk in NYC next Tue
Off-topic: the JuliaCon website still suggests registering for the event. On Wednesday, 29 July 2015 21:10:54 UTC+2, Stefan Karpinski wrote: http://www.meetup.com/julia-nyc/events/224261149/ This will have a fair amount of overlap with his keynote at JuliaCon, so if you saw that you may want to skip this, but if you didn't, it's very interesting and has evolved somewhat since June. Also, there's a Julia meetup group in New York. If you live in the area, join the group! Stefan
Re: [julia-users] John L. Gustafson's UNUMs
I would love to read a summary. I don't think I have time to read the whole book AND contribute, but I might be able to find time to read a summary and contribute. // Spencer BTW, Tom, I was already working on a summary of the book (on an IJulia notebook). I'm on mobile right now so don't have access to it, but I can share it later. I think something like that might be useful to attract more collaborators - we can't expect everyone to read it.
Re: [julia-users] Re: MongoDB and Julia
Good luck! On Thu, Jul 30, 2015 at 1:01 AM, Kevin Liu kevinliu2...@gmail.com wrote: Haha this is my first major project, period! Thanks a lot for putting in the time and effort into guiding. I come from the finance world but became interested in Julia and MongoDB for what they can do with science. It's a hand into understanding so much. Focused on Mongo.jl and will digest your comments after that. I got ahead of myself from Mongo.jl after seeing there was so much more to be done to make the stack fully functioning. Thank you! Will keep you and the community posted. Cheers mate! On Wed, Jul 29, 2015 at 6:50 PM, Kevin Squire kevin.squ...@gmail.com wrote: Hi Kevin, I'm sorry to hear (and see) that you haven't gotten much help on the Julia mailing list. It's probably just that your request hasn't fallen on the ears of anyone who has interest in both Mongo and Julia and the time to help. I'm almost in that category, in that I have a small interest in Mongo (and a lot of interest in Julia), but only so much time. I will say that once you get something working, the mailing list will usually be a good resource, but it works best if you have specific questions or problems (e.g., I'm trying to do this with the following code, but it's not working--what am I doing wrong) vs. general requests (e.g., asking for help wrapping mongo with a mostly empty repo). I'm assuming this is your first major Julia project? If so, my first suggestion is to ignore everything written below (for now), and try to get the Mongo.jl library running on a modern Julia first, and make sure you understand everything that it's doing. Only after that would I try anything below. In truth, I don't know the best way forward, but I can give you a little more information, and maybe it will help you decide. Clang.jl is a general framework for wrapping C libraries. It's never necessary--any C library can be called directly using ccall. But ccalls can be kind of clunky, so Clang.jl provides an API which very closely mimics the C API provided in some header file: functions have very similar signatures, and structs become julia types. For a large library with lots of functions, or complicated structs with lots of members, this can make wrapping relatively easy. While the output of Clang.jl is usually reasonably nice, it's still a very low level of abstraction. Actually, very little abstraction, because it matches the C library, which is usually very low level, and still somewhat inconvenient in Julia (but more convenient than ccalls). To be useful, you'll often want to add a higher level API on top of that, which adds functions and/or types that encompass or simplify the lower level calls. It's also often the case that Clang.jl doesn't give you exactly what you need--e.g., it doesn't know how to wrap certain things, such as unions and C macros. In those cases, you'll have to edit the output by hand, or spend some time programmatically filtering/modifying the results. I do this a bit in the wrapper for VideoIO https://github.com/kmsquire/VideoIO.jl/blob/master/util/wrap_libav_split.jl (and I still have to edit some files by hand at the end). The main alternative is to wrap a subset of useful functions by hand (or simply use ccall directly, which amounts to pretty much the same thing). This is probably closer to what the Mongo folks had in mind when they directed you to the Lua driver. If you only need access to a few functions, or if your code is highly specialized in a way that Clang.jl has trouble with, wrapping by hand can be the way to go. For this, you're basically writing Julia functions which ccall out to external library functions (such as those in mongo), and returns the result (or some modification thereof that matches what would normally be done in Julia). Hopefully this was useful. Please feel free to post back here with questions, and I (and maybe others, if the question is right) will try to answer as we have time. Cheers! Kevin On Wed, Jul 29, 2015 at 3:00 PM, Kevin Liu kevinliu2...@gmail.com wrote: Thanks for the valuable advice and modesty, there is no code in the reps. I'm learning how to do this properly. From the Mongo side I have received a little bit of guidance but from the Julia side, very little. Changed it to CMongo.jl. Clang.jl will be of great help (I had bookmarked it, but didn't realize it was a wrapper). On Wed, Jul 29, 2015 at 3:47 AM, Kevin Squire kevin.squ...@gmail.com wrote: I would suggest CMongo.jl (when renaming this, be careful if you're on a case-insensitive filesystem). Of the fully capitalized names in Julia, most of them imply that they are acronyms. Suggestion: at this point, there isn't much code in the repo. Since not many people have (publicly) responded to your posts, IMHO, it would be good to try to get a basic working system in place, and post your progress when something basic is
[julia-users] Re: parallel threads broken, replacing module
For what it's worth, I'm seeing the same thing: julia @everywhere using LightGraphs WARNING: replacing module LightGraphs WARNING: replacing module LightGraphs WARNING: replacing module LightGraphs WARNING: replacing module LightGraphs exception on 4: ... (Lots of error messages / backtraces removed) ... ERROR: ProcessExitedException() in wait at /usr/local/julia-latest/lib/julia/sys.dylib in sync_end at /usr/local/julia-latest/lib/julia/sys.dylib in anonymous at multi.jl:348 On Wednesday, July 29, 2015 at 4:54:30 PM UTC-7, lapeyre@gmail.com wrote: I can file a bug report. But, I'm not entirely sure what to write. Simpler case: julia -p 4 @everywhere using ZChop (simple modules) The following version is from a completely fresh build today. I don't get a crash immediately, but I do get WARNING: replacing module ZChop (4 times) Version 0.4.0-dev+6394 (2015-07-29 21:58 UTC) Commit a2a218b* (0 days old master) Five days ago, i did git pull and built from an older clone Version 0.4.0-dev+6202 (2015-07-24 01:47 UTC) Commit 53a7f2e* (5 days old master) This version prints the same warning and spews many errors including unknown function (ip: 0x7fe94d596ff7) ERROR (unhandled task failure): EOFError: read end of file The following version will load ZChop with no problem, but when loading Distributions, it still prints many errors and fails. It loads some other modules (i.e with @everywhere, as above), such as Roots with no apparent errors. Version 0.4.0-dev+3965 (2015-03-22 12:24 UTC) Commit e1f0310* (129 days old master) A fresh build today on another machine gives errors with @everywhere using Roots. It fails to load Distributions even with no parallel threads, but this appears to be unrelated. On the same (other) machine, the following version loads and runs parallel code using the modules mentioned above, and more. Version 0.4.0-dev+4096 (2015-03-31 08:05 UTC) Commit a3c0743* (120 days old master) I see no error with 0.3.5-pre+121 (2015-01-07 00:19 UTC) On Wednesday, July 29, 2015 at 11:24:55 PM UTC+2, lapeyre@gmail.com wrote: Parallel threads has stopped working for me. Any ideas ? Code using addprocs and @everywhere include has stopped working (is broken) on one machine. Code that used to work now causes a number of varying errors to be printed, crashes, runaway processes, etc. when it is loaded. Both a recent v0.4 and a 2 month old version that did run the code in the past cause the failure. The same code on a different machine continues to run normally. Maybe my user Julia environment has a problem ? Or maybe I upgraded a system library ? If I use just 2 processes, the code will sometimes both load and run correctly. A minimal example test.jl -- using Distributions -- julia -p 4 @everywhere include(test.jl) WARNING: replacing module Distributions ( 4 times ) signal (11): Segmentation fault unknown function (ip: 0x7f235c7ffd98) jl_module_import at /usr/local/julia-0.4a/bin/../lib/julia/libjulia.so (unknown line) etc. I am running unstable debian linux. Thanks, John
Re: [julia-users] John L. Gustafson's UNUMs
On Wednesday, July 29, 2015 at 10:30:41 AM UTC-4, Tom Breloff wrote: Correct me if I'm wrong, but (fixed-size) decimal floating-point has most of the same issues as floating point in terms of accumulation of errors, right? For certain cases, such as adding 2 prices together, I agree that decimal floating point would work ($1.01 + $2.02 == $3.03), but for those cases it's easier to represent the values as integers: (101 + 202 == 303 ; prec=2), which is basically what I do now. Programming wise, it's a lot easier to deal with decimal floating point instead of decimal fixed point, you don't need to worry about making sure you've scaled things correctly. I implemented a full decimal floating point package (one 32-bit version in IBM/370 assembly, another 16-bit 8086/8, and also pure pre-ANSI C (which worked at 16 or 32, then later 64 bit chunks)), where all the numbers were either: native sized signed integers (16-bit, 32-bit, and 0later 64-bit), or scaled decimal (64-bit value, scaled by 1 byte signed value (10**x)). This was actually faster than binary floating point for most things, because at the time, many of the machines did not have floating point hardware (think PDP-11, and PCs without the 8087 coprocessor), and because of the use case, frequently you were doing things like adding/subtracting things that had the same scale. (like your $1.01 + $2.02 case). You don't have to rewrite your code, if the currency uses 1000ths instead of 100ths, for example, or doesn't use fractions at all, all platforms, no matter the native machine word size, all got exactly the same results, with up to ~19 digits precision, *and* you avoided all binary - decimal conversion issues. (Note: at the time, using binary floating point hardware, on the machines where it was available, like the VAX or IBM or PCs with 8087 chip, would have given different results on each platform, which was not acceptable. Even with the IEEE standard, I think you still can get varying results on different platforms :-( ) In terms of storage and complexity, I would expect that decimal floating point numbers are bloated as compared to floats. You're giving up speed and memory in order to guarantee an exact representation falls in base 10... I could understand how this is occasionally useful, but I can't imagine you'd want that in the general case. Why bloated? There's not really that much difference, a few bits I think, if you are comparing IEEE 64-bit binary to IEEE 64-bit decimal float formats. As far as storage, they actually took less space in the decimal format, because it was also the format for integers (talking about my storage format here) In terms of hardware support... obviously it doesn't exist today, but it could in the future: http://www.theplatform.net/2015/03/12/the-little-chip-that-could-disrupt-exascale-computing/ Either way, I would think there's enough potential to the idea to at least prototype and test, and maybe it will prove to be more useful than you expect. I really would like to see this! On Wed, Jul 29, 2015 at 10:10 AM, Job van der Zwan j.l.van...@gmail.com javascript: wrote: On Wednesday, 29 July 2015 16:50:21 UTC+3, Steven G. Johnson wrote: Regarding, unums, without hardware support, at first glance they don't sound practical compared to the present alternatives (hardware or software fixed-precision float types, or arbitrary precision if you need it). And the ubox method for error analysis, even if it overcomes the problems of interval arithmetic as claimed, sounds too expensive to use on anything except for the smallest-scale problems because of the large number of boxes that you seem to need for each value whose error is being tracked. Well, I don't know enough about traditional methods to say if they're really as limited as Gustafson claims in his book, or if he's just cherry-picking. Same about the cost of using uboxe However, ubound arithmetic tells you that 1 / (0, 1] = [1, inf), and that [1, inf) / inf = 0. The ubounds describing those interval results are effectively just a pair of floating point numbers, plus a ubit to signal whether an endpoint is open or not. That's a very simple thing to implement. Not sure if there's any arbitrary precision method that deal with this so elegantly - you probably know better than I do.
Re: [julia-users] John L. Gustafson's UNUMs
Also to add to Scotts list... there are precisely 2 representations of NaN,
and we could potentially use them as a replacement for Nullable. i.e. the
first NaN could represent 0/0 and similar, while the second NaN could
represent a missing value. So instead of a Array{Nullable{Float64},N},
you would just have Array{Unum{4,7},N}. isnan(u) and isnull(u) could be
mutually exclusive.
I foresee the potential of specialized arrays as well, which could strip
out the size fields of the unum and put that metadata in the array type,
keeping the format fixed within the array. This allows constant indexing
and other niceties, but still allowing for arbitrary-precision intermediate
calcs, with exactness tracking.
On Wed, Jul 29, 2015 at 10:38 AM, Tom Breloff t...@breloff.com wrote:
Scott: Is your number format a public (open source) specification? How
does it differ from decimal floating point?
On Wed, Jul 29, 2015 at 10:30 AM, Tom Breloff t...@breloff.com wrote:
Correct me if I'm wrong, but (fixed-size) decimal floating-point has most
of the same issues as floating point in terms of accumulation of errors,
right? For certain cases, such as adding 2 prices together, I agree that
decimal floating point would work ($1.01 + $2.02 == $3.03), but for those
cases it's easier to represent the values as integers: (101 + 202 == 303 ;
prec=2), which is basically what I do now.
In terms of storage and complexity, I would expect that decimal floating
point numbers are bloated as compared to floats. You're giving up speed
and memory in order to guarantee an exact representation falls in base
10... I could understand how this is occasionally useful, but I can't
imagine you'd want that in the general case.
In terms of hardware support... obviously it doesn't exist today, but it
could in the future:
http://www.theplatform.net/2015/03/12/the-little-chip-that-could-disrupt-exascale-computing/
Either way, I would think there's enough potential to the idea to at
least prototype and test, and maybe it will prove to be more useful than
you expect.
On Wed, Jul 29, 2015 at 10:10 AM, Job van der Zwan
j.l.vanderz...@gmail.com wrote:
On Wednesday, 29 July 2015 16:50:21 UTC+3, Steven G. Johnson wrote:
Regarding, unums, without hardware support, at first glance they don't
sound practical compared to the present alternatives (hardware or software
fixed-precision float types, or arbitrary precision if you need it). And
the ubox method for error analysis, even if it overcomes the problems of
interval arithmetic as claimed, sounds too expensive to use on anything
except for the smallest-scale problems because of the large number of boxes
that you seem to need for each value whose error is being tracked.
Well, I don't know enough about traditional methods to say if they're
really as limited as Gustafson claims in his book, or if he's just
cherry-picking. Same about the cost of using uboxe
However, ubound arithmetic tells you that 1 / (0, 1] = [1, inf), and
that [1, inf) / inf = 0. The ubounds describing those interval results are
effectively just a pair of floating point numbers, plus a ubit to signal
whether an endpoint is open or not. That's a very simple thing to
implement. Not sure if there's any arbitrary precision method that deal
with this so elegantly - you probably know better than I do.
Re: [julia-users] John L. Gustafson's UNUMs
On Wednesday, July 29, 2015 at 10:38:56 AM UTC-4, Tom Breloff wrote: Scott: Is your number format a public (open source) specification? How does it differ from decimal floating point? It was just a packed storage format (think serialize/deserialize in Julia), that I invented to be able to efficiently store Mumps values, some 20+ years ago. The format was documented by sales engineers, and customers certainly figured it out easily enough, even though I wanted it kept opaque (so that I could add things later). It was trivial, a length byte (which included the length byte itself, so 0 was available as a special marker to indicate a longer length was stored (unsigned, LE) in the next two bytes (and later on, that was changed so that if the 2-byte length was 0, it was followed by a 4 byte unsigned length), followed optionally by a type byte, and optionally further data. (a length byte of 1 indicated an undefined or null value). Storing a 0 only took 2 bytes, a length of 2, and the marker for non-negative integer. -1 also took only 2 bytes, length of 2, marker for negative integer. The *format* could handle any arbitrary length integers, although the code only supported 0-8 bytes after the type, so 64-bit max, and values = 2^63, or -2^64, would get an error if read in. Other type bytes indicated binary or 8-bit text string, or UTF16LE Unicode values, or a 1 byte power of 10 scale followed by 0-8 bytes, and later on, IEEE doubles. No big deal, many people have come up with similar schemes, but I optimized it for space, which made it very useful for getting good performance, packing as much data as possible into the B+ tree nodes.
Re: [julia-users] John L. Gustafson's UNUMs
For my stuff, I really want to distinguish between a Null value (i.e.
missing), and any NaN value that could be produced from floating point
calculations.
What would you do about NaN payloads? I want to be able to store them, as
is, just in case somebody has encoded anything important in them (NaN
payloads from the
IEEE binary and floating point standards, that is, the UNUM format doesn't
seem to have NaN payloads, AFAICT).
For efficient handling of Array{Nullable{Float64},N}, I think you are
right, Array{Unum{4,7},N} would be very good.
On Wednesday, July 29, 2015 at 10:55:02 AM UTC-4, Tom Breloff wrote:
Also to add to Scotts list... there are precisely 2 representations of
NaN, and we could potentially use them as a replacement for Nullable. i.e.
the first NaN could represent 0/0 and similar, while the second NaN could
represent a missing value. So instead of a Array{Nullable{Float64},N},
you would just have Array{Unum{4,7},N}. isnan(u) and isnull(u) could be
mutually exclusive.
I foresee the potential of specialized arrays as well, which could strip
out the size fields of the unum and put that metadata in the array type,
keeping the format fixed within the array. This allows constant indexing
and other niceties, but still allowing for arbitrary-precision intermediate
calcs, with exactness tracking.
On Wed, Jul 29, 2015 at 10:38 AM, Tom Breloff t...@breloff.com
javascript: wrote:
Scott: Is your number format a public (open source) specification? How
does it differ from decimal floating point?
On Wed, Jul 29, 2015 at 10:30 AM, Tom Breloff t...@breloff.com
javascript: wrote:
Correct me if I'm wrong, but (fixed-size) decimal floating-point has
most of the same issues as floating point in terms of accumulation of
errors, right? For certain cases, such as adding 2 prices together, I
agree that decimal floating point would work ($1.01 + $2.02 == $3.03), but
for those cases it's easier to represent the values as integers: (101 + 202
== 303 ; prec=2), which is basically what I do now.
In terms of storage and complexity, I would expect that decimal floating
point numbers are bloated as compared to floats. You're giving up speed
and memory in order to guarantee an exact representation falls in base
10... I could understand how this is occasionally useful, but I can't
imagine you'd want that in the general case.
In terms of hardware support... obviously it doesn't exist today, but it
could in the future:
http://www.theplatform.net/2015/03/12/the-little-chip-that-could-disrupt-exascale-computing/
Either way, I would think there's enough potential to the idea to at
least prototype and test, and maybe it will prove to be more useful than
you expect.
On Wed, Jul 29, 2015 at 10:10 AM, Job van der Zwan j.l.van...@gmail.com
javascript: wrote:
On Wednesday, 29 July 2015 16:50:21 UTC+3, Steven G. Johnson wrote:
Regarding, unums, without hardware support, at first glance they don't
sound practical compared to the present alternatives (hardware or
software
fixed-precision float types, or arbitrary precision if you need it). And
the ubox method for error analysis, even if it overcomes the problems
of
interval arithmetic as claimed, sounds too expensive to use on anything
except for the smallest-scale problems because of the large number of
boxes
that you seem to need for each value whose error is being tracked.
Well, I don't know enough about traditional methods to say if they're
really as limited as Gustafson claims in his book, or if he's just
cherry-picking. Same about the cost of using uboxe
However, ubound arithmetic tells you that 1 / (0, 1] = [1, inf), and
that [1, inf) / inf = 0. The ubounds describing those interval results are
effectively just a pair of floating point numbers, plus a ubit to signal
whether an endpoint is open or not. That's a very simple thing to
implement. Not sure if there's any arbitrary precision method that deal
with this so elegantly - you probably know better than I do.
[julia-users] Re: setindex! for matrix of vectors
Is your custom object `: AbstractArray`? Are you on 0.3 or a recent
0.4-dev? Are you implementing custom `setindex!` methods? I don't think
that your workaround should be necessary, but you haven't posted enough
context for me to fully appreciate the issue.
As far as support for Array{Array}s, it's definitely a gradual work in
progress. Transposition, for example, does work in some cases. The
difficulty comes from the fact that it assumes that transposing the
elements won't change their type, which isn't true for Array{Vector}:
julia A = Array[[1,2], [3,4], [5,6], [7,8]]
4-element Array{Array{T,N},1}:
[1,2]
[3,4]
[5,6]
[7,8]
julia A'
1x4 Array{Array{T,N},2}:
1x2 Array{Int64,2}:
1 2 … 1x2 Array{Int64,2}:
7 8
On Wednesday, July 29, 2015 at 9:56:45 AM UTC-4, Linus Härenstam-Nielsen
wrote:
I haven't found a solution that works perfectly yet but I circumvented the
problem by making a check for colons:
rows = inds[1] == Colon() ? (1:sys.ny) : inds[1]
cols = inds[2] == Colon() ? (1:sys.nu) : inds[2]
Not perfect since it limits you to always have to use exactly 2
incidences, but it works in my case.
Maybe the handling of Array{Vector} should be looked at, seems to me that
there are quiet a few cases where you run into problems with that datatype.
For example, I don't think there is a built in way to transpose an
Array{Vector, N1}.
Re: [julia-users] John L. Gustafson's UNUMs
I still having figured out how this would tie in with decimal floating point, but I really do like a few key things: 1. single type for all sizes of integers and floating point values (which I already have, except with decimal values) 2. being able to represent both - and + underflow, unlike the IEEE standard 3. being able to distinguish between +/- overflow, and real +/- Infinite values (and knowing what overflowed #s are bounded by a particular X and infinity) 4. being able to distinguish between exact numbers and inexact numbers 5. variable sized scale (in my own format, I have 0, 1, or 2 bytes, but this is maybe more flexible) 6. compactness of representation (I already have that also, in my own packing format) So, while I already have 1,5,6, this adds 2,3,4 for me. On Wednesday, July 29, 2015 at 9:50:21 AM UTC-4, Steven G. Johnson wrote: On Sunday, July 26, 2015 at 9:09:19 AM UTC-4, Tom Breloff wrote: For example, I have a specialized solution to represent financial prices which have a fixed accuracy, but I want to be able to do floating point arithmetic on them. Why not use decimal floating-point for that? Regarding, unums, without hardware support, at first glance they don't sound practical compared to the present alternatives (hardware or software fixed-precision float types, or arbitrary precision if you need it). And the ubox method for error analysis, even if it overcomes the problems of interval arithmetic as claimed, sounds too expensive to use on anything except for the smallest-scale problems because of the large number of boxes that you seem to need for each value whose error is being tracked.
Re: [julia-users] John L. Gustafson's UNUMs
Correct me if I'm wrong, but (fixed-size) decimal floating-point has most of the same issues as floating point in terms of accumulation of errors, right? For certain cases, such as adding 2 prices together, I agree that decimal floating point would work ($1.01 + $2.02 == $3.03), but for those cases it's easier to represent the values as integers: (101 + 202 == 303 ; prec=2), which is basically what I do now. In terms of storage and complexity, I would expect that decimal floating point numbers are bloated as compared to floats. You're giving up speed and memory in order to guarantee an exact representation falls in base 10... I could understand how this is occasionally useful, but I can't imagine you'd want that in the general case. In terms of hardware support... obviously it doesn't exist today, but it could in the future: http://www.theplatform.net/2015/03/12/the-little-chip-that-could-disrupt-exascale-computing/ Either way, I would think there's enough potential to the idea to at least prototype and test, and maybe it will prove to be more useful than you expect. On Wed, Jul 29, 2015 at 10:10 AM, Job van der Zwan j.l.vanderz...@gmail.com wrote: On Wednesday, 29 July 2015 16:50:21 UTC+3, Steven G. Johnson wrote: Regarding, unums, without hardware support, at first glance they don't sound practical compared to the present alternatives (hardware or software fixed-precision float types, or arbitrary precision if you need it). And the ubox method for error analysis, even if it overcomes the problems of interval arithmetic as claimed, sounds too expensive to use on anything except for the smallest-scale problems because of the large number of boxes that you seem to need for each value whose error is being tracked. Well, I don't know enough about traditional methods to say if they're really as limited as Gustafson claims in his book, or if he's just cherry-picking. Same about the cost of using uboxe However, ubound arithmetic tells you that 1 / (0, 1] = [1, inf), and that [1, inf) / inf = 0. The ubounds describing those interval results are effectively just a pair of floating point numbers, plus a ubit to signal whether an endpoint is open or not. That's a very simple thing to implement. Not sure if there's any arbitrary precision method that deal with this so elegantly - you probably know better than I do.
Re: [julia-users] John L. Gustafson's UNUMs
Scott: Is your number format a public (open source) specification? How does it differ from decimal floating point? On Wed, Jul 29, 2015 at 10:30 AM, Tom Breloff t...@breloff.com wrote: Correct me if I'm wrong, but (fixed-size) decimal floating-point has most of the same issues as floating point in terms of accumulation of errors, right? For certain cases, such as adding 2 prices together, I agree that decimal floating point would work ($1.01 + $2.02 == $3.03), but for those cases it's easier to represent the values as integers: (101 + 202 == 303 ; prec=2), which is basically what I do now. In terms of storage and complexity, I would expect that decimal floating point numbers are bloated as compared to floats. You're giving up speed and memory in order to guarantee an exact representation falls in base 10... I could understand how this is occasionally useful, but I can't imagine you'd want that in the general case. In terms of hardware support... obviously it doesn't exist today, but it could in the future: http://www.theplatform.net/2015/03/12/the-little-chip-that-could-disrupt-exascale-computing/ Either way, I would think there's enough potential to the idea to at least prototype and test, and maybe it will prove to be more useful than you expect. On Wed, Jul 29, 2015 at 10:10 AM, Job van der Zwan j.l.vanderz...@gmail.com wrote: On Wednesday, 29 July 2015 16:50:21 UTC+3, Steven G. Johnson wrote: Regarding, unums, without hardware support, at first glance they don't sound practical compared to the present alternatives (hardware or software fixed-precision float types, or arbitrary precision if you need it). And the ubox method for error analysis, even if it overcomes the problems of interval arithmetic as claimed, sounds too expensive to use on anything except for the smallest-scale problems because of the large number of boxes that you seem to need for each value whose error is being tracked. Well, I don't know enough about traditional methods to say if they're really as limited as Gustafson claims in his book, or if he's just cherry-picking. Same about the cost of using uboxe However, ubound arithmetic tells you that 1 / (0, 1] = [1, inf), and that [1, inf) / inf = 0. The ubounds describing those interval results are effectively just a pair of floating point numbers, plus a ubit to signal whether an endpoint is open or not. That's a very simple thing to implement. Not sure if there's any arbitrary precision method that deal with this so elegantly - you probably know better than I do.
[julia-users] Re: Irregular Interpolation
Besided GridInterpolations, you might want to look at ApproXD.jl, which works up to four dimensions. I once wrote a little script that compares different one- and two-dimensional interpolation schemes in Julia, it can be found here https://github.com/nilshg/LearningModels/blob/master/Test_Interpolations.jl (I might have to update it to include GridInterpolations at some point).
[julia-users] Re: setindex! for matrix of vectors
I haven't found a solution that works perfectly yet but I circumvented the
problem by making a check for colons:
rows = inds[1] == Colon() ? (1:sys.ny) : inds[1]
cols = inds[2] == Colon() ? (1:sys.nu) : inds[2]
Not perfect since it limits you to always have to use exactly 2 incidences,
but it works in my case.
Maybe the handling of Array{Vector} should be looked at, seems to me that
there are quiet a few cases where you run into problems with that datatype.
For example, I don't think there is a built in way to transpose an
Array{Vector, N1}.
Re: [julia-users] John L. Gustafson's UNUMs
On Wednesday, 29 July 2015 16:50:21 UTC+3, Steven G. Johnson wrote: Regarding, unums, without hardware support, at first glance they don't sound practical compared to the present alternatives (hardware or software fixed-precision float types, or arbitrary precision if you need it). And the ubox method for error analysis, even if it overcomes the problems of interval arithmetic as claimed, sounds too expensive to use on anything except for the smallest-scale problems because of the large number of boxes that you seem to need for each value whose error is being tracked. Well, I don't know enough about traditional methods to say if they're really as limited as Gustafson claims in his book, or if he's just cherry-picking. Same about the cost of using uboxe However, ubound arithmetic tells you that 1 / (0, 1] = [1, inf), and that [1, inf) / inf = 0. The ubounds describing those interval results are effectively just a pair of floating point numbers, plus a ubit to signal whether an endpoint is open or not. That's a very simple thing to implement. Not sure if there's any arbitrary precision method that deal with this so elegantly - you probably know better than I do.
Re: [julia-users] John L. Gustafson's UNUMs
On Sunday, July 26, 2015 at 9:09:19 AM UTC-4, Tom Breloff wrote: For example, I have a specialized solution to represent financial prices which have a fixed accuracy, but I want to be able to do floating point arithmetic on them. Why not use decimal floating-point for that? Regarding, unums, without hardware support, at first glance they don't sound practical compared to the present alternatives (hardware or software fixed-precision float types, or arbitrary precision if you need it). And the ubox method for error analysis, even if it overcomes the problems of interval arithmetic as claimed, sounds too expensive to use on anything except for the smallest-scale problems because of the large number of boxes that you seem to need for each value whose error is being tracked.
Re: [julia-users] John L. Gustafson's UNUMs
I'm going to start work on an unum prototype today, but first... a quick poll: what package name is preferred? Unum.jl? Unums.jl? Something else? I expect there to be an Unum type, so following other conventions I think maybe Unums.jl is proper. Also... please get in touch with me if you have interest in collaborating/contributing, or if you have specific test cases you'd like me to try out. On Monday, July 27, 2015 at 3:56:13 PM UTC-4, Scott Jones wrote: I ended up buying the book, haven't got too far into it yet, definitely am going to work these ideas into my storage format. I've actually been using a form of universal numbers for decades, where unlike C, etc. numbers were not typed with certain sizes, actually, numbers originally weren't even a type - semantically, everything was a string, and the operations you performed determined whether you got a numeric interpretation of the string. (typed numbers were added much later, to support IEEE binary floating point, instead of just decimal floating point). On Sunday, July 26, 2015 at 5:06:56 PM UTC-4, Job van der Zwan wrote: Here's a Python port (found in the HN comments), might be worth checking out: https://github.com/jrmuizel/pyunum On Sunday, 26 July 2015 16:50:52 UTC+3, Scott Jones wrote: If you add support for this to Julia, I want to make sure I can add the format to my own record storage format efficiently. (This sounds great!) On Sunday, July 26, 2015 at 9:09:19 AM UTC-4, Tom Breloff wrote: Unums as a general concept seem really interesting. I ordered the book, and may start a julia implementation (unless someone else gets there first). Unified integer and floating point with clear accuracy information could provide nice solutions for certain problems in finance and statistics. For example, I have a specialized solution to represent financial prices which have a fixed accuracy, but I want to be able to do floating point arithmetic on them. This requires lots of converting between int and float, rounding, etc. Unums may completely change those operations. If anyone starts an implementation, please post the package link here so we don't duplicate efforts. On Sunday, July 26, 2015, Scott Jones scott.pa...@gmail.com wrote: On Sunday, July 26, 2015 at 7:51:51 AM UTC-4, Job van der Zwan wrote: So on an impulse I got the ebook, and even for a physics dropout like me it's surprisingly engaging and accessible! There's some stuff in there that isn't mentioned in the online slides that might clarify the idea better. For example, floats already have a way to represent the largest representable number (maxreal) and positive infinity. Add a ubit gives you the following: - maxreal without ubit: largest rep. number - maxreal with ubit: interval between maxreal and infinity - infinity without ubit: infinity - infinity with ubit: the interval between... infinity and beyond? So what that gives you is a way to represent a number that is bigger than what you can represent, but not infinite (maxreal + ubit), and NaN (infinity + ubit). For negative numbers, just add sign bit. On Sunday, 26 July 2015 13:59:33 UTC+3, Scott Jones wrote: Yes, but I could add the information about inexact vs. exact and keeping track of significant figures to my format as well, while still storing many common values in just 1 byte (including Null, , and markers for binary and packed Unicode text). Well, at the very least it seems to inspire some ways you might improve your format! :) Yes, and I am interested in hearing about hardware support for the UNUM format. I'm also curious about how these ideas interact with decimal floating point (which is what I'm more familiar with, because for the sorts of operations important for the use cases I was concerned with, not having rounding/conversion issues between decimal - binary floating point or string - binary floating point was critical).
Re: [julia-users] Irregular Interpolation
For your problem, it looks like GridInterpolations.jl might do what you want. --Tim On Wednesday, July 29, 2015 06:16:59 AM Jude wrote: Hi, I have been using the fantastic grid package by Tim Holy for the past while but I really need to allow for a non-equally spaced grids. It is important to have a bunch of points at some parts of the grid but not at others. I was wondering if anyone knows of any package that allows for irregular interpolation. I know it is possible to do this using Tim's package for one dimension but I want to interpolate in 3 dimensions. Has there been any new packages developed lately or does anyone know a fast way to do this? Thank you
