Re: ANN: babbage 1.0.0, a library for easily gathering data and computing summary measures in a declarative way

[AtKaaZ]

seems a bit similar to https://github.com/Prismatic/plumbing


On Sat, Feb 2, 2013 at 1:12 AM, Ben Wolfson  wrote:

> ReadyForZero is open-sourcing our library for easily gathering data
> and computing summary measures in a declarative way:
>
> https://github.com/ReadyForZero/babbage
>
> The summary measure functionality allows you to compute multiple
> measures over arbitrary partitions of your input data simultaneously
> and in a single pass. You just say what you want to compute:
>
> > (def my-fields {:y (stats :y count)
>   :x (stats :x count)
>   :both (stats #(+ (or (:x %) 0) (or (:y %) 0)) count
> sum mean)})
>
> and the sets that are of interest:
>
> > (def my-sets (-> (sets {:has-y #(contains? % :y})
>(complement :has-y))) ;; could also take
> intersections, unions
>
> And then run it with some data:
>
> > (calculate my-sets my-fields [{:x 1 :y 2} {:x 10} {:x 4 :y 3} {:x 5}])
> {:not-has-y
>  {:y {:count 0}, :x {:count 2}, :both {:mean 7.5, :sum 15, :count 2}},
>  :has-y
>  {:y {:count 2}, :x {:count 2}, :both {:mean 5.0, :sum 10, :count 2}},
>  :all
>  {:y {:count 2}, :x {:count 4}, :both {:mean 6.25, :sum 25, :count 4}}}
>
> The functions :x, :y, and #(+ (or (:x %) 0) (or (:y %) 0)) defined in
> the fields map are called once per input element no matter how many
> sets the element contributes to. The function #(contains? % y) is also
> called once per input element, no matter how many unions,
> intersections, complements, etc. the set :has-y contributes to.
>
> A variety of measure functions, and structured means of combining
> them, are supplied; it's also easy to define additional measures.
>
> babbage also supplies a method for running computations structured as
> dependency graphs; this can make gathering the initial data for
> summarizing simpler to express. To give an example that's probably
> familiar from another context:
>
> > (defgraphfn sum [xs]
> (apply + xs))
> > (defgraphfn sum-squared [xs]
> (sum (map #(* % %) xs)))
> > (defgraphfn count-input :count [xs]
> (count xs))
> > (defgraphfn mean [count sum]
> (double (/ sum count)))
> > (defgraphfn mean2 [count sum-squared]
> (double (/ sum-squared count)))
> > (defgraphfn variance [mean mean2]
> (- mean2 (* mean mean)))
> > (run-graph {:xs [1 2 3 4]} sum variance sum-squared count-input mean
> mean2)
> {:sum 10
>  :count 4
>  :sum-squared 30
>  :mean 2.5
>  :variance 1.25
>  :mean2 7.5
>  :xs [1 2 3 4]}
>
> Options are provided for parallel, sequential, and lazy computation of
> the elements of the result map, and for resolving the dependency graph
> in advance of running the computation for a given input, either at
> runtime or at compile time.
>
> Please see the README at the github repo for more details.
>
> Enjoy!
>
> --
> Ben Wolfson
> "Human kind has used its intelligence to vary the flavour of drinks,
> which may be sweet, aromatic, fermented or spirit-based. ... Family
> and social life also offer numerous other occasions to consume drinks
> for pleasure." [Larousse, "Drink" entry]
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>


-- 
Please correct me if I'm wrong or incomplete,
even if you think I'll subconsciously hate it.

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: emacs - how to wean me off the family of Java IDEs

[John Chijioke]

+1 for ECB. Especially alexott's work. I have been using ecb for over a 
year now and I can assure you it's totally awesome.  Nothing beats it in 
emacs when it comes to exploring source code directory structure. you need 
it. With eshell integrated to switch automatically to your working 
directory you can quickly switch to the eshell buffer and run quick 
commands. Ability to add and remove sources from the sources buffer is 
totally awesome. you can choose between a number of layout options. For me 
working with a minimum of two buffers and one compilation buffer holding my 
eshell has been the most productive. Try it out. Sure you may have a few 
difficulties setting it up as you are bound to when dealing with first time 
unfamiliar things but right after that you learn a few things and are up in 
business. I'm really surprised so few people are using ecb. It's simply the 
best thing that has happened to me in emacs. Immediately emacs is up, for 
me it's M-x ecb-activate and when I need to temporarily hide it it's M-x 
ecb-hide or ecb-de-activate. With nice tips on each start up to remind you 
of various parts of ecb you may not know of.  I can't shout!

John

On Wednesday, January 16, 2013 3:29:36 PM UTC+1, Colin Yates wrote:
>
> Hi all,
>
> After 15 off years of using IDEs I am making the jump into Emacs.  I have 
> read http://dev.clojure.org/display/doc/Getting+Started+with+Emacs and 
> https://github.com/technomancy/emacs-starter-kit and I am just can choose 
> at the point where I have stopped yelling at paredit and starting to 
> appreciate its point.
>
> My current major stumbling block though is navigating my project.  Whilst 
> (I expect) the density and sane namespacing capabilities of Clojure to 
> significantly reduce the number of files, that isn't true of everything. 
>  In particular, ExtJS encourages you to follow the "one file per class". 
>  You don't have to but eventually you will have more than a handful of 
> files regardless.  
>
> So my questions:
>  - is there a decent project explorer.  I really miss the "tree on the 
> left, editor on the right" layout
>  - is there a decent JS and clojure autocompletion aware plugin
>  - other than paredit, nrepl and clojure-mode (and the excellent 
> coffee-mode for coffeescript), what other plugins should I install
>
> Thanks all.
>
> Col
>
> P.S>  Please don't turn this into a flame war :)
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: *read-eval* vulnerability

[Baishampayan Ghose]

Just did some testing with our code-base and Clojure 1.5.0-RC4 (with
and without Chas' read-eval patch).

There is definitely something strange going on. Things worked just
fine with 1.5.0-RC4 but with the read-eval patch `lein swank` is
completely broken but more than that, our code is failing to compile
with this weird error -

eval-reader: (clojure.lang.PersistentArrayMap/create {:author "Joe Dev
", :doc "Some doc here."})
RuntimeException EvalReader not allowed when *read-eval* is false.
clojure.lang.Util.runtimeException (Util.java:219)

Almost all our namespaces have documentation attached via metadata like this -

(ns ^{:doc "Some doc here."
  :author "Joe Dev "}
  com.helpshift.some.ns
  (:require [com.helpshift.other.ns :as chon])
  (:use clojure.test
midje.sweet))

FWIW, the file that will fail to compile is random and I couldn't
reproduce this error on a fresh project with just a couple of files.

It's quite clear that the eval-reader is getting used from inside
Clojure and we need to test out the edge cases a bit more.

This is clearly not a low-impact fix, but IMHO we should take the time
and get it right before 1.5.0

Regards,
BG


On Sat, Feb 2, 2013 at 4:01 AM, Chas Emerick  wrote:
> I have added a patch to CLJ-1153 that appears to address the *read-eval* 
> problem:
>
> http://dev.clojure.org/jira/browse/CLJ-1153?focusedCommentId=30523#comment-30523
>
> code on github: 
> https://github.com/cemerick/clojure/commit/1f5c19c07443d2535ede4ff71d23b40c195d617f
>
> artifact on Clojars: [com.cemerick/clojure "1.5.0-SNAPSHOT"]
>
> The Leiningen dependency above is 1.5.0-RC4 + the patch on the ticket.  Note 
> that you'll need to set your project's global :exclusions to 
> [org.clojure/clojure] in order for the com.cemerick/clojure artifact to 
> supersede it.
>
> It tests well for me, but needs to be exercised as much as possible.  Some 
> have already done so (there's an ongoing discussion on the clojure-dev ML 
> with some initial test experiences: 
> http://groups.google.com/group/clojure-dev/browse_frm/thread/cc6f747919db6c94),
>  but I'm hoping that we can get as many eyes as possible on this — doing both 
> testing as well as code/patch examination — so as to ensure correctness and 
> maximize the chances of 1.5.0 final going out with this vulnerability 
> buttoned up.
>
> Thanks,
>
> - Chas
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your 
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



-- 
Baishampayan Ghose
b.ghose at gmail.com

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: emacs - how to wean me off the family of Java IDEs

[Korny Sietsma]

Actually, that still didn't seem to work.  After a bit of fiddling, I found
that if I install sr-speedbar from melpa, it spits out a couple of
warnings, thinks it is installed, but I don't get any sr-speedbar commands.
 Strange.

I'll stick to just downloading the package from
http://www.emacswiki.org/emacs/sr-speedbar.el and installing it manually -
that seems to work fine (even without disabling make-local-hook).  If/when
I find time to dig further, I'll try to work out what is going wrong for me
with the melpa install.

- Korny



On 2 February 2013 14:04, Korny Sietsma  wrote:

> Phillip - thanks, that works nicely.  (I had gotten it working by
> downloading sr-speedbar.el manually and editing it, but I prefer to stick
> with the packaged version if possible, less for me to remember)
>
> - Korny
>
>
>
> On 25 January 2013 00:38, Phillip Lord wrote:
>
>>
>>
>> make-local-hook has been deprecated for a while now and has now been
>> removed. In general, you can just ignore it and everything will work
>> correctly. So:
>>
>> (defalias 'make-local-hook 'ignore)
>>
>> before you load anything should solve this problem. Whether any other
>> problems remain, I don't know.
>>
>>
>> Korny Sietsma  writes:
>>
>> > On 17 January 2013 17:26, Sean Corfield  wrote:
>> >
>> >> Error: Symbol's function definition is void: make-local-hook
>> >
>> >
>> > bump - anyone know a workaround for this - I was interested in
>> sr-speedbar,
>> > especially for editing over an ssh session, but it doesn't seem to work
>> > with emacs 24?
>> >
>> > - Korny
>> >
>> >
>> > --
>> > Kornelis Sietsma  korny at my surname dot com http://korny.info
>> > "We do not quit playing because we grow old, we grow old because we quit
>> > playing" - O.W. Holmes
>> >
>> > --
>>
>> --
>> Phillip Lord,   Phone: +44 (0) 191 222 7827
>> Lecturer in Bioinformatics, Email:
>> phillip.l...@newcastle.ac.uk
>> School of Computing Science,
>> http://homepages.cs.ncl.ac.uk/phillip.lord
>> Room 914 Claremont Tower,   skype: russet_apples
>> Newcastle University,   msn: m...@russet.org.uk
>> NE1 7RU twitter: phillord
>>
>> --
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>>
>>
>>
>
>
> --
> Kornelis Sietsma  korny at my surname dot com http://korny.info
> "We do not quit playing because we grow old, we grow old because we quit
> playing" - O.W. Holmes
>



-- 
Kornelis Sietsma  korny at my surname dot com http://korny.info
"We do not quit playing because we grow old, we grow old because we quit
playing" - O.W. Holmes

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: emacs - how to wean me off the family of Java IDEs

[Korny Sietsma]

Phillip - thanks, that works nicely.  (I had gotten it working by
downloading sr-speedbar.el manually and editing it, but I prefer to stick
with the packaged version if possible, less for me to remember)

- Korny



On 25 January 2013 00:38, Phillip Lord  wrote:

>
>
> make-local-hook has been deprecated for a while now and has now been
> removed. In general, you can just ignore it and everything will work
> correctly. So:
>
> (defalias 'make-local-hook 'ignore)
>
> before you load anything should solve this problem. Whether any other
> problems remain, I don't know.
>
>
> Korny Sietsma  writes:
>
> > On 17 January 2013 17:26, Sean Corfield  wrote:
> >
> >> Error: Symbol's function definition is void: make-local-hook
> >
> >
> > bump - anyone know a workaround for this - I was interested in
> sr-speedbar,
> > especially for editing over an ssh session, but it doesn't seem to work
> > with emacs 24?
> >
> > - Korny
> >
> >
> > --
> > Kornelis Sietsma  korny at my surname dot com http://korny.info
> > "We do not quit playing because we grow old, we grow old because we quit
> > playing" - O.W. Holmes
> >
> > --
>
> --
> Phillip Lord,   Phone: +44 (0) 191 222 7827
> Lecturer in Bioinformatics, Email:
> phillip.l...@newcastle.ac.uk
> School of Computing Science,
> http://homepages.cs.ncl.ac.uk/phillip.lord
> Room 914 Claremont Tower,   skype: russet_apples
> Newcastle University,   msn: m...@russet.org.uk
> NE1 7RU twitter: phillord
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
>
>
>


-- 
Kornelis Sietsma  korny at my surname dot com http://korny.info
"We do not quit playing because we grow old, we grow old because we quit
playing" - O.W. Holmes

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

ANN: babbage 1.0.0, a library for easily gathering data and computing summary measures in a declarative way

[Ben Wolfson]

ReadyForZero is open-sourcing our library for easily gathering data
and computing summary measures in a declarative way:

https://github.com/ReadyForZero/babbage

The summary measure functionality allows you to compute multiple
measures over arbitrary partitions of your input data simultaneously
and in a single pass. You just say what you want to compute:

> (def my-fields {:y (stats :y count)
  :x (stats :x count)
  :both (stats #(+ (or (:x %) 0) (or (:y %) 0)) count
sum mean)})

and the sets that are of interest:

> (def my-sets (-> (sets {:has-y #(contains? % :y})
   (complement :has-y))) ;; could also take
intersections, unions

And then run it with some data:

> (calculate my-sets my-fields [{:x 1 :y 2} {:x 10} {:x 4 :y 3} {:x 5}])
{:not-has-y
 {:y {:count 0}, :x {:count 2}, :both {:mean 7.5, :sum 15, :count 2}},
 :has-y
 {:y {:count 2}, :x {:count 2}, :both {:mean 5.0, :sum 10, :count 2}},
 :all
 {:y {:count 2}, :x {:count 4}, :both {:mean 6.25, :sum 25, :count 4}}}

The functions :x, :y, and #(+ (or (:x %) 0) (or (:y %) 0)) defined in
the fields map are called once per input element no matter how many
sets the element contributes to. The function #(contains? % y) is also
called once per input element, no matter how many unions,
intersections, complements, etc. the set :has-y contributes to.

A variety of measure functions, and structured means of combining
them, are supplied; it's also easy to define additional measures.

babbage also supplies a method for running computations structured as
dependency graphs; this can make gathering the initial data for
summarizing simpler to express. To give an example that's probably
familiar from another context:

> (defgraphfn sum [xs]
    (apply + xs))
> (defgraphfn sum-squared [xs]
    (sum (map #(* % %) xs)))
> (defgraphfn count-input :count [xs]
    (count xs))
> (defgraphfn mean [count sum]
    (double (/ sum count)))
> (defgraphfn mean2 [count sum-squared]
    (double (/ sum-squared count)))
> (defgraphfn variance [mean mean2]
    (- mean2 (* mean mean)))
> (run-graph {:xs [1 2 3 4]} sum variance sum-squared count-input mean mean2)
{:sum 10
 :count 4
 :sum-squared 30
 :mean 2.5
 :variance 1.25
 :mean2 7.5
 :xs [1 2 3 4]}

Options are provided for parallel, sequential, and lazy computation of
the elements of the result map, and for resolving the dependency graph
in advance of running the computation for a given input, either at
runtime or at compile time.

Please see the README at the github repo for more details.

Enjoy!

-- 
Ben Wolfson
"Human kind has used its intelligence to vary the flavour of drinks,
which may be sweet, aromatic, fermented or spirit-based. ... Family
and social life also offer numerous other occasions to consume drinks
for pleasure." [Larousse, "Drink" entry]

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: *read-eval* vulnerability

[Chas Emerick]

I have added a patch to CLJ-1153 that appears to address the *read-eval* 
problem:

http://dev.clojure.org/jira/browse/CLJ-1153?focusedCommentId=30523#comment-30523

code on github: 
https://github.com/cemerick/clojure/commit/1f5c19c07443d2535ede4ff71d23b40c195d617f

artifact on Clojars: [com.cemerick/clojure "1.5.0-SNAPSHOT"]

The Leiningen dependency above is 1.5.0-RC4 + the patch on the ticket.  Note 
that you'll need to set your project's global :exclusions to 
[org.clojure/clojure] in order for the com.cemerick/clojure artifact to 
supersede it.

It tests well for me, but needs to be exercised as much as possible.  Some have 
already done so (there's an ongoing discussion on the clojure-dev ML with some 
initial test experiences: 
http://groups.google.com/group/clojure-dev/browse_frm/thread/cc6f747919db6c94), 
but I'm hoping that we can get as many eyes as possible on this — doing both 
testing as well as code/patch examination — so as to ensure correctness and 
maximize the chances of 1.5.0 final going out with this vulnerability buttoned 
up.

Thanks,

- Chas

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to read a txt file?

[greg r]

slurp works well, however, it reads the file in as a single string.  The 
result may not be readily useable depending on what you are doing.
 
I read in text files using clojure.java.io functions plus the core function 
line-seq.  What you get is a sequence whose elements are the individual 
lines of the file.
Then you can apply the power of the Clojure sequence library functions to 
the problem you are trying to solve.
This gets the text file directly to a data structure which is easily 
processed.
 
Here's a quick example function:
 
(use 'clojure.java.io)
 
(defn grab-file [file-path]
 (with-open [text-file-reader (reader file-path)]
   (do-all (line-seq text-file-reader
 
This returns a sequence of strings.  The file-path parameter is a string 
which is the path to the file.
Other functions in clojure.java.io are very useful for dealing with files.
Note the reader function is wrapped in a with-open function to make sure it 
is properly closed.
 
Regards,
Greg
 
 

On Friday, February 1, 2013 7:17:43 AM UTC-5, Roger75 wrote:

> I'd like to read a txt file using clojure. How do I do that? Any examples?

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to read a txt file?

[Andy Fingerhut]

Roger, tryclj.com is limited in what it can do.  The Clojure code you type in 
there is running on the web server across the network from you, not on your own 
local machine.  That file isn't accessible there.

Also for that reason many symbols are not allowed to be used in tryclj.com 
expressions, slurp apparently being one of them.  Click on the about link on 
tryclj.com and read the disclaimer.

If you have some version of Java installed on your local machine, or can 
install it, I'd recommend trying to install Leiningen to try out Clojure on 
your own machine, e.g. by following the "Getting Started" instructions on 
clojure-doc.org here: 
http://clojure-doc.org/articles/tutorials/getting_started.html

Andy

On Feb 1, 2013, at 6:12 AM, Roger75 wrote:

> Hi,
> 
> I'm trying to use this console:
> http://tryclj.com/
> 
> the command I wrote:
> (slurp "C:\\Users\\User1\\teste\\teste.txt")
> 
> 
> I get this:
> java.lang.RuntimeException: Unable to resolve symbol: in this context
> 
> On Friday, February 1, 2013 10:17:43 AM UTC-2, Roger75 wrote:
> I'd like to read a txt file using clojure. How do I do that? Any examples?

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to read a txt file?

[Roger75]

Hi,

I'm trying to use this console:
http://tryclj.com/

the command I wrote:
(slurp "C:\\Users\\User1\\teste\\teste.txt")


I get this:
java.lang.RuntimeException: Unable to resolve symbol: in this context

On Friday, February 1, 2013 10:17:43 AM UTC-2, Roger75 wrote:
>
> I'd like to read a txt file using clojure. How do I do that? Any examples?

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: *read-eval* vulnerability

[Chas Emerick]

On Jan 31, 2013, at 8:03 PM, Chas Emerick wrote:

> 
> On Jan 30, 2013, at 5:59 PM, Michał Marczyk wrote:
> 
>> On 30 January 2013 23:32, Chas Emerick  wrote:
>>> On Jan 30, 2013, at 12:23 PM, Michael Fogus wrote:
>>> 
> RuntimeException EvalReader not allowed when *read-eval* is false.
 
 The problem is that the second eval gets ( 1 2
 3) which invokes the right pathway triggering the exception.  You can
 trigger the same exception by:
 
 (binding [*read-eval* false] (eval (list + 1 2 3)))
>>> 
>>> Re-reading this, I'm clearly not grokking something here.  Maybe I'm having 
>>> a slow afternoon; send help. :-P
>>> 
>>> This obviously ends up running through EvalReader — but why?  How is 
>>> LispReader ever involved at all?
>> 
>> I believe the story goes like so:
>> 
>> The eval call here compiles a list of a function object and three
>> numbers. The function object gets compiled to code which effectively
>> calls readString on "#=(clojure.core$_PLUS_. )". (It so happens that
>> print-dup knows how to handle functions; if it didn't, an exception
>> would be thrown during compilation with the message "Can't embed
>> object in code, maybe print-dup not defined: ...".) When the compiled
>> code is executed, readString gets called to reconstruct the function,
>> and since *read-eval* is false, this fails.
> 
> Whoo, sneaky.  If only fns carried (most of) the metadata that their 
> originating vars were defined with, print-dup would be able to emit a 
> fully-qualified symbol instead of that bonkers ctor call...I think.
> 
> This explains why my plan for a nuclear option for "fixing" *read-eval*'s 
> default doesn't work when outside of a bare `java -cp ... clojure.main` REPL:
> 
>   https://gist.github.com/4674181
> 
> Much of the initialization of the nREPL / Leiningen / Reply toolchain 
> involves evaluating code that's been prn'ed, and there may very well be a 
> couple of nested `eval` usages lurking in there similar to what Fogus raised.
> 
> So, getting *read-eval* to a safe default is going to require more than just 
> setting its default to false; all usages of #= in 
> https://github.com/clojure/clojure/blob/master/src/clj/clojure/core_print.clj 
> need to be eliminated.  Will be peeking at that next...

It looks like print-dup is actually significantly relied upon by the compiler 
when emitting constants (e.g. keyword and symbol literals, namespaces, vars, 
etc) for generated classes.  (There might be other usages that I haven't 
uncovered yet, but I don't *think* so.)  I think that that print-dup usage will 
need to be replaced with corresponding asm method emits if we are to default 
*read-eval* to false without introducing eval-related limitations that will 
appear to nearly all users to be arbitrary and capricious.

Per usual, I'm in over my head around compiler issues, but we'll see what next 
week brings.  I suspect others that have done more compiler hacking than I 
would have an easier time of it...

Cheers,

- Chas

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to read a txt file?

[Zack Maril]

Look at slurp. 
http://clojuredocs.org/clojure_core/1.2.0/clojure.core/slurp

On Friday, February 1, 2013 4:17:43 PM UTC+4, Roger75 wrote:
>
> I'd like to read a txt file using clojure. How do I do that? Any examples?

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to read a txt file?

[AtKaaZ]

there are some examples here:

http://clojuredocs.org/clojure_core/clojure.core/slurp

http://clojuredocs.org/clojure_core/clojure.java.io/reader



On Fri, Feb 1, 2013 at 1:44 PM, Baishampayan Ghose wrote:

> clojure.core/slurp
>
> Sent from phone. Please excuse brevity.
> On 1 Feb 2013 18:13, "Roger75"  wrote:
>
>> I'd like to read a txt file using clojure. How do I do that? Any
>> examples?
>>
>> --
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>



-- 
Please correct me if I'm wrong or incomplete,
even if you think I'll subconsciously hate it.

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to read a txt file?

[Baishampayan Ghose]

clojure.core/slurp

Sent from phone. Please excuse brevity.
On 1 Feb 2013 18:13, "Roger75"  wrote:

> I'd like to read a txt file using clojure. How do I do that? Any examples?
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

How to read a txt file?

[Roger75]

I'd like to read a txt file using clojure. How do I do that? Any examples?

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Hooking into doc and source

[Phillip Lord]

Hugo Duncan  writes:

> phillip.l...@newcastle.ac.uk (Phillip Lord) writes:
>
>> I'm build a library which provides a DSL for building ontologies.
>> Underneath this backs onto a Java API. The library works by provide some
>> macros, which create Java objects then intern them into the local
>> namespace. 
>
>> With the documentation, I can just add this to the metadata. But there
>> is a problem; I also need this in the underlying Java objects, so that
>> when the ontology is serialised, all the documentation goes with it. So
>> now I have the documentation in two places. 
>
> Could you initialise the var metadata using the docstring pulled from
> the java object. You would probably need your macros to generate
> alter-var-root forms to add the metadata.


Yes, I thought about that. I already control the macros which create the
vars, so I could add the metadata at that time. However, the link
between the two is then broken. I've already written code, for example,
which allows me to pull documentation in from properties files (for
i18n). So, the documentation on the var would need updating. Again,
doable. But ugly. 

If I could add doc as a function rather than just a string, then then
problem would go away. But, alas, it's not pluggable in this way. 

Phil

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Hooking into doc and source

[Phillip Lord]

Brian Marick  writes:

> On Jan 31, 2013, at 5:50 AM, Phillip Lord  
> wrote:
>> So, I really would like to hook into the doc function so that I can
>> return a documentation string pulled directly from the underlying Java
>> object; I already have a function for doing this, but do not know how to
>> get the native Clojure facilities to call this, rather than just take
>> the documentation directly from the metadata. 
>
>
> Could you wrap clojure.repl/print-doc using something like Robert Hooke?
> https://github.com/technomancy/robert-hooke


I thought about that; my background in Emacs lisp makes me feel this is
a bit nasty; advicing functions never seems like the cleanest way of
doing things. But it might be the best solution in this case. 

Phil

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to solve Collatz Conjecture problem for huge numbers using clojure parallelism tricks?

[Leandro Moreira]

thanks mand

On Friday, February 1, 2013 12:41:44 AM UTC-2, Zack Maril wrote:
>
> Take a look at this gist:
> https://gist.github.com/4688693
>
> It uses memoize to eek out a little bit more performance.
> λ ~/Projects/experiments/collatz > lein run
> Compiling collatz.core
> [9 19]
> "Elapsed time: 30.236 msecs"
> [97 118]
> "Elapsed time: 5.532 msecs"
> [871 178]
> "Elapsed time: 22.529 msecs"
> [6171 261]
> "Elapsed time: 114.061 msecs"
> [77031 350]
> "Elapsed time: 578.955 msecs"
> [837799 524]
> "Elapsed time: 3686.937 msecs"
> [8400511 685]
> "Elapsed time: 40478.64 msecs"
>
> On my machine it is usually significantly faster than when I run the 
> provided code:
> λ ~/Projects/experiments/collatz > lein run
> Compiling collatz.core
> {:n 9, :count 19}
> "Elapsed time: 22.024 msecs"
> {:n 97, :count 118}
> "Elapsed time: 6.838 msecs"
> {:n 871, :count 178}
> "Elapsed time: 56.313 msecs"
> {:n 6171, :count 261}
> "Elapsed time: 293.266 msecs"
> {:n 77031, :count 350}
> "Elapsed time: 962.113 msecs"
> {:n 837799, :count 524}
> "Elapsed time: 9529.107 msecs"
> λ ~/Projects/experiments/collatz > lein run
> Compiling collatz.core
> {:n 9, :count 19}
> "Elapsed time: 28.077 msecs"
> {:n 97, :count 118}
> "Elapsed time: 8.1 msecs"
> {:n 871, :count 178}
> "Elapsed time: 31.023 msecs"
> {:n 6171, :count 261}
> "Elapsed time: 144.956 msecs"
> {:n 77031, :count 350}
> "Elapsed time: 944.857 msecs"
> {:n 837799, :count 524}
> "Elapsed time: 10030.467 msecs"
> {:n 8400511, :count 685}
> "Elapsed time: 113490.494 msecs"
>
> Of course, there is a bunch of optimizations you can take mathematically:
> http://en.wikipedia.org/wiki/Collatz_conjecture
> -Zack
> On Friday, February 1, 2013 4:29:53 AM UTC+4, Leandro Moreira wrote:
>>
>> The problem is known as Collatz Conjecture (also 3n + 1 conjecture).
>> Basically given a n number then you apply the following rule.
>>
>> If n is even then n/2 otherwise 3 * n + 1, you keep applying this until you 
>> reach the number 1.
>> For instance, starting with *n = 6*, one gets the sequence 6, 3, 10, 5, 16, 
>> 8, 4, 2, 1. (with *8 items*)
>>
>> Now the challenge tell the *n* with n descending from 100 to 1 and with 
>> the *greater number of items*.
>>
>> Then I did the program bellow (I'm very happy for feedback since I'm totally 
>> noobie to clj), but it takes forever, there is anyway to make it fast?
>>
>> (defn- apply-collatz-conjecture 
>>  "Given n, it returns n/2 if it's even or n*3+1 if it's odd."
>>  [n]
>>  (if (even? n)
>>  (/ n 2)
>>  (+ (* 3 n) 1)))
>>  
>> (defn- collatz-conjecture-seq
>>  "Given n, it returns the sequence of collatz-conjecture."
>>  [n]
>>  (loop [n n sequence []]
>>  (if (not= n 1)
>>  (recur (apply-collatz-conjecture n) (cons 
>> (apply-collatz-conjecture n) sequence))
>>  (reverse sequence
>>  
>> (defn- collatz-conjecture-number-of-items
>>  "It returns a map with n and number of items on its collatz-conjecture 
>> sequence."
>>  [n]
>>  { :n n :count (count (collatz-conjecture-seq n)) } )
>>  
>> (defn- greater 
>>  "Given x and y, it returns the element with greater count."
>>  [x y]
>>  (if (> (:count x) (:count y))
>>  x
>>  y))
>>  
>> (defn n-with-more-items
>>  "Given n, it applies collatz-conjecture for the range 1..n 
>>  and return the n with more items."
>>  [n]
>>  (reduce greater (pmap collatz-conjecture-number-of-items (range 1 n
>>
>>
>> The only thing I thought was use pmap but it didn't make it super fast.
>>
>> *Using only map*
>> user=> (time (n-with-more-items  99))
>> "Elapsed time: *21191.762883 msecs*"
>> {:n 837799, :count 524}
>>
>> *Using pmap*
>> user=> (time (n-with-more-items  99))
>> "Elapsed time: *13230.919979 msecs*"
>> {:n 837799, :count 524}
>>
>> Any thoughts on how can I apply parallelism to solve this (especially on 
>> my frustrate try of use map and reduce)?
>>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to solve Collatz Conjecture problem for huge numbers using clojure parallelism tricks?

[Leandro Moreira]

thanks

On Friday, February 1, 2013 8:08:20 AM UTC-2, Alan Malloy wrote:
>
> (max-key :power mario luigi)
>
> On Thursday, January 31, 2013 6:08:21 PM UTC-8, Leandro Moreira wrote:
>>
>> Running through this problem I also faced the weird situation, so:
>>
>> Given two maps
>> (def mario {:color "red" :power 45})
>> (def luigi {:color "green" :power 40})
>>
>> I want the max between both but based on :power key.
>> It would be something like this.
>>
>> (max mario luigi)
>> I expect max return not only 45 but the whole map Is there any built 
>> in function for that?
>>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to solve Collatz Conjecture problem for huge numbers using clojure parallelism tricks?

[AtKaaZ]

amalloy, inspirational as always! thank you


On Fri, Feb 1, 2013 at 11:08 AM, Alan Malloy  wrote:

> (max-key :power mario luigi)
>
>
> On Thursday, January 31, 2013 6:08:21 PM UTC-8, Leandro Moreira wrote:
>>
>> Running through this problem I also faced the weird situation, so:
>>
>> Given two maps
>> (def mario {:color "red" :power 45})
>> (def luigi {:color "green" :power 40})
>>
>> I want the max between both but based on :power key.
>> It would be something like this.
>>
>> (max mario luigi)
>> I expect max return not only 45 but the whole map Is there any built
>> in function for that?
>>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>



-- 
Please correct me if I'm wrong or incomplete,
even if you think I'll subconsciously hate it.

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: How to solve Collatz Conjecture problem for huge numbers using clojure parallelism tricks?

[Alan Malloy]

(max-key :power mario luigi)

On Thursday, January 31, 2013 6:08:21 PM UTC-8, Leandro Moreira wrote:
>
> Running through this problem I also faced the weird situation, so:
>
> Given two maps
> (def mario {:color "red" :power 45})
> (def luigi {:color "green" :power 40})
>
> I want the max between both but based on :power key.
> It would be something like this.
>
> (max mario luigi)
> I expect max return not only 45 but the whole map Is there any built 
> in function for that?
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Efficient idioms to handle large lists?

[Marko Topolnik]

Definitely don't recursively apply concat, it will end up with a stack 
overflow due the way the laziness of concat is implemented.

First and foremost, are you absolutely sure you need the full list realized 
in memory? Your first approach with *mapcat* could be the best option if 
you can make sure to process the list only once, without keeping a 
reference to the head of that collection.

If you need to process the list many times, you can try using (into [] 
(mapcat ...)). This will build a vector, which has less overhead per 
element. But if each element allocates anything more than say 300 bytes, 
the overhead will not be your primary problem. You'll just need a bigger 
heap then.


On Friday, February 1, 2013 8:19:17 AM UTC+1, bruce li wrote:
>
> Hello, everyone. I'm experience some performance issue when using clojure. 
> The scenario is as follows:
>
> I have a huge list of xls files to process. I used the 
> org.clojars.boechat107/cloxls  to read the files which for each file 
> generates a list(approximately 65,000 elements). Now I need to concatenate 
> all of them. I used (mapcat read-worksheet files) to get the final list, 
> but it soon reports out of heap space. Then I tried to use mutable 
> structures:
>
>   (doseq [f files]
> (swap! sheet concat (read-worksheet f)))
>
> where sheet is defined as (def sheet (atom [])))
>
> But the concat seems to slow down a lot when the list grows larger. I'm 
> wondering if in clojure there is some efficient idiom to handle such 
> situation such as efficient concatenation?
>
> Thanks,
> Bruce Li
>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: (into [] (take n v)) vs (subvec v 0 n)

[Marko Topolnik]

Also make sure to keep in mind that subvecs are *second class* objects and 
are not equivalent to regular vectors in terms of API. See for example this 
issue. 
 

> I wouldn't worry too much about subvectors. Unless you identify them as a 
> bottleneck with the profiler.


-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Efficient idioms to handle large lists?

[Takahiro Hozumi]

Hi Bruce,
Try `into` instead of `concat`.
Applying concat to growing data is like "Schlemiel the Painter's
algorithm".

Cheers.

On 2月1日, 午後4:19, bruce li  wrote:
> Hello, everyone. I'm experience some performance issue when using clojure.
> The scenario is as follows:
>
> I have a huge list of xls files to process. I used the
> org.clojars.boechat107/cloxls  to read the files which for each file
> generates a list(approximately 65,000 elements). Now I need to concatenate
> all of them. I used (mapcat read-worksheet files) to get the final list,
> but it soon reports out of heap space. Then I tried to use mutable
> structures:
>
>   (doseq [f files]
>     (swap! sheet concat (read-worksheet f)))
>
> where sheet is defined as (def sheet (atom [])))
>
> But the concat seems to slow down a lot when the list grows larger. I'm
> wondering if in clojure there is some efficient idiom to handle such
> situation such as efficient concatenation?
>
> Thanks,
> Bruce Li

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.