Re: Is there any desire or need for a Clojure DataFrame? (X-POST from Numerical Clojure mailing list)

[arthur . maciejewicz]

Dan, 

That's a huge amount of stats packages available for use assuming we 
achieve interop with Renjin's dataframes. I'll look into it as well. My 
priorities are to first get something working for $DAYJOB, and then to 
build a more generally useful package, and finally add extras such as 
interop.

- Arthur

On Wednesday, March 9, 2016 at 7:04:17 PM UTC-5, Daniel Slutsky wrote:
>
> Thank you for raising this question.
>
> By the way, one desired feature for a Clojure dataframe abstraction would 
> be good interop with Renjin's dataframes.
> Renjin is a JVM-based rewrite of (a subset of) R. It offers a large number 
> of JVM-based statistical libraries. Most of them rely on the dataframe 
> abstraction for their data. R is also very Lisp-like in its data 
> representation, so wrapping all this with Clojure would be a delight.
>
>
>
> On Thursday, March 10, 2016 at 1:47:44 AM UTC+2, Christopher Small wrote:
>>
>>
>> If you're going to do any work in this area, I would highly encourage you 
>> to do in as part of the core.matrix library. That is what Incanter is or 
>> will be using for it's dataset implementation. But it's nice that those 
>> abstractions and implementations be separate from Incanter itself, since 
>> Incanter is a rather large dependency.
>>
>> Core.matrix is certainly (in my eyes) becoming the de facto matrix 
>> computation library in the Clojure ecosystem, and I think in the level of 
>> interop between different implementations there, and extent of utilization 
>> by the clojure community, we rival the python offerings. However, while 
>> core.matrix has some dataset protocols, api functions and basic 
>> implementations, there's still some work to get the full expressiveness of 
>> the data.frame pattern as seen in R and Pandas. Specifically, there is no 
>> support for setting rownames (or arbitrary "name" assignments beyond that 
>> of a single dimension (columns...)). This is something I started working on 
>> a while back, but wasn't able to finish. I could potentially push what I 
>> came up with to a fork, but unfortunately, I don't have any more time to 
>> work on the problem at the moment.
>>
>> Mike Anderson is a great project maintainer, and will probably be happy 
>> to help guide you in stitching together a solution.
>>
>> Best
>>
>> Chris
>>
>>
>>
>>
>>
>> On Wednesday, March 9, 2016 at 12:57:31 PM UTC-8, arthur.ma...@gmail.com 
>> wrote:
>>>
>>> Is there any desire or need for a Clojure DataFrame?
>>>
>>>
>>> By DataFrame, I mean a structure similar to R's data.frame, and Python's 
>>> pandas.DataFrame.
>>>
>>> Incanter's DataSet may already be fulfilling this purpose, and if so, 
>>> I'd like to know if and how people are using it.
>>>
>>> From quickly researching, I see that some prior work has been done in 
>>> this space, such as:
>>>
>>> * https://github.com/cardillo/joinery
>>> * https://github.com/mattrepl/data-frame
>>> * 
>>> http://spark.apache.org/docs/latest/sql-programming-guide.html#dataframes
>>>
>>> Rather than going off and creating a competing implementation (
>>> https://xkcd.com/927/), I'd like to know if anyone here is actively 
>>> working on, or would like to work on a DataFrame and related utilities for 
>>> Clojure (and by extension Java)? Is it something that's sorely needed, or 
>>> is everybody happy with using Incanter or some other library that I'm not 
>>> aware of? If there's already a defacto standard out there, would anyone 
>>> care to please point it out?
>>>
>>> As background information:
>>>
>>> My specific use-case is in NLP and ML, where I often explore and 
>>> prototype in Python, but I'm then left to deal with a smattering of 
>>> libraries on the JVM (Mallet, Weka, Mahout, ND4J, DeepLearning4j, CoreNLP, 
>>> etc.), each with their own ad-hoc implementations of algorithms, matrices, 
>>> and utilities for reading data. It would be great to have a unified way to 
>>> explore my data in the Clojure REPL, and then serve the same code and 
>>> models in production.
>>>
>>> I would love for Clojure to have a broadly compatible ecosystem similar 
>>> to Python's Numpy/Pandas/Scikit-*/Scipy/matplotlib/GenSim,etc. Core.Matrix 
>>> and Incanter appear to fulfill a large chunk of those roles, but I am not 
>>> aware if they've yet become the defacto standards in the community.
>>>
>>> Any feedback is greatly appreciated.
>>>
>>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@goog

Re: Understanding init (the zero arity function) for transducers.

[Philos Kim]

I wrote the next example to trace the inner workings of transducer. I hope 
that this will help.

The next filter-t(transducer), map-t(transducer) and conj-t(reducer) 
functions are excerpted from the filter, map and conj from clojure.core and 
then simplified and modified to focus on the understanding of the inner 
workings.

(defn filter-t
  [pred]
  ;; The first fn is a transducer. It receives the reducer rf and returns
  ;; the reducer(the second fn part of this code).  
  (fn [rf]
(fn
  ([]
   (let [r (rf)]
 (println "filter-t [] post: result =" r)
 r))
  ([result]
   (println "filter-t [result] pre: result =" result)
   (let [r (rf result)]
 (println "filter-t [result] post: result =" r)
 r))
  ([result input]
   (println "filter-t [result input] pre: result =" result ", input =" 
input)
   (let [r (if (pred input)
 (rf result input)
 result)]
 (println "filter-t [result input] post: result =" r)
 r)

(defn map-t
  [f]
  (fn [rf]
(fn
  ([]
   (let [r (rf)]
 (println "map-t [] post: result =" r)
 r))
  ([result]
   (println "map-t [result] pre: result =" result)
   (let [r (rf result)]
 (println "map-t [result] post: result =" r)
 r))
  ([result input]
   (println "map-t [result input] pre: result =" result ", input =" 
input)
   (let [r (rf result (f input))]
 (println "map-t [result input] post: result =" r)
 r)

(defn ^:static conj-t
  []
  ;; This is a reducer itself, not a transducer, because it doesn't receive 
the reducer
  ;; and return a reducer as a transducer.
  (fn
([]
 (println "conj-t []: result =" [])
 [])
([result]
 (println "conj-t [result]: result =" result)
 result)
([result input]
 (println "conj-t [result input] pre: result =" result ", input =" 
input)
 (let [r (. clojure.lang.RT (conj result input))]
   (println "conj-t [result input] post: retrun =" r)
   r) )))


The oupput is edited to facilitate the understandings.

(def xform  (comp (filter-t odd?) (map-t #(* % 10

(transduce xform (conj-t) [1 2 3 4 5])
;>> conj-t []: result = []
;
;   filter-t [result input] pre: result = [] , input = 1
; map-t [result input] pre: result = [] , input = 1
;   conj-t [result input] pre: result = [] , input = 10
;   conj-t [result input] post: retrun = [10]
; map-t [result input] post: result = [10]
;   filter-t [result input] post: result = [10]
;
;   filter-t [result input] pre: result = [10] , input = 2
;   filter-t [result input] post: result = [10]
;
;   filter-t [result input] pre: result = [10] , input = 3
; map-t [result input] pre: result = [10] , input = 3
;   conj-t [result input] pre: result = [10] , input = 30
;   conj-t [result input] post: retrun = [10 30]
; map-t [result input] post: result = [10 30]
;   filter-t [result input] post: result = [10 30]
;
;   filter-t [result input] pre: result = [10 30] , input = 4
;   filter-t [result input] post: result = [10 30]
;
;   filter-t [result input] pre: result = [10 30] , input = 5
; map-t [result input] pre: result = [10 30] , input = 5
;   conj-t [result input] pre: result = [10 30] , input = 50
;   conj-t [result input] post: retrun = [10 30 50]
; map-t [result input] post: result = [10 30 50]
;   filter-t [result input] post: result = [10 30 50]
;
;   filter-t [result] pre: result = [10 30 50]
; map-t [result] pre: result = [10 30 50]
;   conj-t [result]: result = [10 30 50]
; map-t [result] post: result = [10 30 50]
;   filter-t [result] post: result = [10 30 50]
;=> [10 30 50]

>From the above output, my conclusion is that the init part(with no 
argument) of reducer is called only in the last reducer(conj-t in this 
case) and never called in the reducers within the transducers(filter-t and 
map-t).

If you give the init value to the transduce function as follows,

(transduce xform (conj-t) [] [1 2 3 4 5])
;>> filter-t [result input] pre: result = [] , input = 1
; map-t [result input] pre: result = [] , input = 1
;   conj-t [result input] pre: result = [] , input = 10
;   conj-t [result input] post: retrun = [10]
; map-t [result input] post: result = [10]
;   filter-t [result input] post: result = [10]
;
;   filter-t [result input] pre: result = [10] , input = 2
;   filter-t [result input] post: result = [10]
;
;   filter-t [result input] pre: result = [10] , input = 3
; map-t [result input] pre: result = [10] , input = 3
;   conj-t [result input] pre: result = [10] , input = 30
;   conj-t [result input] post: retrun = [10 30]
; map-t [result input] post: result = [10 30]
;   filter-t [result input] post: result = [10 30]
;
;   filter-t [result input] pre: result = [10 30] , input = 4
;   filter-t [result input] post: result = [10 30]
;
;   filter-t [result input] pre: result = [10 30] 

Re: Understanding init (the zero arity function) for transducers.

[Sean Corfield]

Can we at least get an example of situation where the zero-arity version would 
be called?

 

Right now it seems that all the transducer literature out there says there must 
be three arities – and that’s how map etc are defined – but it doesn’t seem, 
based on various people’s simple tests, that the zero-arity version is ever 
called… so why would we define it?

 

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood


 

From: Clojure Mailing List  on behalf of Stephen 
Nelson 
Reply-To: Clojure Mailing List 
Date: Wednesday, March 9, 2016 at 12:23 PM
To: Clojure Mailing List 
Subject: Re: Understanding init (the zero arity function) for transducers.

 

This was discussed further in http://dev.clojure.org/jira/browse/CLJ-1569

 

On Wed, Mar 9, 2016 at 3:43 PM Patrick Curran  wrote:

Thanks Alex,

If you ever do get a chance, I'd be curious to know what it was. The more I 
think about it the more I think Dan is correct. Also "scan" seems like a 
natural thing that one should be able to do without having to jump through 
hoops.



On Monday, February 29, 2016 at 5:10:53 PM UTC-5, Alex Miller wrote:

I think that Rich had an objection to this, however in the haziness of time I 
don't recall specifically what it was. If I get a chance, I will ask him this 
week.

On Monday, February 29, 2016 at 3:27:15 PM UTC-6, Patrick Curran wrote:

Hi,

I was trying to write a transducer and the 0-arity part of it never got called, 
which was unexpected. I did some searching and found this post: 
https://groups.google.com/forum/#!msg/clojure/uVKP4_0KMwQ/-oUJahvUarIJ. What 
Dan is proposing in that post would essentially solve my problem, but it 
doesn't look like his proposal has gotten much traction...

Specifically I was trying to implement scan.

(defn scan
  ([f] (scan f (f)))
  ([f init]
   (fn [xf]
 (let [state (volatile! init)]
   (fn
 ([] (xf (xf) init))
 ([result] (xf result))
 ([result input]
  (let [next-state (f @state input)]
(vreset! state next-state)
(xf result next-state


Which results in the following:

(require '[clojure.core.reducers :as r])
(r/reduce ((scan + 3) conj) [1 2 3])
=> [3 4 6 9]
(transduce (scan + 3) conj [1 2 3])
=> [4 6 9]
(transduce (scan + 3) conj (((scan + 3) conj)) [1 2 3])
=> [3 4 6 9]

My expectation would be that we'd always get the 3 at the front of the vector.

I'm actually using core.async and I'm expecting that the initial value be 
available to be taken from the channel.

(require '[clojure.core.async :as a :include-macros true])
(def c (a/chan 1 (scan + 3)))
(a/go (println (a/!! c 1)
=> 4

So this is more of a conceptual thing rather than just how transduce is 
implemented.

I'd love to hear other people's thoughts on this. I'm quite new, but Dan's 
proposal definitely feels "correct" and the current implementation definitely 
feels "wrong".

--Patrick

 

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clo

Re: Is there any desire or need for a Clojure DataFrame? (X-POST from Numerical Clojure mailing list)

[Christopher Small]

Sounds great; and sure thing, will do :-)

The basic idea I had was to implement a bidirectional index mapping names
<-> indices. This requires making sure you keep the index up to date any
time you change the data, but seemed the easiest way forward.

My fork is here: https://github.com/metasoarous/core.matrix/commits/develop

Here are a couple of related issues:

https://github.com/mikera/core.matrix/issues/193
https://github.com/mikera/core.matrix/issues/220

Hope you can come up with something nice!

I would focus first on coming up with what seems like a nice set of
protocols, so that we can be flexible with implementations. Ideally, we'd
be able to just apply some wrapper to any core.matrix array, vector,
matrix, etc that provided named/labeled access to the data, and would be
fairly seamless with the rest of the library. But you should also be able
to wrap something like Renjin's dataframes (as Daniel Slutsky mentioned;
just implement the protocols using their classes, I imagine). There might
have to be some iteration here. Like: initial protocol design -> initial
implementation -> redraft potocols -> try new implementation -> redraft
protocols, etc. I've noticed that it can be difficult to properly abstract
implementation details away from the protocol/API on the first go (though
you might have mastered this more than I :-)).

My 2c

Goodluck!

Chris



On Wed, Mar 9, 2016 at 4:29 PM,  wrote:

> Chris, thanks for the reply.
>
> It's good to know that I'm not the only one who misses this functionality!
> My goal is definitely to be compatible with Incanter and core.matrix, as
> they both seem mature, and I will never have the time to implement that
> functionality from scratch myself. I'll be studying the source of Pandas
> over the next few days, as I want to have a good idea of how they implement
> their dataframes before starting on the Clojure version. My long-term goal
> is for future authors to look to this set of core tools for data analysis
> as the basis for any packages they build.
>
> If you'd like to publish whatever you've written (hacked up code is ok),
> I'll take a look at that as a starting point, or at least as one possible
> design.
>
> - Arthur
>
>
>
> On Wednesday, March 9, 2016 at 6:47:44 PM UTC-5, Christopher Small wrote:
>>
>>
>> If you're going to do any work in this area, I would highly encourage you
>> to do in as part of the core.matrix library. That is what Incanter is or
>> will be using for it's dataset implementation. But it's nice that those
>> abstractions and implementations be separate from Incanter itself, since
>> Incanter is a rather large dependency.
>>
>> Core.matrix is certainly (in my eyes) becoming the de facto matrix
>> computation library in the Clojure ecosystem, and I think in the level of
>> interop between different implementations there, and extent of utilization
>> by the clojure community, we rival the python offerings. However, while
>> core.matrix has some dataset protocols, api functions and basic
>> implementations, there's still some work to get the full expressiveness of
>> the data.frame pattern as seen in R and Pandas. Specifically, there is no
>> support for setting rownames (or arbitrary "name" assignments beyond that
>> of a single dimension (columns...)). This is something I started working on
>> a while back, but wasn't able to finish. I could potentially push what I
>> came up with to a fork, but unfortunately, I don't have any more time to
>> work on the problem at the moment.
>>
>> Mike Anderson is a great project maintainer, and will probably be happy
>> to help guide you in stitching together a solution.
>>
>> Best
>>
>> Chris
>>
>>
>>
>>
>>
>> On Wednesday, March 9, 2016 at 12:57:31 PM UTC-8, arthur.ma...@gmail.com
>> wrote:
>>>
>>> Is there any desire or need for a Clojure DataFrame?
>>>
>>>
>>> By DataFrame, I mean a structure similar to R's data.frame, and Python's
>>> pandas.DataFrame.
>>>
>>> Incanter's DataSet may already be fulfilling this purpose, and if so,
>>> I'd like to know if and how people are using it.
>>>
>>> From quickly researching, I see that some prior work has been done in
>>> this space, such as:
>>>
>>> * https://github.com/cardillo/joinery
>>> * https://github.com/mattrepl/data-frame
>>> *
>>> http://spark.apache.org/docs/latest/sql-programming-guide.html#dataframes
>>>
>>> Rather than going off and creating a competing implementation (
>>> https://xkcd.com/927/), I'd like to know if anyone here is actively
>>> working on, or would like to work on a DataFrame and related utilities for
>>> Clojure (and by extension Java)? Is it something that's sorely needed, or
>>> is everybody happy with using Incanter or some other library that I'm not
>>> aware of? If there's already a defacto standard out there, would anyone
>>> care to please point it out?
>>>
>>> As background information:
>>>
>>> My specific use-case is in NLP and ML, where I often explore and
>>> prototype in Python, but I'm then left to deal wit

Re: Is there any desire or need for a Clojure DataFrame? (X-POST from Numerical Clojure mailing list)

[arthur . maciejewicz]

Chris, thanks for the reply. 

It's good to know that I'm not the only one who misses this functionality! 
My goal is definitely to be compatible with Incanter and core.matrix, as 
they both seem mature, and I will never have the time to implement that 
functionality from scratch myself. I'll be studying the source of Pandas 
over the next few days, as I want to have a good idea of how they implement 
their dataframes before starting on the Clojure version. My long-term goal 
is for future authors to look to this set of core tools for data analysis 
as the basis for any packages they build.

If you'd like to publish whatever you've written (hacked up code is ok), 
I'll take a look at that as a starting point, or at least as one possible 
design.

- Arthur


On Wednesday, March 9, 2016 at 6:47:44 PM UTC-5, Christopher Small wrote:
>
>
> If you're going to do any work in this area, I would highly encourage you 
> to do in as part of the core.matrix library. That is what Incanter is or 
> will be using for it's dataset implementation. But it's nice that those 
> abstractions and implementations be separate from Incanter itself, since 
> Incanter is a rather large dependency.
>
> Core.matrix is certainly (in my eyes) becoming the de facto matrix 
> computation library in the Clojure ecosystem, and I think in the level of 
> interop between different implementations there, and extent of utilization 
> by the clojure community, we rival the python offerings. However, while 
> core.matrix has some dataset protocols, api functions and basic 
> implementations, there's still some work to get the full expressiveness of 
> the data.frame pattern as seen in R and Pandas. Specifically, there is no 
> support for setting rownames (or arbitrary "name" assignments beyond that 
> of a single dimension (columns...)). This is something I started working on 
> a while back, but wasn't able to finish. I could potentially push what I 
> came up with to a fork, but unfortunately, I don't have any more time to 
> work on the problem at the moment.
>
> Mike Anderson is a great project maintainer, and will probably be happy to 
> help guide you in stitching together a solution.
>
> Best
>
> Chris
>
>
>
>
>
> On Wednesday, March 9, 2016 at 12:57:31 PM UTC-8, arthur.ma...@gmail.com 
> wrote:
>>
>> Is there any desire or need for a Clojure DataFrame?
>>
>>
>> By DataFrame, I mean a structure similar to R's data.frame, and Python's 
>> pandas.DataFrame.
>>
>> Incanter's DataSet may already be fulfilling this purpose, and if so, I'd 
>> like to know if and how people are using it.
>>
>> From quickly researching, I see that some prior work has been done in 
>> this space, such as:
>>
>> * https://github.com/cardillo/joinery
>> * https://github.com/mattrepl/data-frame
>> * 
>> http://spark.apache.org/docs/latest/sql-programming-guide.html#dataframes
>>
>> Rather than going off and creating a competing implementation (
>> https://xkcd.com/927/), I'd like to know if anyone here is actively 
>> working on, or would like to work on a DataFrame and related utilities for 
>> Clojure (and by extension Java)? Is it something that's sorely needed, or 
>> is everybody happy with using Incanter or some other library that I'm not 
>> aware of? If there's already a defacto standard out there, would anyone 
>> care to please point it out?
>>
>> As background information:
>>
>> My specific use-case is in NLP and ML, where I often explore and 
>> prototype in Python, but I'm then left to deal with a smattering of 
>> libraries on the JVM (Mallet, Weka, Mahout, ND4J, DeepLearning4j, CoreNLP, 
>> etc.), each with their own ad-hoc implementations of algorithms, matrices, 
>> and utilities for reading data. It would be great to have a unified way to 
>> explore my data in the Clojure REPL, and then serve the same code and 
>> models in production.
>>
>> I would love for Clojure to have a broadly compatible ecosystem similar 
>> to Python's Numpy/Pandas/Scikit-*/Scipy/matplotlib/GenSim,etc. Core.Matrix 
>> and Incanter appear to fulfill a large chunk of those roles, but I am not 
>> aware if they've yet become the defacto standards in the community.
>>
>> Any feedback is greatly appreciated.
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Is there any desire or need for a Clojure DataFrame? (X-POST from Numerical Clojure mailing list)

[Daniel Slutsky]

Thank you for raising this question.

By the way, one desired feature for a Clojure dataframe abstraction would 
be good interop with Renjin's dataframes.
Renjin is a JVM-based rewrite of (a subset of) R. It offers a large number 
of JVM-based statistical libraries. Most of them rely on the dataframe 
abstraction for their data. R is also very Lisp-like in its data 
representation, so wrapping all this with Clojure would be a delight.



On Thursday, March 10, 2016 at 1:47:44 AM UTC+2, Christopher Small wrote:
>
>
> If you're going to do any work in this area, I would highly encourage you 
> to do in as part of the core.matrix library. That is what Incanter is or 
> will be using for it's dataset implementation. But it's nice that those 
> abstractions and implementations be separate from Incanter itself, since 
> Incanter is a rather large dependency.
>
> Core.matrix is certainly (in my eyes) becoming the de facto matrix 
> computation library in the Clojure ecosystem, and I think in the level of 
> interop between different implementations there, and extent of utilization 
> by the clojure community, we rival the python offerings. However, while 
> core.matrix has some dataset protocols, api functions and basic 
> implementations, there's still some work to get the full expressiveness of 
> the data.frame pattern as seen in R and Pandas. Specifically, there is no 
> support for setting rownames (or arbitrary "name" assignments beyond that 
> of a single dimension (columns...)). This is something I started working on 
> a while back, but wasn't able to finish. I could potentially push what I 
> came up with to a fork, but unfortunately, I don't have any more time to 
> work on the problem at the moment.
>
> Mike Anderson is a great project maintainer, and will probably be happy to 
> help guide you in stitching together a solution.
>
> Best
>
> Chris
>
>
>
>
>
> On Wednesday, March 9, 2016 at 12:57:31 PM UTC-8, arthur.ma...@gmail.com 
> wrote:
>>
>> Is there any desire or need for a Clojure DataFrame?
>>
>>
>> By DataFrame, I mean a structure similar to R's data.frame, and Python's 
>> pandas.DataFrame.
>>
>> Incanter's DataSet may already be fulfilling this purpose, and if so, I'd 
>> like to know if and how people are using it.
>>
>> From quickly researching, I see that some prior work has been done in 
>> this space, such as:
>>
>> * https://github.com/cardillo/joinery
>> * https://github.com/mattrepl/data-frame
>> * 
>> http://spark.apache.org/docs/latest/sql-programming-guide.html#dataframes
>>
>> Rather than going off and creating a competing implementation (
>> https://xkcd.com/927/), I'd like to know if anyone here is actively 
>> working on, or would like to work on a DataFrame and related utilities for 
>> Clojure (and by extension Java)? Is it something that's sorely needed, or 
>> is everybody happy with using Incanter or some other library that I'm not 
>> aware of? If there's already a defacto standard out there, would anyone 
>> care to please point it out?
>>
>> As background information:
>>
>> My specific use-case is in NLP and ML, where I often explore and 
>> prototype in Python, but I'm then left to deal with a smattering of 
>> libraries on the JVM (Mallet, Weka, Mahout, ND4J, DeepLearning4j, CoreNLP, 
>> etc.), each with their own ad-hoc implementations of algorithms, matrices, 
>> and utilities for reading data. It would be great to have a unified way to 
>> explore my data in the Clojure REPL, and then serve the same code and 
>> models in production.
>>
>> I would love for Clojure to have a broadly compatible ecosystem similar 
>> to Python's Numpy/Pandas/Scikit-*/Scipy/matplotlib/GenSim,etc. Core.Matrix 
>> and Incanter appear to fulfill a large chunk of those roles, but I am not 
>> aware if they've yet become the defacto standards in the community.
>>
>> Any feedback is greatly appreciated.
>>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Is there any desire or need for a Clojure DataFrame? (X-POST from Numerical Clojure mailing list)

[Christopher Small]

If you're going to do any work in this area, I would highly encourage you 
to do in as part of the core.matrix library. That is what Incanter is or 
will be using for it's dataset implementation. But it's nice that those 
abstractions and implementations be separate from Incanter itself, since 
Incanter is a rather large dependency.

Core.matrix is certainly (in my eyes) becoming the de facto matrix 
computation library in the Clojure ecosystem, and I think in the level of 
interop between different implementations there, and extent of utilization 
by the clojure community, we rival the python offerings. However, while 
core.matrix has some dataset protocols, api functions and basic 
implementations, there's still some work to get the full expressiveness of 
the data.frame pattern as seen in R and Pandas. Specifically, there is no 
support for setting rownames (or arbitrary "name" assignments beyond that 
of a single dimension (columns...)). This is something I started working on 
a while back, but wasn't able to finish. I could potentially push what I 
came up with to a fork, but unfortunately, I don't have any more time to 
work on the problem at the moment.

Mike Anderson is a great project maintainer, and will probably be happy to 
help guide you in stitching together a solution.

Best

Chris





On Wednesday, March 9, 2016 at 12:57:31 PM UTC-8, arthur.ma...@gmail.com 
wrote:
>
> Is there any desire or need for a Clojure DataFrame?
>
>
> By DataFrame, I mean a structure similar to R's data.frame, and Python's 
> pandas.DataFrame.
>
> Incanter's DataSet may already be fulfilling this purpose, and if so, I'd 
> like to know if and how people are using it.
>
> From quickly researching, I see that some prior work has been done in this 
> space, such as:
>
> * https://github.com/cardillo/joinery
> * https://github.com/mattrepl/data-frame
> * 
> http://spark.apache.org/docs/latest/sql-programming-guide.html#dataframes
>
> Rather than going off and creating a competing implementation (
> https://xkcd.com/927/), I'd like to know if anyone here is actively 
> working on, or would like to work on a DataFrame and related utilities for 
> Clojure (and by extension Java)? Is it something that's sorely needed, or 
> is everybody happy with using Incanter or some other library that I'm not 
> aware of? If there's already a defacto standard out there, would anyone 
> care to please point it out?
>
> As background information:
>
> My specific use-case is in NLP and ML, where I often explore and prototype 
> in Python, but I'm then left to deal with a smattering of libraries on the 
> JVM (Mallet, Weka, Mahout, ND4J, DeepLearning4j, CoreNLP, etc.), each with 
> their own ad-hoc implementations of algorithms, matrices, and utilities for 
> reading data. It would be great to have a unified way to explore my data in 
> the Clojure REPL, and then serve the same code and models in production.
>
> I would love for Clojure to have a broadly compatible ecosystem similar to 
> Python's Numpy/Pandas/Scikit-*/Scipy/matplotlib/GenSim,etc. Core.Matrix and 
> Incanter appear to fulfill a large chunk of those roles, but I am not aware 
> if they've yet become the defacto standards in the community.
>
> Any feedback is greatly appreciated.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Is there any desire or need for a Clojure DataFrame? (X-POST from Numerical Clojure mailing list)

[arthur . maciejewicz]

Is there any desire or need for a Clojure DataFrame?


By DataFrame, I mean a structure similar to R's data.frame, and Python's 
pandas.DataFrame.

Incanter's DataSet may already be fulfilling this purpose, and if so, I'd 
like to know if and how people are using it.

>From quickly researching, I see that some prior work has been done in this 
space, such as:

* https://github.com/cardillo/joinery
* https://github.com/mattrepl/data-frame
* http://spark.apache.org/docs/latest/sql-programming-guide.html#dataframes

Rather than going off and creating a competing implementation 
(https://xkcd.com/927/), I'd like to know if anyone here is actively 
working on, or would like to work on a DataFrame and related utilities for 
Clojure (and by extension Java)? Is it something that's sorely needed, or 
is everybody happy with using Incanter or some other library that I'm not 
aware of? If there's already a defacto standard out there, would anyone 
care to please point it out?

As background information:

My specific use-case is in NLP and ML, where I often explore and prototype 
in Python, but I'm then left to deal with a smattering of libraries on the 
JVM (Mallet, Weka, Mahout, ND4J, DeepLearning4j, CoreNLP, etc.), each with 
their own ad-hoc implementations of algorithms, matrices, and utilities for 
reading data. It would be great to have a unified way to explore my data in 
the Clojure REPL, and then serve the same code and models in production.

I would love for Clojure to have a broadly compatible ecosystem similar to 
Python's Numpy/Pandas/Scikit-*/Scipy/matplotlib/GenSim,etc. Core.Matrix and 
Incanter appear to fulfill a large chunk of those roles, but I am not aware 
if they've yet become the defacto standards in the community.

Any feedback is greatly appreciated.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Understanding init (the zero arity function) for transducers.

[Stephen Nelson]

This was discussed further in http://dev.clojure.org/jira/browse/CLJ-1569

On Wed, Mar 9, 2016 at 3:43 PM Patrick Curran 
wrote:

> Thanks Alex,
>
> If you ever do get a chance, I'd be curious to know what it was. The more
> I think about it the more I think Dan is correct. Also "scan" seems like a
> natural thing that one should be able to do without having to jump through
> hoops.
>
>
> On Monday, February 29, 2016 at 5:10:53 PM UTC-5, Alex Miller wrote:
>>
>> I think that Rich had an objection to this, however in the haziness of
>> time I don't recall specifically what it was. If I get a chance, I will ask
>> him this week.
>>
>> On Monday, February 29, 2016 at 3:27:15 PM UTC-6, Patrick Curran wrote:
>>>
>>> Hi,
>>>
>>> I was trying to write a transducer and the 0-arity part of it never got
>>> called, which was unexpected. I did some searching and found this post:
>>> https://groups.google.com/forum/#!msg/clojure/uVKP4_0KMwQ/-oUJahvUarIJ.
>>> What Dan is proposing in that post would essentially solve my problem, but
>>> it doesn't look like his proposal has gotten much traction...
>>>
>>> Specifically I was trying to implement scan
>>> .
>>>
>>> (defn scan
>>>   ([f] (scan f (f)))
>>>   ([f init]
>>>(fn [xf]
>>>  (let [state (volatile! init)]
>>>(fn
>>>  ([] (xf (xf) init))
>>>  ([result] (xf result))
>>>  ([result input]
>>>   (let [next-state (f @state input)]
>>> (vreset! state next-state)
>>> (xf result next-state
>>>
>>> Which results in the following:
>>> (require '[clojure.core.reducers :as r])
>>> (r/reduce ((scan + 3) conj) [1 2 3])
>>> => [3 4 6 9]
>>> (transduce (scan + 3) conj [1 2 3])
>>> => [4 6 9]
>>> (transduce (scan + 3) conj (((scan + 3) conj)) [1 2 3])
>>> => [3 4 6 9]
>>>
>>> My expectation would be that we'd always get the 3 at the front of the
>>> vector.
>>>
>>> I'm actually using core.async and I'm expecting that the initial value
>>> be available to be taken from the channel.
>>> (require '[clojure.core.async :as a :include-macros true])
>>> (def c (a/chan 1 (scan + 3)))
>>> (a/go (println (a/>> ; expecting 3 to immediately be printed.
>>> (a/>!! c 1)
>>> => 4
>>>
>>> So this is more of a conceptual thing rather than just how transduce is
>>> implemented.
>>>
>>> I'd love to hear other people's thoughts on this. I'm quite new, but
>>> Dan's proposal definitely feels "correct" and the current implementation
>>> definitely feels "wrong".
>>>
>>> --Patrick
>>>
>>>
>>> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Similar lisps and emacs reimplementations?

[Sam Halliday]

Hi all,

I have been learning clojure as holiday reading (I'm a scala dev and am one of 
the main authors of ENSIME.org, which brings IDE like support to text editors 
for Java and Scala).

Clojure is amazing! I'm really loving learning it. There is so much good stuff 
in here, plus it's a lisp which is just incredible for me because I've been an 
Emacs hacker for nearly two decades.

I've done enough research to know that the clojure licence is off topic and 
discussions about it make people feel "nauseous", so I'll skip over begging you 
to change it to MPL or Apache 2.0 and tell you that I cannot use EPL at work. 
It is blacklisted by many of my customers and the patent retaliation clause 
gives my legal advisors enough to construct terrifying scenarios that all ended 
up in the end of my career. Also, I can't add clojure support or use clojure in 
ENSIME because of the well known GPL / EPL incompatibility.

So... skipping over that. It seems I can't actually use this beautiful 
language. But I do a fair bit of emacs-lisp so naturally I'd like to know to 
what extent the features have been reimplemented?

I've seen that Emacs 25 is going to have something that looks a bit like 
destructuring, I've used dash (but there is an idiomatic replacement coming 
too) and I've seen some "ports" of the threading (I love this macro so much). 
However direct ports are still subject to the original licence, so it needs to 
be a clean room implementation (or Rich/the author to release those macros user 
GPL as an emacs package).

Is there anything else that is making its way back into Emacs lisp as a result 
of what has been learnt in Clojure?

And are there any other lisps which use STM? Emacs is still single threaded so 
STM is almost useless there. I'd be really interested in a modern lisp with STM 
and a licence that I could use at work.

Best regards,
Sam

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Feedback on idiomatic API design

[Marc Limotte]

With the macro approach, they don't need to escape it.

On Wed, Mar 9, 2016 at 12:52 PM, Johan Haleby 
wrote:

> Thanks a lot for your support Marc, really appreciated.
>
> On Wed, Mar 9, 2016 at 5:33 PM, Marc Limotte  wrote:
>
>> Yes, I was assuming the HTTP calls happen inside the with-fake-routes! block.
>>
>> I missed the part about the random port.  I se 3 options for that:
>>
>> *Assign a port, rather than random*
>>
>> (with-fake-routes!  ...)
>>
>>
>> But then, of course, you have to worry about port already in use.
>>
>> *An atom*
>>
>> (def the-uri (atom nil))
>> (with-fake-routes! the-uri
>>   ...
>>   (http/get @the-uri "/x"))
>>
>> *A macro*
>>
>> A common convention in Clojure would be to pass it a symbol (e.g. `uri`
>> that is bound by the macro), rather implicitly creating `uri`.
>>
>> (with-fake-routes! [uri option-server-instance]
>>
>> route-map
>>
>> (http/get uri "/x"))
>>
>>
> Didn't know about this convention so thanks for the tip. But is your
> snippet above actually working code or does the user need escape "uri" and "
> option-server-instance" using a single-quotes, i.e.
>
> (with-fake-routes! [*'*uri *'*option-server-instance] ...)
>
>
>>
>> or, with a pre-defined server
>>
>> (def fake-server ...)
>> (with-fake-routes!
>>
>> route-map
>>
>> (http/get (:uri fake-server) "/x"))
>>
>>
>> marc
>>
>>
>>
>> On Wed, Mar 9, 2016 at 1:00 AM, Johan Haleby 
>> wrote:
>>
>>>
>>>
>>> On Wed, Mar 9, 2016 at 6:20 AM, Johan Haleby 
>>> wrote:
>>>
 Thanks for your feedback, exactly what I wanted.

 On Tuesday, March 8, 2016 at 3:16:02 PM UTC+1, mlimotte wrote:
>
> I don't think you need a macro here.  In any case, I'd avoid using a
> macro as late as possible.  See how far you get with just functions, and
> then maybe at the end, add one macro if you absolutely need it to add just
> a touch of syntactic sugar.
>
> routes should clearly be some sort of data-structure, rather than
> side-effect setter functions.  Maybe this:
>
> (with-fake-routes!
>   optional-server-instance
>   route-map)
>
>
>>> Hmm now that I come to think of it I don't see how this would actually
>>> work unless you also perform the HTTP request from inside the scope of  
>>> with-fake-routes!,
>>> otherwise the server instance would be closed before you get the chance
>>> to make the request. Since you make an actual HTTP request you need
>>> access to the URI generated when starting the fake-server instance (at
>>> least if the port is chosen randomly). So either I suppose you would
>>> have to do like this (which requires a macro?):
>>>
>>> (with-fake-routes!
>>>   {"/x" {:status 200 :content-type "application/json" :body (slurp
>>> (io/resource "my.json"))}}
>>>   ; Actual HTTP request
>>>   (http/get uri "/x"))
>>>
>>> where "uri" is created by the  with-fake-routes! macro *or* we could
>>> return the generated fake-server. But if so with-fake-routes! cannot
>>> automatically close the fake-server instance since we need the instance
>>> to be alive when we make the call to the generated uri. I suppose it would
>>> have to look something like this:
>>>
>>> (let [fake-server (with-fake-routes! {"/x" {:status 200 :content-type
>>> "application/json" :body (slurp (io/resource "my.json"))}})]
>>> (http/get (:uri fake-server) "/x")
>>> (shutdown! fake-server))
>>>
>>> If so I think that the second option is unnecessary since then you might
>>> just go with:
>>>
>>> (with-fake-routes!
>>>   *required*-server-instance
>>>   route-map)
>>>
>>> instead of having two options. But then we loose the niceness of having
>>> the server instance be automatically created and stopped for us?
>>>
>>>
> Where optional-server-instance, if it exists is, an object returned
> by (fake-server/start!).  If optional-server-instance is not passed
> in, then with-fake-routes! creates it's own and is free to call
> (shutdown!) on it automatically. And route-map is a Map of routes:
>

> {
> "/x"
>   {:status 200 :content-type "application/json" :body (slurp
> (io/resource "my.json"))}
> {:path "/y" :query {:q "something")}}
>   {:status 200 :content-type "application/json" :body (slurp
> (io/resource "my2.json"))}
> }
>
>
 +1. I'm gonna go for this option.


>
> Also, at the risk of scope creep, I could foresee wanting the response
> to be based on the input instead of just a static blob.  So maybe the 
> value
> of :body could be a string or a function of 1 arg, the route-- in your 
> code
> test with (fn?).
>

 That's a good idea indeed. I've already thought about this for matching
 the request. I'd like this to work:

 {
  (fn [request] (= (:path request) "/x"))
   {:status 200 :content-type "application/json" :body (slurp
 (io/resource "my.json"))}
 {:path "/y" :query {:q (fn [q] (clojure.string/starts-with? q
 "some

Re: Feedback on idiomatic API design

[Johan Haleby]

Thanks a lot for your support Marc, really appreciated.

On Wed, Mar 9, 2016 at 5:33 PM, Marc Limotte  wrote:

> Yes, I was assuming the HTTP calls happen inside the with-fake-routes! block.
>
> I missed the part about the random port.  I se 3 options for that:
>
> *Assign a port, rather than random*
>
> (with-fake-routes!  ...)
>
>
> But then, of course, you have to worry about port already in use.
>
> *An atom*
>
> (def the-uri (atom nil))
> (with-fake-routes! the-uri
>   ...
>   (http/get @the-uri "/x"))
>
> *A macro*
>
> A common convention in Clojure would be to pass it a symbol (e.g. `uri`
> that is bound by the macro), rather implicitly creating `uri`.
>
> (with-fake-routes! [uri option-server-instance]
>
> route-map
>
> (http/get uri "/x"))
>
>
Didn't know about this convention so thanks for the tip. But is your
snippet above actually working code or does the user need escape "uri" and "
option-server-instance" using a single-quotes, i.e.

(with-fake-routes! [*'*uri *'*option-server-instance] ...)


>
> or, with a pre-defined server
>
> (def fake-server ...)
> (with-fake-routes!
>
> route-map
>
> (http/get (:uri fake-server) "/x"))
>
>
> marc
>
>
>
> On Wed, Mar 9, 2016 at 1:00 AM, Johan Haleby 
> wrote:
>
>>
>>
>> On Wed, Mar 9, 2016 at 6:20 AM, Johan Haleby 
>> wrote:
>>
>>> Thanks for your feedback, exactly what I wanted.
>>>
>>> On Tuesday, March 8, 2016 at 3:16:02 PM UTC+1, mlimotte wrote:

 I don't think you need a macro here.  In any case, I'd avoid using a
 macro as late as possible.  See how far you get with just functions, and
 then maybe at the end, add one macro if you absolutely need it to add just
 a touch of syntactic sugar.

 routes should clearly be some sort of data-structure, rather than
 side-effect setter functions.  Maybe this:

 (with-fake-routes!
   optional-server-instance
   route-map)


>> Hmm now that I come to think of it I don't see how this would actually
>> work unless you also perform the HTTP request from inside the scope of  
>> with-fake-routes!,
>> otherwise the server instance would be closed before you get the chance
>> to make the request. Since you make an actual HTTP request you need
>> access to the URI generated when starting the fake-server instance (at
>> least if the port is chosen randomly). So either I suppose you would
>> have to do like this (which requires a macro?):
>>
>> (with-fake-routes!
>>   {"/x" {:status 200 :content-type "application/json" :body (slurp
>> (io/resource "my.json"))}}
>>   ; Actual HTTP request
>>   (http/get uri "/x"))
>>
>> where "uri" is created by the  with-fake-routes! macro *or* we could
>> return the generated fake-server. But if so with-fake-routes! cannot
>> automatically close the fake-server instance since we need the instance
>> to be alive when we make the call to the generated uri. I suppose it would
>> have to look something like this:
>>
>> (let [fake-server (with-fake-routes! {"/x" {:status 200 :content-type
>> "application/json" :body (slurp (io/resource "my.json"))}})]
>> (http/get (:uri fake-server) "/x")
>> (shutdown! fake-server))
>>
>> If so I think that the second option is unnecessary since then you might
>> just go with:
>>
>> (with-fake-routes!
>>   *required*-server-instance
>>   route-map)
>>
>> instead of having two options. But then we loose the niceness of having
>> the server instance be automatically created and stopped for us?
>>
>>
 Where optional-server-instance, if it exists is, an object returned by
 (fake-server/start!).  If optional-server-instance is not passed in,
 then with-fake-routes! creates it's own and is free to call (shutdown!)
 on it automatically. And route-map is a Map of routes:

>>>
 {
 "/x"
   {:status 200 :content-type "application/json" :body (slurp
 (io/resource "my.json"))}
 {:path "/y" :query {:q "something")}}
   {:status 200 :content-type "application/json" :body (slurp
 (io/resource "my2.json"))}
 }


>>> +1. I'm gonna go for this option.
>>>
>>>

 Also, at the risk of scope creep, I could foresee wanting the response
 to be based on the input instead of just a static blob.  So maybe the value
 of :body could be a string or a function of 1 arg, the route-- in your code
 test with (fn?).

>>>
>>> That's a good idea indeed. I've already thought about this for matching
>>> the request. I'd like this to work:
>>>
>>> {
>>>  (fn [request] (= (:path request) "/x"))
>>>   {:status 200 :content-type "application/json" :body (slurp
>>> (io/resource "my.json"))}
>>> {:path "/y" :query {:q (fn [q] (clojure.string/starts-with? q "some"))}}
>>>   {:status 200 :content-type "application/json" :body (slurp
>>> (io/resource "my2.json"))}
>>> }
>>>
>>> Thanks a lot for your help and feedback!
>>>
>>>

 This gives you a single api, no macros, optional auto-server start/stop
 or explicit server management.

Re: Feedback on idiomatic API design

[Marc Limotte]

Yes, I was assuming the HTTP calls happen inside the with-fake-routes! block.

I missed the part about the random port.  I se 3 options for that:

*Assign a port, rather than random*

(with-fake-routes!  ...)


But then, of course, you have to worry about port already in use.

*An atom*

(def the-uri (atom nil))
(with-fake-routes! the-uri
  ...
  (http/get @the-uri "/x"))

*A macro*

A common convention in Clojure would be to pass it a symbol (e.g. `uri`
that is bound by the macro), rather implicitly creating `uri`.

(with-fake-routes! [uri option-server-instance]

route-map

(http/get uri "/x"))


or, with a pre-defined server

(def fake-server ...)
(with-fake-routes!

route-map

(http/get (:uri fake-server) "/x"))


marc



On Wed, Mar 9, 2016 at 1:00 AM, Johan Haleby  wrote:

>
>
> On Wed, Mar 9, 2016 at 6:20 AM, Johan Haleby 
> wrote:
>
>> Thanks for your feedback, exactly what I wanted.
>>
>> On Tuesday, March 8, 2016 at 3:16:02 PM UTC+1, mlimotte wrote:
>>>
>>> I don't think you need a macro here.  In any case, I'd avoid using a
>>> macro as late as possible.  See how far you get with just functions, and
>>> then maybe at the end, add one macro if you absolutely need it to add just
>>> a touch of syntactic sugar.
>>>
>>> routes should clearly be some sort of data-structure, rather than
>>> side-effect setter functions.  Maybe this:
>>>
>>> (with-fake-routes!
>>>   optional-server-instance
>>>   route-map)
>>>
>>>
> Hmm now that I come to think of it I don't see how this would actually
> work unless you also perform the HTTP request from inside the scope of  
> with-fake-routes!,
> otherwise the server instance would be closed before you get the chance to
> make the request. Since you make an actual HTTP request you need access
> to the URI generated when starting the fake-server instance (at least if
> the port is chosen randomly). So either I suppose you would have to do
> like this (which requires a macro?):
>
> (with-fake-routes!
>   {"/x" {:status 200 :content-type "application/json" :body (slurp
> (io/resource "my.json"))}}
>   ; Actual HTTP request
>   (http/get uri "/x"))
>
> where "uri" is created by the  with-fake-routes! macro *or* we could
> return the generated fake-server. But if so with-fake-routes! cannot
> automatically close the fake-server instance since we need the instance
> to be alive when we make the call to the generated uri. I suppose it would
> have to look something like this:
>
> (let [fake-server (with-fake-routes! {"/x" {:status 200 :content-type
> "application/json" :body (slurp (io/resource "my.json"))}})]
> (http/get (:uri fake-server) "/x")
> (shutdown! fake-server))
>
> If so I think that the second option is unnecessary since then you might
> just go with:
>
> (with-fake-routes!
>   *required*-server-instance
>   route-map)
>
> instead of having two options. But then we loose the niceness of having
> the server instance be automatically created and stopped for us?
>
>
>>> Where optional-server-instance, if it exists is, an object returned by (
>>> fake-server/start!).  If optional-server-instance is not passed in,
>>> then with-fake-routes! creates it's own and is free to call (shutdown!)
>>> on it automatically. And route-map is a Map of routes:
>>>
>>
>>> {
>>> "/x"
>>>   {:status 200 :content-type "application/json" :body (slurp
>>> (io/resource "my.json"))}
>>> {:path "/y" :query {:q "something")}}
>>>   {:status 200 :content-type "application/json" :body (slurp
>>> (io/resource "my2.json"))}
>>> }
>>>
>>>
>> +1. I'm gonna go for this option.
>>
>>
>>>
>>> Also, at the risk of scope creep, I could foresee wanting the response
>>> to be based on the input instead of just a static blob.  So maybe the value
>>> of :body could be a string or a function of 1 arg, the route-- in your code
>>> test with (fn?).
>>>
>>
>> That's a good idea indeed. I've already thought about this for matching
>> the request. I'd like this to work:
>>
>> {
>>  (fn [request] (= (:path request) "/x"))
>>   {:status 200 :content-type "application/json" :body (slurp (io/resource
>> "my.json"))}
>> {:path "/y" :query {:q (fn [q] (clojure.string/starts-with? q "some"))}}
>>   {:status 200 :content-type "application/json" :body (slurp (io/resource
>> "my2.json"))}
>> }
>>
>> Thanks a lot for your help and feedback!
>>
>>
>>>
>>> This gives you a single api, no macros, optional auto-server start/stop
>>> or explicit server management.
>>>
>>> marc
>>>
>>>
>>> On Tue, Mar 8, 2016 at 3:10 AM, Johan Haleby 
>>> wrote:
>>>
 Hi,

 I've just committed an embryo of an open source project
  to fake http requests by
 starting an actual (programmable) HTTP server. Currently the API looks like
 this (which in my eyes doesn't look very Clojure idiomatic):

 (let [fake-server (fake-server/start!)
 (fake-route! fake-server "/x" {:status 200 :content-type 
 "application/json" :body (slurp (io/resource 

Writing a ClojureScript plugin for and existing project

[Fred Eisele]

I have been writing Clojure code for a while but I am not an expert.
But, my JavaScript skills are even worse.
There is a tool which I need to use for a project.
https://github.com/webgme/webgme

This tool can be extended with plugins written in javascript.
These plugins may run either in the browser or on the server in node.js
I am fairly comfortable with Clojure but not so much with JavaScript.
For that reason I would like to write my plugin in ClojureScript.
The path to do this is not clear to me.

I have found many examples of how to write a ClojureScript project from 
scratch.
What I need is some guidance on augmenting an existing project.
Thanks

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.