Re: Complete Web Development Setup Using Clojure CLI Tools

[Blake Miller]

Jag,

I think what you described is worth sharing. I like the simplicity of that 
approach and the efficiency of the final artifact.

-Blake

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/clojure/8a201cc5-8bf1-4a1f-9103-39af8d5452b2%40googlegroups.com.

Re: New idea & POC for a Kubernetes/Clojure interface

[Blake Miller]

Yes, kops has a tutorial that walks you through it:

https://github.com/kubernetes/kops/blob/master/docs/aws.md

You just need an AWS account to start.

HTH

On Tue, May 22, 2018 at 1:20 PM, Punit Naik <naik.puni...@gmail.com> wrote:

> Is there any documentation around spinning up K8s cluster on Amazon EC2
> instances?
>
>
> On Tuesday, November 7, 2017 at 8:34:35 AM UTC+5:30, Blake Miller wrote:
>>
>> Here's a little something I cooked up this weekend, to interact with a
>> Kubernetes cluster from Clojure:
>>
>> https://github.com/blak3mill3r/keenest-rube
>>
>> It abstracts away the K8s API calls completely. Instead, you get the
>> state of the cluster as a value in an atom. Changes to the state of the
>> cluster are streamed to the Clojure client, which keeps the value in the
>> atom current, and attempts to mutate the atom will cause one cluster
>> resource (one at a time) to be modified/created/destroyed appropriately. So
>> far I'm finding it to be a real pleasure to use compared to `kubectl`
>> (giving it hand-edited json or yaml files) or worse: the Dashboard (poking
>> around at a web app with a mouse).
>>
>> I guess I could've just tried the Python library, but where's the fun in
>> that?
>>
>> I feel like this could turn into a pretty powerful tool for ops work, or
>> even adding abstractions to manage resources automatically. I've been
>> wanting to use Clojure (more) for infrastructure-automation/dev-ops ...
>> but this is one area where the tooling available is a bit lacking, IMO.
>>
>> I've been successfully toying around with this project and a real
>> Kubernetes cluster in an AWS VPC.
>>
>> I'd be glad to get hear any thoughts on this idea. If you're into k8s,
>> please give it a whirl.
>>
>> This is total toy-status right now, by the way, it's just a
>> proof-of-concept. Oh, and it mixes nicely with `cider-enlighten-mode`, if
>> you're into that sort of thing. I went ahead and published it to Clojars.
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/clojure/GqZ04diilkM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ANN] Pyro 0.1.0 - a library for more helpful stacktraces

[Blake Miller]

Very nice! Congrats on the release. I'm going to play with it today.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

New idea & POC for a Kubernetes/Clojure interface

[Blake Miller]

Here's a little something I cooked up this weekend, to interact with a 
Kubernetes cluster from Clojure:

https://github.com/blak3mill3r/keenest-rube

It abstracts away the K8s API calls completely. Instead, you get the state 
of the cluster as a value in an atom. Changes to the state of the cluster 
are streamed to the Clojure client, which keeps the value in the atom 
current, and attempts to mutate the atom will cause one cluster resource 
(one at a time) to be modified/created/destroyed appropriately. So far I'm 
finding it to be a real pleasure to use compared to `kubectl` (giving it 
hand-edited json or yaml files) or worse: the Dashboard (poking around at a 
web app with a mouse).

I guess I could've just tried the Python library, but where's the fun in 
that?

I feel like this could turn into a pretty powerful tool for ops work, or 
even adding abstractions to manage resources automatically. I've been 
wanting to use Clojure (more) for infrastructure-automation/dev-ops ... but 
this is one area where the tooling available is a bit lacking, IMO.

I've been successfully toying around with this project and a real 
Kubernetes cluster in an AWS VPC.

I'd be glad to get hear any thoughts on this idea. If you're into k8s, 
please give it a whirl.

This is total toy-status right now, by the way, it's just a 
proof-of-concept. Oh, and it mixes nicely with `cider-enlighten-mode`, if 
you're into that sort of thing. I went ahead and published it to Clojars.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Is this behavior of clojure.core/pr a bug?

[Blake Miller]

I agree with Herwig in principal ... even though EDN is not meant to cover
the whole set of possible pure Clojure data, if it can be made to cover
more (all other things being equal) that would be a Good Thing.

I think it would be possible to fix these edge cases with reader macro
dispatches without breaking compatibility. The major snag though is
that performance would suffer ... every single keyword or symbol being
`pr`d would have to be tested, even those in the vast majority that don't
need to be emitted in any special way. So my conclusion is it's not worth
trying ...

It sucks that the docstring for pr
https://github.com/clojure/clojure/blob/clojure-1.7.0/src/clj/clojure/core.clj#L3552-L3555
fails to mention that the function may succeed and produce a string that
the reader will barf on, but I think we're pretty much stuck with it.

For posterity: I switched to using Transit for the Clojure(Script) app that
had me run across this issue.

On Thu, Aug 4, 2016 at 2:23 PM, Herwig Hochleitner 
wrote:

> 2016-08-04 13:56 GMT+02:00 Timothy Baldridge :
>
>> The problem is that many do not understand that Clojure data is a
>> superset of EDN. The two were never meant to be completely compatible.
>> There are many things, especially when dealing with keywords and symbols,
>> where its possible to have data that doesn't properly round-trip.
>>
>
> Then fressian and transit are supersets of edn as well. Are those, at
> least, meant to be the same set as clojure data?
> Also, reader tags are a fantastic opportunity to make arbitrary data
> round-trippable.
>
> An added problem when dealing with EDN is that there is only really one or
>> two languages that properly parse it: Clojure and Clojurescript. So it's
>> also a poor choice to use in cases where you desire any sort of interop.
>>
>
> There many edn libraries for various languages: https://github.com/
> edn-format/edn/wiki/Implementations
> It is true, that there is a lack of compatibility, especially in the
> handling of symbols and keywords and the community is hurting for it (I
> remember a couple of tedious discussions on the matter)
>
> see http://dev.clojure.org/jira/browse/CLJ-1527 https://
> github.com/edn-format/edn/issues/67 ...
>
> Add on top of all that that EDN parsing is really slow compared to other
>> approaches, and you have a lot of compelling reasons to, as Herwig put it,
>> "abandon edn, except for hand-written data".
>>
>
> My view is, that those reasons should be eliminated, starting with
> interoperability concerns. I still think edn is a fantastic idea and to me
> it still holds the promise of being a replacement for json and xml, but
> only if we can get our act together and develop it towards that goal.
>
> Please note, that my "except for hand-written data" was meant to be
> hyperbole. Every data is eventually machine-written.
>
> Abandoning edn would send a fatal signal not just to people in the
> community. Especially if we let it slowly die instead of declaring it a
> failed experiment in data exchange.
>
> Imagine if pr wouldn't handle embedded " quotes in strings and the
> inofficial recommendation would be to just avoid that use case or use a
> different encoding.
>
> And yes, the original problem that caused the creation of Transit was "how
>> do we get data from language A to language B while still staying fast, not
>> implementing a ton of code, and keeping rich data (dates should be dates,
>> not strings)."
>>
>
> I like the idea of having various encodings for different uses, but we
> should strife towards compatibility.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/clojure/Rc_b4_Da-KU/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop 

Re: Is this behavior of clojure.core/pr a bug?

[Blake Miller]

Er, I mean "built-in reader macro dispatch".

On Thursday, August 4, 2016 at 1:14:16 AM UTC, Blake Miller wrote:
>
> You're right, Dan. Having mulled it over a little more, it's not clear to 
> me why there ought to be any pure Clojure data (no Java objects) that 
> cannot be serialized as EDN. Emitting a #keyword reader literal for this 
> edge case would make sense to me.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Is this behavior of clojure.core/pr a bug?

[Blake Miller]

You're right, Dan. Having mulled it over a little more, it's not clear to 
me why there ought to be any pure Clojure data (no Java objects) that 
cannot be serialized as EDN. Emitting a #keyword reader literal for this 
edge case would make sense to me.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Is this behavior of clojure.core/pr a bug?

[Blake Miller]

Thanks for that concise explanation, Sean. It makes sense to me that not 
all valid Clojure data is serializable.

There's still something about this that doesn't quite make sense to me, 
though:

clojure.core/pr, rather than throwing an exception when asked to serialize 
an instance of clojure.core.Keyword that cannot be serialized, it simply 
produces bad output. Bad = will cause the reader to throw.

Wouldn't it be preferable for pr to throw in this case?

The way I found out about this was the not-very-informative exception "Map 
literal must contain an even number of forms", because pr was fine with 
making a string that the reader wouldn't accept.

Can anyone think of a good reason why pr should *not* throw an exception on

(pr (keyword "foo bar"))

since there's no way of expressing that keyword as valid EDN?



On Thursday, August 4, 2016 at 12:16:47 AM UTC, Sean Corfield wrote:
>
> You can programmatically create keywords that are illegal as literals, 
> i.e., will not be accepted by the reader.
>
>  
>
> This is not a fault of clojure.core/pr – if it is given a value that uses 
> legal (readable) keywords, its result will indeed be readable by 
> clojure.core/read-string.
>
>  
>
> You can also programmatically create symbols that are illegal as far as 
> the reader is concerned.
>
>  
>
> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
> An Architect's View -- http://corfield.org/
>
> "If you're not annoying somebody, you're not really alive."
> -- Margaret Atwood
>
>  
>
> On 8/3/16, 4:37 PM, "Blake Miller" <clo...@googlegroups.com  
> on behalf of blak3...@gmail.com > wrote:
>
>  
>
> I have tried this with Clojure 1.7.0, 1.8.0 and 1.9.0-alpha10
>
> (clojure.core/read-string (clojure.core/with-out-str (clojure.core/pr 
> (clojure.core/keyword "A valid keyword" ;; => :A
>
>
> This just seems wrong. It's valid to have an instance of 
> clojure.lang.Keyword with a space in its name.
>
> (clojure.core/with-out-str (clojure.core/pr (clojure.core/keyword "A valid 
> keyword"))) => ":A valid keyword"
>
>  
>
> So, it seems like clojure.core/pr and clojure.core/read-string disagree 
> about EDN.
>
>  
>
> Is EDN formally specified? https://github.com/edn-format/edn/issues/56 
> seems to suggest it is not.
>
> I ran into this problem using ptaoussanis/sente to pass EDN over a 
> websocket. The EDN contained a keyword with a space in it, and the 
> clojure(jvm) part of sente had no problem serializing it, but the 
> ClojureScript part of sente barfed on it. I thought it was a bug in sente, 
> however sente simply calls clojure.core/pr to do the serialization... so I 
> played with pr vs read-string and found that they disagree.
>
>  
>
> The serialization that clojure.core/pr does on a keyword with a space in 
> it seems broken to me:
>
>  
>
> user> (clojure.core/with-out-str (clojure.core/pr {:onekey 1
>
>(clojure.core/keyword 
> "two key") 2}))
>
> "{:onekey 1, :two key 2}"
>
> There doesn't seem to be any way to parse that unambiguously.
>
> I think this is a bug. What do you think?
>
>
> https://github.com/ptaoussanis/sente/issues/251
>
>  
>
>  
>
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Is this behavior of clojure.core/pr a bug?

[Blake Miller]

Thanks, Timothy. I'll give transit a try.

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Is this behavior of clojure.core/pr a bug?

[Blake Miller]

The docstring of clojure.core/pr

https://github.com/clojure/clojure/blob/clojure-1.7.0/src/clj/clojure/core.clj#L3552-L3555

actually says (in lieu of a formal EDN specification?)

"pr and prn print in a way that objects can be read by the reader"

...and the example I showed appears to violate that. Here's a minimal 
failing case:

user> (read-string (with-out-str (pr {(clojure.core/keyword "key word") 
1})) )
RuntimeException Map literal must contain an even number of forms 
 clojure.lang.Util.runtimeException (Util.java:221)

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Is this behavior of clojure.core/pr a bug?

[Blake Miller]

I have tried this with Clojure 1.7.0, 1.8.0 and 1.9.0-alpha10

(clojure.core/read-string (clojure.core/with-out-str (clojure.core/pr 
(clojure.core/keyword "A valid keyword" ;; => :A

This just seems wrong. It's valid to have an instance of 
clojure.lang.Keyword with a space in its name.

(clojure.core/with-out-str (clojure.core/pr (clojure.core/keyword "A valid 
keyword"))) => ":A valid keyword"

So, it seems like clojure.core/pr and clojure.core/read-string disagree 
about EDN.

Is EDN formally specified? https://github.com/edn-format/edn/issues/56 
seems to suggest it is not.

I ran into this problem using ptaoussanis/sente to pass EDN over a 
websocket. The EDN contained a keyword with a space in it, and the 
clojure(jvm) part of sente had no problem serializing it, but the 
ClojureScript part of sente barfed on it. I thought it was a bug in sente, 
however sente simply calls clojure.core/pr to do the serialization... so I 
played with pr vs read-string and found that they disagree.

The serialization that clojure.core/pr does on a keyword with a space in it 
seems broken to me:

user> (clojure.core/with-out-str (clojure.core/pr {:onekey 1
   (clojure.core/keyword 
"two key") 2}))
"{:onekey 1, :two key 2}"

There doesn't seem to be any way to parse that unambiguously.

I think this is a bug. What do you think?

https://github.com/ptaoussanis/sente/issues/251

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Secondar Sorting in spark using clojure/flambo

[Blake Miller]

Hi Punit

The behavior you are referring to is a feature of the Scala compiler, which 
is why it does not happen automatically when you try to use it from Clojure.

Please see the note here:

https://github.com/t6/from-scala/blob/4e1752aaa2ef835dd67a8404273bee067510a431/test/t6/from_scala/guide.clj#L161-L166

You may find that library a useful resource, either as a dependency or 
simply as reference material.

What you want to do is find the full method signature, including the 
implicits, and invoke _that_ from clojure, passing values for all implicit 
parameters (in this case, your custom ordering function.

HTH

On Saturday, July 9, 2016 at 6:13:17 AM UTC, Punit Naik wrote:
>
> Hi Ashish
>
> The "package" is indeed the full package name.
> On 09-Jul-2016 11:02 AM, "Ashish Negi"  > wrote:
>
>> Should not be `package` in `:import` be the actual package name of  `
>> RFMCPartitioner` ?
>>
>> see examples at https://clojuredocs.org/clojure.core/import
>>
>> like :
>>
>> (ns foo.bar
>>   (:import (java.util Date
>>   Calendar)
>>(java.util.logging Logger
>>   Level)))
>>
>>
>>
>> (ns xyz
>>   (:import
>> [**  RFMCPartitioner]
>> [** RFMCKey]
>> )
>>   )
>>
>>
>> where ** is package full name.
>>
>>
>>
>> On Friday, 8 July 2016 21:31:27 UTC+5:30, Punit Naik wrote:
>>>
>>>
>>>  
>>>
>>> I have a scala program in which I have implemented a secondary sort 
>>> which works perfectly. The way I have written that program is:
>>>
>>> object rfmc {
>>>   // Custom Key and partitioner
>>>
>>>   case class RFMCKey(cId: String, R: Double, F: Double, M: Double, C: 
>>> Double)
>>>   class RFMCPartitioner(partitions: Int) extends Partitioner {
>>> require(partitions >= 0, "Number of partitions ($partitions) cannot be 
>>> negative.")
>>> override def numPartitions: Int = partitions
>>> override def getPartition(key: Any): Int = {
>>>   val k = key.asInstanceOf[RFMCKey]
>>>   k.cId.hashCode() % numPartitions
>>> }
>>>   }
>>>   object RFMCKey {
>>> implicit def orderingBycId[A <: RFMCKey] : Ordering[A] = {
>>>   Ordering.by(k => (k.R, k.F * -1, k.M * -1, k.C * -1))
>>> }
>>>   }
>>>   // The body of the code
>>>   //
>>>   //
>>>   val x = rdd.map(RFMCKey(cust,r,f,m,c), r+","+f+","+m+","+c)
>>>   val y = x.repartitionAndSortWithinPartitions(new RFMCPartitioner(1))}
>>>
>>> I wanted to implement the same thing using clojure's DSL for spark 
>>> called flambo. Since I can't write partitioner using clojure, I re-used the 
>>> code defind above, compiled it and used it as a dependency in my Clojure 
>>> code.
>>>
>>> Now I am importing the partitioner and the key in my clojure code the 
>>> following way:
>>>
>>> (ns xyz
>>>   (:import
>>> [package RFMCPartitioner]
>>> [package RFMCKey]
>>> )
>>>   )
>>>
>>> But when I try to create RFMCKey by doing (RFMCKey. cust_id r f m c), 
>>> it throws the following error:
>>>
>>> java.lang.ClassCastException: org.formcept.wisdom.RFMCKey cannot be cast to 
>>> java.lang.Comparable
>>> at 
>>> org.spark-project.guava.collect.NaturalOrdering.compare(NaturalOrdering.java:28)
>>> at 
>>> scala.math.LowPriorityOrderingImplicits$$anon$7.compare(Ordering.scala:153)
>>> at 
>>> org.apache.spark.util.collection.ExternalSorter$$anon$8.compare(ExternalSorter.scala:170)
>>> at 
>>> org.apache.spark.util.collection.ExternalSorter$$anon$8.compare(ExternalSorter.scala:164)
>>> at 
>>> org.apache.spark.util.collection.TimSort.countRunAndMakeAscending(TimSort.java:252)
>>> at org.apache.spark.util.collection.TimSort.sort(TimSort.java:110)
>>> at org.apache.spark.util.collection.Sorter.sort(Sorter.scala:37)
>>> at 
>>> org.apache.spark.util.collection.SizeTrackingPairBuffer.destructiveSortedIterator(SizeTrackingPairBuffer.scala:83)
>>> at 
>>> org.apache.spark.util.collection.ExternalSorter.partitionedIterator(ExternalSorter.scala:687)
>>> at 
>>> org.apache.spark.util.collection.ExternalSorter.iterator(ExternalSorter.scala:705)
>>> at 
>>> org.apache.spark.shuffle.hash.HashShuffleReader.read(HashShuffleReader.scala:64)
>>> at org.apache.spark.rdd.ShuffledRDD.compute(ShuffledRDD.scala:92)
>>> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
>>> at org.apache.spark.CacheManager.getOrCompute(CacheManager.scala:70)
>>> at org.apache.spark.rdd.RDD.iterator(RDD.scala:242)
>>> at 
>>> org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:35)
>>> at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:277)
>>> at org.apache.spark.rdd.RDD.iterator(RDD.scala:244)
>>> at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
>>> at org.apache.spark.scheduler.Task.run(Task.scala:64)
>>> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:203)
>>> at 
>>> 

Re: Render Rails templates in Clojure?

[Blake Miller]

Pure ruby haml rendering ought to work in jruby, and you could call it from 
clojure... you would just need to inject values into the jruby scope, taking 
the place of a rails controller.

That seems a bit kludgey, but perhaps if you have tons of templates it might be 
worth it to avoid rewriting then from scratch.

I'd suggest trying to do some static transformation of the templates ... 
parsing the haml and emitting hiccup, turning references to ruby instance 
variables into clojure fn arguments. I think that'd be as easy as trying to 
marry jruby + clojure, and after that one time transformation you're left with 
something simpler.

If there's too much logic in the templates, it could get tricky...

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Recommendations for a schema-based data language for use in Hadoop?

[Blake Miller]

I suggest using prismatic`s schema library, and generating kryo serializers for 
your schematized records at compile time. These serializations can be very 
compact by leveraging the schemas, and kryo is very fast. I've been having 
success with this approach on Apache Spark. If you aren't married to using 
hadoop, and you want performance, l suggest you investigate spark as well.

I'm planning to extract this automatic schema-based kryo serializer macro junk 
and release a lib ... when I get around to it. I'd be glad to share the code if 
you want.

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] edn.el

[Blake Miller]

Lars, Bozhidar,

Thank you so much for the explanation, and for the excellent libs, I
appreciate it! I am using clj-refactor ... and it's rad. I will have to
read about the socketed REPL.

On Wed, Apr 15, 2015 at 10:37 AM, Lars Andersen ex...@expez.com wrote:

 In addition to what Bozhidar mentioned:

 I also work on https://github.com/clojure-emacs/clj-refactor.el which
 communicates with a backend,
 https://github.com/clojure-emacs/refactor-nrepl, which is written in
 clojure.  For now we've been limiting ourselves to data structures which
 are eaily readable in emacs lisp (strings, lists, association lists etc)
 but for more complex values I'd like to use edn and edn.el.

 I've also realized that when I need a 'client' for something, hacking
 together something in emacs is incredibly easy.  I often feel like I get
 more than the proverbial 80% when I invest 20% of my efforts on top of
 emacs :)  Take a look at this incredibly cool demo of a REST client written
 in emacs: http://emacsrocks.com/e15.html


 On Monday, April 13, 2015 at 11:25:20 PM UTC+2, Blake Miller wrote:

 Cool! May I ask what your motivation was for this?

 On Saturday, April 11, 2015 at 3:09:28 AM UTC-7, Lars Andersen wrote:

  https://github.com/expez/edn.el

 is a library for reading an writing edn from emacs lisp.

  --
 You received this message because you are subscribed to the Google
 Groups Clojure group.
 To post to this group, send email to clojure@googlegroups.com
 Note that posts from new members are moderated - please be patient with
 your first post.
 To unsubscribe from this group, send email to
 clojure+unsubscr...@googlegroups.com
 For more options, visit this group at
 http://groups.google.com/group/clojure?hl=en
 ---
 You received this message because you are subscribed to a topic in the
 Google Groups Clojure group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/clojure/Zob2v1pRQAA/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to
 clojure+unsubscr...@googlegroups.com.
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] edn.el

[Blake Miller]

Cool! May I ask what your motivation was for this?

On Saturday, April 11, 2015 at 3:09:28 AM UTC-7, Lars Andersen wrote:

  https://github.com/expez/edn.el 

 is a library for reading an writing edn from emacs lisp.



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Amazonica: Clojure client for the entire AWS api

[Blake Miller]

I know this is an old thread ... but FWIW I think this is awesome. Great 
work, Michael! I am going to play around with it.

On Thursday, March 28, 2013 10:31:00 PM UTC-7, Michael Cohen wrote:

 I ran a quick and dirty benchmark comparing Amazonica with James' rotary 
 library, which uses no explicit reflection. This was run from an EC2 
 instance in East, hitting a Dynamo table in the East region. tl;dr 
  Amazonica averaged 9ms for gets, rotary averaged 6ms, both averaged 13ms 
 for puts. Summary is at https://github.com/mcohen01/amazonica#performance. 
 Benchmark code is at 
 https://github.com/mcohen01/amazonica-benchmark/blob/master/test/benchmark/runner.clj
 . 

 It's pretty simplistic, but I just wanted to see if reflection just 
 completely turned the library into a dog. Seems the contrary, that any 
 reflection performance penalty is basically not even worth mentioning. 
 Maybe some folks who have better understanding of jvm internals can explain 
 if the test is invalid because of some sort of caching of the method 
 lookups or something. 


 On Wednesday, March 27, 2013 7:29:00 AM UTC-7, Herwig Hochleitner wrote:

 2013/3/26 Hugo Duncan dunca...@gmail.com


 Or can the cost be confined to compile time...


 That would be nice to have!
 Generating type-hinted clojure code from the reflection result and 
 emitting that with macros would be an option.

 I think the dynamic use of reflection would be enough to put me off
 using this in something like pallet, for example.


 I agree with Michael on this: Any reflection overhead should pale next to 
 the context switch and network communication, that AWS commands do.
 OTOH, I also agree that driving a generated java api via reflection, to 
 generate xml seems a bit heavy handed.
 Still, first priority should be to get the interface right.

 Regarding that: I think the first context argument should be mandatory.
 We are just saw clojure.java.jdbc painstakenly deprecate a lot of API, to 
 get rid of the dynamic *db* var.
 The reasons against passing context in a dynamic var go double against a 
 global atom: A function parameter can be set at from any data model in 
 every callsite. Everything that's less flexible constrains your users for 
 little gain (in the case of passing context).

 Also, my experience with ClojureQL showed me, that with multiple sources 
 of a context arg, it's hard to get the ordering right.
 E.g. the new implementation seems to prefer dynamically bound credentials 
 over credentials passed as argument.



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
Clojure group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Clojurescript, is it ready yet?

[Blake Miller]

This is somewhat related, though it's not exactly what the OP asked about.

This compiles with GWT:
https://github.com/blak3mill3r/percolator/blob/master/play/src/com/whatsys/test.clj

Also I wanted to point out that you could export a public interface
with the GWT compiler and call it with clojurescript. I'm not sure if
I understand your needs exactly ... not sure if that would help.

I'm picturing writing a bunch of UI components in GWT, describing the
UiBinder xml file as clojure forms, generating the actual xml with
clojure at compile time, and if you wanted to you could also have that
data structure available to your clojurescript client.

Cheers!

-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en

Re: Gwt-Clojure - lightweight browser implementation

[Blake Miller]

I realize this is an old thread ... and the blog link about the gwt-clojure 
project is dead.

I just wanted to mention that I'm working on something similar, with a 
somewhat different approach:

This one is a java metaprogramming toolkit written in clojure. It turns 
clojure forms into java source code which can be compiled with the GWT 
compiler.

It's not production-ready by any means, it is very experimental, but I'd be 
glad to have any feedback on the general concepts or the clojure code (this 
is my first clojure project, I'm probably missing a lot of good uses of the 
sequence libraries and such)

https://github.com/blak3mill3r/percolator/blob/master/play/src/com/whatsys/test.clj

On Thursday, June 17, 2010 7:08:20 AM UTC-7, lpetit wrote:

 Hi,

 Seems real interesting  !

 What I was not able to understand by reading your blog post, is which
 subset of clojure you ported to the client side ?

 Is it just a declarative API for the widgets part, or will it be
 possible, as with GWT java client side code, to embed logic,etc ?

 will it be possible to use atoms for managing state mutation, wiring
 fns as asynchronous callback functions , etc. ?


 2010/6/17 pfisk peter...@gmail.com javascript::
  I will release an open source version of Gwt-Clojure next week.
 
  My approach to building web applications is based on frame
  technology which has been used commercially in mainframe code
  generation for nearly 30 years. You build a library of data structures
  (frames) that describe an application and then send it to a code
  generator to write the application. I have used Clojure to build my
  frame engine. Gwt-Clojure was developed to make scripting easier on
  the client side. In summary, my approach is to build web apps 100% in
  Lisp.
 
 
  On Jun 16, 6:50 pm, Rick Moynihan rick.moyni...@gmail.com wrote:
  Neat!
 
  I'm currently writting a webapp with GWT for the browser client, and
  clojure on the server... What is your approach to doing this, and is
  the code available anywhere yet?
 
  R.
 
  On 15 June 2010 19:48, pfisk peter.f...@gmail.com wrote:
 
 
 
   Gwt-Clojure is a subset of the Clojure language which was developed
   for scripting GWT (Google Windows Toolkit) widgets in the browser
   environment. It is designed to be able to share code with Clojure
   running on the server.
 
   The current deployment size is about 145kb of Javascript - including
   the interpreter and several GWT widget classes.
 
   Test environment:http://wisperweb.appspot.com/
 
   Gwt-Clojure blog post:
 http://framegen.wordpress.com/2010/06/15/gwt-clojure/
 
   --
   You received this message because you are subscribed to the Google
   Groups Clojure group.
   To post to this group, send email to clo...@googlegroups.comjavascript:
   Note that posts from new members are moderated - please be patient 
 with your first post.
   To unsubscribe from this group, send email to
   clojure+u...@googlegroups.com javascript:
   For more options, visit this group at
  http://groups.google.com/group/clojure?hl=en
 
  --
  You received this message because you are subscribed to the Google
  Groups Clojure group.
  To post to this group, send email to clo...@googlegroups.comjavascript:
  Note that posts from new members are moderated - please be patient with 
 your first post.
  To unsubscribe from this group, send email to
  clojure+u...@googlegroups.com javascript:
  For more options, visit this group at
  http://groups.google.com/group/clojure?hl=en



-- 
You received this message because you are subscribed to the Google
Groups Clojure group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en