[ANN] thurber: Clojure on Apache Beam (distributed batch/streaming)

2020-01-21 Thread atdixon
Here is thurber (https://github.com/atdixon/thurber) (at early alpha release) that enables Clojure on Apache Beam platforms like Google Dataflow. thurber's goals include: - Full support for Beam capabilities - AOT-less (AOT not required; full dynamic support for serializing func

Re: Inconsistent AOT classnotfoundexception

2019-09-12 Thread atdixon
Interesting! I had not seen clj-headlights but my org is using Beam + Clojure and we've made some similar decisions as clj-headlights. Specifically we are avoiding AOT and its headaches,. On Thursday, September 12, 2019 at 12:37:24 AM UTC-5, Kimmo Koskinen wrote: > > Hi! > > Not a direct answer,

Re: gen-class/AOT considered harmful (in libraries)???

2019-09-02 Thread atdixon
I was responding to this: > I'm still torn on whether to actually add Clojure as a proper dependency The question to have Clojure as a proper dependency doesn't seem to change whether you AOT or not. I take "proper dependency" to mean a Maven/Lein dependency that ships with the dependency list

Re: gen-class/AOT considered harmful (in libraries)???

2019-09-01 Thread atdixon
Hi, Dimitris - It looks from Clojure source [1] that Clojure compiles to v1.8 class files, so this should be mean you can run in any JVM 1.8 and beyond. When you say you are deliberating including Clojure as a proper dependency. I've noticed that some Clojure libraries will have Clojure listed

Re: Java Interop on steroids?

2019-07-02 Thread atdixon
I'm glad someone else is thinking on this too! #2 - For my case at the moment (Apache Beam), I believe we will always know the types in advance so using a Java class is workable but of course a (proxy++) would be ideal. Beam asks for us to extend abstract generic class so we must use (proxy). I

Re: Clojure is a good choice for Big Data? Which clojure/Hadoop work to use?

2019-07-02 Thread atdixon
I've found Clojure to be an excellent fit for big data processing for a few reasons: - the nature of big data is that it is often unstructured or semi-structured, and Clojure's immutable ad hoc map-based orientation is well suited to this - much of the big data ecosystem is Java or JVM-based (a

Re: Java Interop on steroids?

2019-06-22 Thread atdixon
> Do the framework you're talking about do static analysis of the types? Because generic types are erased at runtime, so there wouldn't ever be a way for proxy to set them in. They aren't entirely erased. They're erased from the code, but Java compilers are obligated to emit generic type signat

Re: Java Interop on steroids?

2019-06-22 Thread atdixon
ents would allow for this not uncommon Java interop need. On Saturday, June 22, 2019 at 5:38:36 AM UTC-5, Matching Socks wrote: > > By "generic type information", you mean the X in List ? > > > On Friday, June 21, 2019 at 12:03:46 AM UTC-4, atdixon wrote: >> >> Ho

Re: Java Interop on steroids?

2019-06-22 Thread atdixon
ents would allow for this not uncommon Java interop need. On Saturday, June 22, 2019 at 5:38:36 AM UTC-5, Matching Socks wrote: > > By "generic type information", you mean the X in List ? > > > On Friday, June 21, 2019 at 12:03:46 AM UTC-4, atdixon wrote: >> >> Ho