Hi, I'm very interested in writing a wrapper library around Apache Crunch for Clojure, something similar to existing Scrunch. How do you recommend to start?
I was looking through Crunch code and it looks like I can pretty easily integrate it in clojure by adding some custom WritableType type. Something like WritableType<Object, ByteWritable> with a custom converter or inputFn/outputFn functions. Regretfully there are several issues with this approach and instead I'd have to duplicate all those type classes for a new type set * WritableType has a package visible constructor so I cannot extend it and cannot instantiate it * Converter is instantiated inside WritableType constructor so in case I need a different converter I'm stuck * Writables has a factory method for WritableType but it's private * it looks like there is an attempt to support additional WritableTypes through EXTENSIONS in Writables but it would only work for cases where in WritableType<T, W> both T and W are hadoop writables So what do you think is a best solution, is it possible to open up the api to support custom WritableTypes or the only option for me is to implement a new ClojurePType and all related classes? Hope I'm not too detailed, but at this stage you all are probably very familiar with the code Thanks, Victor
