Re: Monday Night in Maryland

2015-04-27 Thread David Medinets
I'm in the lobby across from the Commons eatery. On Mon, Apr 27, 2015 at 9:17 AM, Geoffry Roberts threadedb...@gmail.com wrote: David, I'll see you there. On Fri, Apr 24, 2015 at 10:35 PM, David Medinets david.medin...@gmail.com wrote: I'll be staying at the Marriott on Monday night. If

Re: Monday Night in Maryland

2015-04-27 Thread Geoffry Roberts
David, I'll see you there. On Fri, Apr 24, 2015 at 10:35 PM, David Medinets david.medin...@gmail.com wrote: I'll be staying at the Marriott on Monday night. If anyone else will be there as well, let's meet at the bar or find a place for dinner. -- There are ways and there are ways,

Fwd: OpenSource Accumulo Tool Licensing Help

2015-04-27 Thread Andrew Wells
This should have been sent to users... I have been working on a tool, that I am calling Q4A. However, due to my upcoming wedding, I have not had much time to work on it. I was hoping during the Accumulo Hackathon this week, I could actually get some more elbow grease into it and maybe get some

Q4A Project

2015-04-27 Thread Andrew Wells
I have been working on a project, tentatively called Q4A (Query for Accumulo). Another possible name is ASQ (Accumulo Streaming Query) [discus]. This is a streaming query as the query is completed via a stream, should never group data in memory. To batch, intermediate results would be written

Re: Q4A Project

2015-04-27 Thread Corey Nolet
Andrew, Have you considered leveraging existing SQL query layers like Hive or Spark's SQL/DataFrames API? There are some pretty massive optimizations involved in that API making the push-down predicates / selections pretty easy to adapt for Accumulo. On Mon, Apr 27, 2015 at 8:37 PM, Andrew Wells

Re: Q4A Project

2015-04-27 Thread Corey Nolet
I'm always looking for places to help out and integrate/share designs ideas. I look forward to chatting with you about Q4A at the hackathon tomorrow! Have you, by chance, seen the Spark SQL adapter for the Accumulo Recipes Event Entity Stores [1]? At the very least, it's a good example of using

Re: Q4A Project

2015-04-27 Thread Mike Drob
Andrew, This is a cool thing to work on, I hope you have great success! A couple of questions about the motivations behind this, if you don't mind - - There are several SQL implementations already in the Hadoop ecosystem. In what ways do you expect this to improve upon

Re: OpenSource Accumulo Tool Licensing Help

2015-04-27 Thread Andrew Wells
Thanks, I will look into the ASL. Thanks. On Mon, Apr 27, 2015 at 7:37 PM, Josh Elser josh.el...@gmail.com wrote: Disclaimer: IANAL and you should always do your own research before applying software licenses :) The Apache Software License has no bearing on the workings of the ASF.

Re: Custom Iterator output

2015-04-27 Thread vaibhav thapliyal
Thanks Dylan for the help. It helped me a lot. On 18-Apr-2015 10:43 am, Dylan Hutchison dhutc...@mit.edu wrote: Hi Vaibhav, It sounds like you want to emit a single value that is a function of all the entries in the parent iterator. In that case, the following template should solve your

Re: Approach to hold the output of an iterator in memory to do further operations

2015-04-27 Thread Mike Drob
Check out the MinCombiner https://github.com/apache/accumulo/blob/master/core/src/main/java/org/apache/accumulo/core/iterators/user/MinCombiner.java On Mon, Apr 27, 2015 at 12:19 PM, vaibhav thapliyal vaibhav.thapliyal...@gmail.com wrote: Hello everyone. I am trying to carry out max and min

Approach to hold the output of an iterator in memory to do further operations

2015-04-27 Thread vaibhav thapliyal
Hello everyone. I am trying to carry out max and min kind of operations using accumulo. But since the Accumulo iterators only operate on the entries that are lovally hosted I get the local max and local min of the instead of a global max and min. To get this global max and min, I have to

Re: Approach to hold the output of an iterator in memory to do further operations

2015-04-27 Thread Josh Elser
No, there isn't a way to do this that doesn't involve writing it to disk. vaibhav thapliyal wrote: I want to ask if there if a way to store this in memory so as to avoid writing it to a table? Is there a reason for wanting to avoid this (small) amount of work client-side? My experience has

Serialization error

2015-04-27 Thread madhvi
Hi, While connecting to accumulo through spark by making sparkRDD I am getting the following error: object not serializable (class: org.apache.accumulo.core.data.Key) This is due to the 'key' class of accumulo which does not implement serializable interface.How it can be solved and accumulo