Re: [DISCUSS] Abstracting away Hadoop/MapReduce as Data Processing Layer

2014-07-09 Thread Henry Saputra
Internally, Apache Spark can use Hadoop input format for its distributed data structure (a.k.a RDD). So, I guess we could still join the cool kids with Spark via our input format implementation. However, I could think of other improvements that could be useful (apology to Lewis if I hijacked his d

Re: [DISCUSS] Abstracting away Hadoop/MapReduce as Data Processing Layer

2014-07-09 Thread Renato MarroquĂ­n Mogrovejo
2014-07-09 11:10 GMT+02:00 Henry Saputra : > Internally, Apache Spark can use Hadoop input format for its > distributed data structure (a.k.a RDD). > So, I guess we could still join the cool kids with Spark via our input > format implementation. > Cool Henry! I didn't know about we could use Hado

Re: [DISCUSS] Abstracting away Hadoop/MapReduce as Data Processing Layer

2014-07-09 Thread Lewis John Mcgibbney
Hi Henry, On Wed, Jul 9, 2014 at 5:10 AM, wrote: > > However, I could think of other improvements that could be useful > (apology to Lewis if I hijacked his discussion): > 1. Pluggable serialization mechanism to allow other like Thrift or > Protocol Buffer instead of just Avro. > 2. Directly wor

[jira] [Created] (GORA-356) Dead Overview Link on Quickstart Page

2014-07-09 Thread Tyler Palsulich (JIRA)
Tyler Palsulich created GORA-356: Summary: Dead Overview Link on Quickstart Page Key: GORA-356 URL: https://issues.apache.org/jira/browse/GORA-356 Project: Apache Gora Issue Type: Bug

[jira] [Commented] (GORA-354) Clarify lowercasing of DataStoreFactory#findProperty

2014-07-09 Thread Tyler Palsulich (JIRA)
[ https://issues.apache.org/jira/browse/GORA-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14056550#comment-14056550 ] Tyler Palsulich commented on GORA-354: -- I think a better solution than altering {{findP

[jira] [Commented] (GORA-354) Clarify lowercasing of DataStoreFactory#findProperty

2014-07-09 Thread Lewis John McGibbney (JIRA)
[ https://issues.apache.org/jira/browse/GORA-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14056643#comment-14056643 ] Lewis John McGibbney commented on GORA-354: --- Can you submit a PR [~tpalsulich]? I

[jira] [Commented] (GORA-354) Clarify lowercasing of DataStoreFactory#findProperty

2014-07-09 Thread Tyler Palsulich (JIRA)
[ https://issues.apache.org/jira/browse/GORA-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14056675#comment-14056675 ] Tyler Palsulich commented on GORA-354: -- Ah. Just read again. Do you want a PR instead o

[jira] [Updated] (GORA-354) Clarify lowercasing of DataStoreFactory#findProperty

2014-07-09 Thread Tyler Palsulich (JIRA)
[ https://issues.apache.org/jira/browse/GORA-354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Palsulich updated GORA-354: - Attachment: GORA-354.palsulich.070914.patch > Clarify lowercasing of DataStoreFactory#findProperty

[jira] [Comment Edited] (GORA-354) Clarify lowercasing of DataStoreFactory#findProperty

2014-07-09 Thread Tyler Palsulich (JIRA)
[ https://issues.apache.org/jira/browse/GORA-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14056674#comment-14056674 ] Tyler Palsulich edited comment on GORA-354 at 7/9/14 8:39 PM: --

[jira] [Updated] (GORA-354) Clarify lowercasing of DataStoreFactory#findProperty

2014-07-09 Thread Tyler Palsulich (JIRA)
[ https://issues.apache.org/jira/browse/GORA-354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Palsulich updated GORA-354: - Attachment: GORA-354.unit-test.patch Failing unit test that, oddly, doesn't fail the whole suite. J

Re: [DISCUSS] Abstracting away Hadoop/MapReduce as Data Processing Layer

2014-07-09 Thread Henry Saputra
For pluggable serialization, I think if there is not JIRA opened I could open one as reccomended by Lewis. As for low hanging fruit, I am currently not sure. Maybe we could add Gora store manager to Spark to allow read and persist from different NoSQL databases. - Henry On Wed, Jul 9, 2014 at 2:

Re: [DISCUSS] Abstracting away Hadoop/MapReduce as Data Processing Layer

2014-07-09 Thread Henry Saputra
HI Lewis, yeah I could file issues so we could track them for next releases Thanks for starting discussion about new innovation with Gora. +1 - Henry On Wed, Jul 9, 2014 at 3:51 AM, Lewis John Mcgibbney wrote: > Hi Henry, > > On Wed, Jul 9, 2014 at 5:10 AM, wrote: > >> >> However, I could thi

[jira] [Commented] (GORA-354) Clarify lowercasing of DataStoreFactory#findProperty

2014-07-09 Thread Lewis John McGibbney (JIRA)
[ https://issues.apache.org/jira/browse/GORA-354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14057182#comment-14057182 ] Lewis John McGibbney commented on GORA-354: --- Hi [~tpalsulich]... I don't quite get