[jira] [Commented] (DATAFU-114) Make FirstTupleFromBag implement Accumulator
[ https://issues.apache.org/jira/browse/DATAFU-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150679#comment-15150679 ] Matthew Hayes commented on DATAFU-114: -- Okay thanks for confirming, I've updated the README: https://github.com/apache/incubator-datafu#generating-eclipse-files > Make FirstTupleFromBag implement Accumulator > > > Key: DATAFU-114 > URL: https://issues.apache.org/jira/browse/DATAFU-114 > Project: DataFu > Issue Type: Improvement >Affects Versions: 1.3.0 > Environment: All >Reporter: Eyal Allweil >Assignee: Eyal Allweil >Priority: Minor > Labels: easyfix, newbie, performance > Fix For: 1.3.1 > > Attachments: FirstTupleFromBag.java > > > FirstTupleFromBag only needs the first tuple from the bag, but because it > doesn't implement Accumulator the entire bag needs to be passed to it > in-memory. The fix is very minor and will make the UDF support large bags. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DATAFU-114) Make FirstTupleFromBag implement Accumulator
[ https://issues.apache.org/jira/browse/DATAFU-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150312#comment-15150312 ] Eyal Allweil commented on DATAFU-114: - Thanks! After I imported the projects individually, like you suggested, it works fine in Eclipse ... I suggest adding a sentence about it in the base readme file to help out future contributors > Make FirstTupleFromBag implement Accumulator > > > Key: DATAFU-114 > URL: https://issues.apache.org/jira/browse/DATAFU-114 > Project: DataFu > Issue Type: Improvement >Affects Versions: 1.3.0 > Environment: All >Reporter: Eyal Allweil >Assignee: Eyal Allweil >Priority: Minor > Labels: easyfix, newbie, performance > Fix For: 1.3.1 > > Attachments: FirstTupleFromBag.java > > > FirstTupleFromBag only needs the first tuple from the bag, but because it > doesn't implement Accumulator the entire bag needs to be passed to it > in-memory. The fix is very minor and will make the UDF support large bags. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DATAFU-114) Make FirstTupleFromBag implement Accumulator
[ https://issues.apache.org/jira/browse/DATAFU-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149161#comment-15149161 ] Matthew Hayes commented on DATAFU-114: -- Code has been merged to master with commits f7c9b232fe1f7a9a2c445f7c25bb73a3c6a099c1 and a15a15c58164cc39c62235c0da6a7b1d8d7db9e2. > Make FirstTupleFromBag implement Accumulator > > > Key: DATAFU-114 > URL: https://issues.apache.org/jira/browse/DATAFU-114 > Project: DataFu > Issue Type: Improvement >Affects Versions: 1.3.0 > Environment: All >Reporter: Eyal Allweil >Assignee: Eyal Allweil >Priority: Minor > Labels: easyfix, newbie, performance > Fix For: 1.3.1 > > Attachments: FirstTupleFromBag.java > > > FirstTupleFromBag only needs the first tuple from the bag, but because it > doesn't implement Accumulator the entire bag needs to be passed to it > in-memory. The fix is very minor and will make the UDF support large bags. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DATAFU-114) Make FirstTupleFromBag implement Accumulator
[ https://issues.apache.org/jira/browse/DATAFU-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15149120#comment-15149120 ] Matthew Hayes commented on DATAFU-114: -- [~eyal], are you adding the project from the root of the DataFu repo or are you adding each sub-project individually? You need to add each each project separately: build-plugin, datafu-pig, etc. Is this your issue? I don't think this was documented. Also if you follow the instructions at https://cwiki.apache.org/confluence/display/DATAFU/Contributing+to+Apache+DataFu you should be able to generate a patch that RB accepts. This is what I do. > Make FirstTupleFromBag implement Accumulator > > > Key: DATAFU-114 > URL: https://issues.apache.org/jira/browse/DATAFU-114 > Project: DataFu > Issue Type: Improvement >Affects Versions: 1.3.0 > Environment: All >Reporter: Eyal Allweil >Assignee: Eyal Allweil >Priority: Minor > Labels: easyfix, newbie, performance > Fix For: 1.3.1 > > Attachments: FirstTupleFromBag.java > > > FirstTupleFromBag only needs the first tuple from the bag, but because it > doesn't implement Accumulator the entire bag needs to be passed to it > in-memory. The fix is very minor and will make the UDF support large bags. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DATAFU-114) Make FirstTupleFromBag implement Accumulator
[ https://issues.apache.org/jira/browse/DATAFU-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15135730#comment-15135730 ] Eyal Allweil commented on DATAFU-114: - The test looks fine, and so does your patch for DATAFU-95 - I was able to build and test (after adding the test to BagTests.java). What I still can't do is get an Eclipse project working - the gradlew completes, but the project which results doesn't have source folders or dependencies. In the past I had trouble generating patches from Git which RB accepted, but maybe that's been taken care of. Thanks! > Make FirstTupleFromBag implement Accumulator > > > Key: DATAFU-114 > URL: https://issues.apache.org/jira/browse/DATAFU-114 > Project: DataFu > Issue Type: Improvement >Affects Versions: 1.3.0 > Environment: All >Reporter: Eyal Allweil >Assignee: Eyal Allweil >Priority: Minor > Labels: easyfix, newbie, performance > Fix For: 1.3.1 > > Attachments: FirstTupleFromBag.java > > > FirstTupleFromBag only needs the first tuple from the bag, but because it > doesn't implement Accumulator the entire bag needs to be passed to it > in-memory. The fix is very minor and will make the UDF support large bags. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DATAFU-114) Make FirstTupleFromBag implement Accumulator
[ https://issues.apache.org/jira/browse/DATAFU-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15133456#comment-15133456 ] Matthew Hayes commented on DATAFU-114: -- Try the patch in DATAFU-95 I just attached and see if that helps your build issue. It should work with JDK 1.7 now. By the way, I've had good success switching Java versions using jenv (https://github.com/gcuisinier/jenv). Haven't tried it on Ubuntu but it seems like it would work. Also in the future a patch+RB makes it easier to review and apply the changes. > Make FirstTupleFromBag implement Accumulator > > > Key: DATAFU-114 > URL: https://issues.apache.org/jira/browse/DATAFU-114 > Project: DataFu > Issue Type: Improvement >Affects Versions: 1.3.0 > Environment: All >Reporter: Eyal Allweil >Priority: Minor > Labels: easyfix, newbie, performance > Fix For: 1.3.1 > > Attachments: FirstTupleFromBag.java > > > FirstTupleFromBag only needs the first tuple from the bag, but because it > doesn't implement Accumulator the entire bag needs to be passed to it > in-memory. The fix is very minor and will make the UDF support large bags. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DATAFU-114) Make FirstTupleFromBag implement Accumulator
[
https://issues.apache.org/jira/browse/DATAFU-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15133366#comment-15133366
]
Matthew Hayes commented on DATAFU-114:
--
Sorry for the late response. The change looks reasonable to me. There should
be a test for this though (understandable there isn't one since you couldn't
build it). I went ahead and wrote one below. If this test looks reasonable to
you I'll commit both pieces of code. I'm taking a look at DATAFU-95.
{code}
@Test
public void firstTupleFromBagAccumulateTest() throws Exception
{
TupleFactory tf = TupleFactory.getInstance();
BagFactory bf = BagFactory.getInstance();
FirstTupleFromBag op = new FirstTupleFromBag();
Tuple defaultValue = tf.newTuple(1000);
op.accumulate(tf.newTuple(Arrays.asList(bf.newDefaultBag(Arrays.asList(tf.newTuple(4))),
defaultValue)));
op.accumulate(tf.newTuple(Arrays.asList(bf.newDefaultBag(Arrays.asList(tf.newTuple(9))),
defaultValue)));
op.accumulate(tf.newTuple(Arrays.asList(bf.newDefaultBag(Arrays.asList(tf.newTuple(16))),
defaultValue)));
assertEquals(op.getValue(), tf.newTuple(4));
op.cleanup();
op.accumulate(tf.newTuple(Arrays.asList(bf.newDefaultBag(Arrays.asList(tf.newTuple(11))),
defaultValue)));
op.accumulate(tf.newTuple(Arrays.asList(bf.newDefaultBag(Arrays.asList(tf.newTuple(17))),
defaultValue)));
op.accumulate(tf.newTuple(Arrays.asList(bf.newDefaultBag(Arrays.asList(tf.newTuple(5))),
defaultValue)));
assertEquals(op.getValue(), tf.newTuple(11));
op.cleanup();
op.accumulate(tf.newTuple(Arrays.asList(bf.newDefaultBag(), defaultValue)));
assertEquals(op.getValue(), defaultValue);
op.cleanup();
}
{code}
> Make FirstTupleFromBag implement Accumulator
>
>
> Key: DATAFU-114
> URL: https://issues.apache.org/jira/browse/DATAFU-114
> Project: DataFu
> Issue Type: Improvement
>Affects Versions: 1.3.0
> Environment: All
>Reporter: Eyal Allweil
>Priority: Minor
> Labels: easyfix, newbie, performance
> Attachments: FirstTupleFromBag.java
>
>
> FirstTupleFromBag only needs the first tuple from the bag, but because it
> doesn't implement Accumulator the entire bag needs to be passed to it
> in-memory. The fix is very minor and will make the UDF support large bags.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DATAFU-114) Make FirstTupleFromBag implement Accumulator
[ https://issues.apache.org/jira/browse/DATAFU-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131991#comment-15131991 ] Eyal Allweil commented on DATAFU-114: - Anyone? > Make FirstTupleFromBag implement Accumulator > > > Key: DATAFU-114 > URL: https://issues.apache.org/jira/browse/DATAFU-114 > Project: DataFu > Issue Type: Improvement >Affects Versions: 1.3.0 > Environment: All >Reporter: Eyal Allweil >Priority: Minor > Labels: easyfix, newbie, performance > Attachments: FirstTupleFromBag.java > > > FirstTupleFromBag only needs the first tuple from the bag, but because it > doesn't implement Accumulator the entire bag needs to be passed to it > in-memory. The fix is very minor and will make the UDF support large bags. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DATAFU-114) Make FirstTupleFromBag implement Accumulator
[ https://issues.apache.org/jira/browse/DATAFU-114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15114990#comment-15114990 ] Eyal Allweil commented on DATAFU-114: - Any comments? Can this patch be pulled? > Make FirstTupleFromBag implement Accumulator > > > Key: DATAFU-114 > URL: https://issues.apache.org/jira/browse/DATAFU-114 > Project: DataFu > Issue Type: Improvement >Affects Versions: 1.3.0 > Environment: All >Reporter: Eyal Allweil >Priority: Minor > Labels: easyfix, newbie, performance > Attachments: FirstTupleFromBag.java > > > FirstTupleFromBag only needs the first tuple from the bag, but because it > doesn't implement Accumulator the entire bag needs to be passed to it > in-memory. The fix is very minor and will make the UDF support large bags. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
