Let me know if anyone want to collaborate with me on this.
Thanks,
Chandni
On Tue, Nov 10, 2015 at 6:18 PM, Chandni Singh
wrote:
> Have added some more details about a Bucket in the document. Have a look.
>
> On Sun, Nov 8, 2015 at 10:37 PM, Chandni Singh
> wrote:
>
>> Forgot to attach the lin
The RC2 vote is concluded and passes. Thanks everyone for voting and
verifying the release.
It was nice to see the increase in participation and expanded test
activity.
binding +1 (14)
Pramod Immaneni
Bright Chen
David Yan
Sasha Parfenov
Amol Kekre
Isha Arkatkar
Vlad Rozov
Sandesh Hegde
Justin M
Github user PramodSSImmaneni commented on the pull request:
https://github.com/apache/incubator-apex-site/pull/7#issuecomment-156610023
Made some changes @chandnisingh, @vrozov please see
---
If your project is set up for it, you can reply to this email and have your
reply appear on G
Pramod,
Doing an ad-hoc checkpoint may be a possibility.
Amol
On Fri, Nov 13, 2015 at 9:57 AM, Pramod Immaneni
wrote:
> If checkpoint is a multiple of windows and end window tuples are already
> flowing and triggering end windows on the operators is there additional
> knowledge being gained by
Github user tweise commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/51#discussion_r44845177
--- Diff:
library/src/main/java/com/datatorrent/lib/join/AntiJoinOperator.java ---
@@ -0,0 +1,209 @@
+/**
+ * Licensed to the Apache Soft
You need one on line:164 as well.
‹
Chetan
On 11/13/15, 1:21 PM, "ilooner" wrote:
>Github user ilooner commented on the pull request:
>
>
>https://github.com/apache/incubator-apex-core/pull/167#issuecomment-156559
>640
>
>This is for review only @PramodSSImmaneni and @243826 Please t
Github user ilooner commented on the pull request:
https://github.com/apache/incubator-apex-core/pull/167#issuecomment-156559640
This is for review only @PramodSSImmaneni and @243826 Please this is a
small fix. I will work on writing a unit test in the mean time.
---
If your project
GitHub user ilooner opened a pull request:
https://github.com/apache/incubator-apex-core/pull/167
- APEX-263 Fixed case where double checkpointing could occurr.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ilooner/incubator-ap
GitHub user PramodSSImmaneni opened a pull request:
https://github.com/apache/incubator-apex-site/pull/7
Added support for multiple individuals contributing to a pull request
@chandnisingh review
You can merge this pull request into a Git repository by running:
$ git pull https
Hi Atri,
Picked a blocking queue to do a small prototype to check if it helps
in improvement. Blocking queue will handle these by default:
- Stop pushing data from publisher when queue is full. For example if
the downstream operator is slow, we need to suspend publisher for a while.
-
It is same process only till the buffer server thereafter the data is
destined to go to a separate process.
Thanks
On Fri, Nov 13, 2015 at 11:30 AM, Munagala Ramanath
wrote:
> If it is the same container, why bother serializing ?
>
> Ram
>
> On Fri, Nov 13, 2015 at 11:19 AM, Isha Arkatkar
> wr
Github user brightchen commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/82#discussion_r44825680
--- Diff:
contrib/src/main/java/com/datatorrent/contrib/hive/AbstractFSRollingOutputOperator.java
---
@@ -183,12 +184,7 @@ protected void ro
I can't see why it should be blocking.
On 14 Nov 2015 01:08, "Isha Arkatkar" wrote:
> Yes, this is only for inter-container optimization, so we need serialized
> tuples.
>
> The blocking queue is for implementing Publisher-consumer sort of thing,
> where publisher will push the data to the queue
Yes, this is only for inter-container optimization, so we need serialized
tuples.
The blocking queue is for implementing Publisher-consumer sort of thing,
where publisher will push the data to the queue, And on the buffer server
side, another thread loop will pick this data and put it in data lis
The serialization and buffer server is only used for inter-container
communication.
On Fri, Nov 13, 2015 at 11:30 AM, Munagala Ramanath
wrote:
> If it is the same container, why bother serializing ?
>
> Ram
>
> On Fri, Nov 13, 2015 at 11:19 AM, Isha Arkatkar
> wrote:
> > Hi all,
> >
> >For
I am assuming that's what the proposal has based on the initial email.
On 14 Nov 2015 01:00, "Thomas Weise" wrote:
> Yes, but why do we need another blocking queue?
>
> On Fri, Nov 13, 2015 at 11:21 AM, Atri Sharma wrote:
>
> > Apart from file I/O overhead saving , what else can you see that wil
If it is the same container, why bother serializing ?
Ram
On Fri, Nov 13, 2015 at 11:19 AM, Isha Arkatkar wrote:
> Hi all,
>
>For APEX-259 (https://malhar.atlassian.net/browse/APEX-259), I am
> exploring option of passing serialized tuples from publisher to buffer
> server through a blocking
Yes, but why do we need another blocking queue?
On Fri, Nov 13, 2015 at 11:21 AM, Atri Sharma wrote:
> Apart from file I/O overhead saving , what else can you see that will be
> saved?
>
> IMO a blocking queue might add an additional overhead and might tend to
> localise the bottleneck to a sing
Isha, this may be a very good optimization that we discussed a long time
ago.
Any idea about the potential gains?
On Fri, Nov 13, 2015 at 11:19 AM, Isha Arkatkar
wrote:
> Hi all,
>
>For APEX-259 (https://malhar.atlassian.net/browse/APEX-259), I am
> exploring option of passing serialized
Github user chandnisingh commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/51#discussion_r44823026
--- Diff:
library/src/test/java/com/datatorrent/lib/streamquery/AntiJoinOperatorTest.java
---
@@ -0,0 +1,120 @@
+/**
+ * Copyright
Github user chandnisingh commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/51#discussion_r44822719
--- Diff:
library/src/main/java/com/datatorrent/lib/streamquery/AntiJoinOperator.java ---
@@ -0,0 +1,205 @@
+/**
+ * Copyright (C)
Github user chandnisingh commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/51#discussion_r44822769
--- Diff:
library/src/main/java/com/datatorrent/lib/streamquery/AntiJoinOperator.java ---
@@ -0,0 +1,205 @@
+/**
+ * Copyright (C)
Github user chandnisingh commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/51#discussion_r44822807
--- Diff:
library/src/test/java/com/datatorrent/lib/streamquery/AntiJoinOperatorTest.java
---
@@ -0,0 +1,120 @@
+/**
+ * Copyright
Apart from file I/O overhead saving , what else can you see that will be
saved?
IMO a blocking queue might add an additional overhead and might tend to
localise the bottleneck to a single queue. Just my 2c...
On 14 Nov 2015 00:49, "Isha Arkatkar" wrote:
> Hi all,
>
>For APEX-259 (https://mal
Github user chandnisingh commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/51#discussion_r44822442
--- Diff:
library/src/main/java/com/datatorrent/lib/streamquery/AntiJoinOperator.java ---
@@ -0,0 +1,205 @@
+/**
+ * Copyright (C)
Hi all,
For APEX-259 (https://malhar.atlassian.net/browse/APEX-259), I am
exploring option of passing serialized tuples from publisher to buffer
server through a blocking queue.
Right now, publisher and buffer server reside within the same container,
however, communication between the two goes
Github user chandnisingh commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/51#discussion_r44821826
--- Diff:
library/src/main/java/com/datatorrent/lib/streamquery/AntiJoinOperator.java ---
@@ -0,0 +1,205 @@
+/**
+ * Copyright (C)
Github user chandnisingh commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/51#discussion_r44821809
--- Diff:
library/src/main/java/com/datatorrent/lib/streamquery/AntiJoinOperator.java ---
@@ -0,0 +1,205 @@
+/**
+ * Copyright (C)
Sandesh and I have created some slides outlining some more possible design
approaches along with their pros and cons.
https://docs.google.com/presentation/d/1-gWwwq4Dd7g9Mai7XLlzA7R_F9nMqEWg3IKMD3OMbYE/edit?usp=sharing
Please review and comment
Thanks,
Tim
On Wed, Nov 11, 2015 at 11:20 PM, Amol
If checkpoint is a multiple of windows and end window tuples are already
flowing and triggering end windows on the operators is there additional
knowledge being gained by a checkpoint tuple. I can see one advantage that
you can force a checkpoint throughout the system adhoc on a window if the
STRAM
Gaurav i think you replied to the wrong email :)
On Fri, Nov 13, 2015 at 6:52 AM, Gaurav Gupta
wrote:
> Michael,
>
> Here is what you should do..
> Create a new Serializer
>
> import java.io.IOException;
> import java.io.ObjectInputStream;
> import java.util.UUID;
>
> import com.esotericsoftware
There is an additional impact of using checkpoint tuple as opposed to each
StramChild simply checkpointing at pre-known windows. This is the knowledge
of checkpoint flow as per Chetan's #1. Stram will know that the checpoint
tuple has passed through all upstream operators. In non-blocking
checkpoin
Michael,
Here is what you should do..
Create a new Serializer
import java.io.IOException;
import java.io.ObjectInputStream;
import java.util.UUID;
import com.esotericsoftware.kryo.Kryo;
public class CustomeSerializableStreamCodec extends
com.datatorrent.lib.codec.KryoSerializableStreamCo
Michael,
Here is what you should do..
Create a new Serializer
import java.io.IOException;
import java.io.ObjectInputStream;
import java.util.UUID;
import com.esotericsoftware.kryo.Kryo;
public class CustomeSerializableStreamCodec extends
com.datatorrent.lib.codec.KryoSerializableStreamCodec
{
Thomas- Will take a look regarding release date.
Gaurav- I believe this is a browser cache issue
Justin- I'll add KEYS, instructions, and link to README
Thanks for the feedback!
-Andy
On Thursday, November 12, 2015, Justin Mclean wrote:
> Hi,
>
> You might want to place the KEYS file in the di
Hi Gaurav,
I've found this custom serializer:
https://github.com/magro/kryo-serializers/blob/master/src/main/java/de/javakaffee/kryoserializers/UUIDSerializer.java
Question is where would be the correct place with regard to application's
lifecycle to register this serializer.
Michael
On Fri, No
Michael,
I think Kryo is complaining because UUID type doesn’t have default constructor.
Thanks
- Gaurav
> On Nov 13, 2015, at 6:02 AM, Michael Ravits wrote:
>
> Hi,
>
> I'm experimenting with incubating apex 3.1.1.
> I've tried sending an object with a UUID type property and Kryo complained
Hi,
I'm experimenting with incubating apex 3.1.1.
I've tried sending an object with a UUID type property and Kryo complained
that there is no serializer for this type.
Where would be the most appropriate place to register the UUID serializer?
Thanks,
Michael
38 matches
Mail list logo
