[
https://issues.apache.org/jira/browse/APEXCORE-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074751#comment-15074751
]
ASF GitHub Bot commented on APEXCORE-276:
-
Github user chandnisingh commented on
Github user chandnisingh commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/193#discussion_r48590583
--- Diff: api/src/main/java/com/datatorrent/api/Context.java ---
@@ -335,10 +335,16 @@
Attribute APPLICATION_DATA_LINK = new Attribu
Also, since we are not creating separate operators for insert and
update/merge, it seems we don't need an abstract class. We can directly
modify JdbcPOJOOutputOperator.
-Bhupesh
On Wed, Dec 30, 2015 at 11:17 AM, Bhupesh Chawda
wrote:
> Also, a note regarding accepting fieldInfo objects:
>
> Fie
[
https://issues.apache.org/jira/browse/APEXCORE-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074721#comment-15074721
]
ASF GitHub Bot commented on APEXCORE-272:
-
Github user tweise commented on a dif
Github user tweise commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/191#discussion_r48589935
--- Diff:
engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java ---
@@ -1359,7 +1409,8 @@ private void addDAGToCurrentDAG(Modul
Github user tweise commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/191#discussion_r48589840
--- Diff:
engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java ---
@@ -907,6 +912,51 @@ else if (field.getType() == float.clas
[
https://issues.apache.org/jira/browse/APEXCORE-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074716#comment-15074716
]
ASF GitHub Bot commented on APEXCORE-272:
-
Github user tweise commented on a dif
Also, a note regarding accepting fieldInfo objects:
FieldInfo has fields for java type of the column, but not the SQL data type
which is needed to set the parameters in the SQL statement. This still
needs to be derived from the database (as in insert).
In more complex scenarios, as mentioned earli
Chandni: Good point on ordering.
We need to handle the case. Even the ordering is not guaranteed when
upstream operator has multiple instances.
Regards,
Sandeep
On Tue, Dec 29, 2015 at 8:39 PM, Chandni Singh
wrote:
> I agree with Sandeep that at-least once with databases is not the right
> app
GitHub user PramodSSImmaneni opened a pull request:
https://github.com/apache/incubator-apex-malhar/pull/150
MLHR-1959 Set offset manager properties from property files
Added getter for offset manager so that its properties can be set from
property files
You can merge this pull req
[
https://issues.apache.org/jira/browse/APEXCORE-201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074534#comment-15074534
]
ASF GitHub Bot commented on APEXCORE-201:
-
Github user davidyan74 commented on t
[
https://issues.apache.org/jira/browse/APEXCORE-201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074532#comment-15074532
]
ASF GitHub Bot commented on APEXCORE-201:
-
GitHub user davidyan74 opened a pull
Github user davidyan74 commented on the pull request:
https://github.com/apache/incubator-apex-core/pull/194#issuecomment-167915523
@tweise please review
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project do
GitHub user davidyan74 opened a pull request:
https://github.com/apache/incubator-apex-core/pull/194
APEXCORE-201 changed the way latency is calculated and fixed the probâ¦
â¦lem when latency is stalled when an operator falls behind too many
windows
You can merge this pull reques
Github user PramodSSImmaneni commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/132#discussion_r48581225
--- Diff:
contrib/src/main/java/com/datatorrent/contrib/kafka/AbstractKafkaInputOperator.java
---
@@ -392,9 +412,20 @@ public void e
Github user PramodSSImmaneni commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/132#discussion_r48581219
--- Diff:
contrib/src/main/java/com/datatorrent/contrib/kafka/AbstractKafkaInputOperator.java
---
@@ -392,9 +412,20 @@ public void e
[
https://issues.apache.org/jira/browse/APEXCORE-232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sasha Parfenov updated APEXCORE-232:
Labels: roadmap (was: )
> Ability to add new processing code to the DAG
> ---
[
https://issues.apache.org/jira/browse/APEXCORE-233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sasha Parfenov updated APEXCORE-233:
Labels: roadmap (was: )
> Ability to specify single instance objects in configuration
> -
[
https://issues.apache.org/jira/browse/APEXCORE-163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sasha Parfenov updated APEXCORE-163:
Labels: roadmap (was: )
> Dynamic application property changes
>
[
https://issues.apache.org/jira/browse/APEXCORE-231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sasha Parfenov updated APEXCORE-231:
Labels: roadmap (was: )
> Ability to configure attributes dynamically
> -
[
https://issues.apache.org/jira/browse/APEXCORE-295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sasha Parfenov updated APEXCORE-295:
Labels: roadmap (was: )
> Running a Storm topology on Apex.
> ---
[
https://issues.apache.org/jira/browse/APEXCORE-234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sasha Parfenov updated APEXCORE-234:
Labels: roadmap (was: )
> Investigate other ways to specify properties in property files
[
https://issues.apache.org/jira/browse/APEXCORE-119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sasha Parfenov updated APEXCORE-119:
Labels: roadmap (was: )
> Add Support For A New Type Of (Distributed) Operator
>
[
https://issues.apache.org/jira/browse/APEXCORE-202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sasha Parfenov updated APEXCORE-202:
Labels: roadmap (was: )
> Integration with Samoa
> --
>
>
[
https://issues.apache.org/jira/browse/APEXCORE-293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sasha Parfenov updated APEXCORE-293:
Labels: roadmap (was: )
> Add core and malhar documentation to project web site
> ---
[
https://issues.apache.org/jira/browse/APEXCORE-10?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sasha Parfenov updated APEXCORE-10:
---
Labels: roadmap (was: )
> Enable non-affinity of operators per node (not containers)
> -
[
https://issues.apache.org/jira/browse/APEXCORE-235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sasha Parfenov updated APEXCORE-235:
Labels: roadmap (was: )
> Explicit support for batch processing
> ---
[
https://issues.apache.org/jira/browse/APEXCORE-289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sasha Parfenov updated APEXCORE-289:
Labels: roadmap (was: )
> Encrypted Streams in Apex DAG
> -
>
[
https://issues.apache.org/jira/browse/APEXCORE-60?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sasha Parfenov updated APEXCORE-60:
---
Labels: roadmap (was: )
> Iterative processing support
>
>
>
[
https://issues.apache.org/jira/browse/APEXCORE-292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074442#comment-15074442
]
Sasha Parfenov commented on APEXCORE-292:
-
That works. I will go through and up
[
https://issues.apache.org/jira/browse/APEXCORE-3?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sasha Parfenov updated APEXCORE-3:
--
Labels: roadmap (was: )
> Ability for an operator to populate DAG at launch time
>
[
https://issues.apache.org/jira/browse/APEXCORE-292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074436#comment-15074436
]
Thomas Weise commented on APEXCORE-292:
---
Issues that are scheduled for future vers
[
https://issues.apache.org/jira/browse/APEXCORE-267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074367#comment-15074367
]
David Yan commented on APEXCORE-267:
My bad. Those messages were intended to be for
[
https://issues.apache.org/jira/browse/APEXCORE-191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074365#comment-15074365
]
David Yan commented on APEXCORE-191:
Not sure what can be done here since this is a
[
https://issues.apache.org/jira/browse/APEXCORE-276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074356#comment-15074356
]
ASF GitHub Bot commented on APEXCORE-276:
-
Github user davidyan74 commented on t
Github user davidyan74 commented on the pull request:
https://github.com/apache/incubator-apex-core/pull/193#issuecomment-167889209
@chandnisingh please review
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your proj
[
https://issues.apache.org/jira/browse/APEXCORE-267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074353#comment-15074353
]
ASF GitHub Bot commented on APEXCORE-267:
-
GitHub user davidyan74 opened a pull
GitHub user davidyan74 opened a pull request:
https://github.com/apache/incubator-apex-core/pull/193
APEXCORE-267 made METRICS_TRANSPORT pluggable
@tweise please review and merge
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dav
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-apex-malhar/pull/137
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
[
https://issues.apache.org/jira/browse/APEXCORE-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074326#comment-15074326
]
ASF GitHub Bot commented on APEXCORE-272:
-
Github user gauravgopi123 commented o
Github user gauravgopi123 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/191#discussion_r48571279
--- Diff:
engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java ---
@@ -1359,7 +1409,8 @@ private void addDAGToCurrentDA
[
https://issues.apache.org/jira/browse/APEXCORE-292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074310#comment-15074310
]
Sasha Parfenov commented on APEXCORE-292:
-
Do we want to see a summary or the co
[
https://issues.apache.org/jira/browse/APEXCORE-292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sasha Parfenov updated APEXCORE-292:
Attachment: screenshot-2 full list.png
> Add roadmap to web site
> ---
[
https://issues.apache.org/jira/browse/APEXCORE-292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sasha Parfenov updated APEXCORE-292:
Attachment: screenshot-1.png
> Add roadmap to web site
> ---
>
>
[
https://issues.apache.org/jira/browse/APEXCORE-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074303#comment-15074303
]
ASF GitHub Bot commented on APEXCORE-272:
-
Github user gauravgopi123 commented o
Github user gauravgopi123 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/191#discussion_r48570005
--- Diff:
engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java ---
@@ -1374,7 +1425,8 @@ private void addDAGToCurrentDA
Github user gauravgopi123 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/191#discussion_r48569555
--- Diff:
engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java ---
@@ -907,6 +912,51 @@ else if (field.getType() == flo
[
https://issues.apache.org/jira/browse/APEXCORE-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074294#comment-15074294
]
ASF GitHub Bot commented on APEXCORE-272:
-
Github user gauravgopi123 commented o
[
https://issues.apache.org/jira/browse/APEXCORE-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074275#comment-15074275
]
ASF GitHub Bot commented on APEXCORE-272:
-
Github user gauravgopi123 commented o
Github user gauravgopi123 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/191#discussion_r48569051
--- Diff:
engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java ---
@@ -907,6 +912,51 @@ else if (field.getType() == flo
Github user gauravgopi123 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/191#discussion_r48569034
--- Diff:
engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java ---
@@ -907,6 +912,51 @@ else if (field.getType() == flo
[
https://issues.apache.org/jira/browse/APEXCORE-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074274#comment-15074274
]
ASF GitHub Bot commented on APEXCORE-272:
-
Github user gauravgopi123 commented o
[
https://issues.apache.org/jira/browse/APEXCORE-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074273#comment-15074273
]
ASF GitHub Bot commented on APEXCORE-272:
-
Github user gauravgopi123 commented o
Github user gauravgopi123 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/191#discussion_r48568967
--- Diff:
engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java ---
@@ -907,6 +912,51 @@ else if (field.getType() == flo
[
https://issues.apache.org/jira/browse/APEXCORE-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074271#comment-15074271
]
ASF GitHub Bot commented on APEXCORE-272:
-
Github user gauravgopi123 commented o
Github user gauravgopi123 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/191#discussion_r48568829
--- Diff:
engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java ---
@@ -719,6 +719,11 @@ public void
resetStreamPersist
Github user gauravgopi123 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/125#discussion_r48564053
--- Diff: contrib/pom.xml ---
@@ -623,5 +628,11 @@
super-csv-joda
2.3.1
+
+ com.gemstone
Github user ashishtadose commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/125#discussion_r48563172
--- Diff: contrib/pom.xml ---
@@ -623,5 +628,12 @@
super-csv-joda
2.3.1
+
+ com.gemstone.
Github user gauravgopi123 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/125#discussion_r48560143
--- Diff: contrib/pom.xml ---
@@ -623,5 +628,12 @@
super-csv-joda
2.3.1
+
+ com.gemstone
Github user gauravgopi123 commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/125#discussion_r48559989
--- Diff: contrib/pom.xml ---
@@ -623,5 +628,12 @@
super-csv-joda
2.3.1
+
+ com.gemstone
Github user siyuanh commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/148#discussion_r48557887
--- Diff:
contrib/src/main/java/com/datatorrent/contrib/kafka/POJOKafkaOutputOperator.java
---
@@ -0,0 +1,272 @@
+/**
+ * Licensed to
Github user siyuanh commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/148#discussion_r48557730
--- Diff:
contrib/src/main/java/com/datatorrent/contrib/kafka/POJOKafkaOutputOperator.java
---
@@ -0,0 +1,272 @@
+/**
+ * Licensed to
Github user siyuanh commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/148#discussion_r48557364
--- Diff:
contrib/src/main/java/com/datatorrent/contrib/kafka/POJOKafkaOutputOperator.java
---
@@ -0,0 +1,272 @@
+/**
+ * Licensed to
Github user siyuanh commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/148#discussion_r48557321
--- Diff:
contrib/src/main/java/com/datatorrent/contrib/kafka/POJOKafkaOutputOperator.java
---
@@ -0,0 +1,272 @@
+/**
+ * Licensed to
Github user siyuanh commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/148#discussion_r48557083
--- Diff:
contrib/src/main/java/com/datatorrent/contrib/kafka/POJOKafkaOutputOperator.java
---
@@ -0,0 +1,272 @@
+/**
+ * Licensed to
Github user siyuanh commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/132#discussion_r48555633
--- Diff:
contrib/src/main/java/com/datatorrent/contrib/kafka/AbstractKafkaInputOperator.java
---
@@ -392,9 +412,20 @@ public void emitTuples
Github user siyuanh commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/132#discussion_r4899
--- Diff:
contrib/src/main/java/com/datatorrent/contrib/kafka/AbstractKafkaInputOperator.java
---
@@ -392,9 +412,20 @@ public void emitTuples
[
https://issues.apache.org/jira/browse/APEXCORE-272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074077#comment-15074077
]
ASF GitHub Bot commented on APEXCORE-272:
-
Github user vrozov commented on a dif
Github user vrozov commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/191#discussion_r48554311
--- Diff:
engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java ---
@@ -1359,7 +1409,8 @@ private void addDAGToCurrentDAG(Modul
Github user PramodSSImmaneni commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/132#discussion_r48548257
--- Diff:
contrib/src/main/java/com/datatorrent/contrib/kafka/AbstractKafkaInputOperator.java
---
@@ -392,9 +412,20 @@ public void e
Github user PramodSSImmaneni commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/132#discussion_r48548217
--- Diff:
contrib/src/main/java/com/datatorrent/contrib/kafka/AbstractKafkaInputOperator.java
---
@@ -392,9 +412,20 @@ public void e
I agree with Sandeep that at-least once with databases is not the right
approach. All the output adaptors we have in the library are written so
that they do not write duplicate entries. For example FileOutputOperator.
This has nothing to do with the processing mode feature offered by the
platform.
One more option:
We can keep track of windowId & batchId i.e. we save batchId with windowId
when we commit a batch within a window. e.g. we are in window 10 and we
have written second batch in window 10, we commit windowId=10 and batchId=2
to DB. While recovery we won't process batches within last
Not sure if "At least once" is right behavior for databases. We may not
always have primary key to update or insert.
Regards,
Sandeep
On Tue, Dec 29, 2015 at 2:23 PM, Priyanka Gugale wrote:
> Hi,
>
> Thanks for your inputs Chandni. I guess what you are suggesting is similar
> to AbstractJdbcNo
Github user chinmaykolhatkar commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/149#discussion_r48531567
--- Diff:
library/src/main/java/com/datatorrent/lib/io/fs/AbstractReconciler.java ---
@@ -73,9 +73,9 @@ public void process(INPUT inp
GitHub user yogidevendra opened a pull request:
https://github.com/apache/incubator-apex-malhar/pull/149
MLHR-1958: Exposing some fields from AbstractReconciler as protected
1. Making queues as protected for access in derived classes.
You can merge this pull request into a Git repos
Hi,
Thanks for your inputs Chandni. I guess what you are suggesting is similar
to AbstractJdbcNonTransactionableBatchOutputOperator which is batch non
transactional operation. That is one of the good option.
I am also thinking of a possibility of having "At least once" behavior with
Transactional
77 matches
Mail list logo
