GitHub user ishark opened a pull request:
https://github.com/apache/incubator-apex-core/pull/324
Documentation for Affinity Rules
Have added it in Application development guide after Streams section.
You can merge this pull request into a Git repository by running:
$ git pull
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/242#discussion_r59823744
--- Diff: library/src/main/java/org/apache/apex/malhar/lib/wal/WAL.java ---
@@ -0,0 +1,100 @@
+/**
+ * Licensed to the Apache Software
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/242#discussion_r59823701
--- Diff: library/src/main/java/org/apache/apex/malhar/lib/wal/WAL.java ---
@@ -0,0 +1,100 @@
+/**
+ * Licensed to the Apache Software
Github user ishark commented on the pull request:
https://github.com/apache/incubator-apex-core/pull/250#issuecomment-207551258
I have added reassigning of anti-affinity sets for containers in
assignContainers after repartitioning is done. But will create a follow up Jira
to
Github user ishark commented on the pull request:
https://github.com/apache/incubator-apex-core/pull/250#issuecomment-207146683
Have addressed review comments and updated diff.
Tests ran fine locally.
---
If your project is set up for it, you can reply to this email and have
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/250#discussion_r58961724
--- Diff:
engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java ---
@@ -344,9 +346,67 @@ public PhysicalPlan(LogicalPlan dag
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/250#discussion_r58955811
--- Diff:
engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java ---
@@ -357,25 +417,51 @@ public PhysicalPlan(LogicalPlan
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/250#discussion_r58954902
--- Diff:
engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java ---
@@ -386,6 +472,38 @@ public PhysicalPlan(LogicalPlan dag
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/250#discussion_r58954585
--- Diff:
engine/src/main/java/com/datatorrent/stram/plan/physical/PhysicalPlan.java ---
@@ -357,25 +417,51 @@ public PhysicalPlan(LogicalPlan
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/250#discussion_r58952486
--- Diff:
engine/src/main/java/com/datatorrent/stram/StreamingAppMasterService.java ---
@@ -750,6 +752,11 @@ private void execute() throws
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/250#discussion_r58952684
--- Diff:
engine/src/main/java/com/datatorrent/stram/BlacklistBasedResourceRequestHandler.java
---
@@ -0,0 +1,142 @@
+/**
+ * Licensed
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/250#discussion_r58623950
--- Diff: api/src/main/java/com/datatorrent/api/AffinityRule.java ---
@@ -0,0 +1,153 @@
+/**
+ * Licensed to the Apache Software
Github user ishark commented on the pull request:
https://github.com/apache/incubator-apex-core/pull/250#issuecomment-205941259
Have addressed most of the review comments except convenience APIs in DAG.
Did quick sanity testing on Cloudera after changes.
@tweise Could you please
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/204#discussion_r57801373
--- Diff: library/src/main/java/org/apache/apex/malhar/lib/wal/FSWal.java
---
@@ -0,0 +1,178 @@
+/**
+ * Licensed to the Apache
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/250#discussion_r57633606
--- Diff:
engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlanConfiguration.java
---
@@ -2435,7 +2435,7 @@ private void
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/250#discussion_r57633300
--- Diff:
engine/src/main/java/com/datatorrent/stram/ResourceRequestHandler.java ---
@@ -108,6 +113,23 @@ public void updateNodeReports(List
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/250#discussion_r57507443
--- Diff:
engine/src/main/java/com/datatorrent/stram/ContainerRequestHandlerCloudera.java
---
@@ -0,0 +1,138 @@
+/**
+ * Licensed to the
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/250#discussion_r57507404
--- Diff:
engine/src/main/java/com/datatorrent/stram/ContainerRequestHandler.java ---
@@ -0,0 +1,59 @@
+/**
+ * Licensed to the Apache
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/250#discussion_r57506976
--- Diff: api/src/main/java/com/datatorrent/api/AffinityRule.java ---
@@ -0,0 +1,211 @@
+/**
+ * Licensed to the Apache Software
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/250#discussion_r57506922
--- Diff: api/src/main/java/com/datatorrent/api/DAG.java ---
@@ -250,6 +250,24 @@
public abstract void setAttribute(Operator operator
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/250#discussion_r57506744
--- Diff: api/pom.xml ---
@@ -70,14 +70,6 @@
commons-beanutils
- jackson-core-asl
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/274#discussion_r57090295
--- Diff:
engine/src/main/java/com/datatorrent/stram/StreamingAppMasterService.java ---
@@ -950,7 +977,11 @@ private void execute() throws
GitHub user ishark opened a pull request:
https://github.com/apache/incubator-apex-core/pull/275
Disable blacklisting of failed nodes by default
Tested that nodes are not getting blacklisted after 3 failures.
You can merge this pull request into a Git repository by running
GitHub user ishark opened a pull request:
https://github.com/apache/incubator-apex-core/pull/274
APEXCORE-393 #resolve Adding Dag context attributes with increased deâ¦
â¦fault value for blacklisting of failed nodes
Added resetting of failure count for nodes after
Github user ishark commented on the pull request:
https://github.com/apache/incubator-apex-core/pull/274#issuecomment-198587733
Addressed review comments.
@tweise please review.
Have tested following things:
1. Checked that properties set for Blacklist removal time
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/207#discussion_r56096295
--- Diff:
library/src/main/java/com/datatorrent/lib/io/fs/HDFSFileSplitter.java ---
@@ -0,0 +1,184 @@
+/**
+ * Licensed to the Apache
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/207#discussion_r56096099
--- Diff:
library/src/main/java/com/datatorrent/lib/io/block/BlockReader.java ---
@@ -0,0 +1,81 @@
+/**
+ * Licensed to the Apache
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/207#discussion_r55781098
--- Diff:
library/src/main/java/com/datatorrent/lib/io/block/BlockReader.java ---
@@ -0,0 +1,81 @@
+/**
+ * Licensed to the Apache
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/207#discussion_r55781008
--- Diff:
library/src/main/java/com/datatorrent/lib/io/block/BlockReader.java ---
@@ -0,0 +1,81 @@
+/**
+ * Licensed to the Apache
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/207#discussion_r55778161
--- Diff:
library/src/main/java/com/datatorrent/lib/io/fs/HDFSFileSplitter.java ---
@@ -0,0 +1,184 @@
+/**
+ * Licensed to the Apache
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/207#discussion_r55777663
--- Diff:
library/src/main/java/com/datatorrent/lib/io/fs/HDFSFileSplitter.java ---
@@ -0,0 +1,180 @@
+/**
+ * Licensed to the Apache
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/207#discussion_r55777007
--- Diff:
library/src/main/java/com/datatorrent/lib/io/fs/HDFSFileSplitter.java ---
@@ -0,0 +1,184 @@
+/**
+ * Licensed to the Apache
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/207#discussion_r55776429
--- Diff:
library/src/main/java/com/datatorrent/lib/io/fs/HDFSInputModule.java ---
@@ -0,0 +1,253 @@
+/**
+ * Licensed to the Apache
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/207#discussion_r55776427
--- Diff:
library/src/main/java/com/datatorrent/lib/io/fs/HDFSInputModule.java ---
@@ -0,0 +1,253 @@
+/**
+ * Licensed to the Apache
Github user ishark commented on the pull request:
https://github.com/apache/incubator-apex-core/pull/250#issuecomment-191536790
Testing Done:
1. Unit tests for Dag Validation scenarios
2. Unit tests for host allocation based on affinity rules
3. Tested following
Github user ishark commented on the pull request:
https://github.com/apache/incubator-apex-core/pull/250#issuecomment-191536088
@tweise Pleas review
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does
GitHub user ishark opened a pull request:
https://github.com/apache/incubator-apex-core/pull/250
APEXCORE-10 #resolve Changes for supporting anti-affinity in operators
Added list of affinity rules as attribute in DAGContext
Added validation for affinity, anti-affinity and
Github user ishark closed the pull request at:
https://github.com/apache/incubator-apex-core/pull/234
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
GitHub user ishark opened a pull request:
https://github.com/apache/incubator-apex-core/pull/234
(Only for Review) APEXCORE-10 Changes for supporting anti-affinity in
operators
Added list of affinity rules as attribute in DAGContext
Added validation for affinity, anti-affinity
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/137#discussion_r48507863
--- Diff:
library/src/test/java/com/datatorrent/lib/parser/XmlParserTest.java ---
@@ -65,6 +73,19 @@ protected void finished(Description
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/137#discussion_r48425318
--- Diff: library/src/main/java/com/datatorrent/lib/parser/Parser.java ---
@@ -61,25 +70,48 @@ public void setup(PortContext context
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/137#discussion_r48383113
--- Diff: library/src/main/java/com/datatorrent/lib/parser/Parser.java ---
@@ -61,25 +70,48 @@ public void setup(PortContext context
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/137#discussion_r48381493
--- Diff: library/src/main/java/com/datatorrent/lib/parser/Parser.java ---
@@ -61,25 +70,48 @@ public void setup(PortContext context
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/137#discussion_r48319426
--- Diff: library/src/main/java/com/datatorrent/lib/parser/Parser.java ---
@@ -61,25 +70,48 @@ public void setup(PortContext context
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/137#discussion_r48301784
--- Diff: library/src/main/java/com/datatorrent/lib/parser/XmlParser.java
---
@@ -0,0 +1,139 @@
+/**
+ * Licensed to the Apache
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/137#discussion_r48301527
--- Diff: library/src/main/java/com/datatorrent/lib/parser/Parser.java ---
@@ -61,25 +70,48 @@ public void setup(PortContext context
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/137#discussion_r48301231
--- Diff: library/src/main/java/com/datatorrent/lib/parser/Parser.java ---
@@ -61,25 +70,48 @@ public void setup(PortContext context
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/137#discussion_r48301195
--- Diff: library/src/main/java/com/datatorrent/lib/parser/Parser.java ---
@@ -47,10 +47,19 @@
* @since 3.2.0
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/137#discussion_r47726390
--- Diff: library/src/main/java/com/datatorrent/lib/parser/XmlParser.java
---
@@ -0,0 +1,139 @@
+/**
+ * Licensed to the Apache
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/137#discussion_r47726307
--- Diff:
library/src/main/java/com/datatorrent/lib/formatter/XmlFormatter.java ---
@@ -57,45 +55,32 @@ public XmlFormatter
GitHub user ishark opened a pull request:
https://github.com/apache/incubator-apex-malhar/pull/137
Added XSD validation to XML parsers and moved XML and JSON parsers to
Malhar Library
You can merge this pull request into a Git repository by running:
$ git pull https
GitHub user ishark opened a pull request:
https://github.com/apache/incubator-apex-core/pull/178
APEX-274 #resolve Fixing null pointer exception in removing unifier Pâ¦
â¦TOperators from Physical Plan. Updated unit test to check this scenario
You can merge this pull request into
GitHub user ishark opened a pull request:
https://github.com/apache/incubator-apex-core/pull/177
APEX-274 #resolve Fixing null pointer exception in removing unifier Pâ¦
â¦TOperators from Physical Plan. Updated unit test to check this scenario
You can merge this pull request into
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/98#discussion_r44348690
--- Diff:
engine/src/main/java/com/datatorrent/stram/StreamingAppMasterService.java ---
@@ -647,8 +645,8 @@ private void execute() throws
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/158#discussion_r44220893
--- Diff: engine/src/main/java/com/datatorrent/stram/webapp/TypeGraph.java
---
@@ -602,6 +558,14 @@ public void setJarName(String jarName
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/158#discussion_r44220617
--- Diff:
engine/src/test/java/com/datatorrent/stram/webapp/OperatorDiscoveryTest.java ---
@@ -24,6 +24,7 @@
import
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/66#discussion_r42811427
--- Diff:
library/src/main/java/com/datatorrent/lib/appdata/query/WindowBoundedService.java
---
@@ -99,17 +100,31 @@ public void endWindow
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-malhar/pull/66#discussion_r42810195
--- Diff:
library/src/main/java/com/datatorrent/lib/io/PubSubWebSocketAppDataQuery.java
---
@@ -105,11 +105,11 @@ public void endWindow
GitHub user ishark opened a pull request:
https://github.com/apache/incubator-apex-core/pull/131
SPOI-6530 #resolve Adding a check for physical operators list being eâ¦
â¦mpty in fillOperatorInfo
You can merge this pull request into a Git repository by running:
$ git pull
Github user ishark commented on the pull request:
https://github.com/apache/incubator-apex-core/pull/116#issuecomment-148873407
Thanks. Just squashed the commits.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user ishark commented on the pull request:
https://github.com/apache/incubator-apex-core/pull/116#issuecomment-148870778
Yes, one of the operators in Malhar has port annotation but no annotation
properties. And that is part of demo packages. So, uploading of package fails
GitHub user ishark opened a pull request:
https://github.com/apache/incubator-apex-core/pull/116
#SPOI-6505 #resolve Added check of @OutputPortFieldAnnotation with noâ¦
⦠properties. One of the operators in Package had
@OutputPortFieldAnnotation but did not have any properties
Github user ishark commented on the pull request:
https://github.com/apache/incubator-apex-core/pull/113#issuecomment-147898200
Yes, tested shutdown-app from CLI
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
GitHub user ishark opened a pull request:
https://github.com/apache/incubator-apex-core/pull/113
SPOI-5053 APEX-56 #resolve #comment
Fixing removal of terminated operators from physical plan when downanStream
operators are also completed till shutdown window Id
Also fixed
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/98#discussion_r41460626
--- Diff:
engine/src/test/java/com/datatorrent/stram/StreamingAppMasterServiceTest.java
---
@@ -0,0 +1,319 @@
+package com.datatorrent.stram
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/98#discussion_r41460403
--- Diff:
engine/src/main/java/com/datatorrent/stram/StreamingAppMasterService.java ---
@@ -647,8 +645,8 @@ private void execute() throws
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/98#discussion_r41455380
--- Diff:
engine/src/main/java/com/datatorrent/stram/StreamingAppMasterService.java ---
@@ -972,6 +943,52 @@ else if (allocatedContainers.isEmpty
Github user ishark commented on the pull request:
https://github.com/apache/incubator-apex-core/pull/81#issuecomment-145607263
Code changes have been merged with devel-3. Closing the pull request.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user ishark closed the pull request at:
https://github.com/apache/incubator-apex-core/pull/81
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the
GitHub user ishark opened a pull request:
https://github.com/apache/incubator-apex-core/pull/98
APEX-92 Fixing Configuration of attributes
and unit tests for blacklisting consecutive failed nodes using Mockito
You can merge this pull request into a Git repository by running
Github user ishark commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/81#discussion_r40718327
--- Diff: engine/src/test/resources/testTopology.json ---
@@ -25,7 +25,7 @@
"attributes": {
"UN
71 matches
Mail list logo
