[jira] [Commented] (SAMZA-1455) Shutdown coordinator stream producers and consumers cleanly in JobRunner

2017-10-11 Thread Jagadish (JIRA) 

[ 
https://issues.apache.org/jira/browse/SAMZA-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201372#comment-16201372
 ] 

Jagadish commented on SAMZA-1455:
-

The fix is straightforward:
1. Cleanly close the coordinatorStream producer/ consumer in the JobRunner.
2. The finally block that tears these down is an obvious candidate for this.

> Shutdown coordinator stream producers and consumers cleanly in JobRunner
> 
>
> Key: SAMZA-1455
> URL: https://issues.apache.org/jira/browse/SAMZA-1455
> Project: Samza
>  Issue Type: Bug
>Reporter: Jagadish
>
> Currently, we do not cleanly close the producer and consumer in the 
> JobRunner. This means that, any exception happening in the JobRunner will 
> simply exit the main thread but not call tear-down the producers/consumers. 
> For producers and consumers that spawn non-daemon threads (for example, a 
> KafkaConsumer), this has the effect of not shutting down the JVM cleanly. 
> In our production clusters, We have observed that JVM processes 
> (corresponding to the JobRunner) do not shut-down. Often, these processes 
> hold on to deleted file handles leading to multiple resource leaks.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (SAMZA-1455) Shutdown coordinator stream producers and consumers cleanly in JobRunner

2017-10-11 Thread Jagadish (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SAMZA-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jagadish updated SAMZA-1455:

Description: 
Currently, we do not cleanly close the producer and consumer in the JobRunner. 
This means that, any exception happening in the JobRunner will simply exit the 
main thread but not call tear-down the producers/consumers. For producers and 
consumers that spawn non-daemon threads (for example, a KafkaConsumer), this 
has the effect of not shutting down the JVM cleanly. 

In our production clusters, We have observed that JVM processes (corresponding 
to the JobRunner) do not shut-down. Often, these processes hold on to deleted 
file handles leading to multiple resource leaks.

  was:
Currently, we do not cleanly close the producer and consumer in the JobRunner. 
This means that, any exception happening in the JobRunner will simply exit the 
main thread but not call tear-down the producers/consumers. For producers and 
consumers that spawn non-daemon threads (for example, a KafkaConsumer), this 
has the effect of not shutting down the JVM cleanly. 

We have observed that on production clusters, JVM processes corresponding to 
the JobRunner do not shut-down holding on to deleted file handles leading to 
site up issues.


> Shutdown coordinator stream producers and consumers cleanly in JobRunner
> 
>
> Key: SAMZA-1455
> URL: https://issues.apache.org/jira/browse/SAMZA-1455
> Project: Samza
>  Issue Type: Bug
>Reporter: Jagadish
>
> Currently, we do not cleanly close the producer and consumer in the 
> JobRunner. This means that, any exception happening in the JobRunner will 
> simply exit the main thread but not call tear-down the producers/consumers. 
> For producers and consumers that spawn non-daemon threads (for example, a 
> KafkaConsumer), this has the effect of not shutting down the JVM cleanly. 
> In our production clusters, We have observed that JVM processes 
> (corresponding to the JobRunner) do not shut-down. Often, these processes 
> hold on to deleted file handles leading to multiple resource leaks.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (SAMZA-1455) Shutdown coordinator stream producers and consumers cleanly in JobRunner

2017-10-11 Thread Jagadish (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SAMZA-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jagadish updated SAMZA-1455:

Description: 
Currently, we do not cleanly close the producer and consumer in the JobRunner. 
This means that, any exception happening in the JobRunner will simply exit the 
main thread but not call tear-down the producers/consumers. For producers and 
consumers that spawn non-daemon threads (for example, a KafkaConsumer), this 
has the effect of not shutting down the JVM cleanly. 

We have observed that on production clusters, JVM processes corresponding to 
the JobRunner do not shut-down holding on to deleted file handles leading to 
site up issues.

  was:Currently, we do not cleanly close the producer and consumer in the 
JobRunner. This means that, any exception happening in the JobRunner will 
simply exit the main thread but not call tear-down the producers/consumers. For 
producers and consumers that spawn non-daemon threads (for example, a 
KafkaConsumer), this has the effect of not shutting down the JVM cleanly. We 
have observed that on production clusters, JVM processes corresponding to the 
JobRunner do not shut-down holding on to deleted file handles leading to site 
up issues.


> Shutdown coordinator stream producers and consumers cleanly in JobRunner
> 
>
> Key: SAMZA-1455
> URL: https://issues.apache.org/jira/browse/SAMZA-1455
> Project: Samza
>  Issue Type: Bug
>Reporter: Jagadish
>
> Currently, we do not cleanly close the producer and consumer in the 
> JobRunner. This means that, any exception happening in the JobRunner will 
> simply exit the main thread but not call tear-down the producers/consumers. 
> For producers and consumers that spawn non-daemon threads (for example, a 
> KafkaConsumer), this has the effect of not shutting down the JVM cleanly. 
> We have observed that on production clusters, JVM processes corresponding to 
> the JobRunner do not shut-down holding on to deleted file handles leading to 
> site up issues.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (SAMZA-1455) Shutdown coordinator stream producers and consumers cleanly in JobRunner

2017-10-11 Thread Jagadish (JIRA) 
Jagadish created SAMZA-1455:
---

 Summary: Shutdown coordinator stream producers and consumers 
cleanly in JobRunner
 Key: SAMZA-1455
 URL: https://issues.apache.org/jira/browse/SAMZA-1455
 Project: Samza
  Issue Type: Bug
Reporter: Jagadish


Any er



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (SAMZA-1455) Shutdown coordinator stream producers and consumers cleanly in JobRunner

2017-10-11 Thread Jagadish (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SAMZA-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jagadish updated SAMZA-1455:

Description: Currently, we do not cleanly close the producer and consumer 
in the JobRunner. This means that, any exception happening in the JobRunner 
will simply exit the main thread but not call tear-down the 
producers/consumers. For producers and consumers that spawn non-daemon threads 
(for example, a KafkaConsumer), this has the effect of not shutting down the 
JVM cleanly. We have observed that on production clusters, JVM processes 
corresponding to the JobRunner do not shut-down holding on to deleted file 
handles leading to site up issues.  (was: Any er)

> Shutdown coordinator stream producers and consumers cleanly in JobRunner
> 
>
> Key: SAMZA-1455
> URL: https://issues.apache.org/jira/browse/SAMZA-1455
> Project: Samza
>  Issue Type: Bug
>Reporter: Jagadish
>
> Currently, we do not cleanly close the producer and consumer in the 
> JobRunner. This means that, any exception happening in the JobRunner will 
> simply exit the main thread but not call tear-down the producers/consumers. 
> For producers and consumers that spawn non-daemon threads (for example, a 
> KafkaConsumer), this has the effect of not shutting down the JVM cleanly. We 
> have observed that on production clusters, JVM processes corresponding to the 
> JobRunner do not shut-down holding on to deleted file handles leading to site 
> up issues.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (SAMZA-1451) Disable integration tests conditionally in build

2017-10-11 Thread Prateek Maheshwari (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SAMZA-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prateek Maheshwari resolved SAMZA-1451.
---
Resolution: Fixed

> Disable integration tests conditionally in build
> 
>
> Key: SAMZA-1451
> URL: https://issues.apache.org/jira/browse/SAMZA-1451
> Project: Samza
>  Issue Type: Test
>Reporter: Shanthoosh Venkataraman
>Assignee: Shanthoosh Venkataraman
>
> Remove runIntegrationTests gradle property added as a part of SAMZA-1355 and 
> introduce skipIntegrationTests property(which inverts it).
> If skipIntegrationTests gradle project property is enabled, execution of all 
> tests in samza-test project will be skipped from the build.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (SAMZA-1447) YarnCliJobStatusProvider cannot handle a high QPS to the JobsResource REST endpoint

2017-10-11 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/SAMZA-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200948#comment-16200948
 ] 

ASF GitHub Bot commented on SAMZA-1447:
---

Github user asfgit closed the pull request at:

https://github.com/apache/samza/pull/317


> YarnCliJobStatusProvider cannot handle a high QPS to the JobsResource REST 
> endpoint
> ---
>
> Key: SAMZA-1447
> URL: https://issues.apache.org/jira/browse/SAMZA-1447
> Project: Samza
>  Issue Type: Bug
>Reporter: Abhishek Shivanna
>Assignee: Abhishek Shivanna
>
> YarnCliJobStatusProvider shells out to a new process (running 
> org.apache.hadoop.yarn.client.cli.ApplicationCLI) to fetch metadata about 
> applications in the cluster.
> This leads to the host running out of memory when the number of requests 
> being made to the JobsResource endpoint is high.
> The fix should be to use the Resource Manager REST endpoint instead which 
> provides the same data without the new process per call overhead.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

samza git commit: SAMZA-1447; Swapping out CLI JobStatusProvider for REST based implementation in samza-rest

2017-10-11 Thread jagadish 
Repository: samza
Updated Branches:
  refs/heads/master e6eb13f85 -> 072457a2e


SAMZA-1447; Swapping out CLI JobStatusProvider for REST based implementation in 
samza-rest

Removing the YarnCliJobStatusProvider since forking a new shell for every 
request on the JobsResource endpoint is resource intensive.

Author: Abhishek Shivanna 

Reviewers: Jagadish

Closes #317 from abhishekshivanna/samza-rest-oom


Project: http://git-wip-us.apache.org/repos/asf/samza/repo
Commit: http://git-wip-us.apache.org/repos/asf/samza/commit/072457a2
Tree: http://git-wip-us.apache.org/repos/asf/samza/tree/072457a2
Diff: http://git-wip-us.apache.org/repos/asf/samza/diff/072457a2

Branch: refs/heads/master
Commit: 072457a2e44e5e041e87e0aafbef1dfbe332f54c
Parents: e6eb13f
Author: Abhishek Shivanna 
Authored: Wed Oct 11 13:57:20 2017 -0700
Committer: Jagadish 
Committed: Wed Oct 11 13:57:20 2017 -0700

--
 .../learn/documentation/rest/JobsResource.png   | Bin 191199 -> 152138 bytes
 .../versioned/rest/resources/jobs.md|   5 +-
 .../rest/model/yarn/YarnApplicationInfo.java|  86 +++
 .../rest/proxy/job/SimpleYarnJobProxy.java  |   6 +-
 .../proxy/job/YarnCliJobStatusProvider.java | 154 ---
 .../proxy/job/YarnRestJobStatusProvider.java| 146 ++
 .../rest/resources/YarnJobResourceConfig.java   |  53 +++
 7 files changed, 293 insertions(+), 157 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/samza/blob/072457a2/docs/img/versioned/learn/documentation/rest/JobsResource.png
--
diff --git a/docs/img/versioned/learn/documentation/rest/JobsResource.png 
b/docs/img/versioned/learn/documentation/rest/JobsResource.png
index a208c3d..6eebcb4 100644
Binary files a/docs/img/versioned/learn/documentation/rest/JobsResource.png and 
b/docs/img/versioned/learn/documentation/rest/JobsResource.png differ

http://git-wip-us.apache.org/repos/asf/samza/blob/072457a2/docs/learn/documentation/versioned/rest/resources/jobs.md
--
diff --git a/docs/learn/documentation/versioned/rest/resources/jobs.md 
b/docs/learn/documentation/versioned/rest/resources/jobs.md
index 8282a5d..f6fa2f9 100644
--- a/docs/learn/documentation/versioned/rest/resources/jobs.md
+++ b/docs/learn/documentation/versioned/rest/resources/jobs.md
@@ -295,7 +295,7 @@ After validating each request, the JobsResource invokes the 
appropriate JobProxy
 
 The provided 
[SimpleInstallationFinder](../javadocs/org/apache/samza/rest/proxy/installation/SimpleInstallationFinder.html)
 crawls the file system, starting in the directory specified by the 
`job.installations.path` looking for valid Samza job config files. It extracts 
the `job.name` and `job.id` property values and creates an 
[InstallationRecord](../javadocs/org/apache/samza/rest/proxy/installation/InstallationRecord.html)
 for the each job instance. The InstallationRecord contains all the information 
needed to start, stop, and get the status for the job.
 
-The provided 
[YarnCliJobStatusProvider](../javadocs/org/apache/samza/rest/proxy/job/YarnCliJobStatusProvider.html)
 leverages a ScriptRunner to fetch job status using the Yarn ApplicationCLI.
+The provided 
[YarnRestJobStatusProvider](../javadocs/org/apache/samza/rest/proxy/job/YarnRestJobStatusProvider.html)
 uses the Resource Manager's REST API to fetch the job status.
 
 The 
[SimpleYarnJobProxy](../javadocs/org/apache/samza/rest/proxy/job/SimpleYarnJobProxy.html)
 relies on the scripts in the InstallationRecord scriptFilePath (`/bin`) 
directory to start and stop jobs.
 
@@ -323,5 +323,8 @@ The JobsResource properties should be specified in the same 
file as the Samza RE
 
   job.config.factory.classThe config factory to use for 
reading Samza job configs. This is used to fetch the job.name and job.id 
properties for each job instance in the InstallationRecord. It's also used to 
validate that a particular directory within the installations path actually 
contains Samza jobs. If not specified 
org.apache.samza.config.factories.PropertiesConfigFactory will be 
used. 
 
+
+  yarn.resourcemanager.api.endpoint An optional config if 
YARN is used as the cluster manager. This provides the host and port at which 
the YARN ResourceManager REST API is exposed. If not specified 
localhost:8088 will be used.
+
   
 

http://git-wip-us.apache.org/repos/asf/samza/blob/072457a2/samza-rest/src/main/java/org/apache/samza/rest/model/yarn/YarnApplicationInfo.java
--
diff --git 
a/samza-rest/src/main/java/org/apache/samza/rest/model/yarn/YarnApplicationInfo.java

[jira] [Commented] (SAMZA-1451) Disable integration tests conditionally in build

2017-10-11 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/SAMZA-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200945#comment-16200945
 ] 

ASF GitHub Bot commented on SAMZA-1451:
---

Github user asfgit closed the pull request at:

https://github.com/apache/samza/pull/319


> Disable integration tests conditionally in build
> 
>
> Key: SAMZA-1451
> URL: https://issues.apache.org/jira/browse/SAMZA-1451
> Project: Samza
>  Issue Type: Test
>Reporter: Shanthoosh Venkataraman
>Assignee: Shanthoosh Venkataraman
>
> Remove runIntegrationTests gradle property added as a part of SAMZA-1355 and 
> introduce skipIntegrationTests property(which inverts it).
> If skipIntegrationTests gradle project property is enabled, execution of all 
> tests in samza-test project will be skipped from the build.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

samza git commit: SAMZA-1451: Disable integration tests conditionally in build.

2017-10-11 Thread pmaheshwari 
Repository: samza
Updated Branches:
  refs/heads/master 24170389c -> e6eb13f85


SAMZA-1451: Disable integration tests conditionally in build.

Remove runIntegrationTests gradle property added as a part of SAMZA-1355 and 
introduce skipIntegrationTests property(which inverts it).
If skipIntegrationTests gradle project property is enabled, execution of all 
tests in samza-test project will be skipped from the build.

Author: Shanthoosh Venkataraman 

Reviewers: Prateek Maheshwari 

Closes #319 from shanthoosh/skip_integration_tests


Project: http://git-wip-us.apache.org/repos/asf/samza/repo
Commit: http://git-wip-us.apache.org/repos/asf/samza/commit/e6eb13f8
Tree: http://git-wip-us.apache.org/repos/asf/samza/tree/e6eb13f8
Diff: http://git-wip-us.apache.org/repos/asf/samza/diff/e6eb13f8

Branch: refs/heads/master
Commit: e6eb13f8525d69ae1f0736339973d31c619add17
Parents: 2417038
Author: Shanthoosh Venkataraman 
Authored: Wed Oct 11 13:56:24 2017 -0700
Committer: Prateek Maheshwari 
Committed: Wed Oct 11 13:56:24 2017 -0700

--
 README.md| 2 +-
 bin/check-all.sh | 2 +-
 docs/contribute/tests.md | 2 +-
 settings.gradle  | 7 +++
 4 files changed, 6 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/samza/blob/e6eb13f8/README.md
--
diff --git a/README.md b/README.md
index 0164dd3..334c7b4 100644
--- a/README.md
+++ b/README.md
@@ -34,7 +34,7 @@ Samza builds with [Scala](http://www.scala-lang.org/) 2.10 or 
2.11 and [YARN](ht
 
 ### Testing Samza
 
-To run all unit tests:
+To run all tests:
 
 ./gradlew clean test
 

http://git-wip-us.apache.org/repos/asf/samza/blob/e6eb13f8/bin/check-all.sh
--
diff --git a/bin/check-all.sh b/bin/check-all.sh
index ceb1e99..2f9f03c 100755
--- a/bin/check-all.sh
+++ b/bin/check-all.sh
@@ -81,7 +81,7 @@ do
 for yarn_version in "${YARNs[@]}"
 do
   echo "- Running check task against JDK${jdk_number}/Scala 
${scala_version}/YARN ${yarn_version}"
-  ${gradle_file} -PscalaVersion=${scala_version} 
-PyarnVersion=${yarn_version} -Dorg.gradle.java.home=${!i} 
-PrunIntegrationTests clean check $@
+  ${gradle_file} -PscalaVersion=${scala_version} 
-PyarnVersion=${yarn_version} -Dorg.gradle.java.home=${!i} clean check $@
   echo "- Finished running check task against 
JDK${jdk_number}/Scala ${scala_version}/YARN ${yarn_version}"
 done
   done

http://git-wip-us.apache.org/repos/asf/samza/blob/e6eb13f8/docs/contribute/tests.md
--
diff --git a/docs/contribute/tests.md b/docs/contribute/tests.md
index 048e3be..9fe728c 100644
--- a/docs/contribute/tests.md
+++ b/docs/contribute/tests.md
@@ -25,7 +25,7 @@ Samza's unit tests are written on top of 
[JUnit](http://junit.org/), and license
 
 To run all tests, and license checks:
 
-./gradlew clean check -PrunIntegrationTests
+./gradlew clean check
 
 To run a single test:
 

http://git-wip-us.apache.org/repos/asf/samza/blob/e6eb13f8/settings.gradle
--
diff --git a/settings.gradle b/settings.gradle
index 603cd35..4c81d9c 100644
--- a/settings.gradle
+++ b/settings.gradle
@@ -48,18 +48,17 @@ rootProject.children.each {
 }
 
 /**
- * Skips execution of all integration tests in project 'samza-test'.
- * To run integration tests in samza-test: './gradlew clean build 
-PrunIntegrationTests'
+ * To skip integration tests in samza-test: './gradlew clean build 
-PskipIntegrationTests'
  */
 gradle.taskGraph.whenReady { taskGraph ->
   taskGraph.getAllTasks().each { task ->
 def project = task.getProject()
 task.onlyIf {
   /**
-   * Behaves as pass through filter for all tasks when 
`runIntegrationTests` property is turned on.
+   * Behaves as pass through filter for all tasks when 
`skipIntegrationTests` property is turned off.
* Filters 'test' task of 'samza-test' project otherwise.
*/
-  project.hasProperty("runIntegrationTests") || 
!(project.getName().contains("samza-test") && task.getName() == "test")
+  !project.hasProperty("skipIntegrationTests") || 
!(project.getName().contains("samza-test") && task.getName() == "test")
 }
   }
 }

[jira] [Updated] (SAMZA-1451) Disable integration tests conditionally in build

2017-10-11 Thread Shanthoosh Venkataraman (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SAMZA-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shanthoosh Venkataraman updated SAMZA-1451:
---
Description: 
Remove runIntegrationTests gradle property added as a part of SAMZA-1355 and 
introduce skipIntegrationTests property(which inverts it).
If skipIntegrationTests gradle project property is enabled, execution of all 
tests in samza-test project will be skipped from the build.


  was:
Remove runIntegrationTests property and introduce skipIntegrationTests property.
If skipIntegrationTests gradle project property is enabled, all tests in 
samza-test project will be skipped.



> Disable integration tests conditionally in build
> 
>
> Key: SAMZA-1451
> URL: https://issues.apache.org/jira/browse/SAMZA-1451
> Project: Samza
>  Issue Type: Test
>Reporter: Shanthoosh Venkataraman
>Assignee: Shanthoosh Venkataraman
>
> Remove runIntegrationTests gradle property added as a part of SAMZA-1355 and 
> introduce skipIntegrationTests property(which inverts it).
> If skipIntegrationTests gradle project property is enabled, execution of all 
> tests in samza-test project will be skipped from the build.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (SAMZA-1451) Disable integration tests conditionally in build

2017-10-11 Thread Shanthoosh Venkataraman (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SAMZA-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shanthoosh Venkataraman updated SAMZA-1451:
---
Summary: Disable integration tests conditionally in build  (was: Enable 
integration tests conditionally in build)

> Disable integration tests conditionally in build
> 
>
> Key: SAMZA-1451
> URL: https://issues.apache.org/jira/browse/SAMZA-1451
> Project: Samza
>  Issue Type: Test
>Reporter: Shanthoosh Venkataraman
>Assignee: Shanthoosh Venkataraman
>
> Remove runIntegrationTests property and introduce skipIntegrationTests 
> property.
> If skipIntegrationTests gradle project property is enabled, all tests in 
> samza-test project will be skipped.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (SAMZA-1454) Globally unique and user settable IDs for stateful operators

2017-10-11 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/SAMZA-1454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200850#comment-16200850
 ] 

ASF GitHub Bot commented on SAMZA-1454:
---

GitHub user prateekm opened a pull request:

https://github.com/apache/samza/pull/324

SAMZA-1454: Globally unique and user settable IDs for stateful operators



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/prateekm/samza operator-id-uniqueness

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/samza/pull/324.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #324


commit 53a872ce99761a806ab26878366feca938810702
Author: Prateek Maheshwari 
Date:   2017-10-11T19:30:28Z

SAMZA-1454: Globally unique and user settable IDs for stateful operators




> Globally unique and user settable IDs for stateful operators
> 
>
> Key: SAMZA-1454
> URL: https://issues.apache.org/jira/browse/SAMZA-1454
> Project: Samza
>  Issue Type: Bug
>Reporter: Prateek Maheshwari
>Assignee: Prateek Maheshwari
> Fix For: 0.14.0
>
>
> Operators in the the High Level API need globally unique IDs so that we can:
> 1. Create intermediate topics (e.g. for partitionBy).
> 2. Create operator state stores and their changelog topics.
> 3. Start processing from particular points in the DAG (multi-stage).
> Ideally, a particular operator's ID should be:
> 1. The same across job restarts with no code changes.
> 2. The same despite functionality preserving changes in topology (e.g. 
> splitting a map into two maps).
> 3. Different if the semantics of an operator change without a change in 
> topology (e.g. a change in the map function logic).
> 1 is satisfied by a topology based ID, but 2 & 3 are difficult for the 
> framework to determine automatically. All three are required to ensure 
> correctness of durable intermediate state in the DAG across restarts with 
> code changes. In other words, users need to be able handle the following 
> scenarios:
> 1. If they make a code change that preserves the topology but changes the 
> semantics, they should be able to indicate that the downstream stateful 
> operators are now semantically different than before and the previous state 
> should be invalidated.
> 2. If they make a code change that changes the topology but preserves the 
> semantics, they should be able to indicate that the  downstream stateful 
> operators are still the same as earlier and the previous state should be 
> reused.
> To support these two scenarios, we will allow (and recommend) users to 
> provide a custom ID for stateful operators (partitionBy, window and join 
> operators) that's unique in the DAG. We will use this ID as part of the name 
> for any durable state (streams, stores, changelogs) associated with the 
> operator. Whenever users make a code change that changes the topology or 
> semantics of the application, they should identify any affected operators 
> downstream of the change and change or preserve their operator names 
> accordingly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (SAMZA-1391) System generated intermediate streams should have robust naming scheme

2017-10-11 Thread Prateek Maheshwari (JIRA) 

[ 
https://issues.apache.org/jira/browse/SAMZA-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200848#comment-16200848
 ] 

Prateek Maheshwari commented on SAMZA-1391:
---

cc [~bharathkk] to confirm that SAMZA-1454 addresses this issue.

> System generated intermediate streams should have robust naming scheme
> --
>
> Key: SAMZA-1391
> URL: https://issues.apache.org/jira/browse/SAMZA-1391
> Project: Samza
>  Issue Type: Bug
>Reporter: Bharath Kumarasubramanian
>
> In certain class of high level stream applications, Samza takes care of 
> creating intermediate streams. Currently, we use the combination of job name, 
> job id and the operator id to come up with the logical and physical stream 
> name. We need to come up with a unique, robust naming scheme to prevent 
> collision. We need to either use a unique name or detect the collision and 
> perform necessary purge/cleanup steps prior to stream creation.
> An example where this bug is exploited is... 
> Consider a stream application with following phases.
> {quote} input --> partition --> output {quote}
> If the input source were to change or the user provided partition function 
> changes, our DAG would still be the same resulting in the same logical and 
> physical stream name.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (SAMZA-1454) Globally unique and user settable IDs for stateful operators

2017-10-11 Thread Prateek Maheshwari (JIRA) 
Prateek Maheshwari created SAMZA-1454:
-

 Summary: Globally unique and user settable IDs for stateful 
operators
 Key: SAMZA-1454
 URL: https://issues.apache.org/jira/browse/SAMZA-1454
 Project: Samza
  Issue Type: Bug
Reporter: Prateek Maheshwari
Assignee: Prateek Maheshwari
 Fix For: 0.14.0


Operators in the the High Level API need globally unique IDs so that we can:
1. Create intermediate topics (e.g. for partitionBy).
2. Create operator state stores and their changelog topics.
3. Start processing from particular points in the DAG (multi-stage).

Ideally, a particular operator's ID should be:
1. The same across job restarts with no code changes.
2. The same despite functionality preserving changes in topology (e.g. 
splitting a map into two maps).
3. Different if the semantics of an operator change without a change in 
topology (e.g. a change in the map function logic).

1 is satisfied by a topology based ID, but 2 & 3 are difficult for the 
framework to determine automatically. All three are required to ensure 
correctness of durable intermediate state in the DAG across restarts with code 
changes. In other words, users need to be able handle the following scenarios:
1. If they make a code change that preserves the topology but changes the 
semantics, they should be able to indicate that the downstream stateful 
operators are now semantically different than before and the previous state 
should be invalidated.
2. If they make a code change that changes the topology but preserves the 
semantics, they should be able to indicate that the  downstream stateful 
operators are still the same as earlier and the previous state should be reused.

To support these two scenarios, we will allow (and recommend) users to provide 
a custom ID for stateful operators (partitionBy, window and join operators) 
that's unique in the DAG. We will use this ID as part of the name for any 
durable state (streams, stores, changelogs) associated with the operator. 
Whenever users make a code change that changes the topology or semantics of the 
application, they should identify any affected operators downstream of the 
change and change or preserve their operator names accordingly.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (SAMZA-1409) add the missing parts and make guideline clearer for docs/README.md

2017-10-11 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/SAMZA-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200789#comment-16200789
 ] 

ASF GitHub Bot commented on SAMZA-1409:
---

Github user asfgit closed the pull request at:

https://github.com/apache/samza/pull/288


> add the missing parts and make guideline clearer for docs/README.md
> ---
>
> Key: SAMZA-1409
> URL: https://issues.apache.org/jira/browse/SAMZA-1409
> Project: Samza
>  Issue Type: Bug
>Reporter: Fred Ji
>Assignee: Fred Ji
>Priority: Minor
>
> The "Release-new-version Website Checklist" in docs/README.md gives 
> instruction on updating the docs, but there are some issues:
> 1. There is no need for the doc version for minor release.
> 2. We need to update 
> docs/learn/tutorials/versioned/samza-rest-getting-started.md for the release 
> branch as well
> 3. Nothing to change for 
> docs/learn/tutorials/versioned/hello-samza-high-level-code.md for the master 
> branch



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

samza git commit: SAMZA-1409; add missing parts and make guideline clearer for docs/README.md

2017-10-11 Thread jagadish 
Repository: samza
Updated Branches:
  refs/heads/master a47ed9d18 -> 24170389c


SAMZA-1409; add missing parts and make guideline clearer for docs/README.md

Author: Fred Ji 

Reviewers: Jagadish 

Closes #288 from fredji97/master_docs_readme


Project: http://git-wip-us.apache.org/repos/asf/samza/repo
Commit: http://git-wip-us.apache.org/repos/asf/samza/commit/24170389
Tree: http://git-wip-us.apache.org/repos/asf/samza/tree/24170389
Diff: http://git-wip-us.apache.org/repos/asf/samza/diff/24170389

Branch: refs/heads/master
Commit: 24170389cbd04f6f39724fd6e786a2abc655a19c
Parents: a47ed9d
Author: Fred Ji 
Authored: Wed Oct 11 12:09:33 2017 -0700
Committer: Jagadish 
Committed: Wed Oct 11 12:09:33 2017 -0700

--
 docs/README.md | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/samza/blob/24170389/docs/README.md
--
diff --git a/docs/README.md b/docs/README.md
index 1884405..21a4991 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -68,26 +68,26 @@ Following can be done when updating the gradle.properties 
file
 
 1. in x.x.x branch,
 
-* modify the docs/_config.yml to make both the "version" and 
"latest-release" properties to x.x.x
+* if this is a major release, modify the docs/_config.yml to make both the 
"version" and "latest-release" properties to x.x.x
 
 * remove "git checkout latest" line and the "-SNAPSHOT" version suffix in 
each of the tutorials
   * docs/startup/hello-samza/versioned/index.md
   * docs/learn/tutorials/versioned/hello-samza-high-level-code.md
   * docs/learn/tutorials/versioned/hello-samza-high-level-yarn.md
   * docs/learn/tutorials/versioned/hello-samza-high-level-zk.md
+  * docs/learn/tutorials/versioned/samza-rest-getting-started.md
 
 2. in master branch,
 
-* modify the docs/_config.yml to make the "latest-release" to x.x.x
+* if this is a major release, modify the docs/_config.yml to make the 
"latest-release" to x.x.x
 
-* add the x.x.x release to Archive category in docs/_layouts/default.html 
and x.x.x release part in docs/archive/index.html
+* if this is a major release, add the x.x.x release to Archive category in 
docs/_layouts/default.html and x.x.x release part in docs/archive/index.html
 
 * update the download page to use x.x.x release
   * docs/startup/download/index.md
 
 * update the version number in "tar -xvf 
./target/hello-samza-y.y.y-dist.tar.gz -C deploy/samza" in each of the 
tutorials (and search for other uses of version x.x.x which may need to be 
replaced with y.y.y)
   * docs/startup/hello-samza/versioned/index.md
-  * docs/learn/tutorials/versioned/hello-samza-high-level-code.md
   * docs/learn/tutorials/versioned/hello-samza-high-level-yarn.md
   * docs/learn/tutorials/versioned/hello-samza-high-level-zk.md
   * docs/learn/tutorials/versioned/samza-rest-getting-started.md

[jira] [Resolved] (SAMZA-1453) Update README with Travis-CI badge.

2017-10-11 Thread Jagadish (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SAMZA-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jagadish resolved SAMZA-1453.
-
Resolution: Fixed

> Update README with Travis-CI badge.
> ---
>
> Key: SAMZA-1453
> URL: https://issues.apache.org/jira/browse/SAMZA-1453
> Project: Samza
>  Issue Type: Task
>Reporter: Daniel Nishimura
>Assignee: Daniel Nishimura
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Closed] (SAMZA-1453) Update README with Travis-CI badge.

2017-10-11 Thread Jagadish (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SAMZA-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jagadish closed SAMZA-1453.
---

> Update README with Travis-CI badge.
> ---
>
> Key: SAMZA-1453
> URL: https://issues.apache.org/jira/browse/SAMZA-1453
> Project: Samza
>  Issue Type: Task
>Reporter: Daniel Nishimura
>Assignee: Daniel Nishimura
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (SAMZA-1453) Update README with Travis-CI badge.

2017-10-11 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/SAMZA-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200781#comment-16200781
 ] 

ASF GitHub Bot commented on SAMZA-1453:
---

Github user asfgit closed the pull request at:

https://github.com/apache/samza/pull/323


> Update README with Travis-CI badge.
> ---
>
> Key: SAMZA-1453
> URL: https://issues.apache.org/jira/browse/SAMZA-1453
> Project: Samza
>  Issue Type: Task
>Reporter: Daniel Nishimura
>Assignee: Daniel Nishimura
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

samza git commit: SAMZA-1453; Update README with Travis-CI badge.

2017-10-11 Thread jagadish 
Repository: samza
Updated Branches:
  refs/heads/master caec9c536 -> a47ed9d18


SAMZA-1453; Update README with Travis-CI badge.

Added Travis-CI to README.md file.

Author: Daniel Nishimura 

Reviewers: Jagadish 

Closes #323 from dnishimura/samza-1453-readme-travisci


Project: http://git-wip-us.apache.org/repos/asf/samza/repo
Commit: http://git-wip-us.apache.org/repos/asf/samza/commit/a47ed9d1
Tree: http://git-wip-us.apache.org/repos/asf/samza/tree/a47ed9d1
Diff: http://git-wip-us.apache.org/repos/asf/samza/diff/a47ed9d1

Branch: refs/heads/master
Commit: a47ed9d18a104f99ec8036f478f07a0a8b29a35b
Parents: caec9c5
Author: Daniel Nishimura 
Authored: Wed Oct 11 12:03:28 2017 -0700
Committer: Jagadish 
Committed: Wed Oct 11 12:03:28 2017 -0700

--
 README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/samza/blob/a47ed9d1/README.md
--
diff --git a/README.md b/README.md
index cab1826..0164dd3 100644
--- a/README.md
+++ b/README.md
@@ -1,4 +1,4 @@
-## What is Samza?  [![Build 
Status](https://builds.apache.org/view/S-Z/view/Samza/job/samza-freestyle-build/badge/icon)](https://builds.apache.org/view/S-Z/view/Samza/job/samza-freestyle-build/)
+## What is Samza?  [![Build 
Status](https://builds.apache.org/view/S-Z/view/Samza/job/samza-freestyle-build/badge/icon)](https://builds.apache.org/view/S-Z/view/Samza/job/samza-freestyle-build/)
 [![Build 
Status](https://travis-ci.org/apache/samza.svg?branch=master)](https://travis-ci.org/apache/samza)
 
 [Apache Samza](http://samza.apache.org/) is a distributed stream processing 
framework. It uses [Apache Kafka](http://kafka.apache.org) for messaging, and 
[Apache Hadoop 
YARN](http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html)
 to provide fault tolerance, processor isolation, security, and resource 
management.

[samza] Git Push Summary

2017-10-11 Thread nickpan47 
Repository: samza
Updated Branches:
  refs/heads/0.13.2 [deleted] 90fa985ec

[jira] [Commented] (SAMZA-1453) Update README with Travis-CI badge.

2017-10-11 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/SAMZA-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16200617#comment-16200617
 ] 

ASF GitHub Bot commented on SAMZA-1453:
---

GitHub user dnishimura opened a pull request:

https://github.com/apache/samza/pull/323

SAMZA-1453: Update README with Travis-CI badge.

Added Travis-CI to README.md file.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/dnishimura/samza samza-1453-readme-travisci

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/samza/pull/323.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #323


commit aa3c833b22484eb90c3262e2177565b777dff063
Author: Daniel Nishimura 
Date:   2017-10-11T17:13:50Z

Add Travis-CI badge to README.md.




> Update README with Travis-CI badge.
> ---
>
> Key: SAMZA-1453
> URL: https://issues.apache.org/jira/browse/SAMZA-1453
> Project: Samza
>  Issue Type: Task
>Reporter: Daniel Nishimura
>Assignee: Daniel Nishimura
>Priority: Minor
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (SAMZA-1453) Update README with Travis-CI badge.

2017-10-11 Thread Daniel Nishimura (JIRA) 
Daniel Nishimura created SAMZA-1453:
---

 Summary: Update README with Travis-CI badge.
 Key: SAMZA-1453
 URL: https://issues.apache.org/jira/browse/SAMZA-1453
 Project: Samza
  Issue Type: Task
Reporter: Daniel Nishimura
Assignee: Daniel Nishimura
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[CONF] Apache Samza > SEP-9 Add a Kinesis SystemConsumer and SystemProducer

2017-10-11 Thread Aditya Toomula (Confluence) 
Title: Message Title



 
 
 
 
 
 
 

Aditya Toomula edited a page 
 
 
  
 
 
 
 
 
 
 
  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
SEP-9 Add a Kinesis SystemConsumer and SystemProducer 
 
 
  
 
 
 
 
 
 
 ...  4. Shard-2 is split into Shard-6 and Shard-7. ShardPartitionMapper unmaps Shard-2 and since there are no free ssp slots to map the resulting new shards, it will map one of the new shards to an ssp which is already mapped to another shard. The offset of the record read from any of these shards mapped to the same ssp should now contain a list of all the sequence numbers of the latest records for all the other shards that are inserted into BEM, along with the sequence number for the current shard record that is going to be inserted into BEM. Please take a look at Task1:ssp1 below where a record for shard-3 arrives first followed by a record for shard-6.              5. When a job is restarted, the number of ssps in each task will become equal to the shard count and the mapping between ssps and shards willl be 1:1.   Image Added   6. Let’s take the scenario where we add a container to keep up with the Kinesis stream throughput. On Yarn, one needs to make yarn container count config change and restart the job with the new config. This would result in recalculating job model. On stand-alone, adding a container would result in JobCoordinator leader to redo the job model.            Image Removed      ...    Image Added   7. Shard-5 and Shard-6 are merged into Shard-8. ShardPartitionMapper unmaps Shard-5 and Shard-6 and maps the new shard to free ssp as below.  Image RemovedImage Added Limitations   The following limitations apply for Samza jobs consuming from Kinesis streams using the proposed consumer:  ...  
 
 
  
 
 
 
 
 
 
 
 
 
 

View page
• 
 
 
 
 
 
 

Add comment
• 
 
 
 
 
 
 

Like 
 
 
  
 
 
  
 
 
  
 
 
  
 
 
 
 
 
 
 
 
 
 
Stop watching space
• 
 
 
 
 
 
 
Manage notifications 
 
 
 
 
 
 
 
 
 
 
  
 
 
This message was sent by Atlassian Confluence 5.8.17

[jira] [Created] (SAMZA-1452) Clean up interrupted thread bugs

2017-10-11 Thread Jake Maes (JIRA) 
Jake Maes created SAMZA-1452:


 Summary: Clean up interrupted thread bugs
 Key: SAMZA-1452
 URL: https://issues.apache.org/jira/browse/SAMZA-1452
 Project: Samza
  Issue Type: Bug
Reporter: Jake Maes
Assignee: Nacho Solis






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (SAMZA-1452) Clean up interrupted thread bugs

2017-10-11 Thread Jake Maes (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SAMZA-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jake Maes resolved SAMZA-1452.
--
   Resolution: Fixed
Fix Version/s: 0.14.0

Issue resolved by pull request 322
[https://github.com/apache/samza/pull/322]

> Clean up interrupted thread bugs
> 
>
> Key: SAMZA-1452
> URL: https://issues.apache.org/jira/browse/SAMZA-1452
> Project: Samza
>  Issue Type: Bug
>Reporter: Jake Maes
>Assignee: Nacho Solis
> Fix For: 0.14.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

samza git commit: SAMZA-1452: Clean up interrupted thread bugs

2017-10-11 Thread jmakes 
Repository: samza
Updated Branches:
  refs/heads/master 7b0a65b14 -> caec9c536


SAMZA-1452: Clean up interrupted thread bugs

Call Thread.currentThread().interrupt(); when capturing InterruptedException

Author: Nacho Solis 

Reviewers: Jacob Maes ,Jagadish 

Closes #322 from isolis/cleancodebugs


Project: http://git-wip-us.apache.org/repos/asf/samza/repo
Commit: http://git-wip-us.apache.org/repos/asf/samza/commit/caec9c53
Tree: http://git-wip-us.apache.org/repos/asf/samza/tree/caec9c53
Diff: http://git-wip-us.apache.org/repos/asf/samza/diff/caec9c53

Branch: refs/heads/master
Commit: caec9c5362635478a4272199504950789664d0cd
Parents: 7b0a65b
Author: Nacho Solis 
Authored: Wed Oct 11 07:49:52 2017 -0700
Committer: Jacob Maes 
Committed: Wed Oct 11 07:49:52 2017 -0700

--
 .../org/apache/samza/system/SystemStreamPartitionIterator.java | 1 +
 .../java/org/apache/samza/autoscaling/deployer/ConfigManager.java  | 2 ++
 .../java/org/apache/samza/coordinator/AzureJobCoordinator.java | 2 +-
 .../src/main/java/org/apache/samza/coordinator/AzureLock.java  | 2 +-
 .../src/main/java/org/apache/samza/processor/StreamProcessor.java  | 1 +
 .../main/java/org/apache/samza/system/hdfs/HdfsSystemConsumer.java | 1 +
 .../main/java/org/apache/samza/monitor/SamzaMonitorService.java| 1 +
 .../src/main/java/org/apache/samza/rest/script/ScriptRunner.java   | 1 +
 8 files changed, 9 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/samza/blob/caec9c53/samza-api/src/main/java/org/apache/samza/system/SystemStreamPartitionIterator.java
--
diff --git 
a/samza-api/src/main/java/org/apache/samza/system/SystemStreamPartitionIterator.java
 
b/samza-api/src/main/java/org/apache/samza/system/SystemStreamPartitionIterator.java
index d1d61ed..726a5f4 100644
--- 
a/samza-api/src/main/java/org/apache/samza/system/SystemStreamPartitionIterator.java
+++ 
b/samza-api/src/main/java/org/apache/samza/system/SystemStreamPartitionIterator.java
@@ -90,6 +90,7 @@ public class SystemStreamPartitionIterator implements 
Iteratorhttp://git-wip-us.apache.org/repos/asf/samza/blob/caec9c53/samza-autoscaling/src/main/java/org/apache/samza/autoscaling/deployer/ConfigManager.java
--
diff --git 
a/samza-autoscaling/src/main/java/org/apache/samza/autoscaling/deployer/ConfigManager.java
 
b/samza-autoscaling/src/main/java/org/apache/samza/autoscaling/deployer/ConfigManager.java
index 223d1d6..d1b532f 100644
--- 
a/samza-autoscaling/src/main/java/org/apache/samza/autoscaling/deployer/ConfigManager.java
+++ 
b/samza-autoscaling/src/main/java/org/apache/samza/autoscaling/deployer/ConfigManager.java
@@ -131,6 +131,7 @@ public class ConfigManager {
 } catch (InterruptedException e) {
   e.printStackTrace();
   log.warn("Got interrupt in config manager thread, so shutting down");
+  Thread.currentThread().interrupt();
 } finally {
   log.info("Stopping the config manager");
   stop();
@@ -305,6 +306,7 @@ public class ConfigManager {
   }
 } catch (InterruptedException e) {
   e.printStackTrace();
+  Thread.currentThread().interrupt();
 }
 
 log.info("Killed the current job successfully");

http://git-wip-us.apache.org/repos/asf/samza/blob/caec9c53/samza-azure/src/main/java/org/apache/samza/coordinator/AzureJobCoordinator.java
--
diff --git 
a/samza-azure/src/main/java/org/apache/samza/coordinator/AzureJobCoordinator.java
 
b/samza-azure/src/main/java/org/apache/samza/coordinator/AzureJobCoordinator.java
index 9438690..622932f 100644
--- 
a/samza-azure/src/main/java/org/apache/samza/coordinator/AzureJobCoordinator.java
+++ 
b/samza-azure/src/main/java/org/apache/samza/coordinator/AzureJobCoordinator.java
@@ -427,7 +427,7 @@ public class AzureJobCoordinator implements JobCoordinator {
   try {
 Thread.sleep(random.nextInt(5000));
   } catch (InterruptedException e) {
-Thread.interrupted();
+Thread.currentThread().interrupt();
   }
   LOG.info("Checking for barrier state on the blob again...");
   blobBarrierState = leaderBlob.getBarrierState();

http://git-wip-us.apache.org/repos/asf/samza/blob/caec9c53/samza-azure/src/main/java/org/apache/samza/coordinator/AzureLock.java
--
diff --git 
a/samza-azure/src/main/java/org/apache/samza/coordinator/AzureLock.java 
b/samza-azure/src/main/java/org/apache/samza/coordinator/AzureLock.java
index c0d3ff2..172a0f3 100644
---