Join the upcoming stream processing meet-up at LinkedIn - Nov 2

2016-10-17 Thread Jagadish Venkatraman 
LinkedIn is hosting the upcoming Stream processing meetup on Wednesday,
November 2, 2016 at our Sunnyvale campus.

https://www.meetup.com/Stream-Processing-Meetup-LinkedIn/events/234454163/


*Agenda:*

6-6:30PM : Networking - Food & Drinks
6:30-7:05PM : Stream Processing using Apache Samza at LinkedIn: Past,
Present, and Future
7:05-7:40PM: Kafka Cruise Control: Auto Management of the Kafka Clusters
7:40-8:20 PM: Plumber - A data pipeline builder

*Additional Details:*

Please RSVP in the meet-up group if you plan to attend in-person.

Thanks,
Jagadish

Re: Review Request 52962: SAMZA-1029: Prepare release candidate for 0.11.0

2016-10-17 Thread Navina Ramesh 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52962/#review153001
---


Ship it!




Ship It!

- Navina Ramesh


On Oct. 18, 2016, midnight, Xinyu Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52962/
> ---
> 
> (Updated Oct. 18, 2016, midnight)
> 
> 
> Review request for samza and Navina Ramesh.
> 
> 
> Repository: samza
> 
> 
> Description
> ---
> 
> docs update for 0.11.0 release branch
> 
> 
> Diffs
> -
> 
>   docs/_config.yml dc1a66fa743d464c70d92406540fd7122c45272c 
>   docs/learn/tutorials/versioned/deploy-samza-job-from-hdfs.md 
> ca7b5f1a59724bbae9c46c7abd0d68cb3f019e3b 
>   docs/learn/tutorials/versioned/deploy-samza-to-CDH.md 
> daf762bc9f536520cceb503c5053283a80488bb1 
>   docs/learn/tutorials/versioned/remote-debugging-samza.md 
> 40db31a8152a999b549fde8f9155f4541d03147d 
>   docs/learn/tutorials/versioned/run-in-multi-node-yarn.md 
> bf2b59e3f4e0c6a3bfde0187db0b799f76797afb 
>   docs/learn/tutorials/versioned/samza-rest-getting-started.md 
> 942329e968bb02886df44b680bea8f75a221a289 
>   docs/startup/download/index.md 6a0c670bca01e01b9d8a73482af35cc144f1d524 
>   docs/startup/hello-samza/versioned/index.md 
> 8baacd390d41c5c87a426d63eec9ce5028de0cc2 
> 
> Diff: https://reviews.apache.org/r/52962/diff/
> 
> 
> Testing
> ---
> 
> local website.
> 
> 
> Thanks,
> 
> Xinyu Liu
> 
>

Re: Review Request 52960: SAMZA-1029: Prepare release candidate for 0.11.0

2016-10-17 Thread Navina Ramesh 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52960/#review153000
---




docs/learn/tutorials/versioned/samza-rest-getting-started.md (line 51)


Shouldn't this be 0.11.1-SNAPSHOT as well?


- Navina Ramesh


On Oct. 17, 2016, 11:45 p.m., Xinyu Liu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52960/
> ---
> 
> (Updated Oct. 17, 2016, 11:45 p.m.)
> 
> 
> Review request for samza and Navina Ramesh.
> 
> 
> Repository: samza
> 
> 
> Description
> ---
> 
> Doc updates for the master.
> 
> 
> Diffs
> -
> 
>   docs/_config.yml dc1a66fa743d464c70d92406540fd7122c45272c 
>   docs/_layouts/default.html 60e56b5a14f5211a5ff0e2812c1fc331a25ebfe5 
>   docs/archive/index.html b0a44c6ab40f4eeb7ea4dfc17a8c7243b7e6e035 
>   docs/learn/tutorials/versioned/deploy-samza-job-from-hdfs.md 
> ca7b5f1a59724bbae9c46c7abd0d68cb3f019e3b 
>   docs/learn/tutorials/versioned/deploy-samza-to-CDH.md 
> daf762bc9f536520cceb503c5053283a80488bb1 
>   docs/learn/tutorials/versioned/remote-debugging-samza.md 
> 40db31a8152a999b549fde8f9155f4541d03147d 
>   docs/learn/tutorials/versioned/run-in-multi-node-yarn.md 
> bf2b59e3f4e0c6a3bfde0187db0b799f76797afb 
>   docs/learn/tutorials/versioned/samza-rest-getting-started.md 
> 942329e968bb02886df44b680bea8f75a221a289 
>   docs/startup/download/index.md 6a0c670bca01e01b9d8a73482af35cc144f1d524 
>   docs/startup/hello-samza/versioned/index.md 
> 8baacd390d41c5c87a426d63eec9ce5028de0cc2 
>   gradle.properties f032b745a7ceae319996314f22c16fe0b664e705 
> 
> Diff: https://reviews.apache.org/r/52960/diff/
> 
> 
> Testing
> ---
> 
> Local website
> 
> 
> Thanks,
> 
> Xinyu Liu
> 
>

Review Request 52962: SAMZA-1029: Prepare release candidate for 0.11.0

2016-10-17 Thread Xinyu Liu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52962/
---

Review request for samza and Navina Ramesh.


Repository: samza


Description
---

docs update for 0.11.0 release branch


Diffs
-

  docs/_config.yml dc1a66fa743d464c70d92406540fd7122c45272c 
  docs/learn/tutorials/versioned/deploy-samza-job-from-hdfs.md 
ca7b5f1a59724bbae9c46c7abd0d68cb3f019e3b 
  docs/learn/tutorials/versioned/deploy-samza-to-CDH.md 
daf762bc9f536520cceb503c5053283a80488bb1 
  docs/learn/tutorials/versioned/remote-debugging-samza.md 
40db31a8152a999b549fde8f9155f4541d03147d 
  docs/learn/tutorials/versioned/run-in-multi-node-yarn.md 
bf2b59e3f4e0c6a3bfde0187db0b799f76797afb 
  docs/learn/tutorials/versioned/samza-rest-getting-started.md 
942329e968bb02886df44b680bea8f75a221a289 
  docs/startup/download/index.md 6a0c670bca01e01b9d8a73482af35cc144f1d524 
  docs/startup/hello-samza/versioned/index.md 
8baacd390d41c5c87a426d63eec9ce5028de0cc2 

Diff: https://reviews.apache.org/r/52962/diff/


Testing
---

local website.


Thanks,

Xinyu Liu

Re: Review Request 52960: SAMZA-1029: Prepare release candidate for 0.11.0

2016-10-17 Thread Xinyu Liu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52960/
---

(Updated Oct. 17, 2016, 11:45 p.m.)


Review request for samza and Navina Ramesh.


Repository: samza


Description (updated)
---

Doc updates for the master.


Diffs
-

  docs/_config.yml dc1a66fa743d464c70d92406540fd7122c45272c 
  docs/_layouts/default.html 60e56b5a14f5211a5ff0e2812c1fc331a25ebfe5 
  docs/archive/index.html b0a44c6ab40f4eeb7ea4dfc17a8c7243b7e6e035 
  docs/learn/tutorials/versioned/deploy-samza-job-from-hdfs.md 
ca7b5f1a59724bbae9c46c7abd0d68cb3f019e3b 
  docs/learn/tutorials/versioned/deploy-samza-to-CDH.md 
daf762bc9f536520cceb503c5053283a80488bb1 
  docs/learn/tutorials/versioned/remote-debugging-samza.md 
40db31a8152a999b549fde8f9155f4541d03147d 
  docs/learn/tutorials/versioned/run-in-multi-node-yarn.md 
bf2b59e3f4e0c6a3bfde0187db0b799f76797afb 
  docs/learn/tutorials/versioned/samza-rest-getting-started.md 
942329e968bb02886df44b680bea8f75a221a289 
  docs/startup/download/index.md 6a0c670bca01e01b9d8a73482af35cc144f1d524 
  docs/startup/hello-samza/versioned/index.md 
8baacd390d41c5c87a426d63eec9ce5028de0cc2 
  gradle.properties f032b745a7ceae319996314f22c16fe0b664e705 

Diff: https://reviews.apache.org/r/52960/diff/


Testing
---

Local website


Thanks,

Xinyu Liu

Review Request 52960: SAMZA-1029: Prepare release candidate for 0.11.0

2016-10-17 Thread Xinyu Liu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52960/
---

Review request for samza and Navina Ramesh.


Repository: samza


Description
---

SAMZA-1029: Prepare release candidate for 0.11.0


Diffs
-

  docs/_config.yml dc1a66fa743d464c70d92406540fd7122c45272c 
  docs/_layouts/default.html 60e56b5a14f5211a5ff0e2812c1fc331a25ebfe5 
  docs/archive/index.html b0a44c6ab40f4eeb7ea4dfc17a8c7243b7e6e035 
  docs/learn/tutorials/versioned/deploy-samza-job-from-hdfs.md 
ca7b5f1a59724bbae9c46c7abd0d68cb3f019e3b 
  docs/learn/tutorials/versioned/deploy-samza-to-CDH.md 
daf762bc9f536520cceb503c5053283a80488bb1 
  docs/learn/tutorials/versioned/remote-debugging-samza.md 
40db31a8152a999b549fde8f9155f4541d03147d 
  docs/learn/tutorials/versioned/run-in-multi-node-yarn.md 
bf2b59e3f4e0c6a3bfde0187db0b799f76797afb 
  docs/learn/tutorials/versioned/samza-rest-getting-started.md 
942329e968bb02886df44b680bea8f75a221a289 
  docs/startup/download/index.md 6a0c670bca01e01b9d8a73482af35cc144f1d524 
  docs/startup/hello-samza/versioned/index.md 
8baacd390d41c5c87a426d63eec9ce5028de0cc2 
  gradle.properties f032b745a7ceae319996314f22c16fe0b664e705 

Diff: https://reviews.apache.org/r/52960/diff/


Testing
---

Local website


Thanks,

Xinyu Liu

Re: Review Request 52492: Adding monitor to clean up stale local stores of jobs/tasks.

2016-10-17 Thread Jake Maes 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52492/#review152981
---




samza-rest/src/main/java/org/apache/samza/monitor/YarnLocalStoreMonitor.java 
(line 98)


nit: I think this would read better if demorgans law were applied



samza-rest/src/main/java/org/apache/samza/monitor/YarnLocalStoreMonitor.java 
(line 99)


I see tests for the other 2 conditions, but not the preferredHost



samza-rest/src/main/java/org/apache/samza/monitor/YarnLocalStoreMonitor.java 
(lines 164 - 166)


This class is still using YARN terminology when it is actually generic. 
Instead of RM hosts, we just want the hosts which support the Jobs and Tasks 
endpoints. 

JOB_STATUS_SERVERS, perhaps?



samza-rest/src/main/java/org/apache/samza/monitor/YarnLocalStoreMonitorConfig.java
 (line 49)


Same in this class, do we really need to make this Yarn-specific or can we 
make it generic now?


- Jake Maes


On Oct. 13, 2016, 9:01 p.m., Shanthoosh Venkataraman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52492/
> ---
> 
> (Updated Oct. 13, 2016, 9:01 p.m.)
> 
> 
> Review request for samza.
> 
> 
> Repository: samza
> 
> 
> Description
> ---
> 
> This patch contains the samza-rest monitor that periodically cleans up the 
> stale local stores of dead jobs/tasks. It performs the store deletion in two 
> phases. Initially it deletes the offset file in the local task stores if the 
> following condition is true. ((jobIsNotRunning || preferedHost != nmHost) && 
> offsetFilelastModifiedTime is greater than deleteRetention). During the 
> subsequent run, it deletes the local task stores if it does not contain 
> offset file. Please refer to the design doc of SAMZA-656 
> (https://issues.apache.org/jira/secure/attachment/12828083/DESIGN-SAMZA-656.pdf)
>  for more details.
> 
> 
> Diffs
> -
> 
>   build.gradle 2bea27b75288d3103178bc3762b9556f6e69cdd1 
>   
> samza-rest/src/main/java/org/apache/samza/monitor/YarnLocalStoreMonitor.java 
> PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/monitor/YarnLocalStoreMonitorConfig.java
>  PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/monitor/YarnLocalStoreMonitorFactory.java
>  PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/resources/ResourceConstants.java
>  PRE-CREATION 
>   
> samza-rest/src/test/java/org/apache/samza/monitor/TestYarnLocalStoreMonitor.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52492/diff/
> 
> 
> Testing
> ---
> 
> Unit testing and manual testing are done to verify the functionality.
> 
> 
> Thanks,
> 
> Shanthoosh Venkataraman
> 
>

Re: Review Request 52476: Do not load task store which are older than delete tombstones.

2016-10-17 Thread Prateek Maheshwari 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52476/#review152976
---




samza-core/src/main/scala/org/apache/samza/config/StorageConfig.scala (line 33)


Usually more readable if you write this as a multiplication: 1 * 24 * 60 * 
60 * 1000L



samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala (line 
532)


Prefer passing the one config that we need explicitly instead of passing 
the config object.



samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala 
(line 26)


Delete or import explicitly.



samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala 
(line 29)


Unrelated to RB but prefer explicit imports.



samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala 
(line 71)


SystemClock exists so that you can pass a "Clock" to your method/class and 
mock it in tests. Let's either do that (preferred) or use 
System.currentTimeMillis() directly.



samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala 
(line 98)


Looks like we've updated `fileOffset` in `#readOffsetFile` as a side effect 
even when the store is stale. Is that what we want here?



samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala 
(line 100)


Add an INFO message here.



samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala 
(line 106)


Add method description.



samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala 
(line 113)


Another case we ran into on Friday - if the oldest offset in the changelog 
topic is newer than the offset in the OFFSET file. Do you need to handle that 
here?

Nitpick: would isStaleStore be clearer?



samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala 
(line 118)


Looks like this is already logged at line 163?



samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala 
(line 119)


Don't `return` in scala code.



samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala 
(line 121)


Mention somewhere in the message that this means that the store is stale.



samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala 
(line 150)


I'd prefer to split this into two methods - existence check and file read. 
Would be even nicer if fileOffset was updated explicitly (after staleness 
checks etc.) and not as a side effect of reading the file.

If you don't, let's add return type to method signature.



samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala 
(line 155)


Unrelated, but let's make this info.


- Prateek Maheshwari


On Oct. 17, 2016, 3:40 p.m., Shanthoosh Venkataraman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52476/
> ---
> 
> (Updated Oct. 17, 2016, 3:40 p.m.)
> 
> 
> Review request for samza.
> 
> 
> Repository: samza
> 
> 
> Description
> ---
> 
> Every local task store is backed up by a kafka changelog topic. Due to log 
> compaction, delete tombstones of the changelog topic have a ttl of 
> delete.retention.ms. Replaying the events from the changelog that has missing 
> delete tombstones, would result in creation of an inconsistent local 
> store(due to the missing of some delete events). This patch deletes the local 
> stores in which difference between current time and last modified time of the 
> offset file is greater than delete.retention.ms during the container startup.
> 
> 
> Diffs
> -
> 
>   samza-core/src/main/java/org/apache/samza/storage/StorageRecovery.java 
> 9329edf7d724f3a0d9235354bb77936f713e3b5f 
>   samza-core/src/main/scala/org/apache/samza/config/StorageConfig.scala 
> be3f1068f7921c9c8f8697577efea6580672813e 
>   samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala 
> 05a996c98075ea8ed3767af666b9beeb1933f2a6 
>   samza-core/src/main/scala/org/apache/samza/stor

Re: Review Request 52476: Do not load task store which are older than delete tombstones.

2016-10-17 Thread Shanthoosh Venkataraman 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52476/
---

(Updated Oct. 17, 2016, 10:40 p.m.)


Review request for samza.


Repository: samza


Description
---

Every local task store is backed up by a kafka changelog topic. Due to log 
compaction, delete tombstones of the changelog topic have a ttl of 
delete.retention.ms. Replaying the events from the changelog that has missing 
delete tombstones, would result in creation of an inconsistent local store(due 
to the missing of some delete events). This patch deletes the local stores in 
which difference between current time and last modified time of the offset file 
is greater than delete.retention.ms during the container startup.


Diffs (updated)
-

  samza-core/src/main/java/org/apache/samza/storage/StorageRecovery.java 
9329edf7d724f3a0d9235354bb77936f713e3b5f 
  samza-core/src/main/scala/org/apache/samza/config/StorageConfig.scala 
be3f1068f7921c9c8f8697577efea6580672813e 
  samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala 
05a996c98075ea8ed3767af666b9beeb1933f2a6 
  samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala 
0b7bcdda1639eea8239a69c31bdf42558e9077d2 
  
samza-core/src/test/scala/org/apache/samza/storage/TestTaskStorageManager.scala 
4d40f520e54beb643acd8410c772b75e2f6a9162 
  samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala 
973ab8cfb3d248bec7efe5e338f5e667f097556d 

Diff: https://reviews.apache.org/r/52476/diff/


Testing
---

Unit testing and manual testing has been done to verify the functionality.


Thanks,

Shanthoosh Venkataraman

Re: Review Request 52476: Do not load task store which are older than delete tombstones.

2016-10-17 Thread Shanthoosh Venkataraman 


> On Oct. 5, 2016, 2:08 a.m., Jake Maes wrote:
> > Looks better, but I think there's still one major part missing. 
> > 
> > In order to have agreement between a kafka changelog and the task storage, 
> > the changelog should be created with the same delete.retention.ms property. 
> > 
> > There are 2 ways to do this:
> > 1. (preferred) update the kafka system admin to read the samza changelog 
> > property that you've defined (which also needs to be added to the config 
> > table, btw) and create the topic with that value for delete.retention.ms
> > 2. Rename the property so it's one of the "topic-level-property" so it gets 
> > automatically passed to kafka. This is convenient but wouldn't apply to 
> > other systems, which could be useful if those other systems have a delete 
> > retention policy.
> 
> Shanthoosh Venkataraman wrote:
> I think 1) is the only plausible way to accomplish this through job 
> config. delete.retention.ms configuration is associated only with stores 
> changelog, not applicable to topics in general, so making it topic level 
> property might notbe a good idea. Enforcing the delete.retention.ms property 
> is harder to accomplish through config, since kafka is a external system. 
> Ideally, if there's a way to fetch kafka metadata/config(delete.retention.ms) 
> about a topic, during container startups we could fetch that value, rather 
> than expecting the users to specify it.
> 
> Jake Maes wrote:
> Please take a look at 
> org.apache.samza.config.KafkaConfig#getChangelogKafkaProperties

Done.


- Shanthoosh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52476/#review151439
---


On Oct. 17, 2016, 10:40 p.m., Shanthoosh Venkataraman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52476/
> ---
> 
> (Updated Oct. 17, 2016, 10:40 p.m.)
> 
> 
> Review request for samza.
> 
> 
> Repository: samza
> 
> 
> Description
> ---
> 
> Every local task store is backed up by a kafka changelog topic. Due to log 
> compaction, delete tombstones of the changelog topic have a ttl of 
> delete.retention.ms. Replaying the events from the changelog that has missing 
> delete tombstones, would result in creation of an inconsistent local 
> store(due to the missing of some delete events). This patch deletes the local 
> stores in which difference between current time and last modified time of the 
> offset file is greater than delete.retention.ms during the container startup.
> 
> 
> Diffs
> -
> 
>   samza-core/src/main/java/org/apache/samza/storage/StorageRecovery.java 
> 9329edf7d724f3a0d9235354bb77936f713e3b5f 
>   samza-core/src/main/scala/org/apache/samza/config/StorageConfig.scala 
> be3f1068f7921c9c8f8697577efea6580672813e 
>   samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala 
> 05a996c98075ea8ed3767af666b9beeb1933f2a6 
>   samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala 
> 0b7bcdda1639eea8239a69c31bdf42558e9077d2 
>   
> samza-core/src/test/scala/org/apache/samza/storage/TestTaskStorageManager.scala
>  4d40f520e54beb643acd8410c772b75e2f6a9162 
>   samza-kafka/src/main/scala/org/apache/samza/config/KafkaConfig.scala 
> 973ab8cfb3d248bec7efe5e338f5e667f097556d 
> 
> Diff: https://reviews.apache.org/r/52476/diff/
> 
> 
> Testing
> ---
> 
> Unit testing and manual testing has been done to verify the functionality.
> 
> 
> Thanks,
> 
> Shanthoosh Venkataraman
> 
>

Re: Review Request 52168: Tasks endpoint to list the complete details of all tasks related to a job

2016-10-17 Thread Jake Maes 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52168/#review152966
---




samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala (line 
99)


shouldn't this line also use the new Util method to get the container name?


- Jake Maes


On Oct. 13, 2016, 11:57 p.m., Shanthoosh Venkataraman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52168/
> ---
> 
> (Updated Oct. 13, 2016, 11:57 p.m.)
> 
> 
> Review request for samza.
> 
> 
> Repository: samza
> 
> 
> Description
> ---
> 
> This patch contains the following changes
>  * Http get api to list the complete details of all the tasks that belongs to 
> a job. 
>  * Refactored some methods in coordinator stream, to reuse the existing 
> functionality of getting jobConfig from the coordinator stream.
> 
> 
> Diffs
> -
> 
>   docs/learn/documentation/versioned/rest/resource-directory.md 
> 79746d1e2eb3491e4bd26c3c7cf6c7efd150d8ef 
>   docs/learn/documentation/versioned/rest/resources/tasks.md PRE-CREATION 
>   samza-core/src/main/scala/org/apache/samza/config/JobConfig.scala 
> 13b72fae7815ddaea7ae03a24f1a426ca51613cc 
>   samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala 
> 05a996c98075ea8ed3767af666b9beeb1933f2a6 
>   samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinator.scala 
> df63b97e9d598ecd1840111ba490a723e410d089 
>   samza-core/src/main/scala/org/apache/samza/job/JobRunner.scala 
> 022b480856483059cb9f837a08f97a718bc04c31 
>   samza-core/src/main/scala/org/apache/samza/util/Util.scala 
> c4836f202f7eda1d4e71eac94fd48e46207b0316 
>   samza-rest/src/main/java/org/apache/samza/rest/model/Partition.java 
> PRE-CREATION 
>   samza-rest/src/main/java/org/apache/samza/rest/model/Task.java PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/job/AbstractJobProxy.java
>  4d8647f3e1e650632e38b47ba5a8a2dac004f545 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/job/JobProxyFactory.java 
> 067711a74e5b0d7277a9c8b2d2517b56e9cfbcca 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/job/SimpleYarnJobProxy.java
>  a935c98730f85f448c688a6baf2e8ddffdbb2cb4 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/job/SimpleYarnJobProxyFactory.java
>  11d93d4608d23a4e3fb3bfc50dfac35ab6dbdf3c 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/task/SamzaTaskProxy.java 
> PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/task/SamzaTaskProxyFactory.java
>  PRE-CREATION 
>   samza-rest/src/main/java/org/apache/samza/rest/proxy/task/TaskProxy.java 
> PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/task/TaskProxyFactory.java
>  PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/task/TaskResourceConfig.java
>  PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/resources/BaseResourceConfig.java
>  PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/resources/DefaultResourceFactory.java
>  e0224c6bcf4aeaa336e5786ac472482507fcd382 
>   samza-rest/src/main/java/org/apache/samza/rest/resources/JobsResource.java 
> a566db598c284d69ea61af88fdc0851483d5a089 
>   
> samza-rest/src/main/java/org/apache/samza/rest/resources/JobsResourceConfig.java
>  527482d2ee55747e7b3f9c54c8a3b1afe7ad8797 
>   samza-rest/src/main/java/org/apache/samza/rest/resources/Responses.java 
> PRE-CREATION 
>   samza-rest/src/main/java/org/apache/samza/rest/resources/TasksResource.java 
> PRE-CREATION 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/TestJobsResource.java
>  7db437b348ecd286185898b8f8ab0220d59da71a 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/TestTasksResource.java
>  PRE-CREATION 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/mock/MockInstallationFinder.java
>  PRE-CREATION 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/mock/MockResourceFactory.java
>  PRE-CREATION 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/mock/MockTaskProxy.java
>  PRE-CREATION 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/mock/MockTaskProxyFactory.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52168/diff/
> 
> 
> Testing
> ---
> 
> Manual and unit testing has been done to verify the apis.
> 
> 
> Thanks,
> 
> Shanthoosh Venkataraman
> 
>

Re: Review Request 52168: Tasks endpoint to list the complete details of all tasks related to a job

2016-10-17 Thread Jake Maes 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52168/#review152928
---




docs/learn/documentation/versioned/rest/resources/tasks.md (line 23)


*used*



docs/learn/documentation/versioned/rest/resources/tasks.md (line 105)


This comment still looks out of place. Maybe it should just be incorporated 
in the table.



samza-rest/src/main/java/org/apache/samza/rest/resources/Responses.java (line 
62)


This doesn't belong here. It has nothing to do with responses.



samza-rest/src/test/java/org/apache/samza/rest/resources/TestJobsResource.java 
(line 55)


Why has this property been added to this test? It should not be mandatory.


- Jake Maes


On Oct. 13, 2016, 11:57 p.m., Shanthoosh Venkataraman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52168/
> ---
> 
> (Updated Oct. 13, 2016, 11:57 p.m.)
> 
> 
> Review request for samza.
> 
> 
> Repository: samza
> 
> 
> Description
> ---
> 
> This patch contains the following changes
>  * Http get api to list the complete details of all the tasks that belongs to 
> a job. 
>  * Refactored some methods in coordinator stream, to reuse the existing 
> functionality of getting jobConfig from the coordinator stream.
> 
> 
> Diffs
> -
> 
>   docs/learn/documentation/versioned/rest/resource-directory.md 
> 79746d1e2eb3491e4bd26c3c7cf6c7efd150d8ef 
>   docs/learn/documentation/versioned/rest/resources/tasks.md PRE-CREATION 
>   samza-core/src/main/scala/org/apache/samza/config/JobConfig.scala 
> 13b72fae7815ddaea7ae03a24f1a426ca51613cc 
>   samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala 
> 05a996c98075ea8ed3767af666b9beeb1933f2a6 
>   samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinator.scala 
> df63b97e9d598ecd1840111ba490a723e410d089 
>   samza-core/src/main/scala/org/apache/samza/job/JobRunner.scala 
> 022b480856483059cb9f837a08f97a718bc04c31 
>   samza-core/src/main/scala/org/apache/samza/util/Util.scala 
> c4836f202f7eda1d4e71eac94fd48e46207b0316 
>   samza-rest/src/main/java/org/apache/samza/rest/model/Partition.java 
> PRE-CREATION 
>   samza-rest/src/main/java/org/apache/samza/rest/model/Task.java PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/job/AbstractJobProxy.java
>  4d8647f3e1e650632e38b47ba5a8a2dac004f545 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/job/JobProxyFactory.java 
> 067711a74e5b0d7277a9c8b2d2517b56e9cfbcca 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/job/SimpleYarnJobProxy.java
>  a935c98730f85f448c688a6baf2e8ddffdbb2cb4 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/job/SimpleYarnJobProxyFactory.java
>  11d93d4608d23a4e3fb3bfc50dfac35ab6dbdf3c 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/task/SamzaTaskProxy.java 
> PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/task/SamzaTaskProxyFactory.java
>  PRE-CREATION 
>   samza-rest/src/main/java/org/apache/samza/rest/proxy/task/TaskProxy.java 
> PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/task/TaskProxyFactory.java
>  PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/task/TaskResourceConfig.java
>  PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/resources/BaseResourceConfig.java
>  PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/resources/DefaultResourceFactory.java
>  e0224c6bcf4aeaa336e5786ac472482507fcd382 
>   samza-rest/src/main/java/org/apache/samza/rest/resources/JobsResource.java 
> a566db598c284d69ea61af88fdc0851483d5a089 
>   
> samza-rest/src/main/java/org/apache/samza/rest/resources/JobsResourceConfig.java
>  527482d2ee55747e7b3f9c54c8a3b1afe7ad8797 
>   samza-rest/src/main/java/org/apache/samza/rest/resources/Responses.java 
> PRE-CREATION 
>   samza-rest/src/main/java/org/apache/samza/rest/resources/TasksResource.java 
> PRE-CREATION 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/TestJobsResource.java
>  7db437b348ecd286185898b8f8ab0220d59da71a 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/TestTasksResource.java
>  PRE-CREATION 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/mock/MockInstallationFinder.java
>  PRE-CREATION 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/mock/MockResourceFactory.java
>  PRE-CREATION 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/mock/MockTaskProxy.java
>  PRE-CREATIO

Re: Review Request 52168: Tasks endpoint to list the complete details of all tasks related to a job

2016-10-17 Thread Shanthoosh Venkataraman 


> On Oct. 13, 2016, 12:55 a.m., Jake Maes wrote:
> > docs/learn/documentation/versioned/rest/resources/tasks.md, line 55
> > 
> >
> > What is the value of the container name?
> 
> Shanthoosh Venkataraman wrote:
> name of the samza container in which the task is running. With respect to 
> current implementation, container name is unique within a job.
> 
> Jake Maes wrote:
> Sorry, the question was poorly worded. How is container name useful? Do 
> we need it? If so, couldn't we derive it from the container id?

It will be used for debugging purposes in the monitor/client. It logically 
belongs to the task hierarchy. To answer questions like finding list of 
containers running on a particular host. These questions could be answered from 
filter on the container name & preferred host. Adding this also in a way 
completes the entire task model.

Currently container name is of the form samza-container-{containerId}. Hence, 
we could derive it. But the logic used to generate container id is specific to 
samza job model generator and is bound to change. Hence deriving it might not 
be a good idea.


- Shanthoosh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52168/#review152418
---


On Oct. 13, 2016, 11:57 p.m., Shanthoosh Venkataraman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52168/
> ---
> 
> (Updated Oct. 13, 2016, 11:57 p.m.)
> 
> 
> Review request for samza.
> 
> 
> Repository: samza
> 
> 
> Description
> ---
> 
> This patch contains the following changes
>  * Http get api to list the complete details of all the tasks that belongs to 
> a job. 
>  * Refactored some methods in coordinator stream, to reuse the existing 
> functionality of getting jobConfig from the coordinator stream.
> 
> 
> Diffs
> -
> 
>   docs/learn/documentation/versioned/rest/resource-directory.md 
> 79746d1e2eb3491e4bd26c3c7cf6c7efd150d8ef 
>   docs/learn/documentation/versioned/rest/resources/tasks.md PRE-CREATION 
>   samza-core/src/main/scala/org/apache/samza/config/JobConfig.scala 
> 13b72fae7815ddaea7ae03a24f1a426ca51613cc 
>   samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala 
> 05a996c98075ea8ed3767af666b9beeb1933f2a6 
>   samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinator.scala 
> df63b97e9d598ecd1840111ba490a723e410d089 
>   samza-core/src/main/scala/org/apache/samza/job/JobRunner.scala 
> 022b480856483059cb9f837a08f97a718bc04c31 
>   samza-core/src/main/scala/org/apache/samza/util/Util.scala 
> c4836f202f7eda1d4e71eac94fd48e46207b0316 
>   samza-rest/src/main/java/org/apache/samza/rest/model/Partition.java 
> PRE-CREATION 
>   samza-rest/src/main/java/org/apache/samza/rest/model/Task.java PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/job/AbstractJobProxy.java
>  4d8647f3e1e650632e38b47ba5a8a2dac004f545 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/job/JobProxyFactory.java 
> 067711a74e5b0d7277a9c8b2d2517b56e9cfbcca 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/job/SimpleYarnJobProxy.java
>  a935c98730f85f448c688a6baf2e8ddffdbb2cb4 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/job/SimpleYarnJobProxyFactory.java
>  11d93d4608d23a4e3fb3bfc50dfac35ab6dbdf3c 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/task/SamzaTaskProxy.java 
> PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/task/SamzaTaskProxyFactory.java
>  PRE-CREATION 
>   samza-rest/src/main/java/org/apache/samza/rest/proxy/task/TaskProxy.java 
> PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/task/TaskProxyFactory.java
>  PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/task/TaskResourceConfig.java
>  PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/resources/BaseResourceConfig.java
>  PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/resources/DefaultResourceFactory.java
>  e0224c6bcf4aeaa336e5786ac472482507fcd382 
>   samza-rest/src/main/java/org/apache/samza/rest/resources/JobsResource.java 
> a566db598c284d69ea61af88fdc0851483d5a089 
>   
> samza-rest/src/main/java/org/apache/samza/rest/resources/JobsResourceConfig.java
>  527482d2ee55747e7b3f9c54c8a3b1afe7ad8797 
>   samza-rest/src/main/java/org/apache/samza/rest/resources/Responses.java 
> PRE-CREATION 
>   samza-rest/src/main/java/org/apache/samza/rest/resources/TasksResource.java 
> PRE-CREATION 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/TestJobsResource.java
>  7db437b348ecd286185898b8f8ab0220d59da71a 
>   
> samza-rest/sr

Re: Review Request 52168: Tasks endpoint to list the complete details of all tasks related to a job

2016-10-17 Thread Jake Maes 


> On Oct. 13, 2016, 12:55 a.m., Jake Maes wrote:
> > docs/learn/documentation/versioned/rest/resources/tasks.md, line 55
> > 
> >
> > What is the value of the container name?
> 
> Shanthoosh Venkataraman wrote:
> name of the samza container in which the task is running. With respect to 
> current implementation, container name is unique within a job.

Sorry, the question was poorly worded. How is container name useful? Do we need 
it? If so, couldn't we derive it from the container id?


- Jake


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52168/#review152418
---


On Oct. 13, 2016, 11:57 p.m., Shanthoosh Venkataraman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52168/
> ---
> 
> (Updated Oct. 13, 2016, 11:57 p.m.)
> 
> 
> Review request for samza.
> 
> 
> Repository: samza
> 
> 
> Description
> ---
> 
> This patch contains the following changes
>  * Http get api to list the complete details of all the tasks that belongs to 
> a job. 
>  * Refactored some methods in coordinator stream, to reuse the existing 
> functionality of getting jobConfig from the coordinator stream.
> 
> 
> Diffs
> -
> 
>   docs/learn/documentation/versioned/rest/resource-directory.md 
> 79746d1e2eb3491e4bd26c3c7cf6c7efd150d8ef 
>   docs/learn/documentation/versioned/rest/resources/tasks.md PRE-CREATION 
>   samza-core/src/main/scala/org/apache/samza/config/JobConfig.scala 
> 13b72fae7815ddaea7ae03a24f1a426ca51613cc 
>   samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala 
> 05a996c98075ea8ed3767af666b9beeb1933f2a6 
>   samza-core/src/main/scala/org/apache/samza/coordinator/JobCoordinator.scala 
> df63b97e9d598ecd1840111ba490a723e410d089 
>   samza-core/src/main/scala/org/apache/samza/job/JobRunner.scala 
> 022b480856483059cb9f837a08f97a718bc04c31 
>   samza-core/src/main/scala/org/apache/samza/util/Util.scala 
> c4836f202f7eda1d4e71eac94fd48e46207b0316 
>   samza-rest/src/main/java/org/apache/samza/rest/model/Partition.java 
> PRE-CREATION 
>   samza-rest/src/main/java/org/apache/samza/rest/model/Task.java PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/job/AbstractJobProxy.java
>  4d8647f3e1e650632e38b47ba5a8a2dac004f545 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/job/JobProxyFactory.java 
> 067711a74e5b0d7277a9c8b2d2517b56e9cfbcca 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/job/SimpleYarnJobProxy.java
>  a935c98730f85f448c688a6baf2e8ddffdbb2cb4 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/job/SimpleYarnJobProxyFactory.java
>  11d93d4608d23a4e3fb3bfc50dfac35ab6dbdf3c 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/task/SamzaTaskProxy.java 
> PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/task/SamzaTaskProxyFactory.java
>  PRE-CREATION 
>   samza-rest/src/main/java/org/apache/samza/rest/proxy/task/TaskProxy.java 
> PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/task/TaskProxyFactory.java
>  PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/proxy/task/TaskResourceConfig.java
>  PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/resources/BaseResourceConfig.java
>  PRE-CREATION 
>   
> samza-rest/src/main/java/org/apache/samza/rest/resources/DefaultResourceFactory.java
>  e0224c6bcf4aeaa336e5786ac472482507fcd382 
>   samza-rest/src/main/java/org/apache/samza/rest/resources/JobsResource.java 
> a566db598c284d69ea61af88fdc0851483d5a089 
>   
> samza-rest/src/main/java/org/apache/samza/rest/resources/JobsResourceConfig.java
>  527482d2ee55747e7b3f9c54c8a3b1afe7ad8797 
>   samza-rest/src/main/java/org/apache/samza/rest/resources/Responses.java 
> PRE-CREATION 
>   samza-rest/src/main/java/org/apache/samza/rest/resources/TasksResource.java 
> PRE-CREATION 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/TestJobsResource.java
>  7db437b348ecd286185898b8f8ab0220d59da71a 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/TestTasksResource.java
>  PRE-CREATION 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/mock/MockInstallationFinder.java
>  PRE-CREATION 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/mock/MockResourceFactory.java
>  PRE-CREATION 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/mock/MockTaskProxy.java
>  PRE-CREATION 
>   
> samza-rest/src/test/java/org/apache/samza/rest/resources/mock/MockTaskProxyFactory.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52168/diff/
> 
> 
> Testing
> ---
> 
> Manual and unit tes

Re: Review Request 52476: Do not load task store which are older than delete tombstones.

2016-10-17 Thread Jake Maes 


> On Oct. 5, 2016, 2:08 a.m., Jake Maes wrote:
> > Looks better, but I think there's still one major part missing. 
> > 
> > In order to have agreement between a kafka changelog and the task storage, 
> > the changelog should be created with the same delete.retention.ms property. 
> > 
> > There are 2 ways to do this:
> > 1. (preferred) update the kafka system admin to read the samza changelog 
> > property that you've defined (which also needs to be added to the config 
> > table, btw) and create the topic with that value for delete.retention.ms
> > 2. Rename the property so it's one of the "topic-level-property" so it gets 
> > automatically passed to kafka. This is convenient but wouldn't apply to 
> > other systems, which could be useful if those other systems have a delete 
> > retention policy.
> 
> Shanthoosh Venkataraman wrote:
> I think 1) is the only plausible way to accomplish this through job 
> config. delete.retention.ms configuration is associated only with stores 
> changelog, not applicable to topics in general, so making it topic level 
> property might notbe a good idea. Enforcing the delete.retention.ms property 
> is harder to accomplish through config, since kafka is a external system. 
> Ideally, if there's a way to fetch kafka metadata/config(delete.retention.ms) 
> about a topic, during container startups we could fetch that value, rather 
> than expecting the users to specify it.

Please take a look at 
org.apache.samza.config.KafkaConfig#getChangelogKafkaProperties


- Jake


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52476/#review151439
---


On Oct. 4, 2016, 11:33 p.m., Shanthoosh Venkataraman wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52476/
> ---
> 
> (Updated Oct. 4, 2016, 11:33 p.m.)
> 
> 
> Review request for samza.
> 
> 
> Repository: samza
> 
> 
> Description
> ---
> 
> Every local task store is backed up by a kafka changelog topic. Due to log 
> compaction, delete tombstones of the changelog topic have a ttl of 
> delete.retention.ms. Replaying the events from the changelog that has missing 
> delete tombstones, would result in creation of an inconsistent local 
> store(due to the missing of some delete events). This patch deletes the local 
> stores in which difference between current time and last modified time of the 
> offset file is greater than delete.retention.ms during the container startup.
> 
> 
> Diffs
> -
> 
>   samza-core/src/main/java/org/apache/samza/storage/StorageRecovery.java 
> 9329edf7d724f3a0d9235354bb77936f713e3b5f 
>   samza-core/src/main/scala/org/apache/samza/config/StorageConfig.scala 
> be3f1068f7921c9c8f8697577efea6580672813e 
>   samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala 
> 05a996c98075ea8ed3767af666b9beeb1933f2a6 
>   samza-core/src/main/scala/org/apache/samza/storage/TaskStorageManager.scala 
> 0b7bcdda1639eea8239a69c31bdf42558e9077d2 
>   
> samza-core/src/test/scala/org/apache/samza/storage/TestTaskStorageManager.scala
>  4d40f520e54beb643acd8410c772b75e2f6a9162 
> 
> Diff: https://reviews.apache.org/r/52476/diff/
> 
> 
> Testing
> ---
> 
> Unit testing and manual testing has been done to verify the functionality.
> 
> 
> Thanks,
> 
> Shanthoosh Venkataraman
> 
>