[jira] [Commented] (SAMZA-1478) Delete unneeded data from intermediate Kafka topic on offset commit

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SAMZA-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233651#comment-16233651
 ] 

ASF GitHub Bot commented on SAMZA-1478:
---

GitHub user lindong28 opened a pull request:

https://github.com/apache/samza/pull/347

SAMZA-1478; Delete unneeded data from intermediate Kafka topic on offset 
commit



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lindong28/samza SAMZA-1478

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/samza/pull/347.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #347


commit c418d59b45133478e0152625e2f2ec030e3400f3
Author: Dong Lin 
Date:   2017-11-01T03:41:21Z

SAMZA-1478; Delete unneeded data from intermediate Kafka topic on offset 
commit




> Delete unneeded data from intermediate Kafka topic on offset commit
> ---
>
> Key: SAMZA-1478
> URL: https://issues.apache.org/jira/browse/SAMZA-1478
> Project: Samza
>  Issue Type: New Feature
>Reporter: Dong Lin
>Assignee: Dong Lin
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (SAMZA-1478) Delete unneeded data from intermediate Kafka topic on offset commit

[Dong Lin (JIRA)]

Dong Lin created SAMZA-1478:
---

 Summary: Delete unneeded data from intermediate Kafka topic on 
offset commit
 Key: SAMZA-1478
 URL: https://issues.apache.org/jira/browse/SAMZA-1478
 Project: Samza
  Issue Type: New Feature
Reporter: Dong Lin
Assignee: Dong Lin
Priority: Major






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (SAMZA-1477) Fix issues found by BEAM tests

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SAMZA-1477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16227190#comment-16227190
 ] 

ASF GitHub Bot commented on SAMZA-1477:
---

GitHub user xinyuiscool opened a pull request:

https://github.com/apache/samza/pull/345

SAMZA-1477: Fix issues found by BEAM tests

A bunch of issues were found by BEAM tests, which includes:

1) WatermarkFunction needs to be able to return output after 
processWatermark()
2) control message doesn't implement the equals() and hashcode()
3) Some kafka system related code is not scala 2.10 compatible for tests.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xinyuiscool/samza SAMZA-1477

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/samza/pull/345.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #345


commit 735f4b966b19521c3d7003c25209f5344f0fb6c3
Author: xiliu 
Date:   2017-10-31T17:41:56Z

SAMZA-1477: Fix issues found by BEAM tests




> Fix issues found by BEAM tests
> --
>
> Key: SAMZA-1477
> URL: https://issues.apache.org/jira/browse/SAMZA-1477
> Project: Samza
>  Issue Type: Bug
>Affects Versions: 0.14.0
>Reporter: Xinyu Liu
>Assignee: Xinyu Liu
>
> A bunch of issues were found by BEAM tests, which includes:
> 1) WatermarkFunction needs to be able to return output after 
> processWatermark()
> 2) control message doesn't implement the equals() and hashcode()
> 3) Some kafka system related code is not scala 2.10 compatible for tests.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (SAMZA-1477) Fix issues found by BEAM tests

[Xinyu Liu (JIRA)]

Xinyu Liu created SAMZA-1477:


 Summary: Fix issues found by BEAM tests
 Key: SAMZA-1477
 URL: https://issues.apache.org/jira/browse/SAMZA-1477
 Project: Samza
  Issue Type: Bug
Affects Versions: 0.14.0
Reporter: Xinyu Liu
Assignee: Xinyu Liu


A bunch of issues were found by BEAM tests, which includes:

1) WatermarkFunction needs to be able to return output after processWatermark()
2) control message doesn't implement the equals() and hashcode()
3) Some kafka system related code is not scala 2.10 compatible for tests.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

samza git commit: SAMZA-1476: Disable flaky TestStatefulTask.testShouldStartAndRestore

[jagadish]

Repository: samza
Updated Branches:
  refs/heads/master c94a26afe -> 2d1073249


SAMZA-1476: Disable flaky TestStatefulTask.testShouldStartAndRestore


Project: http://git-wip-us.apache.org/repos/asf/samza/repo
Commit: http://git-wip-us.apache.org/repos/asf/samza/commit/2d107324
Tree: http://git-wip-us.apache.org/repos/asf/samza/tree/2d107324
Diff: http://git-wip-us.apache.org/repos/asf/samza/diff/2d107324

Branch: refs/heads/master
Commit: 2d1073249174975f81b9187004aacbd46a91a1d0
Parents: c94a26a
Author: Jagadish 
Authored: Tue Oct 31 08:50:19 2017 -0700
Committer: Jagadish 
Committed: Tue Oct 31 08:50:19 2017 -0700

--
 .../scala/org/apache/samza/test/integration/TestStatefulTask.scala | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/samza/blob/2d107324/samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala
--
diff --git 
a/samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala
 
b/samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala
index 3ef08e5..0b76223 100644
--- 
a/samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala
+++ 
b/samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala
@@ -78,7 +78,7 @@ class TestStatefulTask extends StreamTaskTestUtil {
 // since the second part of the test expects to replay the input streams.
 "systems.kafka.streams.input.samza.reset.offset" -> "true"))
 
-  @Test
+  //@Test
   def testShouldStartAndRestore {
 // Have to do this in one test to guarantee ordering.
 testShouldStartTaskForFirstTime

[jira] [Created] (SAMZA-1476) Flaky test: testShouldStartAndRestore

[Jagadish (JIRA)]

Jagadish created SAMZA-1476:
---

 Summary: Flaky test: testShouldStartAndRestore   
 Key: SAMZA-1476
 URL: https://issues.apache.org/jira/browse/SAMZA-1476
 Project: Samza
  Issue Type: Bug
Reporter: Jagadish


{code}  
java.lang.AssertionError: Timed out of waiting for message rather than received 
one. at org.junit.Assert.fail(Assert.java:91) at 
org.junit.Assert.assertTrue(Assert.java:43) at 
org.apache.samza.test.integration.TestTask.awaitMessage(StreamTaskTestUtil.scala:331)
 at 
org.apache.samza.test.integration.StreamTaskTestUtil.send(StreamTaskTestUtil.scala:235)
 at 
org.apache.samza.test.integration.TestStatefulTask$$anonfun$testShouldStartTaskForFirstTime$1.apply(TestStatefulTask.scala:97)
 at 
org.apache.samza.test.integration.TestStatefulTask$$anonfun$testShouldStartTaskForFirstTime$1.apply(TestStatefulTask.scala:97)
 at scala.collection.immutable.List.foreach(List.scala:381) at 
org.apache.samza.test.integration.TestStatefulTask.testShouldStartTaskForFirstTime(TestStatefulTask.scala:97)
 at 
org.apache.samza.test.integration.TestStatefulTask.testShouldStartAndRestore
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)