[jira] [Commented] (RYA-467) Rya Streams - QueryChangeLog topic deletes content after 7 days.
[ https://issues.apache.org/jira/browse/RYA-467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423016#comment-16423016 ] ASF GitHub Bot commented on RYA-467: Github user asfgit commented on the issue: https://github.com/apache/incubator-rya/pull/286 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/721/ > Rya Streams - QueryChangeLog topic deletes content after 7 days. > > > Key: RYA-467 > URL: https://issues.apache.org/jira/browse/RYA-467 > Project: Rya > Issue Type: Bug >Reporter: Kevin Chilton >Assignee: Eric White >Priority: Major > > The automatically created topic used as the QueryChangeLog for a rya instance > uses the default "cleanup.policy" or "delete", and it does it after 7 days. > The policy for that topic should be "compact" since the query change log > needs to persist forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] incubator-rya issue #286: RYA-467 update topic cleanup.policy
Github user asfgit commented on the issue: https://github.com/apache/incubator-rya/pull/286 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/721/ ---
[jira] [Commented] (RYA-467) Rya Streams - QueryChangeLog topic deletes content after 7 days.
[
https://issues.apache.org/jira/browse/RYA-467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422799#comment-16422799
]
ASF GitHub Bot commented on RYA-467:
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/286#discussion_r178586989
--- Diff:
extras/rya.streams/kafka/src/main/java/org/apache/rya/streams/kafka/interactor/KafkaTopicPropertiesBuilder.java
---
@@ -0,0 +1,46 @@
+package org.apache.rya.streams.kafka.interactor;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.Optional;
+import java.util.Properties;
+
+/**
+ * Properties builder to be used when creating new Kafka Topics.
+ *
+ * Descriptions of properties can be found at
+ * {@link https://kafka.apache.org/documentation/#topicconfigs}
+ */
+public class KafkaTopicPropertiesBuilder {
+/*- Cleanup Policy -*/
+public static final String CLEANUP_POLICY_KEY = "cleanup.policy";
+public static final String CLEANUP_POLICY_DELETE = "cleanup.policy";
--- End diff --
Change to:
= "delete";
or
= LogConfig.Delete();
> Rya Streams - QueryChangeLog topic deletes content after 7 days.
>
>
> Key: RYA-467
> URL: https://issues.apache.org/jira/browse/RYA-467
> Project: Rya
> Issue Type: Bug
>Reporter: Kevin Chilton
>Assignee: Eric White
>Priority: Major
>
> The automatically created topic used as the QueryChangeLog for a rya instance
> uses the default "cleanup.policy" or "delete", and it does it after 7 days.
> The policy for that topic should be "compact" since the query change log
> needs to persist forever.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (RYA-467) Rya Streams - QueryChangeLog topic deletes content after 7 days.
[
https://issues.apache.org/jira/browse/RYA-467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422802#comment-16422802
]
ASF GitHub Bot commented on RYA-467:
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/286#discussion_r178588294
--- Diff:
extras/rya.streams/kafka/src/main/java/org/apache/rya/streams/kafka/interactor/KafkaTopicPropertiesBuilder.java
---
@@ -0,0 +1,46 @@
+package org.apache.rya.streams.kafka.interactor;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.Optional;
+import java.util.Properties;
+
+/**
+ * Properties builder to be used when creating new Kafka Topics.
+ *
+ * Descriptions of properties can be found at
+ * {@link https://kafka.apache.org/documentation/#topicconfigs}
+ */
+public class KafkaTopicPropertiesBuilder {
+/*- Cleanup Policy -*/
+public static final String CLEANUP_POLICY_KEY = "cleanup.policy";
--- End diff --
Maybe change to:
= LogConfig.CleanupPolicyProp();
> Rya Streams - QueryChangeLog topic deletes content after 7 days.
>
>
> Key: RYA-467
> URL: https://issues.apache.org/jira/browse/RYA-467
> Project: Rya
> Issue Type: Bug
>Reporter: Kevin Chilton
>Assignee: Eric White
>Priority: Major
>
> The automatically created topic used as the QueryChangeLog for a rya instance
> uses the default "cleanup.policy" or "delete", and it does it after 7 days.
> The policy for that topic should be "compact" since the query change log
> needs to persist forever.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (RYA-467) Rya Streams - QueryChangeLog topic deletes content after 7 days.
[
https://issues.apache.org/jira/browse/RYA-467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422801#comment-16422801
]
ASF GitHub Bot commented on RYA-467:
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/286#discussion_r178587123
--- Diff:
extras/rya.streams/kafka/src/main/java/org/apache/rya/streams/kafka/interactor/KafkaTopicPropertiesBuilder.java
---
@@ -0,0 +1,46 @@
+package org.apache.rya.streams.kafka.interactor;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.Optional;
+import java.util.Properties;
+
+/**
+ * Properties builder to be used when creating new Kafka Topics.
+ *
+ * Descriptions of properties can be found at
+ * {@link https://kafka.apache.org/documentation/#topicconfigs}
+ */
+public class KafkaTopicPropertiesBuilder {
+/*- Cleanup Policy -*/
+public static final String CLEANUP_POLICY_KEY = "cleanup.policy";
+public static final String CLEANUP_POLICY_DELETE = "cleanup.policy";
+public static final String CLEANUP_POLICY_COMPACT = "cleanup.policy";
--- End diff --
Change to:
= "compact";
or
= LogConfig.Compact();
> Rya Streams - QueryChangeLog topic deletes content after 7 days.
>
>
> Key: RYA-467
> URL: https://issues.apache.org/jira/browse/RYA-467
> Project: Rya
> Issue Type: Bug
>Reporter: Kevin Chilton
>Assignee: Eric White
>Priority: Major
>
> The automatically created topic used as the QueryChangeLog for a rya instance
> uses the default "cleanup.policy" or "delete", and it does it after 7 days.
> The policy for that topic should be "compact" since the query change log
> needs to persist forever.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (RYA-467) Rya Streams - QueryChangeLog topic deletes content after 7 days.
[ https://issues.apache.org/jira/browse/RYA-467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422803#comment-16422803 ] ASF GitHub Bot commented on RYA-467: Github user ejwhite922 commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/286#discussion_r178586652 --- Diff: extras/rya.streams/kafka/src/main/java/org/apache/rya/streams/kafka/interactor/KafkaTopicPropertiesBuilder.java --- @@ -0,0 +1,46 @@ +package org.apache.rya.streams.kafka.interactor; --- End diff -- Add license > Rya Streams - QueryChangeLog topic deletes content after 7 days. > > > Key: RYA-467 > URL: https://issues.apache.org/jira/browse/RYA-467 > Project: Rya > Issue Type: Bug >Reporter: Kevin Chilton >Assignee: Eric White >Priority: Major > > The automatically created topic used as the QueryChangeLog for a rya instance > uses the default "cleanup.policy" or "delete", and it does it after 7 days. > The policy for that topic should be "compact" since the query change log > needs to persist forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] incubator-rya pull request #286: RYA-467 update topic cleanup.policy
Github user ejwhite922 commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/286#discussion_r178586652 --- Diff: extras/rya.streams/kafka/src/main/java/org/apache/rya/streams/kafka/interactor/KafkaTopicPropertiesBuilder.java --- @@ -0,0 +1,46 @@ +package org.apache.rya.streams.kafka.interactor; --- End diff -- Add license ---
[GitHub] incubator-rya pull request #286: RYA-467 update topic cleanup.policy
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/286#discussion_r178588294
--- Diff:
extras/rya.streams/kafka/src/main/java/org/apache/rya/streams/kafka/interactor/KafkaTopicPropertiesBuilder.java
---
@@ -0,0 +1,46 @@
+package org.apache.rya.streams.kafka.interactor;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.Optional;
+import java.util.Properties;
+
+/**
+ * Properties builder to be used when creating new Kafka Topics.
+ *
+ * Descriptions of properties can be found at
+ * {@link https://kafka.apache.org/documentation/#topicconfigs}
+ */
+public class KafkaTopicPropertiesBuilder {
+/*- Cleanup Policy -*/
+public static final String CLEANUP_POLICY_KEY = "cleanup.policy";
--- End diff --
Maybe change to:
= LogConfig.CleanupPolicyProp();
---
[GitHub] incubator-rya pull request #286: RYA-467 update topic cleanup.policy
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/286#discussion_r178586989
--- Diff:
extras/rya.streams/kafka/src/main/java/org/apache/rya/streams/kafka/interactor/KafkaTopicPropertiesBuilder.java
---
@@ -0,0 +1,46 @@
+package org.apache.rya.streams.kafka.interactor;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.Optional;
+import java.util.Properties;
+
+/**
+ * Properties builder to be used when creating new Kafka Topics.
+ *
+ * Descriptions of properties can be found at
+ * {@link https://kafka.apache.org/documentation/#topicconfigs}
+ */
+public class KafkaTopicPropertiesBuilder {
+/*- Cleanup Policy -*/
+public static final String CLEANUP_POLICY_KEY = "cleanup.policy";
+public static final String CLEANUP_POLICY_DELETE = "cleanup.policy";
--- End diff --
Change to:
= "delete";
or
= LogConfig.Delete();
---
[GitHub] incubator-rya pull request #286: RYA-467 update topic cleanup.policy
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/286#discussion_r178587123
--- Diff:
extras/rya.streams/kafka/src/main/java/org/apache/rya/streams/kafka/interactor/KafkaTopicPropertiesBuilder.java
---
@@ -0,0 +1,46 @@
+package org.apache.rya.streams.kafka.interactor;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.Optional;
+import java.util.Properties;
+
+/**
+ * Properties builder to be used when creating new Kafka Topics.
+ *
+ * Descriptions of properties can be found at
+ * {@link https://kafka.apache.org/documentation/#topicconfigs}
+ */
+public class KafkaTopicPropertiesBuilder {
+/*- Cleanup Policy -*/
+public static final String CLEANUP_POLICY_KEY = "cleanup.policy";
+public static final String CLEANUP_POLICY_DELETE = "cleanup.policy";
+public static final String CLEANUP_POLICY_COMPACT = "cleanup.policy";
--- End diff --
Change to:
= "compact";
or
= LogConfig.Compact();
---
[jira] [Commented] (RYA-467) Rya Streams - QueryChangeLog topic deletes content after 7 days.
[ https://issues.apache.org/jira/browse/RYA-467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422786#comment-16422786 ] ASF GitHub Bot commented on RYA-467: Github user asfgit commented on the issue: https://github.com/apache/incubator-rya/pull/286 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/720/Build result: FAILURE[...truncated 1.48 MB...][INFO] Apache Rya Spark Support ... SKIPPED[INFO] Apache Rya Web Projects SKIPPED[INFO] Apache Rya Web Implementation .. SKIPPED[INFO] [INFO] BUILD FAILURE[INFO] [INFO] Total time: 23:07 min[INFO] Finished at: 2018-04-02T17:08:34+00:00[INFO] Final Memory: 335M/3024M[INFO] [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.11:check (check-licenses) on project rya.streams.kafka: Too many files with unapproved license: 1 See RAT report in: /home/jenkins/jenkins-slave/workspace/incubator-rya-master-with-optionals-pull-requests/extras/rya.streams/kafka/target/rat.txt -> [Help 1][ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven using the -X switch to enable full debug logging.[ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException[ERROR] [ERROR] After correcting the problems, you can resume the build with the command[ERROR] mvn -rf :rya.streams.kafkachannel stoppedSetting status of 6565ca4b28c04398c799b5507b0237828a5eb613 to FAILURE with url https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/720/ and message: 'FAILURE 'Using context: Jenkins: clean package -Pgeoindexing > Rya Streams - QueryChangeLog topic deletes content after 7 days. > > > Key: RYA-467 > URL: https://issues.apache.org/jira/browse/RYA-467 > Project: Rya > Issue Type: Bug >Reporter: Kevin Chilton >Assignee: Eric White >Priority: Major > > The automatically created topic used as the QueryChangeLog for a rya instance > uses the default "cleanup.policy" or "delete", and it does it after 7 days. > The policy for that topic should be "compact" since the query change log > needs to persist forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] incubator-rya issue #286: RYA-467 update topic cleanup.policy
Github user asfgit commented on the issue: https://github.com/apache/incubator-rya/pull/286 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/720/Build result: FAILURE[...truncated 1.48 MB...][INFO] Apache Rya Spark Support ... SKIPPED[INFO] Apache Rya Web Projects SKIPPED[INFO] Apache Rya Web Implementation .. SKIPPED[INFO] [INFO] BUILD FAILURE[INFO] [INFO] Total time: 23:07 min[INFO] Finished at: 2018-04-02T17:08:34+00:00[INFO] Final Memory: 335M/3024M[INFO] [ERROR] Failed to execute goal org.apache.rat:apache-rat-plugin:0.11:check (check-licenses) on project rya.streams.kafka: Too many files with unapproved license: 1 See RAT report in: /home/jenkins/jenkins-slave/workspace/incubator-rya-master-with-optionals-pull-requests/extras/rya.stre ams/kafka/target/rat.txt -> [Help 1][ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven using the -X switch to enable full debug logging.[ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException[ERROR] [ERROR] After correcting the problems, you can resume the build with the command[ERROR] mvn -rf :rya.streams.kafkachannel stoppedSetting status of 6565ca4b28c04398c799b5507b0237828a5eb613 to FAILURE with url https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/720/ and message: 'FAILURE 'Using context: Jenkins: clean package -Pgeoindexing ---
[jira] [Commented] (RYA-467) Rya Streams - QueryChangeLog topic deletes content after 7 days.
[ https://issues.apache.org/jira/browse/RYA-467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422739#comment-16422739 ] ASF GitHub Bot commented on RYA-467: GitHub user isper3at opened a pull request: https://github.com/apache/incubator-rya/pull/286 RYA-467 update topic cleanup.policy ## Description >What Changed? Added a topic properties builder. Only one property is currently in the builder, but adding others should be easy. ### Tests >Coverage? ### Links [Jira](https://issues.apache.org/jira/browse/RYA-467) ### Checklist - [ ] Code Review - [x] Squash Commits People To Reivew @kchilton2 @ejwhite922 @pujav65 @dlotts @jessehatfield You can merge this pull request into a Git repository by running: $ git pull https://github.com/isper3at/incubator-rya RYA-467 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-rya/pull/286.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #286 commit 6565ca4b28c04398c799b5507b0237828a5eb613 Author: Andrew Smith Date: 2018-04-02T16:32:40Z RYA-467 update topic cleanup.policy Added a topic properties builder. Only one property is currently in the builder, but adding others should be easy. > Rya Streams - QueryChangeLog topic deletes content after 7 days. > > > Key: RYA-467 > URL: https://issues.apache.org/jira/browse/RYA-467 > Project: Rya > Issue Type: Bug >Reporter: Kevin Chilton >Assignee: Eric White >Priority: Major > > The automatically created topic used as the QueryChangeLog for a rya instance > uses the default "cleanup.policy" or "delete", and it does it after 7 days. > The policy for that topic should be "compact" since the query change log > needs to persist forever. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] incubator-rya pull request #286: RYA-467 update topic cleanup.policy
GitHub user isper3at opened a pull request: https://github.com/apache/incubator-rya/pull/286 RYA-467 update topic cleanup.policy ## Description >What Changed? Added a topic properties builder. Only one property is currently in the builder, but adding others should be easy. ### Tests >Coverage? ### Links [Jira](https://issues.apache.org/jira/browse/RYA-467) ### Checklist - [ ] Code Review - [x] Squash Commits People To Reivew @kchilton2 @ejwhite922 @pujav65 @dlotts @jessehatfield You can merge this pull request into a Git repository by running: $ git pull https://github.com/isper3at/incubator-rya RYA-467 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-rya/pull/286.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #286 commit 6565ca4b28c04398c799b5507b0237828a5eb613 Author: Andrew Smith Date: 2018-04-02T16:32:40Z RYA-467 update topic cleanup.policy Added a topic properties builder. Only one property is currently in the builder, but adding others should be easy. ---
[GitHub] incubator-rya issue #285: RYA-469 Added tests for Rya Streams join iterator ...
Github user asfgit commented on the issue: https://github.com/apache/incubator-rya/pull/285 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/719/ ---
[jira] [Commented] (RYA-469) Rya Streams - Join iterator bug.
[
https://issues.apache.org/jira/browse/RYA-469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422720#comment-16422720
]
ASF GitHub Bot commented on RYA-469:
Github user asfgit commented on the issue:
https://github.com/apache/incubator-rya/pull/285
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/719/
> Rya Streams - Join iterator bug.
>
>
> Key: RYA-469
> URL: https://issues.apache.org/jira/browse/RYA-469
> Project: Rya
> Issue Type: Bug
>Reporter: Kevin Chilton
>Assignee: Kevin Chilton
>Priority: Major
>
> {code:java}
> Exception in thread "StreamThread-3" java.lang.RuntimeException: Problem
> encountered while iterating over the other side's values within the state
> store.
> at
> org.apache.rya.streams.kafka.processors.join.JoinProcessorSupplier$JoinProcessor.process(JoinProcessorSupplier.java:178)
> at
> org.apache.rya.streams.kafka.processors.join.JoinProcessorSupplier$JoinProcessor.process(JoinProcessorSupplier.java:95)
> at
> org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:68)
> at
> org.apache.kafka.streams.processor.internals.StreamTask.forward(StreamTask.java:343)
> at
> org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:187)
> at
> org.apache.rya.streams.kafka.processors.StatementPatternProcessorSupplier$StatementPatternProcessor.process(StatementPatternProcessorSupplier.java:112)
> at
> org.apache.rya.streams.kafka.processors.StatementPatternProcessorSupplier$StatementPatternProcessor.process(StatementPatternProcessorSupplier.java:70)
> at
> org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:68)
> at
> org.apache.kafka.streams.processor.internals.StreamTask.forward(StreamTask.java:343)
> at
> org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:187)
> at
> org.apache.kafka.streams.processor.internals.SourceNode.process(SourceNode.java:64)
> at
> org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:174)
> at
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:320)
> at
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:218)
> Caused by: java.lang.IllegalStateException: The end of range marker must be
> the last key in the iterator.
> at
> org.apache.rya.streams.kafka.processors.join.KeyValueJoinStateStore$1.readNext(KeyValueJoinStateStore.java:221)
> at
> org.apache.rya.streams.kafka.processors.join.KeyValueJoinStateStore$1.next(KeyValueJoinStateStore.java:188)
> at
> org.apache.rya.streams.kafka.processors.join.KeyValueJoinStateStore$1.next(KeyValueJoinStateStore.java:158)
> at
> org.apache.rya.api.function.join.LazyJoiningIterator.next(LazyJoiningIterator.java:77)
> at
> org.apache.rya.api.function.join.LazyJoiningIterator.next(LazyJoiningIterator.java:41)
> at
> org.apache.rya.streams.kafka.processors.join.JoinProcessorSupplier$JoinProcessor.process(JoinProcessorSupplier.java:170)
> ... 13 more{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (RYA-469) Rya Streams - Join iterator bug.
[
https://issues.apache.org/jira/browse/RYA-469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422676#comment-16422676
]
ASF GitHub Bot commented on RYA-469:
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/285#discussion_r178577147
--- Diff:
extras/rya.streams/client/src/test/java/org/apache/rya/streams/client/command/RunQueryCommandIT.java
---
@@ -196,4 +203,74 @@ public void run() {
// Show the read results matched the expected ones.
assertEquals(expected, results);
}
+
+@Test
+public void runQueryFromFile() throws Exception {
+// TODO: Maybe test with
org.apache.rya.api.utils.LubmQuery.LUBM_QUERY_1,2,3,etc...
--- End diff --
Removed
> Rya Streams - Join iterator bug.
>
>
> Key: RYA-469
> URL: https://issues.apache.org/jira/browse/RYA-469
> Project: Rya
> Issue Type: Bug
>Reporter: Kevin Chilton
>Assignee: Kevin Chilton
>Priority: Major
>
> {code:java}
> Exception in thread "StreamThread-3" java.lang.RuntimeException: Problem
> encountered while iterating over the other side's values within the state
> store.
> at
> org.apache.rya.streams.kafka.processors.join.JoinProcessorSupplier$JoinProcessor.process(JoinProcessorSupplier.java:178)
> at
> org.apache.rya.streams.kafka.processors.join.JoinProcessorSupplier$JoinProcessor.process(JoinProcessorSupplier.java:95)
> at
> org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:68)
> at
> org.apache.kafka.streams.processor.internals.StreamTask.forward(StreamTask.java:343)
> at
> org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:187)
> at
> org.apache.rya.streams.kafka.processors.StatementPatternProcessorSupplier$StatementPatternProcessor.process(StatementPatternProcessorSupplier.java:112)
> at
> org.apache.rya.streams.kafka.processors.StatementPatternProcessorSupplier$StatementPatternProcessor.process(StatementPatternProcessorSupplier.java:70)
> at
> org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:68)
> at
> org.apache.kafka.streams.processor.internals.StreamTask.forward(StreamTask.java:343)
> at
> org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:187)
> at
> org.apache.kafka.streams.processor.internals.SourceNode.process(SourceNode.java:64)
> at
> org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:174)
> at
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:320)
> at
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:218)
> Caused by: java.lang.IllegalStateException: The end of range marker must be
> the last key in the iterator.
> at
> org.apache.rya.streams.kafka.processors.join.KeyValueJoinStateStore$1.readNext(KeyValueJoinStateStore.java:221)
> at
> org.apache.rya.streams.kafka.processors.join.KeyValueJoinStateStore$1.next(KeyValueJoinStateStore.java:188)
> at
> org.apache.rya.streams.kafka.processors.join.KeyValueJoinStateStore$1.next(KeyValueJoinStateStore.java:158)
> at
> org.apache.rya.api.function.join.LazyJoiningIterator.next(LazyJoiningIterator.java:77)
> at
> org.apache.rya.api.function.join.LazyJoiningIterator.next(LazyJoiningIterator.java:41)
> at
> org.apache.rya.streams.kafka.processors.join.JoinProcessorSupplier$JoinProcessor.process(JoinProcessorSupplier.java:170)
> ... 13 more{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (RYA-469) Rya Streams - Join iterator bug.
[
https://issues.apache.org/jira/browse/RYA-469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422677#comment-16422677
]
ASF GitHub Bot commented on RYA-469:
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/285#discussion_r178577153
--- Diff: extras/indexingExample/src/main/java/MongoRyaDirectExample.java
---
@@ -805,6 +819,38 @@ public static void testAddAndDeleteNoContext(final
SailRepositoryConnection conn
Validate.isTrue(resultHandler.getCount() == 0);
}
+public static void testLubmFile(final SailRepositoryConnection conn)
throws MalformedQueryException, RepositoryException,
+UpdateExecutionException, QueryEvaluationException,
TupleQueryResultHandlerException, RDFParseException, IOException {
+
+final String query = LubmQuery.LUBM_QUERY_14.getSparqlQuery();
+//"PREFIX lubm: <" + LUBM_PREFIX + "> \n" +
--- End diff --
Done.
> Rya Streams - Join iterator bug.
>
>
> Key: RYA-469
> URL: https://issues.apache.org/jira/browse/RYA-469
> Project: Rya
> Issue Type: Bug
>Reporter: Kevin Chilton
>Assignee: Kevin Chilton
>Priority: Major
>
> {code:java}
> Exception in thread "StreamThread-3" java.lang.RuntimeException: Problem
> encountered while iterating over the other side's values within the state
> store.
> at
> org.apache.rya.streams.kafka.processors.join.JoinProcessorSupplier$JoinProcessor.process(JoinProcessorSupplier.java:178)
> at
> org.apache.rya.streams.kafka.processors.join.JoinProcessorSupplier$JoinProcessor.process(JoinProcessorSupplier.java:95)
> at
> org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:68)
> at
> org.apache.kafka.streams.processor.internals.StreamTask.forward(StreamTask.java:343)
> at
> org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:187)
> at
> org.apache.rya.streams.kafka.processors.StatementPatternProcessorSupplier$StatementPatternProcessor.process(StatementPatternProcessorSupplier.java:112)
> at
> org.apache.rya.streams.kafka.processors.StatementPatternProcessorSupplier$StatementPatternProcessor.process(StatementPatternProcessorSupplier.java:70)
> at
> org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:68)
> at
> org.apache.kafka.streams.processor.internals.StreamTask.forward(StreamTask.java:343)
> at
> org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:187)
> at
> org.apache.kafka.streams.processor.internals.SourceNode.process(SourceNode.java:64)
> at
> org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:174)
> at
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:320)
> at
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:218)
> Caused by: java.lang.IllegalStateException: The end of range marker must be
> the last key in the iterator.
> at
> org.apache.rya.streams.kafka.processors.join.KeyValueJoinStateStore$1.readNext(KeyValueJoinStateStore.java:221)
> at
> org.apache.rya.streams.kafka.processors.join.KeyValueJoinStateStore$1.next(KeyValueJoinStateStore.java:188)
> at
> org.apache.rya.streams.kafka.processors.join.KeyValueJoinStateStore$1.next(KeyValueJoinStateStore.java:158)
> at
> org.apache.rya.api.function.join.LazyJoiningIterator.next(LazyJoiningIterator.java:77)
> at
> org.apache.rya.api.function.join.LazyJoiningIterator.next(LazyJoiningIterator.java:41)
> at
> org.apache.rya.streams.kafka.processors.join.JoinProcessorSupplier$JoinProcessor.process(JoinProcessorSupplier.java:170)
> ... 13 more{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] incubator-rya pull request #285: RYA-469 Added tests for Rya Streams join it...
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/285#discussion_r178577153
--- Diff: extras/indexingExample/src/main/java/MongoRyaDirectExample.java
---
@@ -805,6 +819,38 @@ public static void testAddAndDeleteNoContext(final
SailRepositoryConnection conn
Validate.isTrue(resultHandler.getCount() == 0);
}
+public static void testLubmFile(final SailRepositoryConnection conn)
throws MalformedQueryException, RepositoryException,
+UpdateExecutionException, QueryEvaluationException,
TupleQueryResultHandlerException, RDFParseException, IOException {
+
+final String query = LubmQuery.LUBM_QUERY_14.getSparqlQuery();
+//"PREFIX lubm: <" + LUBM_PREFIX + "> \n" +
--- End diff --
Done.
---
[jira] [Commented] (RYA-469) Rya Streams - Join iterator bug.
[
https://issues.apache.org/jira/browse/RYA-469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422675#comment-16422675
]
ASF GitHub Bot commented on RYA-469:
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/285#discussion_r178576981
--- Diff:
extras/rya.streams/kafka/src/main/java/org/apache/rya/streams/kafka/processors/join/KeyValueJoinStateStore.java
---
@@ -77,30 +77,39 @@
/**
* This is the maximum value of a UTF-8 character.
*/
-private static final String END_RANGE_SUFFIX = new String(new byte[] {
(byte) 0XFF }, Charsets.UTF_8);
+private static final String END_RANGE_SUFFIX = new String(new byte[] {
(byte) 0xFF }, Charsets.UTF_8);
+
+/**
+ * Indicates where the end of the join variables occurs.
+ */
+private static final String JOIN_VAR_END_MARKER = new
String("~!^~".getBytes(Charsets.UTF_8), Charsets.UTF_8);
--- End diff --
Done. Changed to 0x06,0x10,0x03,0x04.
> Rya Streams - Join iterator bug.
>
>
> Key: RYA-469
> URL: https://issues.apache.org/jira/browse/RYA-469
> Project: Rya
> Issue Type: Bug
>Reporter: Kevin Chilton
>Assignee: Kevin Chilton
>Priority: Major
>
> {code:java}
> Exception in thread "StreamThread-3" java.lang.RuntimeException: Problem
> encountered while iterating over the other side's values within the state
> store.
> at
> org.apache.rya.streams.kafka.processors.join.JoinProcessorSupplier$JoinProcessor.process(JoinProcessorSupplier.java:178)
> at
> org.apache.rya.streams.kafka.processors.join.JoinProcessorSupplier$JoinProcessor.process(JoinProcessorSupplier.java:95)
> at
> org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:68)
> at
> org.apache.kafka.streams.processor.internals.StreamTask.forward(StreamTask.java:343)
> at
> org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:187)
> at
> org.apache.rya.streams.kafka.processors.StatementPatternProcessorSupplier$StatementPatternProcessor.process(StatementPatternProcessorSupplier.java:112)
> at
> org.apache.rya.streams.kafka.processors.StatementPatternProcessorSupplier$StatementPatternProcessor.process(StatementPatternProcessorSupplier.java:70)
> at
> org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:68)
> at
> org.apache.kafka.streams.processor.internals.StreamTask.forward(StreamTask.java:343)
> at
> org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:187)
> at
> org.apache.kafka.streams.processor.internals.SourceNode.process(SourceNode.java:64)
> at
> org.apache.kafka.streams.processor.internals.StreamTask.process(StreamTask.java:174)
> at
> org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:320)
> at
> org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:218)
> Caused by: java.lang.IllegalStateException: The end of range marker must be
> the last key in the iterator.
> at
> org.apache.rya.streams.kafka.processors.join.KeyValueJoinStateStore$1.readNext(KeyValueJoinStateStore.java:221)
> at
> org.apache.rya.streams.kafka.processors.join.KeyValueJoinStateStore$1.next(KeyValueJoinStateStore.java:188)
> at
> org.apache.rya.streams.kafka.processors.join.KeyValueJoinStateStore$1.next(KeyValueJoinStateStore.java:158)
> at
> org.apache.rya.api.function.join.LazyJoiningIterator.next(LazyJoiningIterator.java:77)
> at
> org.apache.rya.api.function.join.LazyJoiningIterator.next(LazyJoiningIterator.java:41)
> at
> org.apache.rya.streams.kafka.processors.join.JoinProcessorSupplier$JoinProcessor.process(JoinProcessorSupplier.java:170)
> ... 13 more{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] incubator-rya pull request #285: RYA-469 Added tests for Rya Streams join it...
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/285#discussion_r178577147
--- Diff:
extras/rya.streams/client/src/test/java/org/apache/rya/streams/client/command/RunQueryCommandIT.java
---
@@ -196,4 +203,74 @@ public void run() {
// Show the read results matched the expected ones.
assertEquals(expected, results);
}
+
+@Test
+public void runQueryFromFile() throws Exception {
+// TODO: Maybe test with
org.apache.rya.api.utils.LubmQuery.LUBM_QUERY_1,2,3,etc...
--- End diff --
Removed
---
[GitHub] incubator-rya pull request #285: RYA-469 Added tests for Rya Streams join it...
Github user ejwhite922 commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/285#discussion_r178576981
--- Diff:
extras/rya.streams/kafka/src/main/java/org/apache/rya/streams/kafka/processors/join/KeyValueJoinStateStore.java
---
@@ -77,30 +77,39 @@
/**
* This is the maximum value of a UTF-8 character.
*/
-private static final String END_RANGE_SUFFIX = new String(new byte[] {
(byte) 0XFF }, Charsets.UTF_8);
+private static final String END_RANGE_SUFFIX = new String(new byte[] {
(byte) 0xFF }, Charsets.UTF_8);
+
+/**
+ * Indicates where the end of the join variables occurs.
+ */
+private static final String JOIN_VAR_END_MARKER = new
String("~!^~".getBytes(Charsets.UTF_8), Charsets.UTF_8);
--- End diff --
Done. Changed to 0x06,0x10,0x03,0x04.
---
