[jira] [Commented] (RYA-301) Implement owl:ReflexiveProperty inference
[
https://issues.apache.org/jira/browse/RYA-301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092297#comment-16092297
]
Jesse Hatfield commented on RYA-301:
Given {code}:p rdf:type owl:ReflexiveProperty{code}
We can infer:
{quote}For all x, (x p x) .{quote}
That means queries of the form{code}SELECT ?o WHERE { :s :p ?o }{code} should
be expanded to
{code}SELECT ?o WHERE {
{ :s :p ?o } UNION { BIND(:s as ?o } .
}{code}
And likewise in reverse.
When both subject and object are variables, the implication of {code}SELECT ?s
:p ?o WHERE { ?s :p ?o }{code} is to return a triple for every URI in the
database. This might be a reasonable thing to query for in combination with
other conditions, e.g. "get all pairs of relatives who both have some other
unusual property", but may be challenging to implement efficiently.
At inference engine refresh, query for the reflexive properties just like is
currently done for symmetric and transitive properties:
{code}reflexiveProperties <- Set
for property in query(?property rdf:type owl:ReflexiveProperty):
reflexiveProperties.add(property){code}
At query time, a visitor can handle the case where one of the subject or object
is defined as follows:
{code}meet(StatementPattern originalSP):
if originalSP like (:s :p ?o): //defined subject, variable object
if inferenceEngine.isReflexive(p):
reflexiveSolution <- BindingSet({ Binding(:s as ?o) })
reflexiveSolutionAssignment <- BindingSetAssignment({
reflexiveSolution })
originalSP.replaceWith(InferUnion(originalSP,
reflexiveSolutionAssignment))
else if originalSP like (?s :p :o): //defined object, variable subject
if inferenceEngine.isReflexive(p):
reflexiveSolution <- BindingSet({ Binding(:o as ?s) })
reflexiveSolutionAssignment <- BindingSetAssignment({
reflexiveSolution })
originalSP.replaceWith(InferUnion(originalSP,
reflexiveSolutionAssignment)){code}
> Implement owl:ReflexiveProperty inference
> -
>
> Key: RYA-301
> URL: https://issues.apache.org/jira/browse/RYA-301
> Project: Rya
> Issue Type: Sub-task
> Components: sail
>Reporter: Jesse Hatfield
>
> An *{{owl:ReflexiveProperty}}* is a property that connects every resource to
> itself.
> If the ontology states that {{:hasRelative}} is a reflexive property, i.e.
> that everyone is their own relative, then the inference engine should ensure
> that queries of either form {{:Alice :hasRelative ?x}} or {{?x :hasRelative
> :Alice}} return the binding {{:Alice}} for {{?x}} .
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (RYA-315) ?S ?P queries slow on MongoDB
[
https://issues.apache.org/jira/browse/RYA-315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092255#comment-16092255
]
ASF GitHub Bot commented on RYA-315:
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-rya/pull/178
> ?S ?P queries slow on MongoDB
> -
>
> Key: RYA-315
> URL: https://issues.apache.org/jira/browse/RYA-315
> Project: Rya
> Issue Type: Bug
> Components: dao
>Reporter: Aaron Mihalik
>Assignee: Aaron Mihalik
>
> Queries like
> {noformat}
> select * where { ?s ?p }
> {noformat}
> are extremely slow on mongodb.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya pull request #178: RYA-315 Fixing issue with Core Indices on M...
Github user asfgit closed the pull request at: https://github.com/apache/incubator-rya/pull/178 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] incubator-rya pull request #176: Rya 268
Github user amihalik closed the pull request at: https://github.com/apache/incubator-rya/pull/176 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-296) Implement owl:hasSelf inference
[
https://issues.apache.org/jira/browse/RYA-296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092250#comment-16092250
]
Jesse Hatfield commented on RYA-296:
Note: [The object of the hasSelf triple is
irrelevant|https://www.w3.org/TR/owl2-rdf-based-semantics/#Semantic_Conditions_for_the_Vocabulary_Properties],
though {{"true"^^xsd:boolean}} is recommended.
An owl:hasSelf restriction takes the form {{:A owl:onProperty \:p ; owl:hasSelf
_ .}}, and means{quote}(1) *If* (x p x) *then* x is an A.
(2) *If* x is an A *then* (x p x) .{quote}
We can use both implications. If a query asks for instances of A, then we
rewrite to ask for individuals connected to themselves via p and/or individuals
otherwise known to be instances of A. If a query asks for triples involving
predicate p, then we know to include (x p x) for every x of type A. That is,
{code}SELECT ?x WHERE { ?x rdf:type :A }{code} becomes {code}SELECT ?x WHERE {
{ ?x :p ?x }
UNION
{ ?x rdf:type :A }
}{code}
And{code}SELECT ?s ?o WHERE { ?s :p ?o }{code} becomes {code}SELECT ?s ?o WHERE
{
{
?s rdf:type :A .
BIND(?s AS ?o) .
} UNION {
?s :p ?o .
}
}{code}Or in the query algebra:{code}Union(
Extension(
StatementPattern(?s rdf:type :A),
ExtensionElem(?s, "o")
),
StatementPattern(?s :p ?o)
){code}
(Note: One of s or o could be defined, and if it is o, it should be used in the
join instead, and "?s" bound to its value.)
h4. Proposed Approach
Since the inference can be made in either direction, the inference engine needs
to provide methods to look up the hasSelf information given either the type or
the property. At refresh time, we have:
{code}oneOfByProperty <- Map>
oneOfByType <- Map>
for (restrictionType, property) in propertyRestrictions:
if exists(query(property owl:oneOf ?any)):
oneOfByProperty[property].add(restrictionType }
oneOfByType[restrictionType].add(property){code}
The inference engine needs two methods:
{code}getOneOfImplyingType(type): // return properties that imply this type if
reflexive
properties <- Set
for sufficientType in ( { type } UNION getAllSubClasses(type) ):
properties.addAll(oneOfByType[type])
return properties{code}
{code}getOneOfImplyingProperty(property): // return types that imply this
property to be reflexive
return oneOfByProperty[property] if exists{code}
At query time, apply the visitor:
{code}meet(StatementPattern originalSP):
if originalSP like (?s rdf:type :C1): // require that C1 is defined, i.e.
not a variable
node <- originalSP
for property in getOneOfImplyingType(C1):
node <- InferUnion(node, StatementPattern(?s, property, ?s)).
originalSP.replaceWith(node)
else if originalSP like (s :p o): // where p is not a variable and at
least one of s and o are variables
node <- originalSP
for type in getOneOfImplyingProperty(p):
newNode <- if o is defined: // meaning s is the variable
Extension(StatementPattern(o, rdf:type, type), ExtensionElem(o,
"s"))
else: // o is a variable and s may either be defined or a variable
Extension(StatementPattern(s, rdf:type, type), ExtensionElem(s,
"o"))
node <- InferUnion(node, newNode)
originalSP.replaceWith(node){code}
> Implement owl:hasSelf inference
> ---
>
> Key: RYA-296
> URL: https://issues.apache.org/jira/browse/RYA-296
> Project: Rya
> Issue Type: Sub-task
> Components: sail
>Reporter: Jesse Hatfield
>
> An *{{owl:hasSelf}}* restriction defines the set of resources that are
> connected to themselves by a specific property.
> If the ontology states that a {{:Narcissist}} is a resource that {{:loves}}
> itself, then the inference engine should:
> 1. Rewrite queries of the form {{?x rdf:type :Narcissist}} to find all
> resources matching {{?x :loves ?x}} (as well as anything explicitly stated to
> be a :Narcissist) .
> 2. Rewrite queries of either form {{:A :loves ?y}} or {{?z :loves :A}} to
> match {{:A}} if {{:A}} is known to have the type {{:Narcissist}} .
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (RYA-315) ?S ?P queries slow on MongoDB
[
https://issues.apache.org/jira/browse/RYA-315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092246#comment-16092246
]
ASF GitHub Bot commented on RYA-315:
Github user asfgit commented on the issue:
https://github.com/apache/incubator-rya/pull/178
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/266/
> ?S ?P queries slow on MongoDB
> -
>
> Key: RYA-315
> URL: https://issues.apache.org/jira/browse/RYA-315
> Project: Rya
> Issue Type: Bug
> Components: dao
>Reporter: Aaron Mihalik
>Assignee: Aaron Mihalik
>
> Queries like
> {noformat}
> select * where { ?s ?p }
> {noformat}
> are extremely slow on mongodb.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya issue #178: RYA-315 Fixing issue with Core Indices on MongoDB
Github user asfgit commented on the issue: https://github.com/apache/incubator-rya/pull/178 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/266/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-315) ?S ?P queries slow on MongoDB
[
https://issues.apache.org/jira/browse/RYA-315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092184#comment-16092184
]
ASF GitHub Bot commented on RYA-315:
Github user amihalik commented on the issue:
https://github.com/apache/incubator-rya/pull/178
This is a very strange error... it can't download a dependency... but I
can... :/
Here's the dependency:
[INFO] Downloading:
https://repo.boundlessgeo.com/main/it/geosolutions/imageio-ext/imageio-ext-tiff/1.1.15/imageio-ext-tiff-1.1.15.pom
I'm going to see what happens when I build again.
> ?S ?P queries slow on MongoDB
> -
>
> Key: RYA-315
> URL: https://issues.apache.org/jira/browse/RYA-315
> Project: Rya
> Issue Type: Bug
> Components: dao
>Reporter: Aaron Mihalik
>Assignee: Aaron Mihalik
>
> Queries like
> {noformat}
> select * where { ?s ?p }
> {noformat}
> are extremely slow on mongodb.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya issue #178: RYA-315 Fixing issue with Core Indices on MongoDB
Github user amihalik commented on the issue: https://github.com/apache/incubator-rya/pull/178 This is a very strange error... it can't download a dependency... but I can... :/ Here's the dependency: [INFO] Downloading: https://repo.boundlessgeo.com/main/it/geosolutions/imageio-ext/imageio-ext-tiff/1.1.15/imageio-ext-tiff-1.1.15.pom I'm going to see what happens when I build again. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-315) ?S ?P queries slow on MongoDB
[
https://issues.apache.org/jira/browse/RYA-315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092185#comment-16092185
]
ASF GitHub Bot commented on RYA-315:
Github user amihalik commented on the issue:
https://github.com/apache/incubator-rya/pull/178
asfbot build
> ?S ?P queries slow on MongoDB
> -
>
> Key: RYA-315
> URL: https://issues.apache.org/jira/browse/RYA-315
> Project: Rya
> Issue Type: Bug
> Components: dao
>Reporter: Aaron Mihalik
>Assignee: Aaron Mihalik
>
> Queries like
> {noformat}
> select * where { ?s ?p }
> {noformat}
> are extremely slow on mongodb.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya issue #178: RYA-315 Fixing issue with Core Indices on MongoDB
Github user amihalik commented on the issue: https://github.com/apache/incubator-rya/pull/178 asfbot build --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-315) ?S ?P queries slow on MongoDB
[
https://issues.apache.org/jira/browse/RYA-315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092173#comment-16092173
]
ASF GitHub Bot commented on RYA-315:
Github user asfgit commented on the issue:
https://github.com/apache/incubator-rya/pull/178
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/265/Build
result: FAILURE[...truncated 8.93 MB...][INFO] Apache Rya Spark
Support ... SKIPPED[INFO] Apache Rya Web Projects
SKIPPED[INFO] Apache Rya Web Implementation
.. SKIPPED[INFO]
[INFO]
BUILD FAILURE[INFO]
[INFO]
Total time: 54:22 min[INFO] Finished at: 2017-07-18T20:53:03+00:00[INFO] Final
Memory: 194M/1108M[INFO]
[ERROR]
Failed to execute goal on project rya.geoindexing: Could not resolve
dependencies for project
org.apache.rya:rya.geoindexing:jar:3.2.11-incubating-SNAPSHOT: Failed to
collect dependencies at
org.locationtech.geomesa:geomesa-accumulo-datastore_2.11:jar:1.3.0-m1 ->
org.locationtech.geomesa:geomesa-feature-all_2.11:jar:1.3.0-m1 ->
org.locationtech.geomesa:geomesa-feature-kryo_2.11:jar:1.3.0-m1 ->
org.geotools:gt-process-feature:jar:15.1 -> org.geotools:gt-process:jar:15.1 ->
org.geotools:gt-coverage:jar:15.1 ->
it.geosolutions.imageio-ext:imageio-ext-tiff:jar:1.1.15: Failed to read
artifact descriptor for
it.geosolutions.imageio-ext:imageio-ext-tiff:jar:1.1.15: Could not transfer
artifact it.geosolutions.imageio-ext:imageio-ext-tiff:pom:1.1.15 from/to
geowave-maven-releases (https://s3.amazonaws.com/geowave-maven/release): Access
denied to:
https://s3.amazonaws.com/geowave-maven/release/it/geosolutions/imageio-ext/imageio-ext-tiff/1.1.15/imageio-ext-tiff-1.1.15.pom
, ReasonPhrase:Forbidden. -> [Help 1][ERROR] [ERROR] To see the full stack
trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven using
the -X switch to enable full debug logging.[ERROR] [ERROR] For more information
about the errors and possible solutions, please read the following
articles:[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException[ERROR]
[ERROR] After correcting the problems, you can resume the build with the
command[ERROR] mvn -rf :rya.geoindexingchannel stoppedSetting status
of 7778a4685ae70f67f9b0354fcbec7d30f0beaf7a to FAILURE with url
https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/265/
and message: 'FAILURE 'Using context: Jenkins: clean package -Pgeoindexing
> ?S ?P queries slow on MongoDB
> -
>
> Key: RYA-315
> URL: https://issues.apache.org/jira/browse/RYA-315
> Project: Rya
> Issue Type: Bug
> Components: dao
>Reporter: Aaron Mihalik
>Assignee: Aaron Mihalik
>
> Queries like
> {noformat}
> select * where { ?s ?p }
> {noformat}
> are extremely slow on mongodb.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya issue #178: RYA-315 Fixing issue with Core Indices on MongoDB
Github user asfgit commented on the issue: https://github.com/apache/incubator-rya/pull/178 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/265/Build result: FAILURE[...truncated 8.93 MB...][INFO] Apache Rya Spark Support ... SKIPPED[INFO] Apache Rya Web Projects SKIPPED[INFO] Apache Rya Web Implementation .. SKIPPED[INFO] [INFO] BUILD FAILURE[INFO] [INFO] Total time: 54:22 min[INFO] Finished at: 2017-07-18T20:53:03+00:00[INFO] Final Memory: 194M/1108M[INFO] [ERROR] Failed to execute goal on project rya.geoindexing: Could not resolve dependencies for project org.apache.rya:rya.geoindexing:jar:3.2.11-incubating-SNAPSHOT: Failed to collect dependencies at org.locationtech.geomesa:geomesa-accumulo-datastore_2.11:jar:1.3.0-m1 -> org.locationte ch.geomesa:geomesa-feature-all_2.11:jar:1.3.0-m1 -> org.locationtech.geomesa:geomesa-feature-kryo_2.11:jar:1.3.0-m1 -> org.geotools:gt-process-feature:jar:15.1 -> org.geotools:gt-process:jar:15.1 -> org.geotools:gt-coverage:jar:15.1 -> it.geosolutions.imageio-ext:imageio-ext-tiff:jar:1.1.15: Failed to read artifact descriptor for it.geosolutions.imageio-ext:imageio-ext-tiff:jar:1.1.15: Could not transfer artifact it.geosolutions.imageio-ext:imageio-ext-tiff:pom:1.1.15 from/to geowave-maven-releases (https://s3.amazonaws.com/geowave-maven/release): Access denied to: https://s3.amazonaws.com/geowave-maven/release/it/geosolutions/imageio-ext/imageio-ext-tiff/1.1.15/imageio-ext-tiff-1.1.15.pom , ReasonPhrase:Forbidden. -> [Help 1][ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven using the -X switch to enable full debug logging.[ERROR] [ERROR] For more information about the errors and possible solutions, please read the follow ing articles:[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException[ERROR] [ERROR] After correcting the problems, you can resume the build with the command[ERROR] mvn -rf :rya.geoindexingchannel stoppedSetting status of 7778a4685ae70f67f9b0354fcbec7d30f0beaf7a to FAILURE with url https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/265/ and message: 'FAILURE 'Using context: Jenkins: clean package -Pgeoindexing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-295) Implement owl:allValuesFrom inference
[
https://issues.apache.org/jira/browse/RYA-295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092144#comment-16092144
]
Jesse Hatfield commented on RYA-295:
In general, if we have{quote}:A owl:onProperty :p .
:A owl:allValuesFrom :B {quote}
then this means:{quote}x is an A *if and only if*, for all y, *if* (x p y)
*then* y is a B.{quote}
As described above, we can never deduce the second half ("for all y, if ...
then") just from the data, because of the open-world assumption. Therefore the
only implication we need to consider is
bq. *If* x is an A *then* for all y, *if* (x p y) *then* y is a B.
Equivalently:
bq. *If* there exists some x *such that* (x is an A) AND (x p y) *then* y is a
B.
This is the same kind of expression we can derive from owl:someValuesFrom,
described in [RYA-294], but in the reverse direction: inferring the object's
type if there exist some subject who has it as a value.
We can use the same approach. In the inference engine:
{code}allValuesFromByValueType <- Map
for (restrictionType, property) in propertyRestrictions:
for valueType in query(property owl:allValuesFrom ?valueType):
allValuesFromByValueType[valueType][property] <- restrictionType
...
getAllValuesFromImplying(type): // return (property, restriction type) pairs
that would imply this type
results <- List<(property, restrictionType)>
for sufficientType <- type UNION getAllSubClasses(type):
if sufficientType in allValuesFromByValueType:
results.addAll(allValuesFromByValueType[sufficientType])
return results{code}
And a visitor:
{code}meet(StatementPattern originalSP):
if originalSP like (?object rdf:type :C1): // Assume the type in question
is explicitly given in the query, not a variable
node <- originalSP
for (property, restrictionType) in
inferenceEngine.getAllValuesFromImplying(C1):
option <- InferJoin(StatementPattern(?subject, property, ?object),
StatementPattern(?subject, rdf:type, restrictionType))
node <- InferUnion(node, option)
originalSP.replaceWith(node){code}
> Implement owl:allValuesFrom inference
> -
>
> Key: RYA-295
> URL: https://issues.apache.org/jira/browse/RYA-295
> Project: Rya
> Issue Type: Sub-task
> Components: sail
>Reporter: Jesse Hatfield
>
> An *{{owl:allValuesFrom}}* restriction defines the set of resources for
> which, given a particular predicate and other type, every value of that
> predicate is a member of that type. Note that there may be no values at all.
> For example, the ontology may state that resources of type {{:Person}} have
> all values from {{:Person}} for type {{:parent}}: that is, a person's parents
> are all people as well. Therefore, a pattern of the form {{?x rdf:type
> :Person}} should be expanded to:
> {noformat}
> { ?y rdf:type :Person .
> ?y :parent ?x }
> UNION
> { ?x rdf:type :Person }
> {noformat}
> i.e. we can infer {{?x}}'s personhood from the fact that child {{?y}} is
> known to satisfy the restriction.
> Notes:
> -We can infer "x is a person, therefore all of x's parents are people". But
> we can't infer "all of x's parents are people, therefore x is a person",
> because of the open world semantics: we don't know that the parents given by
> the data are in fact all of x's parents. (If there were also a cardinality
> restriction and we could presume consistency, then we could infer this in the
> right circumstances, but this is outside the scope of basic allValuesFrom
> support.) This differs with most other property restriction rules in that we
> can't infer that an object belongs to the class defined by the restriction,
> but rather use the fact that an object is already known to belong in that
> class in order to infer something about its neighbors in the graph (the types
> of the values).
> -The example above could be applied recursively, but to implement this as a
> simple query rewrite we'll need to limit recursion depth (and interactions
> with other rules, for the same reasons).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (RYA-294) Implement owl:someValuesFrom inference
[
https://issues.apache.org/jira/browse/RYA-294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092094#comment-16092094
]
Jesse Hatfield edited comment on RYA-294 at 7/18/17 8:35 PM:
-
h4. Logic, Limitations
If we have{quote}:A owl:onProperty \:p .
:A owl:someValuesFrom :B .{quote}
then this means:
{quote}(x is an A) *if and only if* there exists some y *such that* (y is a B)
AND (x p y) .{quote}
Or:
{quote}(1) *If* there exists some y *such that* (y is a B) AND (x p y) *then* x
is an A .
(2) *If* x is an A *then* there exists some y such that (y is a B) AND (x p y)
.{quote}
Under the OWL open-world semantics, the correct inference in (2) is that there
is such an individual _y_, even if it isn't represented in the known data. Full
reasoners infer that some unknown _y_ exists and consider the different
individuals it might be, and also consider the possibility that it is some new
individual that isn't explicitly represented in the data store. This is an
example of the kind of non-deterministic implication that makes full reasoning
computationally difficult. In theory, even if nothing were stated to have type
B, a query such as {code}ASK { ?y rdf:type :B . }{code}could use implication
(2) to return true if there were any instance of A: if there is an A, then it
must be connected to some individual who has type B. But we have no way of
knowing what that individual might be, so the query {code}SELECT ?y { ?y
rdf:type :B .}{code}wouldn't be able to return any specific instance.
Non-deterministic implications are outside the scope of the current effort, so
for now we only consider implication (1).
h4. Class Hierarchy
Ideally, other relationships between classes would be taken into account. For
example, in the [actual LUBM
schema|http://swat.cse.lehigh.edu/onto/univ-bench.owl], we have:
{code}lubm:Chair owl:intersectionOf (
lubm:Person
[ a owl:Restriction ; owl:onProperty lubm:headOf ; owl:someValuesFrom
lubm:Department ]
) .
lubm:Chair rdfs:subClassOf lubm:Profesor .{code}
In general, Rya's application of multiple inference rules is limited: the
visitors will skip any joins produced by other visitors. Long-term, we may want
to investigate relaxing that restriction to account for classes like the above.
In the meantime, I propose we at least take subclasses into account with
respect to the type of the property restriction itself, so if A is declared to
be a superclass (or once this is implemented, equivalent class) of the
someValuesFrom expression, we can infer membership in A from a value of the
appropriate type.
That is, if the ontology states:
{code}[ owl:onProperty :p ; owl:someValuesFrom :B ] rdfs:subClassOf :A .
[ owl:onProperty :q ; owl:someValuesFrom :C ] rdfs:subClassOf :A .{code}
Then a query:
{code}SELECT ?x { ?x rdf:type :A . }{code}
Should be rewritten:
{code}SELECT ?x {
{
?x :p ?y .
?y rdf:type :B .
} UNION {
?x :q ?y .
?y rdf:type :C .
} UNION {
?x rdf:type :A .
}
}{code}
For now, the individual joins won't be further expanded (e.g. if q has an
inverse property or B has subclasses), but we'll likely want to consider doing
so to some extent eventually.
h4. Pseudocode
As with most inference rules, we'll need the inference engine to process the
schema at refresh time, and a query visitor to apply the rule at query time. In
the inference engine, at refresh time:
{code}propertyRestrictions <- Map{ restrictionType -> property;
for query(?restrictionType owl:onProperty ?property) }
someValuesFromByRestrictionType <- Map
for (restrictionType, property) in propertyRestrictions:
for valueType in query(property owl:someValuesFrom ?valueType):
someValuesFromByRestrictionType[restrictionType][property] <-
valueType{code}
In the inference engine, provide method:
{code}getSomeValuesFromImplying(type): // return (property, value type) pairs
that would imply this type
results <- List<(property, valueType)>
for sufficientType <- type UNION getAllSubClasses(type):
if sufficientType in someValuesFromByRestrictionType:
results.addAll(someValuesFromByRestrictionType[sufficientType])
return results{code}
Then at query time, we need a visitor to transform statement patterns
accordingly:
{code}meet(StatementPattern originalSP):
if originalSP like (?subject rdf:type :C1): // Assume the type in question
is explicitly given in the query, not a variable
node <- originalSP
for (property, valueType) in
inferenceEngine.getSomeValuesFromImplying(C1):
option <- InferJoin(StatementPattern(?subject, property, ?object),
StatementPattern(?object, rdf:type, valueType))
node <- InferUnion(node, option)
originalSP.replaceWith(node){code}
was (Author: jhatfiel):
h4. Logic, Limitations
If we have{quote}:A owl:onProperty \:p .
:A owl:
[jira] [Comment Edited] (RYA-294) Implement owl:someValuesFrom inference
[
https://issues.apache.org/jira/browse/RYA-294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092094#comment-16092094
]
Jesse Hatfield edited comment on RYA-294 at 7/18/17 8:28 PM:
-
h4. Logic, Limitations
If we have{quote}:A owl:onProperty \:p .
:A owl:someValuesFrom :B .{quote}
then this means:
{quote}(x is an A) *if and only if* there exists some y *such that* (y is a B)
AND (x p y) .{quote}
Or:
{quote}(1) *If* there exists some y *such that* (y is a B) AND (x p y) *then* x
is an A .
(2) *If* x is an A *then* there exists some y such that (y is a B) AND (x p y)
.{quote}
Under the OWL open-world semantics, the correct inference in (2) is that there
is such an individual _y_, even if it isn't represented in the known data. Full
reasoners infer that some unknown _y_ exists and consider the different
individuals it might be, and also consider the possibility that it is some new
individual that isn't explicitly represented in the data store. This is an
example of the kind of non-deterministic implication that makes full reasoning
computationally difficult. In theory, even if nothing were stated to have type
B, a query such as {code}ASK { ?y rdf:type :B . }{code}could use implication
(2) to return true if there were any instance of A: if there is an A, then it
must be connected to some individual who has type B. But we have no way of
knowing what that individual might be, so the query {code}SELECT ?y { ?y
rdf:type :B .}{code}wouldn't be able to return any specific instance.
Non-deterministic implications are outside the scope of the current effort, so
for now we only consider implication (1).
h4. Class Hierarchy
Ideally, other relationships between classes would be taken into account. For
example, in the [actual LUBM
schema|http://swat.cse.lehigh.edu/onto/univ-bench.owl], we have:
{code}lubm:Chair owl:intersectionOf (
lubm:Person
[ a owl:Restriction ; owl:onProperty lubm:headOf ; owl:someValuesFrom
lubm:Department ]
) .
lubm:Chair rdfs:subClassOf lubm:Profesor .{code}
In general, Rya's application of multiple inference rules is limited: the
visitors will skip any joins produced by other visitors. Long-term, we may want
to investigate relaxing that restriction to account for classes like the above.
In the meantime, I propose we at least take subclasses into account with
respect to the type of the property restriction itself, so if A is declared to
be a superclass (or once this is implemented, equivalent class) of the
someValuesFrom expression, we can infer membership in A from a value of the
appropriate type.
That is, if the ontology states:
{code}[ owl:onProperty :p ; owl:someValuesFrom :B ] rdfs:subClassOf :A .
[ owl:onProperty :q ; owl:someValuesFrom :C ] rdfs:subClassOf :A .{code}
Then a query:
{code}SELECT ?x { ?x rdf:type :A . }{code}
Should be rewritten:
{code}SELECT ?x {
{
?x :p ?y .
?y rdf:type :B .
} UNION {
?x :q ?y .
?y rdf:type :C .
} UNION {
?x rdf:type :A .
}
}{code}
For now, the individual joins won't be further expanded (e.g. if q has an
inverse property or B has subclasses), but we'll likely want to consider doing
so to some extent eventually.
h4. Pseudocode
As with most inference rules, we'll need the inference engine to process the
schema at refresh time, and a query visitor to apply the rule at query time. In
the inference engine, at refresh time:
{code}propertyRestrictions <- Map{ restrictionType -> property;
for query(?restrictionType owl:onProperty ?property) }
someValuesFromByRestrictionType <- Map
for (restrictionType, property) in propertyRestrictions:
for valueType in query(property owl:someValuesFrom ?valueType):
someValuesFromByRestrictionType[restrictionType][property] <-
valueType{code}
In the inference engine, provide method:
{code}getSomeValuesFromImplying(type): // return (property, value type) pairs
that would imply this type
results <- List<(property, valueType)>
typesImplyingType <- type UNION getAllSubclasses(type)
for sufficientType <- type UNION getAllSubClasses(type):
if sufficientType in someValuesFromByRestrictionType:
results.addAll(someValuesFromByRestrictionType[sufficientType])
return results{code}
Then at query time, we need a visitor to transform statement patterns
accordingly:
{code}meet(StatementPattern originalSP):
if originalSP like (?subject rdf:type :C1): // Assume the type in question
is explicitly given in the query, not a variable
node <- originalSP
for (property, valueType) in
inferenceEngine.getSomeValuesFromImplying(C1):
option <- InferJoin(StatementPattern(?subject, property, ?object),
StatementPattern(?object, rdf:type, valueType))
node <- InferUnion(node, option)
originalSP.replaceWith(node){code}
was (Author: jhatfiel):
h4. Logic, L
[jira] [Commented] (RYA-294) Implement owl:someValuesFrom inference
[
https://issues.apache.org/jira/browse/RYA-294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092094#comment-16092094
]
Jesse Hatfield commented on RYA-294:
h4. Logic, Limitations
If we have{quote}:A owl:onProperty \:p .
:A owl:someValuesFrom :B .{quote}
then this means:
{quote}(x is an A) *if and only if* there exists some y *such that* (y is a B)
AND (x p y) .{quote}
Or:
{quote}(1) *If* there exists some y *such that* (y is a B) AND (x p y) *then* x
is an A .
(2) *If* x is an A *then* there exists some y such that (y is a B) AND (x p y)
.{quote}
Under the OWL open-world semantics, the correct inference in (2) is that there
is such an individual _y_, even if it isn't represented in the known data. Full
reasoners infer that some unknown _y_ exists and consider the different
individuals it might be, and also consider the possibility that it is some new
individual that isn't explicitly represented in the data store. This is an
example of the kind of non-deterministic implication that makes full reasoning
computationally difficult. In theory, even if nothing were stated to have type
B, a query such as {code}ASK \{ ?y rdf:type :B . \}{code}could use implication
(2) to return true if there were any instance of A: if there is an A, then it
must be connected to some individual who has type B. But we have no way of
knowing what that individual might be, so the query {code}SELECT ?y \{ ?y
rdf:type :B .\}{code}wouldn't be able to return any specific instance.
Non-deterministic implications are outside the scope of the current effort, so
for now we only consider implication (1).
h4. Class Hierarchy
Ideally, other relationships between classes would be taken into account. For
example, in the [actual LUBM
schema|http://swat.cse.lehigh.edu/onto/univ-bench.owl], we have:
{code}lubm:Chair owl:intersectionOf (
lubm:Person
[ a owl:Restriction ; owl:onProperty lubm:headOf ; owl:someValuesFrom
lubm:Department ]
) .
lubm:Chair rdfs:subClassOf lubm:Profesor .{code}
In general, Rya's application of multiple inference rules is limited: the
visitors will skip any joins produced by other visitors. Long-term, we may want
to investigate relaxing that restriction to account for classes like the above.
In the meantime, I propose we at least take subclasses into account with
respect to the type of the property restriction itself, so if A is declared to
be a superclass (or once this is implemented, equivalent class) of the
someValuesFrom expression, we can infer membership in A from a value of the
appropriate type.
That is, if the ontology states:
{code}[ owl:onProperty :p ; owl:someValuesFrom :B ] rdfs:subClassOf :A .
[ owl:onProperty :q ; owl:someValuesFrom :C ] rdfs:subClassOf :A .{code}
Then a query:
{code}SELECT ?x { ?x rdf:type :A . }{code}
Should be rewritten:
{code}SELECT ?x {
{
?x :p ?y .
?y rdf:type :B .
} UNION {
?x :q ?y .
?y rdf:type :C .
} UNION {
?x rdf:type :A .
}
}{code}
For now, the individual joins won't be further expanded (e.g. if q has an
inverse property or B has subclasses), but we'll likely want to consider doing
so to some extent eventually.
h4. Pseudocode
As with most inference rules, we'll need the inference engine to process the
schema at refresh time, and a query visitor to apply the rule at query time. In
the inference engine, at refresh time:
{code}propertyRestrictions <- Map{ restrictionType -> property;
for query(?restrictionType owl:onProperty ?property) }
someValuesFromByRestrictionType <- Map
for (restrictionType, property) in propertyRestrictions:
for valueType in query(property owl:someValuesFrom ?valueType):
someValuesFromByRestrictionType[restrictionType][property] <-
valueType{code}
In the inference engine, provide method:
{code}getSomeValuesFromImplying(type): // return (property, value type) pairs
that would imply this type
results <- List<(property, valueType)>
typesImplyingType <- type UNION getAllSubclasses(type)
for sufficientType <- type UNION getAllSubClasses(type):
if sufficientType in someValuesFromByRestrictionType:
results.addAll(someValuesFromByRestrictionType[sufficientType])
return results{code}
Then at query time, we need a visitor to transform statement patterns
accordingly:
{code}meet(StatementPattern originalSP):
if originalSP like (?subject rdf:type :C1): // Assume the type in question
is explicitly given in the query, not a variable
node <- originalSP
for (property, valueType) in
inferenceEngine.getSomeValuesFromImplying(C1):
option <- InferJoin(StatementPattern(?subject, property, ?object),
StatementPattern(?object, rdf:type, valueType))
node <- InferUnion(node, option)
originalSP.replaceWith(node){code}
> Implement owl:someValuesFrom inference
> --
[jira] [Commented] (RYA-315) ?S ?P queries slow on MongoDB
[
https://issues.apache.org/jira/browse/RYA-315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16092085#comment-16092085
]
ASF GitHub Bot commented on RYA-315:
GitHub user amihalik opened a pull request:
https://github.com/apache/incubator-rya/pull/178
RYA-315 Fixing issue with Core Indices on MongoDB
## Description
Fixed issue with building OS index. This was causing the very slow query
response on the ?s ?p queries
### Links
[Jira](https://issues.apache.org/jira/browse/RYA-315)
### Checklist
- [ ] Code Review
- [ ] Squash Commits
People To Review
@isper3at
@meiercaleb
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/amihalik/incubator-rya RYA-315
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-rya/pull/178.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #178
commit 7778a4685ae70f67f9b0354fcbec7d30f0beaf7a
Author: Aaron Mihalik
Date: 2017-07-18T19:55:47Z
RYA-315 Fixing issue with Core Indices on MongoDB
> ?S ?P queries slow on MongoDB
> -
>
> Key: RYA-315
> URL: https://issues.apache.org/jira/browse/RYA-315
> Project: Rya
> Issue Type: Bug
> Components: dao
>Reporter: Aaron Mihalik
>Assignee: Aaron Mihalik
>
> Queries like
> {noformat}
> select * where { ?s ?p }
> {noformat}
> are extremely slow on mongodb.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya pull request #178: RYA-315 Fixing issue with Core Indices on M...
GitHub user amihalik opened a pull request: https://github.com/apache/incubator-rya/pull/178 RYA-315 Fixing issue with Core Indices on MongoDB ## Description Fixed issue with building OS index. This was causing the very slow query response on the ?s ?p queries ### Links [Jira](https://issues.apache.org/jira/browse/RYA-315) ### Checklist - [ ] Code Review - [ ] Squash Commits People To Review @isper3at @meiercaleb You can merge this pull request into a Git repository by running: $ git pull https://github.com/amihalik/incubator-rya RYA-315 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-rya/pull/178.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #178 commit 7778a4685ae70f67f9b0354fcbec7d30f0beaf7a Author: Aaron Mihalik Date: 2017-07-18T19:55:47Z RYA-315 Fixing issue with Core Indices on MongoDB --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (RYA-315) ?S ?P queries slow on MongoDB
Aaron Mihalik created RYA-315:
-
Summary: ?S ?P queries slow on MongoDB
Key: RYA-315
URL: https://issues.apache.org/jira/browse/RYA-315
Project: Rya
Issue Type: Bug
Components: dao
Reporter: Aaron Mihalik
Queries like
{noformat}
select * where { ?s ?p }
{noformat}
are extremely slow on mongodb.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (RYA-315) ?S ?P queries slow on MongoDB
[
https://issues.apache.org/jira/browse/RYA-315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron Mihalik reassigned RYA-315:
-
Assignee: Aaron Mihalik
> ?S ?P queries slow on MongoDB
> -
>
> Key: RYA-315
> URL: https://issues.apache.org/jira/browse/RYA-315
> Project: Rya
> Issue Type: Bug
> Components: dao
>Reporter: Aaron Mihalik
>Assignee: Aaron Mihalik
>
> Queries like
> {noformat}
> select * where { ?s ?p }
> {noformat}
> are extremely slow on mongodb.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (RYA-314) Update Rya Console
Caleb Meier created RYA-314: --- Summary: Update Rya Console Key: RYA-314 URL: https://issues.apache.org/jira/browse/RYA-314 Project: Rya Issue Type: Bug Components: clients Affects Versions: 3.2.10 Reporter: Caleb Meier Assignee: Jeff Dasch The Rya console is out of date. There are a number of features that could be added such as the ability to add an RDF file (this underlying class is implemented, it just needs to be hooked in), the ability to issue SPARQL queries through the shell. In addition, there are a large number of additions to the Rya Fluo application that need to be incorporated into the console, such as the ability to create Periodic Queries and the ability to create Construct queries. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RYA-313) Rya Mongo Blows up on Large result sets
[
https://issues.apache.org/jira/browse/RYA-313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron Mihalik updated RYA-313:
--
Description:
Simple queries that return a lot of results fail because mongo is trying to
send all of the results back at once. For instance, if I have a lot of data
and run something like:
{noformat}
SELECT * WHERE
{
?s a ?t.
}
{noformat}
I will get this exception.
{noformat}
Caused by: com.mongodb.MongoCommandException: Command failed with error 16389:
'aggregation result exceeds maximum document size (16MB)' on server
localhost:27017. The full response is { "ok" : 0.0, "errmsg" : "aggregation
result exceeds maximum document size (16MB)", "code" : 16389 }
{noformat}
I think we need to toss in a "AggregationOptions with Batch = 1000", but I
couldn't get that to work immediately. Somebody with more mongo experience
needs to look at this.
[Here is the line of
code|https://github.com/apache/incubator-rya/blob/master/dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/RyaStatementBindingSetCursorIterator.java#L114]
was:
Simple queries that return a lot of results fail because mongo is trying to
send all of the results back at once. For instance, if I have a lot of data
and run something like:
'''
SELECT * WHERE
{
?s a ?t.
}
'''
I will get this exception.
Caused by: com.mongodb.MongoCommandException: Command failed with error 16389:
'aggregation result exceeds maximum document size (16MB)' on server
localhost:27017. The full response is { "ok" : 0.0, "errmsg" : "aggregation
result exceeds maximum document size (16MB)", "code" : 16389 }
I think we need to toss in a "AggregationOptions with Batch = 1000", but I
couldn't get that to work immediately. Somebody with more mongo experience
needs to look at this.
[Here is the line of
code|https://github.com/apache/incubator-rya/blob/master/dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/RyaStatementBindingSetCursorIterator.java#L114
]
> Rya Mongo Blows up on Large result sets
> ---
>
> Key: RYA-313
> URL: https://issues.apache.org/jira/browse/RYA-313
> Project: Rya
> Issue Type: Bug
> Components: dao
>Affects Versions: 3.2.10
> Environment: Mongo DB with Rya 3.2.11-SNAPSHOT with a lot of data in
> Rya
>Reporter: Aaron Mihalik
>Assignee: Andrew Smith
>
> Simple queries that return a lot of results fail because mongo is trying to
> send all of the results back at once. For instance, if I have a lot of data
> and run something like:
> {noformat}
> SELECT * WHERE
> {
> ?s a ?t.
> }
> {noformat}
> I will get this exception.
> {noformat}
> Caused by: com.mongodb.MongoCommandException: Command failed with error
> 16389: 'aggregation result exceeds maximum document size (16MB)' on server
> localhost:27017. The full response is { "ok" : 0.0, "errmsg" : "aggregation
> result exceeds maximum document size (16MB)", "code" : 16389 }
> {noformat}
> I think we need to toss in a "AggregationOptions with Batch = 1000", but I
> couldn't get that to work immediately. Somebody with more mongo experience
> needs to look at this.
> [Here is the line of
> code|https://github.com/apache/incubator-rya/blob/master/dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/RyaStatementBindingSetCursorIterator.java#L114]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (RYA-313) Rya Mongo Blows up on Large result sets
Aaron Mihalik created RYA-313:
-
Summary: Rya Mongo Blows up on Large result sets
Key: RYA-313
URL: https://issues.apache.org/jira/browse/RYA-313
Project: Rya
Issue Type: Bug
Components: dao
Affects Versions: 3.2.10
Environment: Mongo DB with Rya 3.2.11-SNAPSHOT with a lot of data in
Rya
Reporter: Aaron Mihalik
Assignee: Andrew Smith
Simple queries that return a lot of results fail because mongo is trying to
send all of the results back at once. For instance, if I have a lot of data
and run something like:
'''
SELECT * WHERE
{
?s a ?t.
}
'''
I will get this exception.
Caused by: com.mongodb.MongoCommandException: Command failed with error 16389:
'aggregation result exceeds maximum document size (16MB)' on server
localhost:27017. The full response is { "ok" : 0.0, "errmsg" : "aggregation
result exceeds maximum document size (16MB)", "code" : 16389 }
I think we need to toss in a "AggregationOptions with Batch = 1000", but I
couldn't get that to work immediately. Somebody with more mongo experience
needs to look at this.
[Here is the line of
code|https://github.com/apache/incubator-rya/blob/master/dao/mongodb.rya/src/main/java/org/apache/rya/mongodb/iter/RyaStatementBindingSetCursorIterator.java#L114
]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (RYA-305) Wrap PeriodicNotificationApplication in Twill and Test in a Distributed Environment
[ https://issues.apache.org/jira/browse/RYA-305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Meier reassigned RYA-305: --- Assignee: Jeff Dasch (was: Caleb Meier) > Wrap PeriodicNotificationApplication in Twill and Test in a Distributed > Environment > --- > > Key: RYA-305 > URL: https://issues.apache.org/jira/browse/RYA-305 > Project: Rya > Issue Type: Sub-task > Components: clients >Affects Versions: 3.2.10 >Reporter: Caleb Meier >Assignee: Jeff Dasch >Priority: Critical > > In order to run the PeriodicNotificationApplication in a distributed > environment, the application needs to be wrapped in a Twill Application so > that it can be run on Yarn. After creating the Twill application, it needs > to be tested along with an updated version of the Rya-Fluo application to > verify that the service works in a timely manner. There may be some need to > adjust the Fluo config parameter "fluo.impl.ScanTask.maxSleep" which sets the > longest amount of time that Fluo sleeps for until it wakes up and processes > new results. A balance needs to be struck between the need for Fluo to be > timely and preventing a large number of unnecessary checks for "dirty cells". -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RYA-305) Wrap PeriodicNotificationApplication in Twill and Test in a Distributed Environment
[ https://issues.apache.org/jira/browse/RYA-305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Mihalik updated RYA-305: -- Reporter: Caleb Meier (was: Daniel Schierbeck) > Wrap PeriodicNotificationApplication in Twill and Test in a Distributed > Environment > --- > > Key: RYA-305 > URL: https://issues.apache.org/jira/browse/RYA-305 > Project: Rya > Issue Type: Sub-task > Components: clients >Affects Versions: 3.2.10 >Reporter: Caleb Meier >Assignee: Caleb Meier >Priority: Critical > > In order to run the PeriodicNotificationApplication in a distributed > environment, the application needs to be wrapped in a Twill Application so > that it can be run on Yarn. After creating the Twill application, it needs > to be tested along with an updated version of the Rya-Fluo application to > verify that the service works in a timely manner. There may be some need to > adjust the Fluo config parameter "fluo.impl.ScanTask.maxSleep" which sets the > longest amount of time that Fluo sleeps for until it wakes up and processes > new results. A balance needs to be struck between the need for Fluo to be > timely and preventing a large number of unnecessary checks for "dirty cells". -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (RYA-305) Wrap PeriodicNotificationApplication in Twill and Test in a Distributed Environment
[ https://issues.apache.org/jira/browse/RYA-305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron Mihalik updated RYA-305: -- Reporter: Daniel Schierbeck (was: Caleb Meier) > Wrap PeriodicNotificationApplication in Twill and Test in a Distributed > Environment > --- > > Key: RYA-305 > URL: https://issues.apache.org/jira/browse/RYA-305 > Project: Rya > Issue Type: Sub-task > Components: clients >Affects Versions: 3.2.10 >Reporter: Daniel Schierbeck >Assignee: Caleb Meier >Priority: Critical > > In order to run the PeriodicNotificationApplication in a distributed > environment, the application needs to be wrapped in a Twill Application so > that it can be run on Yarn. After creating the Twill application, it needs > to be tested along with an updated version of the Rya-Fluo application to > verify that the service works in a timely manner. There may be some need to > adjust the Fluo config parameter "fluo.impl.ScanTask.maxSleep" which sets the > longest amount of time that Fluo sleeps for until it wakes up and processes > new results. A balance needs to be struck between the need for Fluo to be > timely and preventing a large number of unnecessary checks for "dirty cells". -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091990#comment-16091990
]
ASF GitHub Bot commented on RYA-280:
Github user isper3at commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128063437
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/export/kafka/KafkaBindingSetExporter.java
---
@@ -36,10 +36,11 @@
* Incrementally exports SPARQL query results to Kafka topics.
*/
public class KafkaBindingSetExporter implements
IncrementalBindingSetExporter {
+
--- End diff --
I meant the file had no changes in it, so you could revert it and remove it
from the PR
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user isper3at commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128063437
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/export/kafka/KafkaBindingSetExporter.java
---
@@ -36,10 +36,11 @@
* Incrementally exports SPARQL query results to Kafka topics.
*/
public class KafkaBindingSetExporter implements
IncrementalBindingSetExporter {
+
--- End diff --
I meant the file had no changes in it, so you could revert it and remove it
from the PR
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091988#comment-16091988
]
ASF GitHub Bot commented on RYA-280:
Github user isper3at commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128063325
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/SpanBatchBindingSetUpdater.java
---
@@ -0,0 +1,128 @@
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import java.util.Iterator;
+import java.util.Optional;
+
+import org.apache.fluo.api.client.TransactionBase;
+import org.apache.fluo.api.client.scanner.ColumnScanner;
+import org.apache.fluo.api.client.scanner.RowScanner;
+import org.apache.fluo.api.data.Bytes;
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.ColumnValue;
+import org.apache.fluo.api.data.RowColumn;
+import org.apache.fluo.api.data.Span;
+import org.apache.log4j.Logger;
+import org.apache.rya.indexing.pcj.fluo.app.batch.BatchInformation.Task;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class processes {@link SpanBatchDeleteInformation} objects by
+ * deleting the entries in the Fluo Column corresponding to the {@link
Span}
+ * of the BatchInformation object. This class will delete entries until
the
+ * batch size is met, and then create a new SpanBatchDeleteInformation
object
+ * with an updated Span whose starting point is the stopping point of this
+ * batch. If the batch limit is not met, then a new batch is not created
and
+ * the task is complete.
+ *
+ */
+public class SpanBatchBindingSetUpdater extends
AbstractBatchBindingSetUpdater {
+
+private static final Logger log =
Logger.getLogger(SpanBatchBindingSetUpdater.class);
+
+/**
+ * Process SpanBatchDeleteInformation objects by deleting all entries
indicated
+ * by Span until batch limit is met.
+ * @param tx - Fluo Transaction
+ * @param row - Byte row identifying BatchInformation
+ * @param batch - SpanBatchDeleteInformation object to be processed
+ */
+@Override
+public void processBatch(TransactionBase tx, Bytes row,
BatchInformation batch) throws Exception {
+super.processBatch(tx, row, batch);
+Preconditions.checkArgument(batch instanceof
SpanBatchDeleteInformation);
+SpanBatchDeleteInformation spanBatch =
(SpanBatchDeleteInformation) batch;
+Task task = spanBatch.getTask();
+int batchSize = spanBatch.getBatchSize();
+Span span = spanBatch.getSpan();
+Column column = batch.getColumn();
+Optional rowCol = Optional.empty();
+
+switch (task) {
+case Add:
+log.trace("The Task Add is not supported for
SpanBatchBindingSetUpdater. Batch " + batch + " will not be processed.");
--- End diff --
fair enough
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
>
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user isper3at commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128063325
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/SpanBatchBindingSetUpdater.java
---
@@ -0,0 +1,128 @@
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import java.util.Iterator;
+import java.util.Optional;
+
+import org.apache.fluo.api.client.TransactionBase;
+import org.apache.fluo.api.client.scanner.ColumnScanner;
+import org.apache.fluo.api.client.scanner.RowScanner;
+import org.apache.fluo.api.data.Bytes;
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.ColumnValue;
+import org.apache.fluo.api.data.RowColumn;
+import org.apache.fluo.api.data.Span;
+import org.apache.log4j.Logger;
+import org.apache.rya.indexing.pcj.fluo.app.batch.BatchInformation.Task;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class processes {@link SpanBatchDeleteInformation} objects by
+ * deleting the entries in the Fluo Column corresponding to the {@link
Span}
+ * of the BatchInformation object. This class will delete entries until
the
+ * batch size is met, and then create a new SpanBatchDeleteInformation
object
+ * with an updated Span whose starting point is the stopping point of this
+ * batch. If the batch limit is not met, then a new batch is not created
and
+ * the task is complete.
+ *
+ */
+public class SpanBatchBindingSetUpdater extends
AbstractBatchBindingSetUpdater {
+
+private static final Logger log =
Logger.getLogger(SpanBatchBindingSetUpdater.class);
+
+/**
+ * Process SpanBatchDeleteInformation objects by deleting all entries
indicated
+ * by Span until batch limit is met.
+ * @param tx - Fluo Transaction
+ * @param row - Byte row identifying BatchInformation
+ * @param batch - SpanBatchDeleteInformation object to be processed
+ */
+@Override
+public void processBatch(TransactionBase tx, Bytes row,
BatchInformation batch) throws Exception {
+super.processBatch(tx, row, batch);
+Preconditions.checkArgument(batch instanceof
SpanBatchDeleteInformation);
+SpanBatchDeleteInformation spanBatch =
(SpanBatchDeleteInformation) batch;
+Task task = spanBatch.getTask();
+int batchSize = spanBatch.getBatchSize();
+Span span = spanBatch.getSpan();
+Column column = batch.getColumn();
+Optional rowCol = Optional.empty();
+
+switch (task) {
+case Add:
+log.trace("The Task Add is not supported for
SpanBatchBindingSetUpdater. Batch " + batch + " will not be processed.");
--- End diff --
fair enough
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[ https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091970#comment-16091970 ] ASF GitHub Bot commented on RYA-280: Github user asfgit commented on the issue: https://github.com/apache/incubator-rya/pull/177 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/264/Build result: FAILURE[...truncated 8.58 MB...][INFO] Apache Rya Spark Support ... SKIPPED[INFO] Apache Rya Web Projects SKIPPED[INFO] Apache Rya Web Implementation .. SKIPPED[INFO] [INFO] BUILD FAILURE[INFO] [INFO] Total time: 57:05 min[INFO] Finished at: 2017-07-18T18:37:52+00:00[INFO] Final Memory: 204M/1043M[INFO] [ERROR] Failed to execute goal on project rya.geoindexing: Could not resolve dependencies for project org.apache.rya:rya.geoindexing:jar:3.2.11-incubating-SNAPSHOT: Failed to collect dependencies at org.locationtech.geomesa:geomesa-accumulo-datastore_2.11:jar:1.3.0-m1 -> org.locationtech.geomesa:geomesa-feature-all_2.11:jar:1.3.0-m1 -> org.locationtech.geomesa:geomesa-feature-kryo_2.11:jar:1.3.0-m1 -> org.geotools:gt-process-feature:jar:15.1 -> org.geotools:gt-process:jar:15.1 -> org.geotools:gt-coverage:jar:15.1 -> it.geosolutions.imageio-ext:imageio-ext-tiff:jar:1.1.15: Failed to read artifact descriptor for it.geosolutions.imageio-ext:imageio-ext-tiff:jar:1.1.15: Could not transfer artifact it.geosolutions.imageio-ext:imageio-ext-tiff:pom:1.1.15 from/to geowave-maven-releases (https://s3.amazonaws.com/geowave-maven/release): Access denied to: https://s3.amazonaws.com/geowave-maven/release/it/geosolutions/imageio-ext/imageio-ext-tiff/1.1.15/imageio-ext-tiff-1.1.15.pom , ReasonPhrase:Forbidden. -> [Help 1][ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven using the -X switch to enable full debug logging.[ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException[ERROR] [ERROR] After correcting the problems, you can resume the build with the command[ERROR] mvn -rf :rya.geoindexingchannel stoppedSetting status of fa951132ef77f8f5849134f7f2d5b44acaf191e2 to FAILURE with url https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/264/ and message: 'FAILURE 'Using context: Jenkins: clean package -Pgeoindexing > PeriodicQuery Support for Fluo > -- > > Key: RYA-280 > URL: https://issues.apache.org/jira/browse/RYA-280 > Project: Rya > Issue Type: New Feature > Components: clients >Affects Versions: 3.2.10 >Reporter: Caleb Meier >Assignee: Caleb Meier > > Add the capability to Rya-Fluo App to provide periodic updates for queries > registered with Fluo. That is, provide the application with the ability to > satisfy the standing query "tell me every 12 hours about all of the events of > a particular type that occurred within the last 24 hours". Given that Fluo > operates using a push based notification system, some external service needs > to be implemented to periodically notify Fluo to generate a result > notification (this helps handle the non-event event that occurs when nothing > happens in a given period of time). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] incubator-rya issue #177: RYA-280-Periodic Query Service
Github user asfgit commented on the issue: https://github.com/apache/incubator-rya/pull/177 Refer to this link for build results (access rights to CI server needed): https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/264/Build result: FAILURE[...truncated 8.58 MB...][INFO] Apache Rya Spark Support ... SKIPPED[INFO] Apache Rya Web Projects SKIPPED[INFO] Apache Rya Web Implementation .. SKIPPED[INFO] [INFO] BUILD FAILURE[INFO] [INFO] Total time: 57:05 min[INFO] Finished at: 2017-07-18T18:37:52+00:00[INFO] Final Memory: 204M/1043M[INFO] [ERROR] Failed to execute goal on project rya.geoindexing: Could not resolve dependencies for project org.apache.rya:rya.geoindexing:jar:3.2.11-incubating-SNAPSHOT: Failed to collect dependencies at org.locationtech.geomesa:geomesa-accumulo-datastore_2.11:jar:1.3.0-m1 -> org.locationte ch.geomesa:geomesa-feature-all_2.11:jar:1.3.0-m1 -> org.locationtech.geomesa:geomesa-feature-kryo_2.11:jar:1.3.0-m1 -> org.geotools:gt-process-feature:jar:15.1 -> org.geotools:gt-process:jar:15.1 -> org.geotools:gt-coverage:jar:15.1 -> it.geosolutions.imageio-ext:imageio-ext-tiff:jar:1.1.15: Failed to read artifact descriptor for it.geosolutions.imageio-ext:imageio-ext-tiff:jar:1.1.15: Could not transfer artifact it.geosolutions.imageio-ext:imageio-ext-tiff:pom:1.1.15 from/to geowave-maven-releases (https://s3.amazonaws.com/geowave-maven/release): Access denied to: https://s3.amazonaws.com/geowave-maven/release/it/geosolutions/imageio-ext/imageio-ext-tiff/1.1.15/imageio-ext-tiff-1.1.15.pom , ReasonPhrase:Forbidden. -> [Help 1][ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.[ERROR] Re-run Maven using the -X switch to enable full debug logging.[ERROR] [ERROR] For more information about the errors and possible solutions, please read the follow ing articles:[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException[ERROR] [ERROR] After correcting the problems, you can resume the build with the command[ERROR] mvn -rf :rya.geoindexingchannel stoppedSetting status of fa951132ef77f8f5849134f7f2d5b44acaf191e2 to FAILURE with url https://builds.apache.org/job/incubator-rya-master-with-optionals-pull-requests/264/ and message: 'FAILURE 'Using context: Jenkins: clean package -Pgeoindexing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091867#comment-16091867
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128043592
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/pruner/PeriodicQueryPruner.java
---
@@ -0,0 +1,111 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.periodic.notification.pruner;
+
+import java.util.HashSet;
+import java.util.Set;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import org.apache.fluo.api.client.FluoClient;
+import org.apache.fluo.api.client.Snapshot;
+import org.apache.fluo.api.client.SnapshotBase;
+import org.apache.fluo.api.data.Bytes;
+import org.apache.log4j.Logger;
+import org.apache.rya.indexing.pcj.fluo.app.query.FluoQueryColumns;
+import org.apache.rya.indexing.pcj.fluo.app.util.PeriodicQueryUtil;
+import org.apache.rya.periodic.notification.api.BinPruner;
+import org.apache.rya.periodic.notification.api.NodeBin;
+
+import jline.internal.Preconditions;
+
+/**
+ * Implementation of {@link BinPruner} that deletes old, already processed
+ * Periodic Query results from Fluo and the PCJ table to which the Fluo
results
+ * are exported.
+ *
+ */
+public class PeriodicQueryPruner implements BinPruner, Runnable {
+
+private static final Logger log =
Logger.getLogger(PeriodicQueryPruner.class);
+private FluoClient client;
+private AccumuloBinPruner accPruner;
+private FluoBinPruner fluoPruner;
+private BlockingQueue bins;
+private AtomicBoolean closed = new AtomicBoolean(false);
+private int threadNumber;
+
+public PeriodicQueryPruner(FluoBinPruner fluoPruner, AccumuloBinPruner
accPruner, FluoClient client, BlockingQueue bins, int threadNumber) {
+Preconditions.checkNotNull(fluoPruner);
+Preconditions.checkNotNull(accPruner);
+Preconditions.checkNotNull(client);
+this.client = client;
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128043592
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/pruner/PeriodicQueryPruner.java
---
@@ -0,0 +1,111 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.periodic.notification.pruner;
+
+import java.util.HashSet;
+import java.util.Set;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import org.apache.fluo.api.client.FluoClient;
+import org.apache.fluo.api.client.Snapshot;
+import org.apache.fluo.api.client.SnapshotBase;
+import org.apache.fluo.api.data.Bytes;
+import org.apache.log4j.Logger;
+import org.apache.rya.indexing.pcj.fluo.app.query.FluoQueryColumns;
+import org.apache.rya.indexing.pcj.fluo.app.util.PeriodicQueryUtil;
+import org.apache.rya.periodic.notification.api.BinPruner;
+import org.apache.rya.periodic.notification.api.NodeBin;
+
+import jline.internal.Preconditions;
+
+/**
+ * Implementation of {@link BinPruner} that deletes old, already processed
+ * Periodic Query results from Fluo and the PCJ table to which the Fluo
results
+ * are exported.
+ *
+ */
+public class PeriodicQueryPruner implements BinPruner, Runnable {
+
+private static final Logger log =
Logger.getLogger(PeriodicQueryPruner.class);
+private FluoClient client;
+private AccumuloBinPruner accPruner;
+private FluoBinPruner fluoPruner;
+private BlockingQueue bins;
+private AtomicBoolean closed = new AtomicBoolean(false);
+private int threadNumber;
+
+public PeriodicQueryPruner(FluoBinPruner fluoPruner, AccumuloBinPruner
accPruner, FluoClient client, BlockingQueue bins, int threadNumber) {
+Preconditions.checkNotNull(fluoPruner);
+Preconditions.checkNotNull(accPruner);
+Preconditions.checkNotNull(client);
+this.client = client;
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091864#comment-16091864
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128043280
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/processor/TimestampedNotificationProcessor.java
---
@@ -0,0 +1,206 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.periodic.notification.processor;
+
+import java.util.Optional;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import org.apache.log4j.Logger;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage;
+import
org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage.CloseableIterator;
+import org.apache.rya.periodic.notification.api.BinPruner;
+import org.apache.rya.periodic.notification.api.NodeBin;
+import org.apache.rya.periodic.notification.api.NotificationProcessor;
+import org.apache.rya.periodic.notification.exporter.BindingSetRecord;
+import
org.apache.rya.periodic.notification.exporter.KafkaPeriodicBindingSetExporter;
+import
org.apache.rya.periodic.notification.notification.TimestampedNotification;
+import org.openrdf.query.BindingSet;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * Implementation of {@link NotificationProcessor} that uses the id
indicated by
+ * the {@link TimestampedNotification} to obtain results from the
+ * {@link PeriodicQueryResultStorage} layer containing the results of the
+ * Periodic Query. The TimestampedNotificationProcessor then parses the
results
+ * and adds them to work queues to be processed by the {@link BinPruner}
and the
+ * {@link KafkaPeriodicBindingSetExporter}.
+ *
+ */
+public class TimestampedNotificationProcessor implements
NotificationProcessor, Runnable {
+
+private static final Logger log =
Logger.getLogger(TimestampedNotificationProcessor.class);
+private PeriodicQueryResultStorage periodicStorage;
+private BlockingQueue notifications; //
notifications
+ // to
process
+private BlockingQueue bins; // entries to delete from Fluo
+private BlockingQueue bindingSets; // query results
to export
+private AtomicBoolean closed = new AtomicBoolean(false);
+private int threadNumber;
+
+
+public TimestampedNotificationProcessor(PeriodicQueryResultStorage
periodicStorage,
+BlockingQueue notifications,
BlockingQueue bins, BlockingQueue bindingSets,
+int threadNumber) {
+Preconditions.checkNotNull(notifications);
+Preconditions.checkNotNull(bins);
+Preconditions.checkNotNull(bindingSets);
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This m
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128043280
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/processor/TimestampedNotificationProcessor.java
---
@@ -0,0 +1,206 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.periodic.notification.processor;
+
+import java.util.Optional;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import org.apache.log4j.Logger;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage;
+import
org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage.CloseableIterator;
+import org.apache.rya.periodic.notification.api.BinPruner;
+import org.apache.rya.periodic.notification.api.NodeBin;
+import org.apache.rya.periodic.notification.api.NotificationProcessor;
+import org.apache.rya.periodic.notification.exporter.BindingSetRecord;
+import
org.apache.rya.periodic.notification.exporter.KafkaPeriodicBindingSetExporter;
+import
org.apache.rya.periodic.notification.notification.TimestampedNotification;
+import org.openrdf.query.BindingSet;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * Implementation of {@link NotificationProcessor} that uses the id
indicated by
+ * the {@link TimestampedNotification} to obtain results from the
+ * {@link PeriodicQueryResultStorage} layer containing the results of the
+ * Periodic Query. The TimestampedNotificationProcessor then parses the
results
+ * and adds them to work queues to be processed by the {@link BinPruner}
and the
+ * {@link KafkaPeriodicBindingSetExporter}.
+ *
+ */
+public class TimestampedNotificationProcessor implements
NotificationProcessor, Runnable {
+
+private static final Logger log =
Logger.getLogger(TimestampedNotificationProcessor.class);
+private PeriodicQueryResultStorage periodicStorage;
+private BlockingQueue notifications; //
notifications
+ // to
process
+private BlockingQueue bins; // entries to delete from Fluo
+private BlockingQueue bindingSets; // query results
to export
+private AtomicBoolean closed = new AtomicBoolean(false);
+private int threadNumber;
+
+
+public TimestampedNotificationProcessor(PeriodicQueryResultStorage
periodicStorage,
+BlockingQueue notifications,
BlockingQueue bins, BlockingQueue bindingSets,
+int threadNumber) {
+Preconditions.checkNotNull(notifications);
+Preconditions.checkNotNull(bins);
+Preconditions.checkNotNull(bindingSets);
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091862#comment-16091862
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128042903
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/processor/NotificationProcessorExecutor.java
---
@@ -0,0 +1,117 @@
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */package org.apache.rya.periodic.notification.processor;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.log4j.Logger;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage;
+import org.apache.rya.periodic.notification.api.LifeCycle;
+import org.apache.rya.periodic.notification.api.NodeBin;
+import org.apache.rya.periodic.notification.exporter.BindingSetRecord;
+import
org.apache.rya.periodic.notification.notification.TimestampedNotification;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * Executor service that runs {@link TimestampedNotificationProcessor}s
with basic
+ * functionality for starting, stopping, and determining whether
notification processors are
+ * being executed.
+ *
+ */
+public class NotificationProcessorExecutor implements LifeCycle {
+
+private static final Logger log =
Logger.getLogger(TimestampedNotificationProcessor.class);
+private BlockingQueue notifications; //
notifications
+private BlockingQueue bins; // entries to delete from Fluo
+private BlockingQueue bindingSets; // query results
to
+ // export
+private PeriodicQueryResultStorage periodicStorage;
+private List processors;
+private int numberThreads;
+private ExecutorService executor;
+private boolean running = false;
+
+/**
+ * Creates NotificationProcessorExecutor.
+ * @param periodicStorage - storage layer that periodic results are
read from
+ * @param notifications - notifications are pulled from this queue,
and the timestamp indicates which bin of results to query for
+ * @param bins - after notifications are processed, they are added to
the bin to be deleted
+ * @param bindingSets - results read from the storage layer to be
exported
+ * @param numberThreads - number of threads used for processing
+ */
+public NotificationProcessorExecutor(PeriodicQueryResultStorage
periodicStorage, BlockingQueue notifications,
+BlockingQueue bins, BlockingQueue
bindingSets, int numberThreads) {
+Preconditions.checkNotNull(notifications);
+Preconditions.checkNotNull(bins);
+Preconditions.checkNotNull(bindingSets);
+this.notifications = notifications;
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo t
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128042903
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/processor/NotificationProcessorExecutor.java
---
@@ -0,0 +1,117 @@
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */package org.apache.rya.periodic.notification.processor;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.log4j.Logger;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage;
+import org.apache.rya.periodic.notification.api.LifeCycle;
+import org.apache.rya.periodic.notification.api.NodeBin;
+import org.apache.rya.periodic.notification.exporter.BindingSetRecord;
+import
org.apache.rya.periodic.notification.notification.TimestampedNotification;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * Executor service that runs {@link TimestampedNotificationProcessor}s
with basic
+ * functionality for starting, stopping, and determining whether
notification processors are
+ * being executed.
+ *
+ */
+public class NotificationProcessorExecutor implements LifeCycle {
+
+private static final Logger log =
Logger.getLogger(TimestampedNotificationProcessor.class);
+private BlockingQueue notifications; //
notifications
+private BlockingQueue bins; // entries to delete from Fluo
+private BlockingQueue bindingSets; // query results
to
+ // export
+private PeriodicQueryResultStorage periodicStorage;
+private List processors;
+private int numberThreads;
+private ExecutorService executor;
+private boolean running = false;
+
+/**
+ * Creates NotificationProcessorExecutor.
+ * @param periodicStorage - storage layer that periodic results are
read from
+ * @param notifications - notifications are pulled from this queue,
and the timestamp indicates which bin of results to query for
+ * @param bins - after notifications are processed, they are added to
the bin to be deleted
+ * @param bindingSets - results read from the storage layer to be
exported
+ * @param numberThreads - number of threads used for processing
+ */
+public NotificationProcessorExecutor(PeriodicQueryResultStorage
periodicStorage, BlockingQueue notifications,
+BlockingQueue bins, BlockingQueue
bindingSets, int numberThreads) {
+Preconditions.checkNotNull(notifications);
+Preconditions.checkNotNull(bins);
+Preconditions.checkNotNull(bindingSets);
+this.notifications = notifications;
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091857#comment-16091857
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128042579
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/notification/PeriodicNotification.java
---
@@ -0,0 +1,185 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.periodic.notification.notification;
+
+import java.util.Objects;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.rya.periodic.notification.api.Notification;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * Notification Object used by the Periodic Query Service to inform
workers to
+ * process results for a given Periodic Query with the indicated id.
+ * Additionally, this Object contains a period that indicates a frequency
at
+ * which regular updates are generated.
+ *
+ */
+public class PeriodicNotification implements Notification {
+
+private String id;
+private long period;
+private TimeUnit periodTimeUnit;
+private long initialDelay;
+
+/**
+ * Creates a PeriodicNotification.
+ * @param id - Fluo Query Id that this notification is associated with
+ * @param period - period at which notifications are generated
+ * @param periodTimeUnit - time unit associated with the period and
delay
+ * @param initialDelay - amount of time to wait before generating the
first notification
+ */
+public PeriodicNotification(String id, long period, TimeUnit
periodTimeUnit, long initialDelay) {
+Preconditions.checkNotNull(id);
+Preconditions.checkNotNull(periodTimeUnit);
+Preconditions.checkArgument(period > 0 && initialDelay >= 0);
+this.id = id;
+this.period = period;
+this.periodTimeUnit = periodTimeUnit;
+this.initialDelay = initialDelay;
+}
+
+
+/**
+ * Create a PeriodicNotification
+ * @param other - other PeriodicNotification used in copy constructor
+ */
+public PeriodicNotification(PeriodicNotification other) {
+this(other.id, other.period, other.periodTimeUnit,
other.initialDelay);
+}
+
+public String getId() {
+return id;
+}
+
+/**
+ * @return - period at which regular notifications are generated
+ */
+public long getPeriod() {
+return period;
+}
+
+/**
+ * @return time unit of period and initial delay
+ */
+public TimeUnit getTimeUnit() {
+return periodTimeUnit;
+}
+
+/**
+ * @return amount of time to delay before beginning to generate
notifications
+ */
+public long getInitialDelay() {
+return initialDelay;
+}
+
+@Override
+public String toString() {
+StringBuilder builder = new StringBuilder();
+String delim = "=";
+String delim2 = ";";
+return
builder.append("id").append(delim).append(id).append(delim2).append("period").append(delim).append(period).append(delim2)
+
.append("periodTimeUnit").append(delim).append(periodTimeUnit).append(delim2).append("initialDelay").append(delim)
+.append(initialDelay).toString();
+}
+
+@Override
+public boolean equals(Object other) {
+if (this == other) {
+return true;
+}
+
+if (!(other instanceof PeriodicNotification)) {
+return false;
+}
+
+PeriodicNotification notification = (PeriodicNotification) other;
+
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128042579
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/notification/PeriodicNotification.java
---
@@ -0,0 +1,185 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.periodic.notification.notification;
+
+import java.util.Objects;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.rya.periodic.notification.api.Notification;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * Notification Object used by the Periodic Query Service to inform
workers to
+ * process results for a given Periodic Query with the indicated id.
+ * Additionally, this Object contains a period that indicates a frequency
at
+ * which regular updates are generated.
+ *
+ */
+public class PeriodicNotification implements Notification {
+
+private String id;
+private long period;
+private TimeUnit periodTimeUnit;
+private long initialDelay;
+
+/**
+ * Creates a PeriodicNotification.
+ * @param id - Fluo Query Id that this notification is associated with
+ * @param period - period at which notifications are generated
+ * @param periodTimeUnit - time unit associated with the period and
delay
+ * @param initialDelay - amount of time to wait before generating the
first notification
+ */
+public PeriodicNotification(String id, long period, TimeUnit
periodTimeUnit, long initialDelay) {
+Preconditions.checkNotNull(id);
+Preconditions.checkNotNull(periodTimeUnit);
+Preconditions.checkArgument(period > 0 && initialDelay >= 0);
+this.id = id;
+this.period = period;
+this.periodTimeUnit = periodTimeUnit;
+this.initialDelay = initialDelay;
+}
+
+
+/**
+ * Create a PeriodicNotification
+ * @param other - other PeriodicNotification used in copy constructor
+ */
+public PeriodicNotification(PeriodicNotification other) {
+this(other.id, other.period, other.periodTimeUnit,
other.initialDelay);
+}
+
+public String getId() {
+return id;
+}
+
+/**
+ * @return - period at which regular notifications are generated
+ */
+public long getPeriod() {
+return period;
+}
+
+/**
+ * @return time unit of period and initial delay
+ */
+public TimeUnit getTimeUnit() {
+return periodTimeUnit;
+}
+
+/**
+ * @return amount of time to delay before beginning to generate
notifications
+ */
+public long getInitialDelay() {
+return initialDelay;
+}
+
+@Override
+public String toString() {
+StringBuilder builder = new StringBuilder();
+String delim = "=";
+String delim2 = ";";
+return
builder.append("id").append(delim).append(id).append(delim2).append("period").append(delim).append(period).append(delim2)
+
.append("periodTimeUnit").append(delim).append(periodTimeUnit).append(delim2).append("initialDelay").append(delim)
+.append(initialDelay).toString();
+}
+
+@Override
+public boolean equals(Object other) {
+if (this == other) {
+return true;
+}
+
+if (!(other instanceof PeriodicNotification)) {
+return false;
+}
+
+PeriodicNotification notification = (PeriodicNotification) other;
+return Objects.equals(this.id, notification.id) && (this.period ==
notification.period)
+&& Objects.equals(this.periodTimeUnit,
notification.periodTimeUnit) && (this.initialDelay ==
notification.initialDelay);
+
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128042239
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/notification/PeriodicNotification.java
---
@@ -0,0 +1,185 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.periodic.notification.notification;
+
+import java.util.Objects;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.rya.periodic.notification.api.Notification;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * Notification Object used by the Periodic Query Service to inform
workers to
+ * process results for a given Periodic Query with the indicated id.
+ * Additionally, this Object contains a period that indicates a frequency
at
+ * which regular updates are generated.
+ *
+ */
+public class PeriodicNotification implements Notification {
+
+private String id;
+private long period;
+private TimeUnit periodTimeUnit;
+private long initialDelay;
+
+/**
+ * Creates a PeriodicNotification.
+ * @param id - Fluo Query Id that this notification is associated with
+ * @param period - period at which notifications are generated
+ * @param periodTimeUnit - time unit associated with the period and
delay
+ * @param initialDelay - amount of time to wait before generating the
first notification
+ */
+public PeriodicNotification(String id, long period, TimeUnit
periodTimeUnit, long initialDelay) {
+Preconditions.checkNotNull(id);
+Preconditions.checkNotNull(periodTimeUnit);
+Preconditions.checkArgument(period > 0 && initialDelay >= 0);
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091855#comment-16091855
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128042239
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/notification/PeriodicNotification.java
---
@@ -0,0 +1,185 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.periodic.notification.notification;
+
+import java.util.Objects;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.rya.periodic.notification.api.Notification;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * Notification Object used by the Periodic Query Service to inform
workers to
+ * process results for a given Periodic Query with the indicated id.
+ * Additionally, this Object contains a period that indicates a frequency
at
+ * which regular updates are generated.
+ *
+ */
+public class PeriodicNotification implements Notification {
+
+private String id;
+private long period;
+private TimeUnit periodTimeUnit;
+private long initialDelay;
+
+/**
+ * Creates a PeriodicNotification.
+ * @param id - Fluo Query Id that this notification is associated with
+ * @param period - period at which notifications are generated
+ * @param periodTimeUnit - time unit associated with the period and
delay
+ * @param initialDelay - amount of time to wait before generating the
first notification
+ */
+public PeriodicNotification(String id, long period, TimeUnit
periodTimeUnit, long initialDelay) {
+Preconditions.checkNotNull(id);
+Preconditions.checkNotNull(periodTimeUnit);
+Preconditions.checkArgument(period > 0 && initialDelay >= 0);
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091853#comment-16091853
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128041892
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/notification/CommandNotification.java
---
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.periodic.notification.notification;
+
+import org.apache.rya.periodic.notification.api.Notification;
+
+import com.google.common.base.Objects;
+import com.google.common.base.Preconditions;
+
+/**
+ * This Object contains a Notification Object used by the Periodic Query
Service
+ * to inform workers to process results for a given Periodic Query with the
+ * indicated id. Additionally, the CommandNotification contains a
+ * {@link Command} about which action the
+ * {@link NotificationCoordinatorExecutor} should take (adding or
deleting).
+ * CommandNotifications are meant to be added to an external work queue
(such as
+ * Kafka) to be processed by the NotificationCoordinatorExecutor.
+ *
+ */
+public class CommandNotification implements Notification {
+
+private Notification notification;
+private Command command;
+
+public enum Command {
+ADD, DELETE
+};
+
+/**
+ * Creates a new CommandNotification
+ * @param command - the command associated with this notification
(either add, update, or delete)
+ * @param notification - the underlying notification associated with
this command
+ */
+public CommandNotification(Command command, Notification notification)
{
+Preconditions.checkNotNull(notification);
+Preconditions.checkNotNull(command);
+this.command = command;
+this.notification = notification;
+}
+
+@Override
+public String getId() {
+return notification.getId();
+}
+
+/**
+ * Returns {@link Notification} contained by this CommmandNotification.
+ * @return - Notification contained by this Object
+ */
+public Notification getNotification() {
+return this.notification;
+}
+
+/**
+ * @return Command contained by this Object (either add or delete)
+ */
+public Command getCommand() {
+return this.command;
+}
+
+@Override
+public boolean equals(Object other) {
+if (this == other) {
+return true;
+}
+if (other instanceof CommandNotification) {
+CommandNotification cn = (CommandNotification) other;
+return Objects.equal(this.command, cn.command) &&
Objects.equal(this.notification, cn.notification);
+} else {
+return false;
+}
+}
+
+@Override
+public int hashCode() {
+int result = 17;
+result = 31 * result + Objects.hashCode(command);
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128041892
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/notification/CommandNotification.java
---
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.periodic.notification.notification;
+
+import org.apache.rya.periodic.notification.api.Notification;
+
+import com.google.common.base.Objects;
+import com.google.common.base.Preconditions;
+
+/**
+ * This Object contains a Notification Object used by the Periodic Query
Service
+ * to inform workers to process results for a given Periodic Query with the
+ * indicated id. Additionally, the CommandNotification contains a
+ * {@link Command} about which action the
+ * {@link NotificationCoordinatorExecutor} should take (adding or
deleting).
+ * CommandNotifications are meant to be added to an external work queue
(such as
+ * Kafka) to be processed by the NotificationCoordinatorExecutor.
+ *
+ */
+public class CommandNotification implements Notification {
+
+private Notification notification;
+private Command command;
+
+public enum Command {
+ADD, DELETE
+};
+
+/**
+ * Creates a new CommandNotification
+ * @param command - the command associated with this notification
(either add, update, or delete)
+ * @param notification - the underlying notification associated with
this command
+ */
+public CommandNotification(Command command, Notification notification)
{
+Preconditions.checkNotNull(notification);
+Preconditions.checkNotNull(command);
+this.command = command;
+this.notification = notification;
+}
+
+@Override
+public String getId() {
+return notification.getId();
+}
+
+/**
+ * Returns {@link Notification} contained by this CommmandNotification.
+ * @return - Notification contained by this Object
+ */
+public Notification getNotification() {
+return this.notification;
+}
+
+/**
+ * @return Command contained by this Object (either add or delete)
+ */
+public Command getCommand() {
+return this.command;
+}
+
+@Override
+public boolean equals(Object other) {
+if (this == other) {
+return true;
+}
+if (other instanceof CommandNotification) {
+CommandNotification cn = (CommandNotification) other;
+return Objects.equal(this.command, cn.command) &&
Objects.equal(this.notification, cn.notification);
+} else {
+return false;
+}
+}
+
+@Override
+public int hashCode() {
+int result = 17;
+result = 31 * result + Objects.hashCode(command);
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091848#comment-16091848
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128041556
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/notification/CommandNotification.java
---
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.periodic.notification.notification;
+
+import org.apache.rya.periodic.notification.api.Notification;
+
+import com.google.common.base.Objects;
+import com.google.common.base.Preconditions;
+
+/**
+ * This Object contains a Notification Object used by the Periodic Query
Service
+ * to inform workers to process results for a given Periodic Query with the
+ * indicated id. Additionally, the CommandNotification contains a
+ * {@link Command} about which action the
+ * {@link NotificationCoordinatorExecutor} should take (adding or
deleting).
+ * CommandNotifications are meant to be added to an external work queue
(such as
+ * Kafka) to be processed by the NotificationCoordinatorExecutor.
+ *
+ */
+public class CommandNotification implements Notification {
+
+private Notification notification;
+private Command command;
+
+public enum Command {
+ADD, DELETE
+};
+
+/**
+ * Creates a new CommandNotification
+ * @param command - the command associated with this notification
(either add, update, or delete)
+ * @param notification - the underlying notification associated with
this command
+ */
+public CommandNotification(Command command, Notification notification)
{
+Preconditions.checkNotNull(notification);
+Preconditions.checkNotNull(command);
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128041556
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/notification/CommandNotification.java
---
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.periodic.notification.notification;
+
+import org.apache.rya.periodic.notification.api.Notification;
+
+import com.google.common.base.Objects;
+import com.google.common.base.Preconditions;
+
+/**
+ * This Object contains a Notification Object used by the Periodic Query
Service
+ * to inform workers to process results for a given Periodic Query with the
+ * indicated id. Additionally, the CommandNotification contains a
+ * {@link Command} about which action the
+ * {@link NotificationCoordinatorExecutor} should take (adding or
deleting).
+ * CommandNotifications are meant to be added to an external work queue
(such as
+ * Kafka) to be processed by the NotificationCoordinatorExecutor.
+ *
+ */
+public class CommandNotification implements Notification {
+
+private Notification notification;
+private Command command;
+
+public enum Command {
+ADD, DELETE
+};
+
+/**
+ * Creates a new CommandNotification
+ * @param command - the command associated with this notification
(either add, update, or delete)
+ * @param notification - the underlying notification associated with
this command
+ */
+public CommandNotification(Command command, Notification notification)
{
+Preconditions.checkNotNull(notification);
+Preconditions.checkNotNull(command);
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091844#comment-16091844
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128041076
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/notification/BasicNotification.java
---
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.periodic.notification.notification;
+
+import org.apache.rya.periodic.notification.api.Notification;
+
+import com.google.common.base.Objects;
+
+/**
+ * Notification Object used by the Periodic Query Service
+ * to inform workers to process results for a given Periodic
+ * Query with the indicated id.
+ *
+ */
+public class BasicNotification implements Notification {
+
+private String id;
+
+/**
+ * Creates a BasicNotification
+ * @param id - Fluo query id associated with this Notification
+ */
+public BasicNotification(String id) {
+this.id = id;
+}
+
+/**
+ * @return the Fluo Query Id that this notification will generate
results for
+ */
+@Override
+public String getId() {
+return id;
+}
+
+@Override
+public boolean equals(Object other) {
+if (this == other) {
+return true;
+}
+
+if (other instanceof BasicNotification) {
+BasicNotification not = (BasicNotification) other;
+return Objects.equal(this.id, not.id);
+}
+
+return false;
+}
+
+@Override
+public int hashCode() {
+int result = 17;
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128041076
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/notification/BasicNotification.java
---
@@ -0,0 +1,78 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.periodic.notification.notification;
+
+import org.apache.rya.periodic.notification.api.Notification;
+
+import com.google.common.base.Objects;
+
+/**
+ * Notification Object used by the Periodic Query Service
+ * to inform workers to process results for a given Periodic
+ * Query with the indicated id.
+ *
+ */
+public class BasicNotification implements Notification {
+
+private String id;
+
+/**
+ * Creates a BasicNotification
+ * @param id - Fluo query id associated with this Notification
+ */
+public BasicNotification(String id) {
+this.id = id;
+}
+
+/**
+ * @return the Fluo Query Id that this notification will generate
results for
+ */
+@Override
+public String getId() {
+return id;
+}
+
+@Override
+public boolean equals(Object other) {
+if (this == other) {
+return true;
+}
+
+if (other instanceof BasicNotification) {
+BasicNotification not = (BasicNotification) other;
+return Objects.equal(this.id, not.id);
+}
+
+return false;
+}
+
+@Override
+public int hashCode() {
+int result = 17;
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091673#comment-16091673
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128001523
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/application/PeriodicNotificationApplication.java
---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.periodic.notification.application;
+
+import org.apache.log4j.Logger;
+import org.apache.rya.indexing.pcj.fluo.app.util.PeriodicQueryUtil;
+import org.apache.rya.periodic.notification.api.BinPruner;
+import org.apache.rya.periodic.notification.api.LifeCycle;
+import org.apache.rya.periodic.notification.api.NodeBin;
+import
org.apache.rya.periodic.notification.api.NotificationCoordinatorExecutor;
+import org.apache.rya.periodic.notification.exporter.BindingSetRecord;
+import org.apache.rya.periodic.notification.exporter.KafkaExporterExecutor;
+import
org.apache.rya.periodic.notification.processor.NotificationProcessorExecutor;
+import
org.apache.rya.periodic.notification.pruner.PeriodicQueryPrunerExecutor;
+import
org.apache.rya.periodic.notification.registration.kafka.KafkaNotificationProvider;
+import org.openrdf.query.algebra.evaluation.function.Function;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * The PeriodicNotificationApplication runs the key components of the
Periodic
+ * Query Service. It consists of a {@link KafkaNotificationProvider}, a
+ * {@link NotificationCoordinatorExecutor}, a
+ * {@link NotificationProcessorExecutor}, a {@link KafkaExporterExecutor},
and a
+ * {@link PeriodicQueryPrunerExecutor}. These services run in coordination
with
+ * one another to perform the following tasks in the indicated order:
+ * Retrieve new requests to generate periodic notifications from Kafka
+ * Register them with the {@link NotificationCoordinatorExecutor} to
+ * generate the periodic notifications
+ * As notifications are generated, they are added to a work queue that
is
+ * monitored by the {@link NotificationProcessorExecutor}.
+ * The processor processes the notifications by reading all of the
query
+ * results corresponding to the bin and query id indicated by the
notification.
+ * After reading the results, the processor adds a {@link
BindingSetRecord}
+ * to a work queue monitored by the {@link KafkaExporterExecutor}.
+ * The processor then adds a {@link NodeBin} to a workqueue monitored
by the
+ * {@link BinPruner}
+ * The exporter processes the BindingSetRecord by exporing the result
to
+ * Kafka
+ * The BinPruner processes the NodeBin by cleaning up the results for
the
+ * indicated bin and query in Accumulo and Fluo.
+ *
+ * The purpose of this Periodic Query Service is to facilitate the ability
to
+ * answer Periodic Queries using the Rya Fluo application, where a Periodic
+ * Query is any query requesting periodic updates about events that
occurred
+ * within a given window of time of this instant. This is also known as a
+ * rolling window query. Period Queries can be expressed using SPARQL by
+ * including the {@link Function} indicated by the URI
+ * {@link PeriodicQueryUtil#PeriodicQueryURI}. The user must provide this
+ * Function with the following arguments: the temporal variable in the
query
+ * that will be filtered on, the window of time that events must occur
within,
+ * the period at which the user wants to receive updates, and the time
unit. The
+ * following query requests all observations that occurred within the last
+ * minute and requests updates every 15 seconds. It also performs a count
on
+ * those observations.
+ *
+ * prefix functi
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128001523
--- Diff:
extras/rya.periodic.service/periodic.service.notification/src/main/java/org/apache/rya/periodic/notification/application/PeriodicNotificationApplication.java
---
@@ -0,0 +1,212 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.periodic.notification.application;
+
+import org.apache.log4j.Logger;
+import org.apache.rya.indexing.pcj.fluo.app.util.PeriodicQueryUtil;
+import org.apache.rya.periodic.notification.api.BinPruner;
+import org.apache.rya.periodic.notification.api.LifeCycle;
+import org.apache.rya.periodic.notification.api.NodeBin;
+import
org.apache.rya.periodic.notification.api.NotificationCoordinatorExecutor;
+import org.apache.rya.periodic.notification.exporter.BindingSetRecord;
+import org.apache.rya.periodic.notification.exporter.KafkaExporterExecutor;
+import
org.apache.rya.periodic.notification.processor.NotificationProcessorExecutor;
+import
org.apache.rya.periodic.notification.pruner.PeriodicQueryPrunerExecutor;
+import
org.apache.rya.periodic.notification.registration.kafka.KafkaNotificationProvider;
+import org.openrdf.query.algebra.evaluation.function.Function;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * The PeriodicNotificationApplication runs the key components of the
Periodic
+ * Query Service. It consists of a {@link KafkaNotificationProvider}, a
+ * {@link NotificationCoordinatorExecutor}, a
+ * {@link NotificationProcessorExecutor}, a {@link KafkaExporterExecutor},
and a
+ * {@link PeriodicQueryPrunerExecutor}. These services run in coordination
with
+ * one another to perform the following tasks in the indicated order:
+ * Retrieve new requests to generate periodic notifications from Kafka
+ * Register them with the {@link NotificationCoordinatorExecutor} to
+ * generate the periodic notifications
+ * As notifications are generated, they are added to a work queue that
is
+ * monitored by the {@link NotificationProcessorExecutor}.
+ * The processor processes the notifications by reading all of the
query
+ * results corresponding to the bin and query id indicated by the
notification.
+ * After reading the results, the processor adds a {@link
BindingSetRecord}
+ * to a work queue monitored by the {@link KafkaExporterExecutor}.
+ * The processor then adds a {@link NodeBin} to a workqueue monitored
by the
+ * {@link BinPruner}
+ * The exporter processes the BindingSetRecord by exporing the result
to
+ * Kafka
+ * The BinPruner processes the NodeBin by cleaning up the results for
the
+ * indicated bin and query in Accumulo and Fluo.
+ *
+ * The purpose of this Periodic Query Service is to facilitate the ability
to
+ * answer Periodic Queries using the Rya Fluo application, where a Periodic
+ * Query is any query requesting periodic updates about events that
occurred
+ * within a given window of time of this instant. This is also known as a
+ * rolling window query. Period Queries can be expressed using SPARQL by
+ * including the {@link Function} indicated by the URI
+ * {@link PeriodicQueryUtil#PeriodicQueryURI}. The user must provide this
+ * Function with the following arguments: the temporal variable in the
query
+ * that will be filtered on, the window of time that events must occur
within,
+ * the period at which the user wants to receive updates, and the time
unit. The
+ * following query requests all observations that occurred within the last
+ * minute and requests updates every 15 seconds. It also performs a count
on
+ * those observations.
+ *
+ * prefix function: http://org.apache.rya/function#
+ * "prefix time: http://www.w3.org/2006/time#
+ * "select (count(?obs) as ?total) where {
+ * "Filter(function:periodic(?time, 1, .25, time:minutes))
+ * "?obs uri:hasTime ?time.
+ * "?obs u
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091671#comment-16091671
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128001253
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/test/java/org/apache/rya/indexing/pcj/fluo/app/batch/serializer/BatchInformationSerializerTest.java
---
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.fluo.app.batch.serializer;
+
+import static org.junit.Assert.assertEquals;
+
+import java.util.Arrays;
+import java.util.Optional;
+
+import org.apache.fluo.api.data.Bytes;
+import org.apache.fluo.api.data.Span;
+import org.apache.rya.indexing.pcj.fluo.app.JoinResultUpdater.Side;
+import org.apache.rya.indexing.pcj.fluo.app.batch.BatchInformation;
+import org.apache.rya.indexing.pcj.fluo.app.batch.BatchInformation.Task;
+import org.apache.rya.indexing.pcj.fluo.app.batch.JoinBatchInformation;
+import
org.apache.rya.indexing.pcj.fluo.app.batch.SpanBatchDeleteInformation;
+import org.apache.rya.indexing.pcj.fluo.app.query.FluoQueryColumns;
+import org.apache.rya.indexing.pcj.fluo.app.query.JoinMetadata.JoinType;
+import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder;
+import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSet;
+import org.junit.Test;
+import org.openrdf.model.impl.URIImpl;
+import org.openrdf.query.algebra.evaluation.QueryBindingSet;
+
+public class BatchInformationSerializerTest {
+
+@Test
+public void testSpanBatchInformationSerialization() {
+
+SpanBatchDeleteInformation batch =
SpanBatchDeleteInformation.builder().setBatchSize(1000)
+
.setColumn(FluoQueryColumns.PERIODIC_QUERY_BINDING_SET).setSpan(Span.prefix(Bytes.of("prefix"))).build();
+System.out.println(batch);
+byte[] batchBytes = BatchInformationSerializer.toBytes(batch);
+Optional decodedBatch =
BatchInformationSerializer.fromBytes(batchBytes);
+System.out.println(decodedBatch);
+assertEquals(batch, decodedBatch.get());
+}
+
+@Test
+public void testJoinBatchInformationSerialization() {
+
+QueryBindingSet bs = new QueryBindingSet();
+bs.addBinding("a", new URIImpl("urn:123"));
+bs.addBinding("b", new URIImpl("urn:456"));
+VisibilityBindingSet vBis = new VisibilityBindingSet(bs, "FOUO");
+
+JoinBatchInformation batch =
JoinBatchInformation.builder().setBatchSize(1000).setTask(Task.Update)
+
.setColumn(FluoQueryColumns.PERIODIC_QUERY_BINDING_SET).setSpan(Span.prefix(Bytes.of("prefix346")))
+
.setJoinType(JoinType.LEFT_OUTER_JOIN).setSide(Side.RIGHT).setVarOrder(new
VariableOrder(Arrays.asList("a", "b")))
+.setBs(vBis).build();
+
+System.out.println(batch);
--- End diff --
deleted
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128001253
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/test/java/org/apache/rya/indexing/pcj/fluo/app/batch/serializer/BatchInformationSerializerTest.java
---
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.fluo.app.batch.serializer;
+
+import static org.junit.Assert.assertEquals;
+
+import java.util.Arrays;
+import java.util.Optional;
+
+import org.apache.fluo.api.data.Bytes;
+import org.apache.fluo.api.data.Span;
+import org.apache.rya.indexing.pcj.fluo.app.JoinResultUpdater.Side;
+import org.apache.rya.indexing.pcj.fluo.app.batch.BatchInformation;
+import org.apache.rya.indexing.pcj.fluo.app.batch.BatchInformation.Task;
+import org.apache.rya.indexing.pcj.fluo.app.batch.JoinBatchInformation;
+import
org.apache.rya.indexing.pcj.fluo.app.batch.SpanBatchDeleteInformation;
+import org.apache.rya.indexing.pcj.fluo.app.query.FluoQueryColumns;
+import org.apache.rya.indexing.pcj.fluo.app.query.JoinMetadata.JoinType;
+import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder;
+import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSet;
+import org.junit.Test;
+import org.openrdf.model.impl.URIImpl;
+import org.openrdf.query.algebra.evaluation.QueryBindingSet;
+
+public class BatchInformationSerializerTest {
+
+@Test
+public void testSpanBatchInformationSerialization() {
+
+SpanBatchDeleteInformation batch =
SpanBatchDeleteInformation.builder().setBatchSize(1000)
+
.setColumn(FluoQueryColumns.PERIODIC_QUERY_BINDING_SET).setSpan(Span.prefix(Bytes.of("prefix"))).build();
+System.out.println(batch);
+byte[] batchBytes = BatchInformationSerializer.toBytes(batch);
+Optional decodedBatch =
BatchInformationSerializer.fromBytes(batchBytes);
+System.out.println(decodedBatch);
+assertEquals(batch, decodedBatch.get());
+}
+
+@Test
+public void testJoinBatchInformationSerialization() {
+
+QueryBindingSet bs = new QueryBindingSet();
+bs.addBinding("a", new URIImpl("urn:123"));
+bs.addBinding("b", new URIImpl("urn:456"));
+VisibilityBindingSet vBis = new VisibilityBindingSet(bs, "FOUO");
+
+JoinBatchInformation batch =
JoinBatchInformation.builder().setBatchSize(1000).setTask(Task.Update)
+
.setColumn(FluoQueryColumns.PERIODIC_QUERY_BINDING_SET).setSpan(Span.prefix(Bytes.of("prefix346")))
+
.setJoinType(JoinType.LEFT_OUTER_JOIN).setSide(Side.RIGHT).setVarOrder(new
VariableOrder(Arrays.asList("a", "b")))
+.setBs(vBis).build();
+
+System.out.println(batch);
--- End diff --
deleted
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091669#comment-16091669
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128000960
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/query/PeriodicQueryNode.java
---
@@ -0,0 +1,157 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.fluo.app.query;
+
+import java.util.Objects;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.commons.lang3.builder.EqualsBuilder;
+import org.apache.rya.indexing.pcj.fluo.app.util.PeriodicQueryUtil;
+import org.openrdf.query.algebra.QueryModelVisitor;
+import org.openrdf.query.algebra.TupleExpr;
+import org.openrdf.query.algebra.UnaryTupleOperator;
+import org.openrdf.query.algebra.evaluation.function.Function;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+import static com.google.common.base.Preconditions.checkArgument;
+
+/**
+ * This is a {@link UnaryTupleOperator} that gets placed in the parsed
query
+ * {@link TupleExpr} when a {@link Filter} is encountered in the SPARQL
String that
+ * contains the Periodic {@link Function} {@link
PeriodicQueryUtil#PeriodicQueryURI}.
+ * The PeiodicQueryNode is created from the arguments passed to the
Periodic Function,
+ * which consist of a time unit, a temporal period, a temporal window of
time, and the
+ * temporal variable in the query, which assumes a value indicated by the
+ * Time ontology: http://www.w3.org/2006/time. The purpose of the
PeriodicQueryNode
+ * is to filter out all events that did not occur within the specified
window of time
+ * of this instant and to generate notifications at a regular interval
indicated by the period.
+ *
+ */
+public class PeriodicQueryNode extends UnaryTupleOperator {
+
+private TimeUnit unit;
+private long windowDuration;
+private long periodDuration;
+private String temporalVar;
+
+/**
+ * Creates a PeriodicQueryNode from the specified values.
+ * @param window - specifies the window of time that event must occur
within from this instant
+ * @param period - regular interval at which notifications are
generated (must be leq window).
+ * @param unit - time unit of the period and window
+ * @param temporalVar - temporal variable in query used for filtering
+ * @param arg - child of PeriodicQueryNode in parsed query
+ */
+public PeriodicQueryNode(long window, long period, TimeUnit unit,
String temporalVar, TupleExpr arg) {
+super(arg);
+checkArgument(period <= window);
+checkNotNull(temporalVar);
+checkNotNull(arg);
+checkNotNull(unit);
+this.windowDuration = window;
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128000960
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/query/PeriodicQueryNode.java
---
@@ -0,0 +1,157 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.fluo.app.query;
+
+import java.util.Objects;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.commons.lang3.builder.EqualsBuilder;
+import org.apache.rya.indexing.pcj.fluo.app.util.PeriodicQueryUtil;
+import org.openrdf.query.algebra.QueryModelVisitor;
+import org.openrdf.query.algebra.TupleExpr;
+import org.openrdf.query.algebra.UnaryTupleOperator;
+import org.openrdf.query.algebra.evaluation.function.Function;
+
+import static com.google.common.base.Preconditions.checkNotNull;
+import static com.google.common.base.Preconditions.checkArgument;
+
+/**
+ * This is a {@link UnaryTupleOperator} that gets placed in the parsed
query
+ * {@link TupleExpr} when a {@link Filter} is encountered in the SPARQL
String that
+ * contains the Periodic {@link Function} {@link
PeriodicQueryUtil#PeriodicQueryURI}.
+ * The PeiodicQueryNode is created from the arguments passed to the
Periodic Function,
+ * which consist of a time unit, a temporal period, a temporal window of
time, and the
+ * temporal variable in the query, which assumes a value indicated by the
+ * Time ontology: http://www.w3.org/2006/time. The purpose of the
PeriodicQueryNode
+ * is to filter out all events that did not occur within the specified
window of time
+ * of this instant and to generate notifications at a regular interval
indicated by the period.
+ *
+ */
+public class PeriodicQueryNode extends UnaryTupleOperator {
+
+private TimeUnit unit;
+private long windowDuration;
+private long periodDuration;
+private String temporalVar;
+
+/**
+ * Creates a PeriodicQueryNode from the specified values.
+ * @param window - specifies the window of time that event must occur
within from this instant
+ * @param period - regular interval at which notifications are
generated (must be leq window).
+ * @param unit - time unit of the period and window
+ * @param temporalVar - temporal variable in query used for filtering
+ * @param arg - child of PeriodicQueryNode in parsed query
+ */
+public PeriodicQueryNode(long window, long period, TimeUnit unit,
String temporalVar, TupleExpr arg) {
+super(arg);
+checkArgument(period <= window);
+checkNotNull(temporalVar);
+checkNotNull(arg);
+checkNotNull(unit);
+this.windowDuration = window;
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091667#comment-16091667
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128000243
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/query/PeriodicQueryMetadata.java
---
@@ -0,0 +1,291 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.fluo.app.query;
+
+import java.util.concurrent.TimeUnit;
+
+import org.apache.commons.lang3.builder.EqualsBuilder;
+import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder;
+
+import com.google.common.base.Objects;
+import com.google.common.base.Preconditions;
+
+/**
+ * Metadata that is required for periodic queries in the Rya Fluo
Application.
+ * If a periodic query is registered with the Rya Fluo application, the
BindingSets
+ * are placed into temporal bins according to whether they occur within
the window of
+ * a period's ending time. This Metadata is used to create a Bin Id,
which is equivalent
+ * to the period's ending time, to be inserted into each BindingSet that
occurs within that
+ * bin. This is to allow the AggregationUpdater to aggregate the bins by
grouping on the
+ * Bin Id.
+ *
+ */
+public class PeriodicQueryMetadata extends CommonNodeMetadata {
+
+private String parentNodeId;
+private String childNodeId;
+private long windowSize;
+private long period;
+private TimeUnit unit;
+private String temporalVariable;
+
+/**
+ * Constructs an instance of PeriodicQueryMetadata
+ * @param nodeId - id of periodic query node
+ * @param varOrder - variable order indicating the order the
BindingSet results are written in
+ * @param parentNodeId - id of parent node
+ * @param childNodeId - id of child node
+ * @param windowSize - size of window used for filtering
+ * @param period - period size that indicates frequency of
notifications
+ * @param unit - TimeUnit corresponding to window and period
+ * @param temporalVariable - temporal variable that periodic
conditions are applied to
+ */
+public PeriodicQueryMetadata(String nodeId, VariableOrder varOrder,
String parentNodeId, String childNodeId, long windowSize, long period,
+TimeUnit unit, String temporalVariable) {
+super(nodeId, varOrder);
+Preconditions.checkNotNull(parentNodeId);
+Preconditions.checkNotNull(childNodeId);
+Preconditions.checkNotNull(temporalVariable);
+Preconditions.checkNotNull(unit);
+Preconditions.checkNotNull(period > 0);
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r128000243
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/query/PeriodicQueryMetadata.java
---
@@ -0,0 +1,291 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.fluo.app.query;
+
+import java.util.concurrent.TimeUnit;
+
+import org.apache.commons.lang3.builder.EqualsBuilder;
+import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder;
+
+import com.google.common.base.Objects;
+import com.google.common.base.Preconditions;
+
+/**
+ * Metadata that is required for periodic queries in the Rya Fluo
Application.
+ * If a periodic query is registered with the Rya Fluo application, the
BindingSets
+ * are placed into temporal bins according to whether they occur within
the window of
+ * a period's ending time. This Metadata is used to create a Bin Id,
which is equivalent
+ * to the period's ending time, to be inserted into each BindingSet that
occurs within that
+ * bin. This is to allow the AggregationUpdater to aggregate the bins by
grouping on the
+ * Bin Id.
+ *
+ */
+public class PeriodicQueryMetadata extends CommonNodeMetadata {
+
+private String parentNodeId;
+private String childNodeId;
+private long windowSize;
+private long period;
+private TimeUnit unit;
+private String temporalVariable;
+
+/**
+ * Constructs an instance of PeriodicQueryMetadata
+ * @param nodeId - id of periodic query node
+ * @param varOrder - variable order indicating the order the
BindingSet results are written in
+ * @param parentNodeId - id of parent node
+ * @param childNodeId - id of child node
+ * @param windowSize - size of window used for filtering
+ * @param period - period size that indicates frequency of
notifications
+ * @param unit - TimeUnit corresponding to window and period
+ * @param temporalVariable - temporal variable that periodic
conditions are applied to
+ */
+public PeriodicQueryMetadata(String nodeId, VariableOrder varOrder,
String parentNodeId, String childNodeId, long windowSize, long period,
+TimeUnit unit, String temporalVariable) {
+super(nodeId, varOrder);
+Preconditions.checkNotNull(parentNodeId);
+Preconditions.checkNotNull(childNodeId);
+Preconditions.checkNotNull(temporalVariable);
+Preconditions.checkNotNull(unit);
+Preconditions.checkNotNull(period > 0);
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[ https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091663#comment-16091663 ] ASF GitHub Bot commented on RYA-280: Github user meiercaleb commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/177#discussion_r127999634 --- Diff: extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/observers/PeriodicQueryObserver.java --- @@ -0,0 +1,63 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.rya.indexing.pcj.fluo.app.observers; + +import static java.util.Objects.requireNonNull; + +import org.apache.fluo.api.client.TransactionBase; +import org.apache.fluo.api.data.Bytes; +import org.apache.rya.indexing.pcj.fluo.app.BindingSetRow; +import org.apache.rya.indexing.pcj.fluo.app.query.FluoQueryColumns; +import org.apache.rya.indexing.pcj.fluo.app.query.FluoQueryMetadataDAO; +import org.apache.rya.indexing.pcj.fluo.app.query.PeriodicQueryMetadata; +import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSet; +import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSetSerDe; + --- End diff -- done > PeriodicQuery Support for Fluo > -- > > Key: RYA-280 > URL: https://issues.apache.org/jira/browse/RYA-280 > Project: Rya > Issue Type: New Feature > Components: clients >Affects Versions: 3.2.10 >Reporter: Caleb Meier >Assignee: Caleb Meier > > Add the capability to Rya-Fluo App to provide periodic updates for queries > registered with Fluo. That is, provide the application with the ability to > satisfy the standing query "tell me every 12 hours about all of the events of > a particular type that occurred within the last 24 hours". Given that Fluo > operates using a push based notification system, some external service needs > to be implemented to periodically notify Fluo to generate a result > notification (this helps handle the non-event event that occurs when nothing > happens in a given period of time). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/177#discussion_r127999634 --- Diff: extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/observers/PeriodicQueryObserver.java --- @@ -0,0 +1,63 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.rya.indexing.pcj.fluo.app.observers; + +import static java.util.Objects.requireNonNull; + +import org.apache.fluo.api.client.TransactionBase; +import org.apache.fluo.api.data.Bytes; +import org.apache.rya.indexing.pcj.fluo.app.BindingSetRow; +import org.apache.rya.indexing.pcj.fluo.app.query.FluoQueryColumns; +import org.apache.rya.indexing.pcj.fluo.app.query.FluoQueryMetadataDAO; +import org.apache.rya.indexing.pcj.fluo.app.query.PeriodicQueryMetadata; +import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSet; +import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSetSerDe; + --- End diff -- done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091654#comment-16091654
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127997700
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/export/kafka/KafkaBindingSetExporter.java
---
@@ -36,10 +36,11 @@
* Incrementally exports SPARQL query results to Kafka topics.
*/
public class KafkaBindingSetExporter implements
IncrementalBindingSetExporter {
+
--- End diff --
Okay. Well that's good. Can't tell if you're being sarcastic or
something. No need to leave a comment if nothing needs to be changed.
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127997700
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/export/kafka/KafkaBindingSetExporter.java
---
@@ -36,10 +36,11 @@
* Incrementally exports SPARQL query results to Kafka topics.
*/
public class KafkaBindingSetExporter implements
IncrementalBindingSetExporter {
+
--- End diff --
Okay. Well that's good. Can't tell if you're being sarcastic or
something. No need to leave a comment if nothing needs to be changed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091653#comment-16091653
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127997246
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/SpanBatchDeleteInformation.java
---
@@ -0,0 +1,92 @@
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.Span;
+
+/**
+ * This class represents a batch order to delete all entries in the Fluo
table indicated
+ * by the given Span and Column. These batch orders are processed by the
{@link BatchObserver},
+ * which uses this batch information along with the nodeId passed into the
Observer to perform
+ * batch deletes.
+ *
+ */
+public class SpanBatchDeleteInformation extends
AbstractSpanBatchInformation {
+
+private static final BatchBindingSetUpdater updater = new
SpanBatchBindingSetUpdater();
+
+public SpanBatchDeleteInformation(int batchSize, Column column, Span
span) {
+super(batchSize, Task.Delete, column, span);
+}
+
+/**
+ * @return Updater that applies the {@link Task} to the given {@link
Span} and {@link Column}
+ */
+@Override
+public BatchBindingSetUpdater getBatchUpdater() {
+return updater;
+}
+
+
+public static Builder builder() {
+return new Builder();
+}
+
+public static class Builder {
+
+private int batchSize = DEFAULT_BATCH_SIZE;
+private Column column;
+private Span span;
+
+/**
+ * @param batchSize - {@link Task}s are applied in batches of this
size
+ */
+public Builder setBatchSize(int batchSize) {
+this.batchSize = batchSize;
+return this;
+}
+
+/**
+ * Sets column to apply batch {@link Task} to
+ * @param column - column batch Task will be applied to
+ * @return
+ */
+public Builder setColumn(Column column) {
+this.column = column;
+return this;
+}
+
+/**
+ * @param span - span that batch {@link Task} will be applied to
+ *
+ */
+public Builder setSpan(Span span) {
+this.span = span;
+return this;
+}
+
+
+public SpanBatchDeleteInformation build() {
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091652#comment-16091652
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127997010
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/SpanBatchDeleteInformation.java
---
@@ -0,0 +1,92 @@
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.Span;
+
+/**
+ * This class represents a batch order to delete all entries in the Fluo
table indicated
+ * by the given Span and Column. These batch orders are processed by the
{@link BatchObserver},
+ * which uses this batch information along with the nodeId passed into the
Observer to perform
+ * batch deletes.
+ *
+ */
+public class SpanBatchDeleteInformation extends
AbstractSpanBatchInformation {
+
+private static final BatchBindingSetUpdater updater = new
SpanBatchBindingSetUpdater();
+
+public SpanBatchDeleteInformation(int batchSize, Column column, Span
span) {
+super(batchSize, Task.Delete, column, span);
+}
+
+/**
+ * @return Updater that applies the {@link Task} to the given {@link
Span} and {@link Column}
+ */
+@Override
+public BatchBindingSetUpdater getBatchUpdater() {
+return updater;
+}
+
+
+public static Builder builder() {
--- End diff --
I think this one is pretty clear. Not providing class docs here.
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127997246
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/SpanBatchDeleteInformation.java
---
@@ -0,0 +1,92 @@
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.Span;
+
+/**
+ * This class represents a batch order to delete all entries in the Fluo
table indicated
+ * by the given Span and Column. These batch orders are processed by the
{@link BatchObserver},
+ * which uses this batch information along with the nodeId passed into the
Observer to perform
+ * batch deletes.
+ *
+ */
+public class SpanBatchDeleteInformation extends
AbstractSpanBatchInformation {
+
+private static final BatchBindingSetUpdater updater = new
SpanBatchBindingSetUpdater();
+
+public SpanBatchDeleteInformation(int batchSize, Column column, Span
span) {
+super(batchSize, Task.Delete, column, span);
+}
+
+/**
+ * @return Updater that applies the {@link Task} to the given {@link
Span} and {@link Column}
+ */
+@Override
+public BatchBindingSetUpdater getBatchUpdater() {
+return updater;
+}
+
+
+public static Builder builder() {
+return new Builder();
+}
+
+public static class Builder {
+
+private int batchSize = DEFAULT_BATCH_SIZE;
+private Column column;
+private Span span;
+
+/**
+ * @param batchSize - {@link Task}s are applied in batches of this
size
+ */
+public Builder setBatchSize(int batchSize) {
+this.batchSize = batchSize;
+return this;
+}
+
+/**
+ * Sets column to apply batch {@link Task} to
+ * @param column - column batch Task will be applied to
+ * @return
+ */
+public Builder setColumn(Column column) {
+this.column = column;
+return this;
+}
+
+/**
+ * @param span - span that batch {@link Task} will be applied to
+ *
+ */
+public Builder setSpan(Span span) {
+this.span = span;
+return this;
+}
+
+
+public SpanBatchDeleteInformation build() {
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127997010
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/SpanBatchDeleteInformation.java
---
@@ -0,0 +1,92 @@
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.Span;
+
+/**
+ * This class represents a batch order to delete all entries in the Fluo
table indicated
+ * by the given Span and Column. These batch orders are processed by the
{@link BatchObserver},
+ * which uses this batch information along with the nodeId passed into the
Observer to perform
+ * batch deletes.
+ *
+ */
+public class SpanBatchDeleteInformation extends
AbstractSpanBatchInformation {
+
+private static final BatchBindingSetUpdater updater = new
SpanBatchBindingSetUpdater();
+
+public SpanBatchDeleteInformation(int batchSize, Column column, Span
span) {
+super(batchSize, Task.Delete, column, span);
+}
+
+/**
+ * @return Updater that applies the {@link Task} to the given {@link
Span} and {@link Column}
+ */
+@Override
+public BatchBindingSetUpdater getBatchUpdater() {
+return updater;
+}
+
+
+public static Builder builder() {
--- End diff --
I think this one is pretty clear. Not providing class docs here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091651#comment-16091651
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127996635
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/SpanBatchBindingSetUpdater.java
---
@@ -0,0 +1,128 @@
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import java.util.Iterator;
+import java.util.Optional;
+
+import org.apache.fluo.api.client.TransactionBase;
+import org.apache.fluo.api.client.scanner.ColumnScanner;
+import org.apache.fluo.api.client.scanner.RowScanner;
+import org.apache.fluo.api.data.Bytes;
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.ColumnValue;
+import org.apache.fluo.api.data.RowColumn;
+import org.apache.fluo.api.data.Span;
+import org.apache.log4j.Logger;
+import org.apache.rya.indexing.pcj.fluo.app.batch.BatchInformation.Task;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class processes {@link SpanBatchDeleteInformation} objects by
+ * deleting the entries in the Fluo Column corresponding to the {@link
Span}
+ * of the BatchInformation object. This class will delete entries until
the
+ * batch size is met, and then create a new SpanBatchDeleteInformation
object
+ * with an updated Span whose starting point is the stopping point of this
+ * batch. If the batch limit is not met, then a new batch is not created
and
+ * the task is complete.
+ *
+ */
+public class SpanBatchBindingSetUpdater extends
AbstractBatchBindingSetUpdater {
+
+private static final Logger log =
Logger.getLogger(SpanBatchBindingSetUpdater.class);
+
+/**
+ * Process SpanBatchDeleteInformation objects by deleting all entries
indicated
+ * by Span until batch limit is met.
+ * @param tx - Fluo Transaction
+ * @param row - Byte row identifying BatchInformation
+ * @param batch - SpanBatchDeleteInformation object to be processed
+ */
+@Override
+public void processBatch(TransactionBase tx, Bytes row,
BatchInformation batch) throws Exception {
+super.processBatch(tx, row, batch);
+Preconditions.checkArgument(batch instanceof
SpanBatchDeleteInformation);
+SpanBatchDeleteInformation spanBatch =
(SpanBatchDeleteInformation) batch;
+Task task = spanBatch.getTask();
+int batchSize = spanBatch.getBatchSize();
+Span span = spanBatch.getSpan();
+Column column = batch.getColumn();
+Optional rowCol = Optional.empty();
+
+switch (task) {
+case Add:
+log.trace("The Task Add is not supported for
SpanBatchBindingSetUpdater. Batch " + batch + " will not be processed.");
--- End diff --
I don't think you want to kill the whole application here. If an invalid
batch request is made, the application simply logs that it can't process it,
then moves on.
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service n
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127996635
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/SpanBatchBindingSetUpdater.java
---
@@ -0,0 +1,128 @@
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import java.util.Iterator;
+import java.util.Optional;
+
+import org.apache.fluo.api.client.TransactionBase;
+import org.apache.fluo.api.client.scanner.ColumnScanner;
+import org.apache.fluo.api.client.scanner.RowScanner;
+import org.apache.fluo.api.data.Bytes;
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.ColumnValue;
+import org.apache.fluo.api.data.RowColumn;
+import org.apache.fluo.api.data.Span;
+import org.apache.log4j.Logger;
+import org.apache.rya.indexing.pcj.fluo.app.batch.BatchInformation.Task;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class processes {@link SpanBatchDeleteInformation} objects by
+ * deleting the entries in the Fluo Column corresponding to the {@link
Span}
+ * of the BatchInformation object. This class will delete entries until
the
+ * batch size is met, and then create a new SpanBatchDeleteInformation
object
+ * with an updated Span whose starting point is the stopping point of this
+ * batch. If the batch limit is not met, then a new batch is not created
and
+ * the task is complete.
+ *
+ */
+public class SpanBatchBindingSetUpdater extends
AbstractBatchBindingSetUpdater {
+
+private static final Logger log =
Logger.getLogger(SpanBatchBindingSetUpdater.class);
+
+/**
+ * Process SpanBatchDeleteInformation objects by deleting all entries
indicated
+ * by Span until batch limit is met.
+ * @param tx - Fluo Transaction
+ * @param row - Byte row identifying BatchInformation
+ * @param batch - SpanBatchDeleteInformation object to be processed
+ */
+@Override
+public void processBatch(TransactionBase tx, Bytes row,
BatchInformation batch) throws Exception {
+super.processBatch(tx, row, batch);
+Preconditions.checkArgument(batch instanceof
SpanBatchDeleteInformation);
+SpanBatchDeleteInformation spanBatch =
(SpanBatchDeleteInformation) batch;
+Task task = spanBatch.getTask();
+int batchSize = spanBatch.getBatchSize();
+Span span = spanBatch.getSpan();
+Column column = batch.getColumn();
+Optional rowCol = Optional.empty();
+
+switch (task) {
+case Add:
+log.trace("The Task Add is not supported for
SpanBatchBindingSetUpdater. Batch " + batch + " will not be processed.");
--- End diff --
I don't think you want to kill the whole application here. If an invalid
batch request is made, the application simply logs that it can't process it,
then moves on.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091647#comment-16091647
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127996045
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/JoinBatchInformation.java
---
@@ -0,0 +1,262 @@
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import java.util.Objects;
+
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.Span;
+import org.apache.rya.indexing.pcj.fluo.app.JoinResultUpdater.Side;
+import org.apache.rya.indexing.pcj.fluo.app.query.JoinMetadata.JoinType;
+import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder;
+import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSet;
+import org.openrdf.query.Binding;
+
+import jline.internal.Preconditions;
+
+/**
+ * This class updates join results based on parameters specified for the
join's
+ * children. The join has two children, and for one child a
VisibilityBindingSet
+ * is specified along with the Side of that child. This BindingSet
represents an
+ * update to that join child. For the other child, a Span, Column and
+ * VariableOrder are specified. This is so that the sibling node (the node
that
+ * wasn't updated) can be scanned to obtain results that can be joined
with the
+ * VisibilityBindingSet. The assumption here is that the Span is derived
from
+ * the {@link Binding}s of common variables between the join children, with
+ * Values ordered according to the indicated {@link VariableOrder}. This
class
+ * represents a batch order to perform a given task on join BindingSet
results.
+ * The {@link Task} is to Add, Delete, or Update. This batch order is
processed
+ * by the {@link BatchObserver} and used with the nodeId provided to the
+ * Observer to process the Task specified by the batch order. If the Task
is to
+ * add, the BatchBindingSetUpdater returned by
+ * {@link JoinBatchInformation#getBatchUpdater()} will scan the join's
child for
+ * results using the indicated Span and Column. These results are joined
with
+ * the indicated VisibilityBindingSet, and the results are added to the
parent
+ * join. The other Tasks are performed analogously.
+ *
+ */
+public class JoinBatchInformation extends AbstractSpanBatchInformation {
+
+private static final BatchBindingSetUpdater updater = new
JoinBatchBindingSetUpdater();
+private VisibilityBindingSet bs; //update for join child indicated by
side
+private VariableOrder varOrder; //variable order for child indicated
by Span
+private Side side; //join child that was updated by bs
+private JoinType join;
+/**
+ * @param batchSize - batch size that Tasks are performed in
+ * @param task - Add, Delete, or Update
+ * @param column - Column of join child to be scanned
+ * @param span - span of join child to be scanned (derived from common
variables of left and right join children)
+ * @param bs - BindingSet to be joined with results of child scan
+ * @param varOrder - VariableOrder used to form join (order for join
child corresponding to Span)
+ * @param side - The side of the child that the VisibilityBindingSet
update occurred at
+ * @param join - JoinType (left, right, natural inner)
+ */
+public JoinBatchInformation(int batchSize, Task task, Column column,
Span span, VisibilityBindingSet bs, VariableOrder varOrder, Side side, JoinType
join) {
+super(batchSize, task, column, span);
+Preconditions.checkNotNull(bs);
+Preconditions.checkNotNull(varOrder);
+Preconditions.checkNotNull(side);
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127996045
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/JoinBatchInformation.java
---
@@ -0,0 +1,262 @@
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import java.util.Objects;
+
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.Span;
+import org.apache.rya.indexing.pcj.fluo.app.JoinResultUpdater.Side;
+import org.apache.rya.indexing.pcj.fluo.app.query.JoinMetadata.JoinType;
+import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder;
+import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSet;
+import org.openrdf.query.Binding;
+
+import jline.internal.Preconditions;
+
+/**
+ * This class updates join results based on parameters specified for the
join's
+ * children. The join has two children, and for one child a
VisibilityBindingSet
+ * is specified along with the Side of that child. This BindingSet
represents an
+ * update to that join child. For the other child, a Span, Column and
+ * VariableOrder are specified. This is so that the sibling node (the node
that
+ * wasn't updated) can be scanned to obtain results that can be joined
with the
+ * VisibilityBindingSet. The assumption here is that the Span is derived
from
+ * the {@link Binding}s of common variables between the join children, with
+ * Values ordered according to the indicated {@link VariableOrder}. This
class
+ * represents a batch order to perform a given task on join BindingSet
results.
+ * The {@link Task} is to Add, Delete, or Update. This batch order is
processed
+ * by the {@link BatchObserver} and used with the nodeId provided to the
+ * Observer to process the Task specified by the batch order. If the Task
is to
+ * add, the BatchBindingSetUpdater returned by
+ * {@link JoinBatchInformation#getBatchUpdater()} will scan the join's
child for
+ * results using the indicated Span and Column. These results are joined
with
+ * the indicated VisibilityBindingSet, and the results are added to the
parent
+ * join. The other Tasks are performed analogously.
+ *
+ */
+public class JoinBatchInformation extends AbstractSpanBatchInformation {
+
+private static final BatchBindingSetUpdater updater = new
JoinBatchBindingSetUpdater();
+private VisibilityBindingSet bs; //update for join child indicated by
side
+private VariableOrder varOrder; //variable order for child indicated
by Span
+private Side side; //join child that was updated by bs
+private JoinType join;
+/**
+ * @param batchSize - batch size that Tasks are performed in
+ * @param task - Add, Delete, or Update
+ * @param column - Column of join child to be scanned
+ * @param span - span of join child to be scanned (derived from common
variables of left and right join children)
+ * @param bs - BindingSet to be joined with results of child scan
+ * @param varOrder - VariableOrder used to form join (order for join
child corresponding to Span)
+ * @param side - The side of the child that the VisibilityBindingSet
update occurred at
+ * @param join - JoinType (left, right, natural inner)
+ */
+public JoinBatchInformation(int batchSize, Task task, Column column,
Span span, VisibilityBindingSet bs, VariableOrder varOrder, Side side, JoinType
join) {
+super(batchSize, task, column, span);
+Preconditions.checkNotNull(bs);
+Preconditions.checkNotNull(varOrder);
+Preconditions.checkNotNull(side);
+Preconditions.checkNotNull(join);
+this.bs = bs;
+this.varOrder = varOrder;
+this.join = join;
+this.side = side;
+}
+
+public JoinBatchInformation(Task task, Colum
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091644#comment-16091644
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127995755
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/JoinBatchInformation.java
---
@@ -0,0 +1,262 @@
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import java.util.Objects;
+
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.Span;
+import org.apache.rya.indexing.pcj.fluo.app.JoinResultUpdater.Side;
+import org.apache.rya.indexing.pcj.fluo.app.query.JoinMetadata.JoinType;
+import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder;
+import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSet;
+import org.openrdf.query.Binding;
+
+import jline.internal.Preconditions;
+
+/**
+ * This class updates join results based on parameters specified for the
join's
+ * children. The join has two children, and for one child a
VisibilityBindingSet
+ * is specified along with the Side of that child. This BindingSet
represents an
+ * update to that join child. For the other child, a Span, Column and
+ * VariableOrder are specified. This is so that the sibling node (the node
that
+ * wasn't updated) can be scanned to obtain results that can be joined
with the
+ * VisibilityBindingSet. The assumption here is that the Span is derived
from
+ * the {@link Binding}s of common variables between the join children, with
+ * Values ordered according to the indicated {@link VariableOrder}. This
class
+ * represents a batch order to perform a given task on join BindingSet
results.
+ * The {@link Task} is to Add, Delete, or Update. This batch order is
processed
+ * by the {@link BatchObserver} and used with the nodeId provided to the
+ * Observer to process the Task specified by the batch order. If the Task
is to
+ * add, the BatchBindingSetUpdater returned by
+ * {@link JoinBatchInformation#getBatchUpdater()} will scan the join's
child for
+ * results using the indicated Span and Column. These results are joined
with
+ * the indicated VisibilityBindingSet, and the results are added to the
parent
+ * join. The other Tasks are performed analogously.
+ *
+ */
+public class JoinBatchInformation extends AbstractSpanBatchInformation {
+
+private static final BatchBindingSetUpdater updater = new
JoinBatchBindingSetUpdater();
+private VisibilityBindingSet bs; //update for join child indicated by
side
+private VariableOrder varOrder; //variable order for child indicated
by Span
+private Side side; //join child that was updated by bs
+private JoinType join;
+/**
+ * @param batchSize - batch size that Tasks are performed in
+ * @param task - Add, Delete, or Update
+ * @param column - Column of join child to be scanned
+ * @param span - span of join child to be scanned (derived from common
variables of left and right join children)
+ * @param bs - BindingSet to be joined with results of child scan
+ * @param varOrder - VariableOrder used to form join (order for join
child corresponding to Span)
+ * @param side - The side of the child that the VisibilityBindingSet
update occurred at
+ * @param join - JoinType (left, right, natural inner)
+ */
+public JoinBatchInformation(int batchSize, Task task, Column column,
Span span, VisibilityBindingSet bs, VariableOrder varOrder, Side side, JoinType
join) {
+super(batchSize, task, column, span);
+Preconditions.checkNotNull(bs);
+Preconditions.checkNotNull(varOrder);
+Preconditions.checkNotNull(side);
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127995755
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/JoinBatchInformation.java
---
@@ -0,0 +1,262 @@
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import java.util.Objects;
+
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.Span;
+import org.apache.rya.indexing.pcj.fluo.app.JoinResultUpdater.Side;
+import org.apache.rya.indexing.pcj.fluo.app.query.JoinMetadata.JoinType;
+import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder;
+import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSet;
+import org.openrdf.query.Binding;
+
+import jline.internal.Preconditions;
+
+/**
+ * This class updates join results based on parameters specified for the
join's
+ * children. The join has two children, and for one child a
VisibilityBindingSet
+ * is specified along with the Side of that child. This BindingSet
represents an
+ * update to that join child. For the other child, a Span, Column and
+ * VariableOrder are specified. This is so that the sibling node (the node
that
+ * wasn't updated) can be scanned to obtain results that can be joined
with the
+ * VisibilityBindingSet. The assumption here is that the Span is derived
from
+ * the {@link Binding}s of common variables between the join children, with
+ * Values ordered according to the indicated {@link VariableOrder}. This
class
+ * represents a batch order to perform a given task on join BindingSet
results.
+ * The {@link Task} is to Add, Delete, or Update. This batch order is
processed
+ * by the {@link BatchObserver} and used with the nodeId provided to the
+ * Observer to process the Task specified by the batch order. If the Task
is to
+ * add, the BatchBindingSetUpdater returned by
+ * {@link JoinBatchInformation#getBatchUpdater()} will scan the join's
child for
+ * results using the indicated Span and Column. These results are joined
with
+ * the indicated VisibilityBindingSet, and the results are added to the
parent
+ * join. The other Tasks are performed analogously.
+ *
+ */
+public class JoinBatchInformation extends AbstractSpanBatchInformation {
+
+private static final BatchBindingSetUpdater updater = new
JoinBatchBindingSetUpdater();
+private VisibilityBindingSet bs; //update for join child indicated by
side
+private VariableOrder varOrder; //variable order for child indicated
by Span
+private Side side; //join child that was updated by bs
+private JoinType join;
+/**
+ * @param batchSize - batch size that Tasks are performed in
+ * @param task - Add, Delete, or Update
+ * @param column - Column of join child to be scanned
+ * @param span - span of join child to be scanned (derived from common
variables of left and right join children)
+ * @param bs - BindingSet to be joined with results of child scan
+ * @param varOrder - VariableOrder used to form join (order for join
child corresponding to Span)
+ * @param side - The side of the child that the VisibilityBindingSet
update occurred at
+ * @param join - JoinType (left, right, natural inner)
+ */
+public JoinBatchInformation(int batchSize, Task task, Column column,
Span span, VisibilityBindingSet bs, VariableOrder varOrder, Side side, JoinType
join) {
+super(batchSize, task, column, span);
+Preconditions.checkNotNull(bs);
+Preconditions.checkNotNull(varOrder);
+Preconditions.checkNotNull(side);
+Preconditions.checkNotNull(join);
+this.bs = bs;
+this.varOrder = varOrder;
+this.join = join;
+this.side = side;
+}
+
+public JoinBatchInformation(Task task, Colum
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091641#comment-16091641
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127995235
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/JoinBatchInformation.java
---
@@ -0,0 +1,262 @@
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import java.util.Objects;
+
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.Span;
+import org.apache.rya.indexing.pcj.fluo.app.JoinResultUpdater.Side;
+import org.apache.rya.indexing.pcj.fluo.app.query.JoinMetadata.JoinType;
+import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder;
+import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSet;
+import org.openrdf.query.Binding;
+
+import jline.internal.Preconditions;
+
+/**
+ * This class updates join results based on parameters specified for the
join's
+ * children. The join has two children, and for one child a
VisibilityBindingSet
+ * is specified along with the Side of that child. This BindingSet
represents an
+ * update to that join child. For the other child, a Span, Column and
+ * VariableOrder are specified. This is so that the sibling node (the node
that
+ * wasn't updated) can be scanned to obtain results that can be joined
with the
+ * VisibilityBindingSet. The assumption here is that the Span is derived
from
+ * the {@link Binding}s of common variables between the join children, with
+ * Values ordered according to the indicated {@link VariableOrder}. This
class
+ * represents a batch order to perform a given task on join BindingSet
results.
+ * The {@link Task} is to Add, Delete, or Update. This batch order is
processed
+ * by the {@link BatchObserver} and used with the nodeId provided to the
+ * Observer to process the Task specified by the batch order. If the Task
is to
+ * add, the BatchBindingSetUpdater returned by
+ * {@link JoinBatchInformation#getBatchUpdater()} will scan the join's
child for
+ * results using the indicated Span and Column. These results are joined
with
+ * the indicated VisibilityBindingSet, and the results are added to the
parent
+ * join. The other Tasks are performed analogously.
+ *
+ */
+public class JoinBatchInformation extends AbstractSpanBatchInformation {
+
+private static final BatchBindingSetUpdater updater = new
JoinBatchBindingSetUpdater();
+private VisibilityBindingSet bs; //update for join child indicated by
side
+private VariableOrder varOrder; //variable order for child indicated
by Span
+private Side side; //join child that was updated by bs
+private JoinType join;
+/**
+ * @param batchSize - batch size that Tasks are performed in
+ * @param task - Add, Delete, or Update
+ * @param column - Column of join child to be scanned
+ * @param span - span of join child to be scanned (derived from common
variables of left and right join children)
+ * @param bs - BindingSet to be joined with results of child scan
+ * @param varOrder - VariableOrder used to form join (order for join
child corresponding to Span)
+ * @param side - The side of the child that the VisibilityBindingSet
update occurred at
+ * @param join - JoinType (left, right, natural inner)
+ */
+public JoinBatchInformation(int batchSize, Task task, Column column,
Span span, VisibilityBindingSet bs, VariableOrder varOrder, Side side, JoinType
join) {
+super(batchSize, task, column, span);
+Preconditions.checkNotNull(bs);
+Preconditions.checkNotNull(varOrder);
+Preconditions.checkNotNull(side);
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091638#comment-16091638
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127995043
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/BasicBatchInformation.java
---
@@ -0,0 +1,82 @@
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import org.apache.fluo.api.data.Column;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class contains all of the common info contained in other
implementations
+ * of BatchInformation.
+ *
+ */
+public abstract class BasicBatchInformation implements BatchInformation {
+
+private int batchSize;
+private Task task;
+private Column column;
+
+/**
+ * Create BasicBatchInformation object
+ * @param batchSize - size of batch to be processed
+ * @param task - task to be processed
+ * @param column - Column in which data is proessed
+ */
+public BasicBatchInformation(int batchSize, Task task, Column column )
{
+Preconditions.checkNotNull(task);
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127995235
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/JoinBatchInformation.java
---
@@ -0,0 +1,262 @@
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import java.util.Objects;
+
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.Span;
+import org.apache.rya.indexing.pcj.fluo.app.JoinResultUpdater.Side;
+import org.apache.rya.indexing.pcj.fluo.app.query.JoinMetadata.JoinType;
+import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder;
+import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSet;
+import org.openrdf.query.Binding;
+
+import jline.internal.Preconditions;
+
+/**
+ * This class updates join results based on parameters specified for the
join's
+ * children. The join has two children, and for one child a
VisibilityBindingSet
+ * is specified along with the Side of that child. This BindingSet
represents an
+ * update to that join child. For the other child, a Span, Column and
+ * VariableOrder are specified. This is so that the sibling node (the node
that
+ * wasn't updated) can be scanned to obtain results that can be joined
with the
+ * VisibilityBindingSet. The assumption here is that the Span is derived
from
+ * the {@link Binding}s of common variables between the join children, with
+ * Values ordered according to the indicated {@link VariableOrder}. This
class
+ * represents a batch order to perform a given task on join BindingSet
results.
+ * The {@link Task} is to Add, Delete, or Update. This batch order is
processed
+ * by the {@link BatchObserver} and used with the nodeId provided to the
+ * Observer to process the Task specified by the batch order. If the Task
is to
+ * add, the BatchBindingSetUpdater returned by
+ * {@link JoinBatchInformation#getBatchUpdater()} will scan the join's
child for
+ * results using the indicated Span and Column. These results are joined
with
+ * the indicated VisibilityBindingSet, and the results are added to the
parent
+ * join. The other Tasks are performed analogously.
+ *
+ */
+public class JoinBatchInformation extends AbstractSpanBatchInformation {
+
+private static final BatchBindingSetUpdater updater = new
JoinBatchBindingSetUpdater();
+private VisibilityBindingSet bs; //update for join child indicated by
side
+private VariableOrder varOrder; //variable order for child indicated
by Span
+private Side side; //join child that was updated by bs
+private JoinType join;
+/**
+ * @param batchSize - batch size that Tasks are performed in
+ * @param task - Add, Delete, or Update
+ * @param column - Column of join child to be scanned
+ * @param span - span of join child to be scanned (derived from common
variables of left and right join children)
+ * @param bs - BindingSet to be joined with results of child scan
+ * @param varOrder - VariableOrder used to form join (order for join
child corresponding to Span)
+ * @param side - The side of the child that the VisibilityBindingSet
update occurred at
+ * @param join - JoinType (left, right, natural inner)
+ */
+public JoinBatchInformation(int batchSize, Task task, Column column,
Span span, VisibilityBindingSet bs, VariableOrder varOrder, Side side, JoinType
join) {
+super(batchSize, task, column, span);
+Preconditions.checkNotNull(bs);
+Preconditions.checkNotNull(varOrder);
+Preconditions.checkNotNull(side);
+Preconditions.checkNotNull(join);
+this.bs = bs;
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127995043
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/BasicBatchInformation.java
---
@@ -0,0 +1,82 @@
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+import org.apache.fluo.api.data.Column;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class contains all of the common info contained in other
implementations
+ * of BatchInformation.
+ *
+ */
+public abstract class BasicBatchInformation implements BatchInformation {
+
+private int batchSize;
+private Task task;
+private Column column;
+
+/**
+ * Create BasicBatchInformation object
+ * @param batchSize - size of batch to be processed
+ * @param task - task to be processed
+ * @param column - Column in which data is proessed
+ */
+public BasicBatchInformation(int batchSize, Task task, Column column )
{
+Preconditions.checkNotNull(task);
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091637#comment-16091637
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127994577
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/AbstractSpanBatchInformation.java
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+
+import java.util.Objects;
+
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.Span;
+
+import jline.internal.Preconditions;
+
+/**
+ * Abstract class for generating span based notifications. A spanned
notification
+ * uses a {@link Span} to begin processing a Fluo Column at the position
designated by the Span.
+ *
+ */
+public abstract class AbstractSpanBatchInformation extends
BasicBatchInformation {
+
+private Span span;
+
+/**
+ * Create AbstractBatchInformation
+ * @param batchSize - size of batch to be processed
+ * @param task - type of task processed (Add, Delete, Udpate)
+ * @param column - Cpolumn that Span notification is applied
+ * @param span - span used to indicate where processing should begin
+ */
+public AbstractSpanBatchInformation(int batchSize, Task task, Column
column, Span span) {
+super(batchSize, task, column);
+Preconditions.checkNotNull(span);
+this.span = span;
+}
+
+public AbstractSpanBatchInformation(Task task, Column column, Span
span) {
+this(DEFAULT_BATCH_SIZE, task, column, span);
+}
+
+/**
+ * @return Span that batch Task will be applied to
+ */
+public Span getSpan() {
+return span;
+}
+
+/**
+ * Sets span to which batch Task will be applied
+ * @param span
+ */
+public void setSpan(Span span) {
+this.span = span;
+}
+
+@Override
+public String toString() {
+return new StringBuilder()
+.append("Span Batch Information {\n")
+.append("Span: " + span + "\n")
+.append("Batch Size: " + super.getBatchSize() + "\n")
+.append("Task: " + super.getTask() + "\n")
+.append("Column: " + super.getColumn() + "\n")
+.append("}")
+.toString();
+}
+
+@Override
+public boolean equals(Object other) {
+if (this == other) {
+return true;
+}
+
+if (!(other instanceof AbstractSpanBatchInformation)) {
+return false;
+}
+
+AbstractSpanBatchInformation batch =
(AbstractSpanBatchInformation) other;
+return (super.getBatchSize() == batch.getBatchSize()) &&
Objects.equals(super.getColumn(), batch.getColumn()) &&
Objects.equals(this.span, batch.span)
+&& Objects.equals(super.getTask(), batch.getTask());
+}
+
+@Override
+public int hashCode() {
+int result = 17;
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127994577
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/AbstractSpanBatchInformation.java
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+
+import java.util.Objects;
+
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.Span;
+
+import jline.internal.Preconditions;
+
+/**
+ * Abstract class for generating span based notifications. A spanned
notification
+ * uses a {@link Span} to begin processing a Fluo Column at the position
designated by the Span.
+ *
+ */
+public abstract class AbstractSpanBatchInformation extends
BasicBatchInformation {
+
+private Span span;
+
+/**
+ * Create AbstractBatchInformation
+ * @param batchSize - size of batch to be processed
+ * @param task - type of task processed (Add, Delete, Udpate)
+ * @param column - Cpolumn that Span notification is applied
+ * @param span - span used to indicate where processing should begin
+ */
+public AbstractSpanBatchInformation(int batchSize, Task task, Column
column, Span span) {
+super(batchSize, task, column);
+Preconditions.checkNotNull(span);
+this.span = span;
+}
+
+public AbstractSpanBatchInformation(Task task, Column column, Span
span) {
+this(DEFAULT_BATCH_SIZE, task, column, span);
+}
+
+/**
+ * @return Span that batch Task will be applied to
+ */
+public Span getSpan() {
+return span;
+}
+
+/**
+ * Sets span to which batch Task will be applied
+ * @param span
+ */
+public void setSpan(Span span) {
+this.span = span;
+}
+
+@Override
+public String toString() {
+return new StringBuilder()
+.append("Span Batch Information {\n")
+.append("Span: " + span + "\n")
+.append("Batch Size: " + super.getBatchSize() + "\n")
+.append("Task: " + super.getTask() + "\n")
+.append("Column: " + super.getColumn() + "\n")
+.append("}")
+.toString();
+}
+
+@Override
+public boolean equals(Object other) {
+if (this == other) {
+return true;
+}
+
+if (!(other instanceof AbstractSpanBatchInformation)) {
+return false;
+}
+
+AbstractSpanBatchInformation batch =
(AbstractSpanBatchInformation) other;
+return (super.getBatchSize() == batch.getBatchSize()) &&
Objects.equals(super.getColumn(), batch.getColumn()) &&
Objects.equals(this.span, batch.span)
+&& Objects.equals(super.getTask(), batch.getTask());
+}
+
+@Override
+public int hashCode() {
+int result = 17;
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091633#comment-16091633
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127993938
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/AbstractSpanBatchInformation.java
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+
+import java.util.Objects;
+
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.Span;
+
+import jline.internal.Preconditions;
+
+/**
+ * Abstract class for generating span based notifications. A spanned
notification
+ * uses a {@link Span} to begin processing a Fluo Column at the position
designated by the Span.
+ *
+ */
+public abstract class AbstractSpanBatchInformation extends
BasicBatchInformation {
+
+private Span span;
+
+/**
+ * Create AbstractBatchInformation
+ * @param batchSize - size of batch to be processed
+ * @param task - type of task processed (Add, Delete, Udpate)
+ * @param column - Cpolumn that Span notification is applied
+ * @param span - span used to indicate where processing should begin
+ */
+public AbstractSpanBatchInformation(int batchSize, Task task, Column
column, Span span) {
+super(batchSize, task, column);
+Preconditions.checkNotNull(span);
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091632#comment-16091632
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127993723
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/AbstractBatchBindingSetUpdater.java
---
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+
+import org.apache.fluo.api.client.TransactionBase;
+import org.apache.fluo.api.data.Bytes;
+import org.apache.fluo.api.data.RowColumn;
+import org.apache.fluo.api.data.Span;
+import org.apache.rya.indexing.pcj.fluo.app.query.FluoQueryColumns;
+
+public abstract class AbstractBatchBindingSetUpdater implements
BatchBindingSetUpdater {
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127993938
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/AbstractSpanBatchInformation.java
---
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+
+import java.util.Objects;
+
+import org.apache.fluo.api.data.Column;
+import org.apache.fluo.api.data.Span;
+
+import jline.internal.Preconditions;
+
+/**
+ * Abstract class for generating span based notifications. A spanned
notification
+ * uses a {@link Span} to begin processing a Fluo Column at the position
designated by the Span.
+ *
+ */
+public abstract class AbstractSpanBatchInformation extends
BasicBatchInformation {
+
+private Span span;
+
+/**
+ * Create AbstractBatchInformation
+ * @param batchSize - size of batch to be processed
+ * @param task - type of task processed (Add, Delete, Udpate)
+ * @param column - Cpolumn that Span notification is applied
+ * @param span - span used to indicate where processing should begin
+ */
+public AbstractSpanBatchInformation(int batchSize, Task task, Column
column, Span span) {
+super(batchSize, task, column);
+Preconditions.checkNotNull(span);
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127993723
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/batch/AbstractBatchBindingSetUpdater.java
---
@@ -0,0 +1,51 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.fluo.app.batch;
+
+import org.apache.fluo.api.client.TransactionBase;
+import org.apache.fluo.api.data.Bytes;
+import org.apache.fluo.api.data.RowColumn;
+import org.apache.fluo.api.data.Span;
+import org.apache.rya.indexing.pcj.fluo.app.query.FluoQueryColumns;
+
+public abstract class AbstractBatchBindingSetUpdater implements
BatchBindingSetUpdater {
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[ https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091631#comment-16091631 ] ASF GitHub Bot commented on RYA-280: Github user meiercaleb commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/177#discussion_r127993037 --- Diff: extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/JoinResultUpdater.java --- @@ -43,6 +43,7 @@ import org.apache.rya.indexing.pcj.storage.accumulo.BindingSetConverter.BindingSetConversionException; import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder; import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSet; +import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSetSerDe; --- End diff -- yep > PeriodicQuery Support for Fluo > -- > > Key: RYA-280 > URL: https://issues.apache.org/jira/browse/RYA-280 > Project: Rya > Issue Type: New Feature > Components: clients >Affects Versions: 3.2.10 >Reporter: Caleb Meier >Assignee: Caleb Meier > > Add the capability to Rya-Fluo App to provide periodic updates for queries > registered with Fluo. That is, provide the application with the ability to > satisfy the standing query "tell me every 12 hours about all of the events of > a particular type that occurred within the last 24 hours". Given that Fluo > operates using a push based notification system, some external service needs > to be implemented to periodically notify Fluo to generate a result > notification (this helps handle the non-event event that occurs when nothing > happens in a given period of time). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request: https://github.com/apache/incubator-rya/pull/177#discussion_r127993037 --- Diff: extras/rya.pcj.fluo/pcj.fluo.app/src/main/java/org/apache/rya/indexing/pcj/fluo/app/JoinResultUpdater.java --- @@ -43,6 +43,7 @@ import org.apache.rya.indexing.pcj.storage.accumulo.BindingSetConverter.BindingSetConversionException; import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder; import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSet; +import org.apache.rya.indexing.pcj.storage.accumulo.VisibilityBindingSetSerDe; --- End diff -- yep --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091627#comment-16091627
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127992511
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.api/src/main/java/org/apache/rya/indexing/pcj/fluo/api/DeletePcj.java
---
@@ -67,7 +68,8 @@
/**
* Constructs an instance of {@link DeletePcj}.
*
- * @param batchSize - The number of entries that will be deleted at a
time. (> 0)
+ * @param batchSize
+ *- The number of entries that will be deleted at a time.
(> 0)
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127992511
--- Diff:
extras/rya.pcj.fluo/pcj.fluo.api/src/main/java/org/apache/rya/indexing/pcj/fluo/api/DeletePcj.java
---
@@ -67,7 +68,8 @@
/**
* Constructs an instance of {@link DeletePcj}.
*
- * @param batchSize - The number of entries that will be deleted at a
time. (> 0)
+ * @param batchSize
+ *- The number of entries that will be deleted at a time.
(> 0)
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091620#comment-16091620
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127990899
--- Diff:
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/accumulo/VisibilityBindingSetSerDe.java
---
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage.accumulo;
+
+import static java.util.Objects.requireNonNull;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.ObjectInputStream;
+import java.io.ObjectOutputStream;
+
+import org.apache.fluo.api.data.Bytes;
+
+import edu.umd.cs.findbugs.annotations.DefaultAnnotation;
+import edu.umd.cs.findbugs.annotations.NonNull;
+
+/**
+ * Serializes and deserializes a {@link VisibilityBindingSet} to and from
{@link Bytes} objects.
+ */
+@DefaultAnnotation(NonNull.class)
+public class VisibilityBindingSetSerDe {
--- End diff --
This is actually a duplicate class. I think I needed to migrate the
original to this project, but forgot to delete the original. Deleted the
original. Not my class, so I didn't decide on the naming conventions here.
Leaving as is.
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127990899
--- Diff:
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/accumulo/VisibilityBindingSetSerDe.java
---
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage.accumulo;
+
+import static java.util.Objects.requireNonNull;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.ObjectInputStream;
+import java.io.ObjectOutputStream;
+
+import org.apache.fluo.api.data.Bytes;
+
+import edu.umd.cs.findbugs.annotations.DefaultAnnotation;
+import edu.umd.cs.findbugs.annotations.NonNull;
+
+/**
+ * Serializes and deserializes a {@link VisibilityBindingSet} to and from
{@link Bytes} objects.
+ */
+@DefaultAnnotation(NonNull.class)
+public class VisibilityBindingSetSerDe {
--- End diff --
This is actually a duplicate class. I think I needed to migrate the
original to this project, but forgot to delete the original. Deleted the
original. Not my class, so I didn't decide on the naming conventions here.
Leaving as is.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091610#comment-16091610
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127988392
--- Diff:
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/accumulo/PeriodicQueryTableNameFactory.java
---
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage.accumulo;
+
+import static java.util.Objects.requireNonNull;
+
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage;
+
+/**
+ * Class for creating the names of {@link PeriodicQueryResultStorage}
tables.
+ *
+ */
+public class PeriodicQueryTableNameFactory {
+
+public static final String PeriodicTableSuffix = "PERIODIC_QUERY_";
--- End diff --
It actually is a suffix. This follows the rya prefix in the table name. A
UUID is appended after this in the table name, which is why the underscore is
added.
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127988392
--- Diff:
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/accumulo/PeriodicQueryTableNameFactory.java
---
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage.accumulo;
+
+import static java.util.Objects.requireNonNull;
+
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage;
+
+/**
+ * Class for creating the names of {@link PeriodicQueryResultStorage}
tables.
+ *
+ */
+public class PeriodicQueryTableNameFactory {
+
+public static final String PeriodicTableSuffix = "PERIODIC_QUERY_";
--- End diff --
It actually is a suffix. This follows the rya prefix in the table name. A
UUID is appended after this in the table name, which is why the underscore is
added.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091604#comment-16091604
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127987608
--- Diff:
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/accumulo/PcjTables.java
---
@@ -412,9 +413,9 @@ private void writeResults(
// Row ID = binding set values, Column Family = variable
order of the binding set.
final Mutation addResult = new Mutation(rowKey);
final String visibility = result.getVisibility();
-addResult.put(varOrder.toString(), "", new
ColumnVisibility(visibility), "");
+addResult.put(varOrder.toString(), "", new
ColumnVisibility(visibility), new Value(bsSerDe.serialize(result).toArray()));
mutations.add(addResult);
-} catch(final BindingSetConversionException e) {
+} catch(Exception e) {
--- End diff --
Just converted back to the original Exception.
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127987608
--- Diff:
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/accumulo/PcjTables.java
---
@@ -412,9 +413,9 @@ private void writeResults(
// Row ID = binding set values, Column Family = variable
order of the binding set.
final Mutation addResult = new Mutation(rowKey);
final String visibility = result.getVisibility();
-addResult.put(varOrder.toString(), "", new
ColumnVisibility(visibility), "");
+addResult.put(varOrder.toString(), "", new
ColumnVisibility(visibility), new Value(bsSerDe.serialize(result).toArray()));
mutations.add(addResult);
-} catch(final BindingSetConversionException e) {
+} catch(Exception e) {
--- End diff --
Just converted back to the original Exception.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127986867
--- Diff:
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/accumulo/AccumuloPeriodicQueryResultStorage.java
---
@@ -0,0 +1,271 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage.accumulo;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.BatchDeleter;
+import org.apache.accumulo.core.client.BatchWriterConfig;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.hadoop.io.Text;
+import org.apache.rya.indexing.pcj.storage.PCJIdFactory;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryStorageException;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryStorageMetadata;
+import
org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage.CloseableIterator;
+import
org.apache.rya.indexing.pcj.storage.accumulo.BindingSetConverter.BindingSetConversionException;
+import org.openrdf.model.impl.LiteralImpl;
+import org.openrdf.model.vocabulary.XMLSchema;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.algebra.AggregateOperatorBase;
+import org.openrdf.query.algebra.ExtensionElem;
+import org.openrdf.query.algebra.TupleExpr;
+import org.openrdf.query.algebra.evaluation.QueryBindingSet;
+import org.openrdf.query.algebra.helpers.QueryModelVisitorBase;
+import org.openrdf.query.parser.sparql.SPARQLParser;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is the Accumulo implementation of {@link
PeriodicQueryResultStorage} for
+ * creating, deleting, and interacting with tables where PeriodicQuery
results are stored.
+ */
+public class AccumuloPeriodicQueryResultStorage implements
PeriodicQueryResultStorage {
+
+private String ryaInstance;
+private Connector accumuloConn;
+private Authorizations auths;
+private final PCJIdFactory pcjIdFactory = new PCJIdFactory();
+private final AccumuloPcjSerializer converter = new
AccumuloPcjSerializer();
+private static final PcjTables pcjTables = new PcjTables();
+private static final PeriodicQueryTableNameFactory tableNameFactory =
new PeriodicQueryTableNameFactory();
+
+/**
+ * Creates a AccumuloPeriodicQueryResultStorage Object.
+ * @param accumuloConn - Accumulo Connector for connecting to an
Accumulo instance
+ * @param ryaInstance - Rya Instance name for connecting to Rya
+ */
+public AccumuloPeriodicQueryResultStorage(Connector accumuloConn,
String ryaInstance) {
+this.accumuloConn = accumuloConn;
+this.ryaInstance = ryaInstance;
+String user = accumuloConn.whoami();
+try {
+this.auths =
accumuloConn.securityOperations().getUserAuthorizations(user);
+} catch (AccumuloException | AccumuloSecurityException e) {
+throw new RuntimeException("Unable access user: " + user +
"authorizations.");
+}
+}
+
+@Override
+public String createPeriodicQuery(String sparql) throws
PeriodicQueryStorageException {
+Preconditions.checkNotNull(sparql);
+String queryId = pcjIdFactory.nextId();
+r
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091592#comment-16091592
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127986867
--- Diff:
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/accumulo/AccumuloPeriodicQueryResultStorage.java
---
@@ -0,0 +1,271 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage.accumulo;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.BatchDeleter;
+import org.apache.accumulo.core.client.BatchWriterConfig;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.hadoop.io.Text;
+import org.apache.rya.indexing.pcj.storage.PCJIdFactory;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryStorageException;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryStorageMetadata;
+import
org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage.CloseableIterator;
+import
org.apache.rya.indexing.pcj.storage.accumulo.BindingSetConverter.BindingSetConversionException;
+import org.openrdf.model.impl.LiteralImpl;
+import org.openrdf.model.vocabulary.XMLSchema;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.algebra.AggregateOperatorBase;
+import org.openrdf.query.algebra.ExtensionElem;
+import org.openrdf.query.algebra.TupleExpr;
+import org.openrdf.query.algebra.evaluation.QueryBindingSet;
+import org.openrdf.query.algebra.helpers.QueryModelVisitorBase;
+import org.openrdf.query.parser.sparql.SPARQLParser;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is the Accumulo implementation of {@link
PeriodicQueryResultStorage} for
+ * creating, deleting, and interacting with tables where PeriodicQuery
results are stored.
+ */
+public class AccumuloPeriodicQueryResultStorage implements
PeriodicQueryResultStorage {
+
+private String ryaInstance;
+private Connector accumuloConn;
+private Authorizations auths;
+private final PCJIdFactory pcjIdFactory = new PCJIdFactory();
+private final AccumuloPcjSerializer converter = new
AccumuloPcjSerializer();
+private static final PcjTables pcjTables = new PcjTables();
+private static final PeriodicQueryTableNameFactory tableNameFactory =
new PeriodicQueryTableNameFactory();
+
+/**
+ * Creates a AccumuloPeriodicQueryResultStorage Object.
+ * @param accumuloConn - Accumulo Connector for connecting to an
Accumulo instance
+ * @param ryaInstance - Rya Instance name for connecting to Rya
+ */
+public AccumuloPeriodicQueryResultStorage(Connector accumuloConn,
String ryaInstance) {
+this.accumuloConn = accumuloConn;
+this.ryaInstance = ryaInstance;
+String user = accumuloConn.whoami();
+try {
+this.auths =
accumuloConn.securityOperations().getUserAuthorizations(user);
+} catch (AccumuloException | AccumuloSecurityException e) {
+throw new RuntimeException("Unable access user: " + user +
"authorizations.");
+}
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091590#comment-16091590
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127986607
--- Diff:
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/accumulo/AccumuloPeriodicQueryResultStorage.java
---
@@ -0,0 +1,271 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage.accumulo;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.BatchDeleter;
+import org.apache.accumulo.core.client.BatchWriterConfig;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.hadoop.io.Text;
+import org.apache.rya.indexing.pcj.storage.PCJIdFactory;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryStorageException;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryStorageMetadata;
+import
org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage.CloseableIterator;
+import
org.apache.rya.indexing.pcj.storage.accumulo.BindingSetConverter.BindingSetConversionException;
+import org.openrdf.model.impl.LiteralImpl;
+import org.openrdf.model.vocabulary.XMLSchema;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.algebra.AggregateOperatorBase;
+import org.openrdf.query.algebra.ExtensionElem;
+import org.openrdf.query.algebra.TupleExpr;
+import org.openrdf.query.algebra.evaluation.QueryBindingSet;
+import org.openrdf.query.algebra.helpers.QueryModelVisitorBase;
+import org.openrdf.query.parser.sparql.SPARQLParser;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is the Accumulo implementation of {@link
PeriodicQueryResultStorage} for
+ * creating, deleting, and interacting with tables where PeriodicQuery
results are stored.
+ */
+public class AccumuloPeriodicQueryResultStorage implements
PeriodicQueryResultStorage {
+
+private String ryaInstance;
+private Connector accumuloConn;
+private Authorizations auths;
+private final PCJIdFactory pcjIdFactory = new PCJIdFactory();
+private final AccumuloPcjSerializer converter = new
AccumuloPcjSerializer();
+private static final PcjTables pcjTables = new PcjTables();
+private static final PeriodicQueryTableNameFactory tableNameFactory =
new PeriodicQueryTableNameFactory();
+
+/**
+ * Creates a AccumuloPeriodicQueryResultStorage Object.
+ * @param accumuloConn - Accumulo Connector for connecting to an
Accumulo instance
+ * @param ryaInstance - Rya Instance name for connecting to Rya
+ */
+public AccumuloPeriodicQueryResultStorage(Connector accumuloConn,
String ryaInstance) {
+this.accumuloConn = accumuloConn;
+this.ryaInstance = ryaInstance;
+String user = accumuloConn.whoami();
+try {
+this.auths =
accumuloConn.securityOperations().getUserAuthorizations(user);
+} catch (AccumuloException | AccumuloSecurityException e) {
+throw new RuntimeException("Unable access user: " + user +
"authorizations.");
+}
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127986607
--- Diff:
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/accumulo/AccumuloPeriodicQueryResultStorage.java
---
@@ -0,0 +1,271 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage.accumulo;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.BatchDeleter;
+import org.apache.accumulo.core.client.BatchWriterConfig;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.hadoop.io.Text;
+import org.apache.rya.indexing.pcj.storage.PCJIdFactory;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryStorageException;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryStorageMetadata;
+import
org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage.CloseableIterator;
+import
org.apache.rya.indexing.pcj.storage.accumulo.BindingSetConverter.BindingSetConversionException;
+import org.openrdf.model.impl.LiteralImpl;
+import org.openrdf.model.vocabulary.XMLSchema;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.algebra.AggregateOperatorBase;
+import org.openrdf.query.algebra.ExtensionElem;
+import org.openrdf.query.algebra.TupleExpr;
+import org.openrdf.query.algebra.evaluation.QueryBindingSet;
+import org.openrdf.query.algebra.helpers.QueryModelVisitorBase;
+import org.openrdf.query.parser.sparql.SPARQLParser;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is the Accumulo implementation of {@link
PeriodicQueryResultStorage} for
+ * creating, deleting, and interacting with tables where PeriodicQuery
results are stored.
+ */
+public class AccumuloPeriodicQueryResultStorage implements
PeriodicQueryResultStorage {
+
+private String ryaInstance;
+private Connector accumuloConn;
+private Authorizations auths;
+private final PCJIdFactory pcjIdFactory = new PCJIdFactory();
+private final AccumuloPcjSerializer converter = new
AccumuloPcjSerializer();
+private static final PcjTables pcjTables = new PcjTables();
+private static final PeriodicQueryTableNameFactory tableNameFactory =
new PeriodicQueryTableNameFactory();
+
+/**
+ * Creates a AccumuloPeriodicQueryResultStorage Object.
+ * @param accumuloConn - Accumulo Connector for connecting to an
Accumulo instance
+ * @param ryaInstance - Rya Instance name for connecting to Rya
+ */
+public AccumuloPeriodicQueryResultStorage(Connector accumuloConn,
String ryaInstance) {
+this.accumuloConn = accumuloConn;
+this.ryaInstance = ryaInstance;
+String user = accumuloConn.whoami();
+try {
+this.auths =
accumuloConn.securityOperations().getUserAuthorizations(user);
+} catch (AccumuloException | AccumuloSecurityException e) {
+throw new RuntimeException("Unable access user: " + user +
"authorizations.");
+}
+}
+
+@Override
+public String createPeriodicQuery(String sparql) throws
PeriodicQueryStorageException {
+Preconditions.checkNotNull(sparql);
+String queryId = pcjIdFactory.nextId();
+r
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091583#comment-16091583
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127984352
--- Diff:
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/accumulo/AccumuloPeriodicQueryResultStorage.java
---
@@ -0,0 +1,271 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage.accumulo;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.BatchDeleter;
+import org.apache.accumulo.core.client.BatchWriterConfig;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.hadoop.io.Text;
+import org.apache.rya.indexing.pcj.storage.PCJIdFactory;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryStorageException;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryStorageMetadata;
+import
org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage.CloseableIterator;
+import
org.apache.rya.indexing.pcj.storage.accumulo.BindingSetConverter.BindingSetConversionException;
+import org.openrdf.model.impl.LiteralImpl;
+import org.openrdf.model.vocabulary.XMLSchema;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.algebra.AggregateOperatorBase;
+import org.openrdf.query.algebra.ExtensionElem;
+import org.openrdf.query.algebra.TupleExpr;
+import org.openrdf.query.algebra.evaluation.QueryBindingSet;
+import org.openrdf.query.algebra.helpers.QueryModelVisitorBase;
+import org.openrdf.query.parser.sparql.SPARQLParser;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is the Accumulo implementation of {@link
PeriodicQueryResultStorage} for
+ * creating, deleting, and interacting with tables where PeriodicQuery
results are stored.
+ */
+public class AccumuloPeriodicQueryResultStorage implements
PeriodicQueryResultStorage {
+
+private String ryaInstance;
+private Connector accumuloConn;
+private Authorizations auths;
+private final PCJIdFactory pcjIdFactory = new PCJIdFactory();
+private final AccumuloPcjSerializer converter = new
AccumuloPcjSerializer();
+private static final PcjTables pcjTables = new PcjTables();
+private static final PeriodicQueryTableNameFactory tableNameFactory =
new PeriodicQueryTableNameFactory();
+
+/**
+ * Creates a AccumuloPeriodicQueryResultStorage Object.
+ * @param accumuloConn - Accumulo Connector for connecting to an
Accumulo instance
+ * @param ryaInstance - Rya Instance name for connecting to Rya
+ */
+public AccumuloPeriodicQueryResultStorage(Connector accumuloConn,
String ryaInstance) {
+this.accumuloConn = accumuloConn;
+this.ryaInstance = ryaInstance;
+String user = accumuloConn.whoami();
+try {
+this.auths =
accumuloConn.securityOperations().getUserAuthorizations(user);
+} catch (AccumuloException | AccumuloSecurityException e) {
+throw new RuntimeException("Unable access user: " + user +
"authorizations.");
+}
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127984352
--- Diff:
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/accumulo/AccumuloPeriodicQueryResultStorage.java
---
@@ -0,0 +1,271 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage.accumulo;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.BatchDeleter;
+import org.apache.accumulo.core.client.BatchWriterConfig;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.hadoop.io.Text;
+import org.apache.rya.indexing.pcj.storage.PCJIdFactory;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryStorageException;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryStorageMetadata;
+import
org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage.CloseableIterator;
+import
org.apache.rya.indexing.pcj.storage.accumulo.BindingSetConverter.BindingSetConversionException;
+import org.openrdf.model.impl.LiteralImpl;
+import org.openrdf.model.vocabulary.XMLSchema;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.algebra.AggregateOperatorBase;
+import org.openrdf.query.algebra.ExtensionElem;
+import org.openrdf.query.algebra.TupleExpr;
+import org.openrdf.query.algebra.evaluation.QueryBindingSet;
+import org.openrdf.query.algebra.helpers.QueryModelVisitorBase;
+import org.openrdf.query.parser.sparql.SPARQLParser;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is the Accumulo implementation of {@link
PeriodicQueryResultStorage} for
+ * creating, deleting, and interacting with tables where PeriodicQuery
results are stored.
+ */
+public class AccumuloPeriodicQueryResultStorage implements
PeriodicQueryResultStorage {
+
+private String ryaInstance;
+private Connector accumuloConn;
+private Authorizations auths;
+private final PCJIdFactory pcjIdFactory = new PCJIdFactory();
+private final AccumuloPcjSerializer converter = new
AccumuloPcjSerializer();
+private static final PcjTables pcjTables = new PcjTables();
+private static final PeriodicQueryTableNameFactory tableNameFactory =
new PeriodicQueryTableNameFactory();
+
+/**
+ * Creates a AccumuloPeriodicQueryResultStorage Object.
+ * @param accumuloConn - Accumulo Connector for connecting to an
Accumulo instance
+ * @param ryaInstance - Rya Instance name for connecting to Rya
+ */
+public AccumuloPeriodicQueryResultStorage(Connector accumuloConn,
String ryaInstance) {
+this.accumuloConn = accumuloConn;
+this.ryaInstance = ryaInstance;
+String user = accumuloConn.whoami();
+try {
+this.auths =
accumuloConn.securityOperations().getUserAuthorizations(user);
+} catch (AccumuloException | AccumuloSecurityException e) {
+throw new RuntimeException("Unable access user: " + user +
"authorizations.");
+}
+}
+
+@Override
+public String createPeriodicQuery(String sparql) throws
PeriodicQueryStorageException {
+Preconditions.checkNotNull(sparql);
+String queryId = pcjIdFactory.nextId();
+r
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091572#comment-16091572
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127981764
--- Diff:
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/accumulo/AccumuloPeriodicQueryResultStorage.java
---
@@ -0,0 +1,271 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage.accumulo;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.BatchDeleter;
+import org.apache.accumulo.core.client.BatchWriterConfig;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.hadoop.io.Text;
+import org.apache.rya.indexing.pcj.storage.PCJIdFactory;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryStorageException;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryStorageMetadata;
+import
org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage.CloseableIterator;
+import
org.apache.rya.indexing.pcj.storage.accumulo.BindingSetConverter.BindingSetConversionException;
+import org.openrdf.model.impl.LiteralImpl;
+import org.openrdf.model.vocabulary.XMLSchema;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.algebra.AggregateOperatorBase;
+import org.openrdf.query.algebra.ExtensionElem;
+import org.openrdf.query.algebra.TupleExpr;
+import org.openrdf.query.algebra.evaluation.QueryBindingSet;
+import org.openrdf.query.algebra.helpers.QueryModelVisitorBase;
+import org.openrdf.query.parser.sparql.SPARQLParser;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is the Accumulo implementation of {@link
PeriodicQueryResultStorage} for
+ * creating, deleting, and interacting with tables where PeriodicQuery
results are stored.
+ */
+public class AccumuloPeriodicQueryResultStorage implements
PeriodicQueryResultStorage {
+
+private String ryaInstance;
+private Connector accumuloConn;
+private Authorizations auths;
+private final PCJIdFactory pcjIdFactory = new PCJIdFactory();
+private final AccumuloPcjSerializer converter = new
AccumuloPcjSerializer();
+private static final PcjTables pcjTables = new PcjTables();
+private static final PeriodicQueryTableNameFactory tableNameFactory =
new PeriodicQueryTableNameFactory();
+
+/**
+ * Creates a AccumuloPeriodicQueryResultStorage Object.
+ * @param accumuloConn - Accumulo Connector for connecting to an
Accumulo instance
+ * @param ryaInstance - Rya Instance name for connecting to Rya
+ */
+public AccumuloPeriodicQueryResultStorage(Connector accumuloConn,
String ryaInstance) {
+this.accumuloConn = accumuloConn;
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
[GitHub] incubator-rya pull request #177: RYA-280-Periodic Query Service
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127981764
--- Diff:
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/accumulo/AccumuloPeriodicQueryResultStorage.java
---
@@ -0,0 +1,271 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage.accumulo;
+
+import static java.util.Objects.requireNonNull;
+
+import java.util.ArrayList;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.BatchDeleter;
+import org.apache.accumulo.core.client.BatchWriterConfig;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.hadoop.io.Text;
+import org.apache.rya.indexing.pcj.storage.PCJIdFactory;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryResultStorage;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryStorageException;
+import org.apache.rya.indexing.pcj.storage.PeriodicQueryStorageMetadata;
+import
org.apache.rya.indexing.pcj.storage.PrecomputedJoinStorage.CloseableIterator;
+import
org.apache.rya.indexing.pcj.storage.accumulo.BindingSetConverter.BindingSetConversionException;
+import org.openrdf.model.impl.LiteralImpl;
+import org.openrdf.model.vocabulary.XMLSchema;
+import org.openrdf.query.BindingSet;
+import org.openrdf.query.MalformedQueryException;
+import org.openrdf.query.algebra.AggregateOperatorBase;
+import org.openrdf.query.algebra.ExtensionElem;
+import org.openrdf.query.algebra.TupleExpr;
+import org.openrdf.query.algebra.evaluation.QueryBindingSet;
+import org.openrdf.query.algebra.helpers.QueryModelVisitorBase;
+import org.openrdf.query.parser.sparql.SPARQLParser;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * This class is the Accumulo implementation of {@link
PeriodicQueryResultStorage} for
+ * creating, deleting, and interacting with tables where PeriodicQuery
results are stored.
+ */
+public class AccumuloPeriodicQueryResultStorage implements
PeriodicQueryResultStorage {
+
+private String ryaInstance;
+private Connector accumuloConn;
+private Authorizations auths;
+private final PCJIdFactory pcjIdFactory = new PCJIdFactory();
+private final AccumuloPcjSerializer converter = new
AccumuloPcjSerializer();
+private static final PcjTables pcjTables = new PcjTables();
+private static final PeriodicQueryTableNameFactory tableNameFactory =
new PeriodicQueryTableNameFactory();
+
+/**
+ * Creates a AccumuloPeriodicQueryResultStorage Object.
+ * @param accumuloConn - Accumulo Connector for connecting to an
Accumulo instance
+ * @param ryaInstance - Rya Instance name for connecting to Rya
+ */
+public AccumuloPeriodicQueryResultStorage(Connector accumuloConn,
String ryaInstance) {
+this.accumuloConn = accumuloConn;
--- End diff --
done
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (RYA-280) PeriodicQuery Support for Fluo
[
https://issues.apache.org/jira/browse/RYA-280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091571#comment-16091571
]
ASF GitHub Bot commented on RYA-280:
Github user meiercaleb commented on a diff in the pull request:
https://github.com/apache/incubator-rya/pull/177#discussion_r127981484
--- Diff:
extras/rya.indexing.pcj/src/main/java/org/apache/rya/indexing/pcj/storage/PeriodicQueryStorageMetadata.java
---
@@ -0,0 +1,101 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.rya.indexing.pcj.storage;
+
+import java.util.Objects;
+
+import org.apache.commons.lang3.builder.EqualsBuilder;
+import org.apache.rya.indexing.pcj.storage.accumulo.VariableOrder;
+
+import com.google.common.base.Preconditions;
+
+/**
+ * Metadata for a given PeriodicQueryStorage table.
+ */
+public class PeriodicQueryStorageMetadata {
+
+private String sparql;
+private VariableOrder varOrder;
+
+/**
+ * Create a PeriodicQueryStorageMetadat object
+ * @param sparql - SPARQL query whose results are stored in table
+ * @param varOrder - order that BindingSet values are written in in
table
+ */
+public PeriodicQueryStorageMetadata(String sparql, VariableOrder
varOrder) {
+Preconditions.checkNotNull(sparql);
--- End diff --
done
> PeriodicQuery Support for Fluo
> --
>
> Key: RYA-280
> URL: https://issues.apache.org/jira/browse/RYA-280
> Project: Rya
> Issue Type: New Feature
> Components: clients
>Affects Versions: 3.2.10
>Reporter: Caleb Meier
>Assignee: Caleb Meier
>
> Add the capability to Rya-Fluo App to provide periodic updates for queries
> registered with Fluo. That is, provide the application with the ability to
> satisfy the standing query "tell me every 12 hours about all of the events of
> a particular type that occurred within the last 24 hours". Given that Fluo
> operates using a push based notification system, some external service needs
> to be implemented to periodically notify Fluo to generate a result
> notification (this helps handle the non-event event that occurs when nothing
> happens in a given period of time).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
