[
https://issues.apache.org/jira/browse/LENS-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14651447#comment-14651447
]
Amareshwari Sriramadasu commented on LENS-124:
----------------------------------------------
bq. 4. Allow gating of the scheduled queries based on partition availability.
It shouldn't happen that ideally query is run in low cost engine and due to
data availability delays, the system chooses a higher cost engine because of
availability of data earlier in that system. In some cases that may be
acceptable as well. But the choice should be conscious
bq. Need more inputs on this. Can you give me an example please?
Here is an example: Say you have two storages HDFS and DB. And accessing DB is
of lower cost than HDFS. But the data is available in HDFS and not in DB yet
(might be available in some time). Then scheduled query can be executed on
HDFS. For this scenario, the requirement says "the system chooses a higher cost
engine because of availability of data earlier in that system. In some cases
that may be acceptable as well. But the choice should be conscious".
bq. Handling query failures in the schedule and being able to run them again
through administrative levers
bq. How is this different from re-running failed queries ?
Running failed query as first class api is not available in lens. If user
submits a query and its fails, he will simply resubmit the query as a new
query. But this requirement says accept "re-run" as first class on a scheduled
query attempt.
bq. Does this mean ability to selectively choose schedules that can be run
again?
Yes.
I will look at the PDF attached and provides comments soon.
> Add scheduler service on lens
> -----------------------------
>
> Key: LENS-124
> URL: https://issues.apache.org/jira/browse/LENS-124
> Project: Apache Lens
> Issue Type: New Feature
> Components: server
> Reporter: Amareshwari Sriramadasu
> Labels: Hackathon-July
> Attachments: Lens Scheduler API.pdf
>
>
> Query scheduler service should provide following services
> * Schedule a query
> ** Periodically
> ** On data availability, for ex: whenever a day's data is available
> * Look at status of scheduled query
> ** Mostly the current run status
> * Look at the output of scheduled query - through email?
> * Update scheduled query
> ** Update the query
> ** Update the frequency
> ** Update the configuration
> * Look at stats of scheduled query
> ** number of failures
> ** number of reruns
> ** Get handles for completed queries
> * Cancel a scheduled query
> * Pause and resume a scheduled query
> * Look at all scheduled queries
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
