[
https://issues.apache.org/jira/browse/LENS-124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642632#comment-14642632
]
Bala Nathan commented on LENS-124:
----------------------------------
Merging Comments from two threads:
1. Parameterization of queries to allow fixed, moving time windows
- Pre-requisite is that the queries are already saved and prepared in Lens and
can be executed later. Not sure if bind arg support is available
2. Allow Cron style scheduling & specific weekday or specific day of month etc
- Similar to the quartz scheduler format (supports cron)
3. Allow scheduled queries to be paced in such a way that regular queries that
users are submitting point in time don't suffer
- Should be possible to solve this by having a dedicated scheduler instance for
running scheduled queries
4. Allow gating of the scheduled queries based on partition availability. It
shouldn't happen that ideally query is run in low cost engine and due to data
availability delays, the system chooses a higher cost engine because of
availability of data earlier in that system. In some cases that may be
acceptable as well. But the choice should be conscious
- Need more inputs on this. Can you give me an example please?
5.Handling query failures in the schedule and being able to run them again
through administrative levers
- How is this different from re-running failed queries ?
6. Administrative levers to repeating the entire schedule due to errors in
data, requiring re-publishing the reports
- Does this mean ability to selectively choose schedules that can be run again?
7. Support query stats analysis over scheduled queries
- Need more inputs on What kind of analysis can be run on scheduled queries ?
8. Limit number of schedules per user/app/client
9. Schedule should be aware of data availability
- Can this not be handled as part of a pre-condition to running a schedule.
Assuming there is a proper expression syntax supported for specifying a
pre-condition (query/script/set of rules), the schedule instance itself can
either be executed/rejected based on the output of a pre-condition check.
10. Schedule should get retried on non data availability and failures and send
appropriate alerts to an end point
- I'm assuming this means "Schedule should not run" and an alert should be
sent.
11. User will set schedule start time in his/her time zone
12. Ability to expire a schedule
13. Pause and resume of schedule
14. Cancel a running instance
15. Upper limit on number of scheduled jobs which will be submitted to the grid
at a time. Otherwise it might end up overloading the cluster
- Somewhat similar to #8 ?
> Add scheduler service on lens
> -----------------------------
>
> Key: LENS-124
> URL: https://issues.apache.org/jira/browse/LENS-124
> Project: Apache Lens
> Issue Type: New Feature
> Components: server
> Reporter: Amareshwari Sriramadasu
> Labels: Hackathon-July
>
> Query scheduler service should provide following services
> * Schedule a query
> ** Periodically
> ** On data availability, for ex: whenever a day's data is available
> * Look at status of scheduled query
> ** Mostly the current run status
> * Look at the output of scheduled query - through email?
> * Update scheduled query
> ** Update the query
> ** Update the frequency
> ** Update the configuration
> * Look at stats of scheduled query
> ** number of failures
> ** number of reruns
> ** Get handles for completed queries
> * Cancel a scheduled query
> * Pause and resume a scheduled query
> * Look at all scheduled queries
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
