[
https://issues.apache.org/jira/browse/LENS-619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14598913#comment-14598913
]
Himanshu Gahlaut commented on LENS-619:
---------------------------------------
Refined model present in description is presenting the design. Thanks for
pointing the persistence leak in model description. Will include that waiting
queries will be persisted across restarts in the model description. There is no
waiting queue. The model description describes that waiting queries are in
waiting queries list.
Initial Thought which is discussing limiting queries by user is not in work
anymore. It is there for historical documentation. Will add (not in work) to it
in model description. However, since the refined model is mentioning about
applying query running constraint, one implementation of Query Running
Constraint could be to limit on queries per user basis as well, if someone
needs it in future.
Currently the query cost returned in lens is only 0 (for JDBC) and 1 (for Hive)
which is not much helpful. Making cumulative query cost on top of this and
limiting queries on basis of same wouldn't have been any useful as well.
However, to overcome this, LENS-630 has been parked yesterday, which will give
better query cost than 1 returned by Hive Driver currently. Now since query
cost is going to be improved, discussions are on to see if a constraint based
on cumulative query cost can be used as default Query Running Constraint.
However, its not finalized yet.
FIFO, Priority Based Queue are scheduling policies. Query Running Constraints
are being built as a separate concern than scheduling.
Also LENS-631 is parked yesterday which will improve priority queue scheduling
to use query cost to determine the query with lowest cost as the highest
priority query while scheduling.
LENS-619, LENS-630 and LENS-631 are being worked upon such that anyone can be
checked in first without affecting the current functionality of lens, hence
they are not related or blocked on each other.
> Applying Query Running Constraint before allowing a query to run
> ----------------------------------------------------------------
>
> Key: LENS-619
> URL: https://issues.apache.org/jira/browse/LENS-619
> Project: Apache Lens
> Issue Type: New Feature
> Reporter: Himanshu Gahlaut
> Assignee: Himanshu Gahlaut
>
> [June 23. 2015]:
> This is the refined model after knowledge crunching:
> Lens will always accept a new query from a user and put it in a priority
> queue for processing.
> Next candidate query picked up from priority queue for processing will be run
> only if a query running constraint evaluated on running queries and candidate
> query allows the candidate query to be run, otherwise the candidate query
> will be moved to waiting queries list.
> When any running query is finished, result of evaluation of query constraint
> might change, waiting queries might become eligible to be run, hence waiting
> queries will be moved back to priority queue to be re-processed.
> [Initial thought]: Fair Usage Policy for using Lens
> After a user has used the lens system beyond fair usage policy, he will get
> lens features in degraded mode.
> One of the forms of fair usage policy could be that at any point in time only
> N queries can be present in RUNNING state for a user.
> If N queries of a user are in RUNNING state, then a new query submitted by
> same user will stay in QUEUED state until one of the N queries in RUNNING
> state have moved to SUCCESSFUL, FAILED, OR CANCELLED state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
