[
https://issues.apache.org/jira/browse/LENS-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16754826#comment-16754826
]
Hudson commented on LENS-1538:
------------------------------
SUCCESS: Integrated in Jenkins build Lens-Commit #1494 (See
[https://builds.apache.org/job/Lens-Commit/1494/])
LENS-1538 : Lens changes for HA to persist and restore sessions and
(amareshwari: rev f64a0f5984216a61f0a2c5ef90510a88390acde0)
* (edit)
lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java
* (edit) lens-server/src/test/java/org/apache/lens/server/query/TestLensDAO.java
* (edit)
lens-server/src/main/java/org/apache/lens/server/query/QueryServiceResource.java
* (edit)
lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java
* (edit)
lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java
* (edit)
lens-server/src/main/java/org/apache/lens/server/metastore/CubeMetastoreServiceImpl.java
* (edit)
lens-client/src/test/java/org/apache/lens/server/MockQueryExecutionServiceImpl.java
* (edit)
lens-server/src/main/java/org/apache/lens/server/session/HiveSessionService.java
* (add)
lens-server/src/test/java/org/apache/lens/server/TestBaseLensService.java
* (edit) lens-server/src/main/java/org/apache/lens/server/BaseLensService.java
> Lens HA changes
> ---------------
>
> Key: LENS-1538
> URL: https://issues.apache.org/jira/browse/LENS-1538
> Project: Apache Lens
> Issue Type: Improvement
> Components: server
> Reporter: Ankit Kailaswar
> Assignee: Ankit Kailaswar
> Priority: Major
> Fix For: 2.8
>
> Attachments: Lens-1538_8.diff, Lens-1538_9.diff
>
>
> #
> h1. Implementation
> #
> h2. Session
> # We need to persist all client session in database.
> # This involve persisting client session’s id and configuration in mysql
> server. persisting entire session object is not required. We need to do this
> as first thing while creating user session.
> # For each request from client lens server should not validate session id
> from its in memory map. It should rather validate it from persisted session
> in mysql server.
> if
> session is present in mysql and not present in lens server’s in memory
> map(session manager) then it should create a lens session with same session
> id and add it to in memory map
> else If
> session is not present in mysql and present in server’s in memory map then it
> signifies that user have initiated close session from other host and we will
> need to remove this session from current host’s in memory map as well
> else if
> session is not present in mysql and in server’s in memory map then return
> “invalid session”.
> # Whenever session expired/closed then we need to remove this session from
> lens server’s in memory map and mysql.
>
> #
> h2. *Query*
> # For sync queries lens server will close connection as soon as it is
> stopped or failed with appropriate failure message. Client will be retrying
> the query.
> # For async-light queries and async-heavy queries lens server takes care of
> rescheduling all queries which were in running or queued state at the time of
> restart. we can use this to make query ids available with both servers.
> # For async queries we need to persist all non finished async user queries
> in mysql server in new table current_query.
> # If user executes query on host1 and it goes down and nginx points to host2
> then user should be able to poll on query status. Lens server will first
> check query id in its inmemory maps if it is not present then it will check
> in “finished_query” table in mysql else in “current_query” table in mysql. In
> this case we will continue showing old status of query since it is scheduled
> by host1. As soon as host1 comes up it will reschedule these queries and will
> change the status. Optionally we can have host2 to move these queries in
> “allqueries” map of its query service which will take care of recovering, in
> this case we don't need to wait for host1 to come up and reschedule.
> # While moving query from ”finished_quey” map to “finished_query” table in
> mysql we will need to remove it from “current_query” tables as well.
> # This flow will remain same for request ids/query ids generated for query
> plan/estimate.
>
> #
> h1. Issue addressed
> # Lens downtime will be zero in case of deployment and if primary fail.
> # User will not have to create a new session whenever switch happens
> # User will be able to get status of all async queries seamlessly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
