[
https://issues.apache.org/jira/browse/LENS-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16754694#comment-16754694
]
Ankit Kailaswar commented on LENS-1538:
---------------------------------------
Jenkins build summary for patch
{code:java}
[INFO] Reactor Summary:
[INFO]
[INFO] Lens Checkstyle Rules ............................. SUCCESS [2.954s]
[INFO] Lens .............................................. SUCCESS [5.297s]
[INFO] Lens API .......................................... SUCCESS [34.868s]
[INFO] Lens API for server and extensions ................ SUCCESS [27.002s]
[INFO] Lens Cube ......................................... SUCCESS [4:07.103s]
[INFO] Lens DB storage ................................... SUCCESS [12.991s]
[INFO] Lens Query Library ................................ SUCCESS [8.972s]
[INFO] Lens Hive Driver .................................. SUCCESS [1:06.959s]
[INFO] Lens Driver for JDBC .............................. SUCCESS [45.168s]
[INFO] Lens Elastic Search Driver ........................ SUCCESS [18.904s]
[INFO] Lens Server ....................................... SUCCESS [11:20.731s]
[INFO] Lens client ....................................... SUCCESS [1:46.624s]
[INFO] Lens CLI .......................................... SUCCESS [1:31.769s]
[INFO] Lens Examples ..................................... SUCCESS [9.015s]
[INFO] Lens Ship Jars to Distributed Cache ............... SUCCESS [0.911s]
[INFO] Lens Distribution ................................. SUCCESS [12.214s]
[INFO] Lens ML Lib ....................................... SUCCESS [1:12.594s]
[INFO] Lens ML Ext Distribution .......................... SUCCESS [1.718s]
[INFO] Lens Regression ................................... SUCCESS [11.010s]
[INFO] Lens UI ........................................... SUCCESS [32.495s]
[INFO] Lens Contrib ...................................... SUCCESS [0.275s]
[INFO] Lens Contributed Clients .......................... SUCCESS [0.284s]
[INFO] Lens Python Client ................................ SUCCESS [0.271s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 24:51.067s
[INFO] Finished at: Mon Jan 28 19:19:53 UTC 2019
[INFO] Final Memory: 286M/2934M
[INFO]
------------------------------------------------------------------------{code}
> Lens HA changes
> ---------------
>
> Key: LENS-1538
> URL: https://issues.apache.org/jira/browse/LENS-1538
> Project: Apache Lens
> Issue Type: Improvement
> Components: server
> Reporter: Ankit Kailaswar
> Assignee: Ankit Kailaswar
> Priority: Major
> Attachments: Lens-1538_8.diff
>
>
> #
> h1. Implementation
> #
> h2. Session
> # We need to persist all client session in database.
> # This involve persisting client session’s id and configuration in mysql
> server. persisting entire session object is not required. We need to do this
> as first thing while creating user session.
> # For each request from client lens server should not validate session id
> from its in memory map. It should rather validate it from persisted session
> in mysql server.
> if
> session is present in mysql and not present in lens server’s in memory
> map(session manager) then it should create a lens session with same session
> id and add it to in memory map
> else If
> session is not present in mysql and present in server’s in memory map then it
> signifies that user have initiated close session from other host and we will
> need to remove this session from current host’s in memory map as well
> else if
> session is not present in mysql and in server’s in memory map then return
> “invalid session”.
> # Whenever session expired/closed then we need to remove this session from
> lens server’s in memory map and mysql.
>
> #
> h2. *Query*
> # For sync queries lens server will close connection as soon as it is
> stopped or failed with appropriate failure message. Client will be retrying
> the query.
> # For async-light queries and async-heavy queries lens server takes care of
> rescheduling all queries which were in running or queued state at the time of
> restart. we can use this to make query ids available with both servers.
> # For async queries we need to persist all non finished async user queries
> in mysql server in new table current_query.
> # If user executes query on host1 and it goes down and nginx points to host2
> then user should be able to poll on query status. Lens server will first
> check query id in its inmemory maps if it is not present then it will check
> in “finished_query” table in mysql else in “current_query” table in mysql. In
> this case we will continue showing old status of query since it is scheduled
> by host1. As soon as host1 comes up it will reschedule these queries and will
> change the status. Optionally we can have host2 to move these queries in
> “allqueries” map of its query service which will take care of recovering, in
> this case we don't need to wait for host1 to come up and reschedule.
> # While moving query from ”finished_quey” map to “finished_query” table in
> mysql we will need to remove it from “current_query” tables as well.
> # This flow will remain same for request ids/query ids generated for query
> plan/estimate.
>
> #
> h1. Issue addressed
> # Lens downtime will be zero in case of deployment and if primary fail.
> # User will not have to create a new session whenever switch happens
> # User will be able to get status of all async queries seamlessly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
