[GitHub] drill issue #904: DRILL-5717: change some date time test cases with specific...
Github user weijietong commented on the issue: https://github.com/apache/drill/pull/904 @vvysotskyi the problem with mocking strategy is that when I run the test case just with the single method `testToDateForTimeStamp` it will pass. but when I run the whole test cases of the `TestCastFunctions ` , the included `testToDateForTimeStamp` method will fail. ---
[jira] [Created] (DRILL-5897) Support Query Cancellation when WebConnection is closed on client side both for authenticated and unauthenticated user's
Sorabh Hamirwasia created DRILL-5897: Summary: Support Query Cancellation when WebConnection is closed on client side both for authenticated and unauthenticated user's Key: DRILL-5897 URL: https://issues.apache.org/jira/browse/DRILL-5897 Project: Apache Drill Issue Type: Task Components: Web Server Reporter: Sorabh Hamirwasia Today there is no session created (using cookies) for unauthenticated Webuser whereas for authenticated user's session is created. Also when a user submits a query then we wait until entire results is gathered on WebServer side and then send the entire Webpage in the response (probably that's how ftl works). For authenticated user's we only cancel the queries in-flight when the session is invalidated (either by timeout or logout). However in absence of session we do nothing for unauthenticated user's so once a query is submitted it will run until it's failed or successful. The only way to explicitly cancel a query is from profile page which will not work when profiles are disabled. We should research more on if it's possible to get the underlying WebConnection (not session) close event and cancel queries running as part of that connection close event. Also since today we will wait for entire query to finish on backend server and then send the response back, which is when a bad connection is detected it doesn't makes sense to cancel at that point (there is 1:1 mapping between request and connection) since query is already completed. Instead we can send header followed by batches of data (pagination) then we can detect early enough if connection is valid or not and cancel the query in response to that. More research is needed in this area along with knowledge of Jetty on how this can be achieved to make our WebServer more performant. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] drill pull request #993: DRILL-5874: NPE in AnonWebUserConnection.cleanupSes...
Github user sohami commented on a diff in the pull request: https://github.com/apache/drill/pull/993#discussion_r146112692 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/WebUserConnection.java --- @@ -124,6 +124,13 @@ public void sendData(RpcOutcomeListener listener, QueryWritableBatch result } } + /** + * Returns a DefaultChannelPromise which doesn't have reference to any actual channel but has an EventExecutor + * associated with it. In this case we use EventExecutor out of BitServer EventLoopGroup. Since there is no actual + * connection established using this class, hence the close event will never be fired by underlying layer and close + * future is set only when the WebSessionResources are closed. --- End diff -- Based on previous conversation the assumption was unsecure path also maintains cookie or session. On looking more this proved to be false and we don't maintain any session for unsecure case. This means that the requests submitted without authentication are stateless and once the connection is broken by closing the browser session then the query will still continue running until explicitly cancelled from profile page. I didn't found any way in Jetty to get notification about connection close event. And looks like that won't be much helpful right now since we send the response back only after the query is completed and we have entire result. I have opened another JIRA [DRILL-5897](https://issues.apache.org/jira/browse/DRILL-5897) to see if we can improve the way we send result back to client or make unsecure case to also use session concept such that we can cancel the in-flight queries based on when session is invalidated. ---
[GitHub] drill issue #993: DRILL-5874: NPE in AnonWebUserConnection.cleanupSession()
Github user sohami commented on the issue: https://github.com/apache/drill/pull/993 Updated the PR with review comments and opened a JIRA based on in-person discussion. ---
[GitHub] drill pull request #996: DRILL-5878: TableNotFound exception is being report...
Github user HanumathRao commented on a diff in the pull request:
https://github.com/apache/drill/pull/996#discussion_r146118332
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/SchemaUtilites.java
---
@@ -77,6 +77,22 @@ public static SchemaPlus findSchema(final SchemaPlus
defaultSchema, final String
return findSchema(defaultSchema, schemaPathAsList);
}
+ /**
+ * Utility function to get the commonPrefix schema between two supplied
schemas.
+ * @param defaultSchema default schema
+ * @param schemaPath current schema path
+ * @param isCaseSensitive true if caseSensitive comparision is required.
+ * @return common prefix schemaPath
+ */
+ public static String getPrefixSchemaPath(final String defaultSchema,
final String schemaPath, final boolean isCaseSensitive) {
--- End diff --
@paul-rogers Thanks for the review. For this API I am assuming the caller
will supply the information whether it needs to be a case sensitive or case
insensitive comparison. The caller for this function is passing the
parserConfig case sensitive parameter.
---
[GitHub] drill pull request #996: DRILL-5878: TableNotFound exception is being report...
Github user HanumathRao commented on a diff in the pull request:
https://github.com/apache/drill/pull/996#discussion_r146118431
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/SchemaUtilites.java
---
@@ -77,6 +77,22 @@ public static SchemaPlus findSchema(final SchemaPlus
defaultSchema, final String
return findSchema(defaultSchema, schemaPathAsList);
}
+ /**
+ * Utility function to get the commonPrefix schema between two supplied
schemas.
+ * @param defaultSchema default schema
+ * @param schemaPath current schema path
+ * @param isCaseSensitive true if caseSensitive comparision is required.
+ * @return common prefix schemaPath
+ */
+ public static String getPrefixSchemaPath(final String defaultSchema,
final String schemaPath, final boolean isCaseSensitive) {
+if (!isCaseSensitive) {
+ return Strings.commonPrefix(defaultSchema.toLowerCase(),
schemaPath.toLowerCase());
--- End diff --
@paul-rogers Thank you for this catch. I did try out some examples which
were throwing schemanotfound exception. I have changed the code to handle it.
getDefaultSchemaPath of session object returns the defaultSchema as String,
hence I converted the SchemaPath to String by SchemaPathJOINER. As pointed out
by your comment, this will loose some context information if the "." is part of
the name. However, I also think that there is a bug in the defaultSchema code
path as it is also loosing this information by converting it to String. For now
I am not fixing that bug (which I am planning to fix in another JIRA) as I am
not sure about the changes in setting defaultSchema.
Coming on to the specific valid use case which was throwing schemaNotFound
exception is
in dfs I have introduced "tmp.tmp1" in place of tmp.
And if I query select * from dfs.`tmp.tmp1`.`employee.json` limit 1; this
query reports SchemaNotFound exception even though a workspace with name
tmp.tmp1 exists.
This is currently working with new changes.
---
