[jira] [Commented] (HIVE-6364) HiveServer2 - Request serving thread should get class loader from existing SessionState
[
https://issues.apache.org/jira/browse/HIVE-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13941027#comment-13941027
]
Ashutosh Chauhan commented on HIVE-6364:
I think [~navis] 's patch on HIVE-3969 is more complete and subsumes current
patch. So, I guess we should try to get that one and close this one. Although,
[~jaideepdhok] 's [earlier point |
https://issues.apache.org/jira/browse/HIVE-6364?focusedCommentId=13892132page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13892132]
about jars on a thread being visible across session is still there. Though, I
am not sure how will that negatively impact query execution and way to fix it.
That probably can be taken up as follow-up of HIVE-3969
HiveServer2 - Request serving thread should get class loader from existing
SessionState
---
Key: HIVE-6364
URL: https://issues.apache.org/jira/browse/HIVE-6364
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: Jaideep Dhok
Attachments: HIVE-6364.1.patch
SessionState is created for each session in HS2. If we do any add jars, a
class loader is set in the SessionState's conf object. This class loader
should also be set in each thread that serves request of the same session.
Scenario (both requests are in the same session)-
{noformat}
// req 1
add jar foo.jar // Served by thread th1, this updates class loader and sets
in SessionState.conf
// req2 served by th2, such that th1 != th2
CREATE TEMPORARY FUNCTION foo_udf AS 'some class in foo.jar'
// This can throw class not found error, because although
// the new thread (th2) gets the same session state as th1,
// the class loader is different (Thread.currentThread.getContextClassLoader()
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6364) HiveServer2 - Request serving thread should get class loader from existing SessionState
[
https://issues.apache.org/jira/browse/HIVE-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13939467#comment-13939467
]
Ashutosh Chauhan commented on HIVE-6364:
Although, HIVE-3969 is marked as duplicate, I don't think it is a duplicate.
This one fixes the problem of having right class loader for a thread serving
the query, whereas HIVE-3969 talks about unloading registered jars. So, it
seems there are two independent problem, both of which needs to be fixed.
[~jaideepdhok] would you like to rebase your patch.
HiveServer2 - Request serving thread should get class loader from existing
SessionState
---
Key: HIVE-6364
URL: https://issues.apache.org/jira/browse/HIVE-6364
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: Jaideep Dhok
Attachments: HIVE-6364.1.patch
SessionState is created for each session in HS2. If we do any add jars, a
class loader is set in the SessionState's conf object. This class loader
should also be set in each thread that serves request of the same session.
Scenario (both requests are in the same session)-
{noformat}
// req 1
add jar foo.jar // Served by thread th1, this updates class loader and sets
in SessionState.conf
// req2 served by th2, such that th1 != th2
CREATE TEMPORARY FUNCTION foo_udf AS 'some class in foo.jar'
// This can throw class not found error, because although
// the new thread (th2) gets the same session state as th1,
// the class loader is different (Thread.currentThread.getContextClassLoader()
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6364) HiveServer2 - Request serving thread should get class loader from existing SessionState
[
https://issues.apache.org/jira/browse/HIVE-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13939592#comment-13939592
]
Jason Dere commented on HIVE-6364:
--
Hi Jaideep, when I tried debugging hiveserver2 due to HIVE-6672, it appeared
that there was a thread running for each connection (session). Non-SQL
commands (such as ADD JAR), were being run within this session thread and so
the classloader for the session thread had the JARs loaded. When a SQL command
was executed the session thread would start a new thread, and it appeared that
this new thread was using the same classloader (and had the added JARs in the
classloader's list of URLs). Were you seeing different behavior in your
testing (I was running this on Mac, I think with jdk 1.6, not sure if it would
have been different)?
In the patch, the thread's classloader is getting set to the HiveConf's
classloader .. where is the HiveConf's classloader getting set from? Do we need
to worry about having to make sure this classloader is updated whenever a JAR
is added to the classpath?
HiveServer2 - Request serving thread should get class loader from existing
SessionState
---
Key: HIVE-6364
URL: https://issues.apache.org/jira/browse/HIVE-6364
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: Jaideep Dhok
Attachments: HIVE-6364.1.patch
SessionState is created for each session in HS2. If we do any add jars, a
class loader is set in the SessionState's conf object. This class loader
should also be set in each thread that serves request of the same session.
Scenario (both requests are in the same session)-
{noformat}
// req 1
add jar foo.jar // Served by thread th1, this updates class loader and sets
in SessionState.conf
// req2 served by th2, such that th1 != th2
CREATE TEMPORARY FUNCTION foo_udf AS 'some class in foo.jar'
// This can throw class not found error, because although
// the new thread (th2) gets the same session state as th1,
// the class loader is different (Thread.currentThread.getContextClassLoader()
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6364) HiveServer2 - Request serving thread should get class loader from existing SessionState
[
https://issues.apache.org/jira/browse/HIVE-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13940018#comment-13940018
]
Jaideep Dhok commented on HIVE-6364:
[~ashutoshc] I will put up a new patch.
[~jdere] Add jar will always update the class loader. That's the current
behaviour. I think the first class loader is set using the conf.getClassLoader
method, if nothing is set it will return the default class loader.
HiveServer2 - Request serving thread should get class loader from existing
SessionState
---
Key: HIVE-6364
URL: https://issues.apache.org/jira/browse/HIVE-6364
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: Jaideep Dhok
Attachments: HIVE-6364.1.patch
SessionState is created for each session in HS2. If we do any add jars, a
class loader is set in the SessionState's conf object. This class loader
should also be set in each thread that serves request of the same session.
Scenario (both requests are in the same session)-
{noformat}
// req 1
add jar foo.jar // Served by thread th1, this updates class loader and sets
in SessionState.conf
// req2 served by th2, such that th1 != th2
CREATE TEMPORARY FUNCTION foo_udf AS 'some class in foo.jar'
// This can throw class not found error, because although
// the new thread (th2) gets the same session state as th1,
// the class loader is different (Thread.currentThread.getContextClassLoader()
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6364) HiveServer2 - Request serving thread should get class loader from existing SessionState
[
https://issues.apache.org/jira/browse/HIVE-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13892132#comment-13892132
]
Jaideep Dhok commented on HIVE-6364:
There's another issue - In SessionState.registerJar class loader is obtained
from the current thread. If the current thread was used during another session,
then the current session can get some jars from that session.
HiveServer2 - Request serving thread should get class loader from existing
SessionState
---
Key: HIVE-6364
URL: https://issues.apache.org/jira/browse/HIVE-6364
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: Jaideep Dhok
Assignee: Jaideep Dhok
Attachments: HIVE-6364.1.patch
SessionState is created for each session in HS2. If we do any add jars, a
class loader is set in the SessionState's conf object. This class loader
should also be set in each thread that serves request of the same session.
Scenario (both requests are in the same session)-
{noformat}
// req 1
add jar foo.jar // Served by thread th1, this updates class loader and sets
in SessionState.conf
// req2 served by th2, such that th1 != th2
CREATE TEMPORARY FUNCTION foo_udf AS 'some class in foo.jar'
// This can throw class not found error, because although
// the new thread (th2) gets the same session state as th1,
// the class loader is different (Thread.currentThread.getContextClassLoader()
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HIVE-6364) HiveServer2 - Request serving thread should get class loader from existing SessionState
[
https://issues.apache.org/jira/browse/HIVE-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13890625#comment-13890625
]
Jaideep Dhok commented on HIVE-6364:
Raised review board request - https://reviews.apache.org/r/17708/
HiveServer2 - Request serving thread should get class loader from existing
SessionState
---
Key: HIVE-6364
URL: https://issues.apache.org/jira/browse/HIVE-6364
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: Jaideep Dhok
Attachments: HIVE-6364.1.patch
SessionState is created for each session in HS2. If we do any add jars, a
class loader is set in the SessionState's conf object. This class loader
should also be set in each thread that serves request of the same session.
Scenario (both requests are in the same session)-
{noformat}
// req 1
add jar foo.jar // Served by thread th1, this updates class loader and sets
in SessionState.conf
// req2 served by th2, such that th1 != th2
CREATE TEMPORARY FUNCTION foo_udf AS 'some class in foo.jar'
// This can throw class not found error, because although
// the new thread (th2) gets the same session state as th1,
// the class loader is different (Thread.currentThread.getContextClassLoader()
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
