[jira] [Commented] (PIG-2815) class loader management in PigContext
[
https://issues.apache.org/jira/browse/PIG-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13525809#comment-13525809
]
Julien Le Dem commented on PIG-2815:
0.11/trunk sounds good to me
class loader management in PigContext
-
Key: PIG-2815
URL: https://issues.apache.org/jira/browse/PIG-2815
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.9.0
Reporter: Raghu Angadi
Assignee: Raghu Angadi
Fix For: 0.11
Attachments: PIG-2815-branch-0.9.patch, PIG-2815-branch-0.9.patch,
PIG-2815.patch, PIG-2815.patch
The way {{PigContext.classloader}} and resolveClassName() are managed can
lead to strange class loading issues, especially when not all {{register}}
statements are at the top (example in the first comment).
Two factors contribute to this: sometimes only one of them and sometimes
together:
# a new classloader (CL) is created after registering each jar.
** but the new jar's parent is the root CL rather than previous CL,
effectively throwing previous CL away.
# resolveClassName() caches classes based on just the name
** A class is not defined by name alone. Classes loaded by two different
unrelated CLs are different objects even if both extract the class from same
physical jar file.
** because of (1), the cached class is not necessarily same as the class
that would be loaded based on 'current' CL
having different class objects for same class have many subtle side effects.
e.g. there would be two instances of static variables.
I think both should be fixed.. thought fixing one of them might be good
enough in many cases. I will add a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2815) class loader management in PigContext
[
https://issues.apache.org/jira/browse/PIG-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13508451#comment-13508451
]
Cheolsoo Park commented on PIG-2815:
It seems that the patch is committed to 0.11/trunk but not to 0.9/0.10.
Please let me know if anyone wants this to be committed to 0.9/0.10. Or I will
close the jira.
Thanks!
class loader management in PigContext
-
Key: PIG-2815
URL: https://issues.apache.org/jira/browse/PIG-2815
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.9.0
Reporter: Raghu Angadi
Assignee: Raghu Angadi
Fix For: 0.11
Attachments: PIG-2815-branch-0.9.patch, PIG-2815-branch-0.9.patch,
PIG-2815.patch, PIG-2815.patch
The way {{PigContext.classloader}} and resolveClassName() are managed can
lead to strange class loading issues, especially when not all {{register}}
statements are at the top (example in the first comment).
Two factors contribute to this: sometimes only one of them and sometimes
together:
# a new classloader (CL) is created after registering each jar.
** but the new jar's parent is the root CL rather than previous CL,
effectively throwing previous CL away.
# resolveClassName() caches classes based on just the name
** A class is not defined by name alone. Classes loaded by two different
unrelated CLs are different objects even if both extract the class from same
physical jar file.
** because of (1), the cached class is not necessarily same as the class
that would be loaded based on 'current' CL
having different class objects for same class have many subtle side effects.
e.g. there would be two instances of static variables.
I think both should be fixed.. thought fixing one of them might be good
enough in many cases. I will add a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2815) class loader management in PigContext
[
https://issues.apache.org/jira/browse/PIG-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500567#comment-13500567
]
Julien Le Dem commented on PIG-2815:
It sounds like this has been committed. Can you resolve it?
class loader management in PigContext
-
Key: PIG-2815
URL: https://issues.apache.org/jira/browse/PIG-2815
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.9.0
Reporter: Raghu Angadi
Assignee: Raghu Angadi
Fix For: 0.11
Attachments: PIG-2815-branch-0.9.patch, PIG-2815-branch-0.9.patch,
PIG-2815.patch, PIG-2815.patch
The way {{PigContext.classloader}} and resolveClassName() are managed can
lead to strange class loading issues, especially when not all {{register}}
statements are at the top (example in the first comment).
Two factors contribute to this: sometimes only one of them and sometimes
together:
# a new classloader (CL) is created after registering each jar.
** but the new jar's parent is the root CL rather than previous CL,
effectively throwing previous CL away.
# resolveClassName() caches classes based on just the name
** A class is not defined by name alone. Classes loaded by two different
unrelated CLs are different objects even if both extract the class from same
physical jar file.
** because of (1), the cached class is not necessarily same as the class
that would be loaded based on 'current' CL
having different class objects for same class have many subtle side effects.
e.g. there would be two instances of static variables.
I think both should be fixed.. thought fixing one of them might be good
enough in many cases. I will add a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2815) class loader management in PigContext
[
https://issues.apache.org/jira/browse/PIG-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13414933#comment-13414933
]
Ashutosh Chauhan commented on PIG-2815:
---
[~rangadi] Thanks for digging into it. Till now atleast in some cases if I
registered the jar and then if I want to use it in Pig client frontend, I also
necessarily need to add it in {{PIG_CLASSPATH}}. Does this patch removes that
restriction? Put another way, will registering the jar will automatically add
the jar in pig's classpath?
class loader management in PigContext
-
Key: PIG-2815
URL: https://issues.apache.org/jira/browse/PIG-2815
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.9.0
Reporter: Raghu Angadi
Fix For: 0.11
Attachments: PIG-2815.patch, PIG-2815.patch
The way {{PigContext.classloader}} and resolveClassName() are managed can
lead to strange class loading issues, especially when not all {{register}}
statements are at the top (example in the first comment).
Two factors contribute to this: sometimes only one of them and sometimes
together:
# a new classloader (CL) is created after registering each jar.
** but the new jar's parent is the root CL rather than previous CL,
effectively throwing previous CL away.
# resolveClassName() caches classes based on just the name
** A class is not defined by name alone. Classes loaded by two different
unrelated CLs are different objects even if both extract the class from same
physical jar file.
** because of (1), the cached class is not necessarily same as the class
that would be loaded based on 'current' CL
having different class objects for same class have many subtle side effects.
e.g. there would be two instances of static variables.
I think both should be fixed.. thought fixing one of them might be good
enough in many cases. I will add a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2815) class loader management in PigContext
[
https://issues.apache.org/jira/browse/PIG-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415430#comment-13415430
]
Raghu Angadi commented on PIG-2815:
---
Ashutosh, that issue is fixed in PIG-2532. Let us know otherwise.
Still there are some corner cases: e.g. if you use {{ObjectSerializer}} on
frontend, that class should be in CLASSPATH (the fix is to extend
ObjectSerializer to take a classloader or it should use current threads
classloader).
class loader management in PigContext
-
Key: PIG-2815
URL: https://issues.apache.org/jira/browse/PIG-2815
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.9.0
Reporter: Raghu Angadi
Assignee: Raghu Angadi
Fix For: 0.11
Attachments: PIG-2815.patch, PIG-2815.patch
The way {{PigContext.classloader}} and resolveClassName() are managed can
lead to strange class loading issues, especially when not all {{register}}
statements are at the top (example in the first comment).
Two factors contribute to this: sometimes only one of them and sometimes
together:
# a new classloader (CL) is created after registering each jar.
** but the new jar's parent is the root CL rather than previous CL,
effectively throwing previous CL away.
# resolveClassName() caches classes based on just the name
** A class is not defined by name alone. Classes loaded by two different
unrelated CLs are different objects even if both extract the class from same
physical jar file.
** because of (1), the cached class is not necessarily same as the class
that would be loaded based on 'current' CL
having different class objects for same class have many subtle side effects.
e.g. there would be two instances of static variables.
I think both should be fixed.. thought fixing one of them might be good
enough in many cases. I will add a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2815) class loader management in PigContext
[
https://issues.apache.org/jira/browse/PIG-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415467#comment-13415467
]
Jonathan Coveney commented on PIG-2815:
---
+1, will commit once test-commit passes
class loader management in PigContext
-
Key: PIG-2815
URL: https://issues.apache.org/jira/browse/PIG-2815
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.9.0
Reporter: Raghu Angadi
Assignee: Raghu Angadi
Fix For: 0.11
Attachments: PIG-2815.patch, PIG-2815.patch
The way {{PigContext.classloader}} and resolveClassName() are managed can
lead to strange class loading issues, especially when not all {{register}}
statements are at the top (example in the first comment).
Two factors contribute to this: sometimes only one of them and sometimes
together:
# a new classloader (CL) is created after registering each jar.
** but the new jar's parent is the root CL rather than previous CL,
effectively throwing previous CL away.
# resolveClassName() caches classes based on just the name
** A class is not defined by name alone. Classes loaded by two different
unrelated CLs are different objects even if both extract the class from same
physical jar file.
** because of (1), the cached class is not necessarily same as the class
that would be loaded based on 'current' CL
having different class objects for same class have many subtle side effects.
e.g. there would be two instances of static variables.
I think both should be fixed.. thought fixing one of them might be good
enough in many cases. I will add a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2815) class loader management in PigContext
[
https://issues.apache.org/jira/browse/PIG-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415514#comment-13415514
]
Ashutosh Chauhan commented on PIG-2815:
---
[~rangadi] I think I have hit on corner cases. Will file jira for it. Do you
happen to know any other places where it will manifest ?
class loader management in PigContext
-
Key: PIG-2815
URL: https://issues.apache.org/jira/browse/PIG-2815
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.9.0
Reporter: Raghu Angadi
Assignee: Raghu Angadi
Fix For: 0.11
Attachments: PIG-2815.patch, PIG-2815.patch
The way {{PigContext.classloader}} and resolveClassName() are managed can
lead to strange class loading issues, especially when not all {{register}}
statements are at the top (example in the first comment).
Two factors contribute to this: sometimes only one of them and sometimes
together:
# a new classloader (CL) is created after registering each jar.
** but the new jar's parent is the root CL rather than previous CL,
effectively throwing previous CL away.
# resolveClassName() caches classes based on just the name
** A class is not defined by name alone. Classes loaded by two different
unrelated CLs are different objects even if both extract the class from same
physical jar file.
** because of (1), the cached class is not necessarily same as the class
that would be loaded based on 'current' CL
having different class objects for same class have many subtle side effects.
e.g. there would be two instances of static variables.
I think both should be fixed.. thought fixing one of them might be good
enough in many cases. I will add a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2815) class loader management in PigContext
[
https://issues.apache.org/jira/browse/PIG-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13415522#comment-13415522
]
Jonathan Coveney commented on PIG-2815:
---
Committed. Thanks Raghu! Also: want to make a ticket for that ObjectSerializer
CLASSPATH issue you mentioned?
class loader management in PigContext
-
Key: PIG-2815
URL: https://issues.apache.org/jira/browse/PIG-2815
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.9.0
Reporter: Raghu Angadi
Assignee: Raghu Angadi
Fix For: 0.11
Attachments: PIG-2815.patch, PIG-2815.patch
The way {{PigContext.classloader}} and resolveClassName() are managed can
lead to strange class loading issues, especially when not all {{register}}
statements are at the top (example in the first comment).
Two factors contribute to this: sometimes only one of them and sometimes
together:
# a new classloader (CL) is created after registering each jar.
** but the new jar's parent is the root CL rather than previous CL,
effectively throwing previous CL away.
# resolveClassName() caches classes based on just the name
** A class is not defined by name alone. Classes loaded by two different
unrelated CLs are different objects even if both extract the class from same
physical jar file.
** because of (1), the cached class is not necessarily same as the class
that would be loaded based on 'current' CL
having different class objects for same class have many subtle side effects.
e.g. there would be two instances of static variables.
I think both should be fixed.. thought fixing one of them might be good
enough in many cases. I will add a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2815) class loader management in PigContext
[
https://issues.apache.org/jira/browse/PIG-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13413532#comment-13413532
]
Raghu Angadi commented on PIG-2815:
---
An example:
{noformat}
register elephant-bird.jar; -- for working with Thrift objects.
-- (1)
register T_One.jar;
-- (2)
-- ThriftPigLoader takes name of a Thrift class that corresponds to input.
a = load '/logs/T_One' using ThriftPigLoader('thrift.gen.T_One');
-- (3)
register second.jar;
-- (4)
b = load '/logs/T_two' using ThriftPigLoader('thrift.gen.T_two');
-- (5)
-- FAIL!
{noformat}
* (1): new classlaoder cl_A is created with root classloader as the parent.
* (2): cl_B is created with root as the parent.
* (3): {{ThirftPigLoader.class}} is instantiated with cl_B and cached.
* (4): cl_C is created with root as the parent.
* (5): {{thrift.gen.T_two.class}} is instantiated with cl_C, but
'{{ThriftPigLoader.class}}' from cl_B is reused by Pig. So all the Thrift
classes seen by ThriftPigLoader are entirely different from all the Thrift
classes seen by {{thrift.gen.T_two}} since cl_B is not a parent of cl_C. That
can lead to a number of issues and it does.
class loader management in PigContext
-
Key: PIG-2815
URL: https://issues.apache.org/jira/browse/PIG-2815
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.9.0
Reporter: Raghu Angadi
Fix For: 0.11
The way {{PigContext.classloader}} and resolveClassName() are managed can
lead to strange class loading issues, especially when not all {{register}}
statements are at the top (example in the first comment).
Two factors contribute to this: sometimes only one of them and sometimes
together:
# a new classloader (CL) is created after registering each jar.
** but the new jar's parent is the root CL rather than previous CL,
effectively throwing previous CL away.
# resolveClassName() caches classes based on just the name
** A class is not defined by name alone. Classes loaded by two different
unrelated CLs are different objects even if both extract the class from same
physical jar file.
** because of (1), the cached class is not necessarily same as the class
that would be loaded based on 'current' CL
having different class objects for same class have many subtle side effects.
e.g. there would be two instances of static variables.
I think both should be fixed.. thought fixing one of them might be good
enough in many cases. I will add a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2815) class loader management in PigContext
[
https://issues.apache.org/jira/browse/PIG-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13414029#comment-13414029
]
Jonathan Coveney commented on PIG-2815:
---
Raghu,
This is awesome. Is there any hope that you could add a test that fails in the
current setup? I think that would be important as far as conveying a concrete
use case when this fails (which your example does, but if we have a test that
fails it'll be super super concrete.
Thanks for doing this. Awesome.
Jon
class loader management in PigContext
-
Key: PIG-2815
URL: https://issues.apache.org/jira/browse/PIG-2815
Project: Pig
Issue Type: Bug
Components: impl
Affects Versions: 0.9.0
Reporter: Raghu Angadi
Fix For: 0.11
Attachments: PIG-2815.patch
The way {{PigContext.classloader}} and resolveClassName() are managed can
lead to strange class loading issues, especially when not all {{register}}
statements are at the top (example in the first comment).
Two factors contribute to this: sometimes only one of them and sometimes
together:
# a new classloader (CL) is created after registering each jar.
** but the new jar's parent is the root CL rather than previous CL,
effectively throwing previous CL away.
# resolveClassName() caches classes based on just the name
** A class is not defined by name alone. Classes loaded by two different
unrelated CLs are different objects even if both extract the class from same
physical jar file.
** because of (1), the cached class is not necessarily same as the class
that would be loaded based on 'current' CL
having different class objects for same class have many subtle side effects.
e.g. there would be two instances of static variables.
I think both should be fixed.. thought fixing one of them might be good
enough in many cases. I will add a patch.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
