+1 That's definitely the right way forward. These thread local variables should be verboten.
I also agree with Brock's assessment that this will need a design doc first. My guess is that this will be quite a bit of work. Thanks, Gunther. On Sun, Aug 11, 2013 at 12:34 PM, Prasad Mujumdar <pras...@cloudera.com>wrote: > +1 ! > That's should be the right approach IMO too. > > thanks > Prasad > > > > On Sun, Aug 11, 2013 at 10:03 AM, Brock Noland <br...@cloudera.com> wrote: > > > HIVE-4226 is a very welcome patch but in the long run I think we want to > > move away from using static thread local everywhere which that patch > uses. > > Rightfully so at this juncture. > > > > Meaning I think that HIVE-4226 is a good interim solution but long term I > > think we want to be passing around some kind of context object as Edward > > suggests in his original mail. This will make the code cleaner and more > > testable. > > > > This is a big effort and will require some analysis and planning before > > execution. > > > > > > On Sat, Aug 10, 2013 at 11:25 PM, Navis류승우 <navis....@nexr.com> wrote: > > > > > https://issues.apache.org/jira/browse/HIVE-4226 seemed addressing > this. > > > > > > 2013/8/11 Brock Noland <br...@cloudera.com>: > > > > I would love to get rid of the static thread local stuff. It was > > required > > > > to make hive work in a server model but isn't the correct solution to > > > this > > > > problem. > > > > > > > > I do think it will be a large amount of work so it'd be great to see > > > > whoever leads this effort to have a high level plan as opposed to an > > > adhoc > > > > effort. > > > > > > > > > > > > On Sat, Aug 10, 2013 at 12:32 PM, Edward Capriolo < > > edlinuxg...@gmail.com > > > >wrote: > > > > > > > >> I just committed https://issues.apache.org/jira/browse/HIVE-3772. > > > >> > > > >> For hive-server2 Carl and others did a lot of work to clean up un > > thread > > > >> safe things from hive. > > > >> > > > >> Hive was originally build as a fat client so it is not surprising > that > > > many > > > >> such constructs exist. Now since we have retrofitted > > multi-threaded-ness > > > >> onto the project we have a number of edge case bugs. > > > >> > > > >> My suggestions here would be for that the next release 0.13 we make > a > > > push > > > >> to remove all possible non thread safe code and explicitly pass > > context > > > >> objects or serialized structures everywhere thread safety is needed. > > > >> > > > >> I can see this would start with something like the Function > Registry, > > > this > > > >> would be a per session object passed around rather then a global > > object > > > >> with static hashmap instances in it. > > > >> > > > >> I know that this probably will not be as simple as removing all > static > > > >> members from our codebase, but does anyone know of specific > challenges > > > that > > > >> will be intrinsically hard to solve? > > > >> > > > >> Please comment. > > > >> > > > > > > > > > > > > > > > > -- > > > > Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org > > > > > > > > > > > -- > > Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org > > >
