Sorry about not getting to this, Mike! Dmitry's idea is a good one. I still think that something is axing the connections, possible between different map slots on the same mapper. I would log very liberally and see what comes of it. And I'll try and carve out some time.
2013/3/26 Dmitriy Ryaboy <dvrya...@gmail.com> > Mike, have you tried adding logging to any EvalFunc methods that > communicate with Mongo to see which of them is calling it after finish() ? > Are you sure something else doesn't close Mongo connection for you? > > > On Fri, Mar 22, 2013 at 8:28 AM, Mike Sukmanowsky <m...@parsely.com> > wrote: > > > Bump - any thoughts? > > > > > > On Mon, Mar 18, 2013 at 4:53 PM, Mike Sukmanowsky <m...@parsely.com> > > wrote: > > > > > Sure thing - here you go > > > https://gist.github.com/msukmanowsky/198c878e1deaf4e470a9 > > > > > > When initialized, the ParselyMetadataService creates a new Mongo and > > Jedis > > > instance which the EvalFunc queries using a public method fetch(). > > > Instance of ParselyMetadataService also have a close() function which > > > simply calls: > > > > > > this.mongoClient.close(); > > > this.redisClient.disconnect(); > > > > > > > > > On Thu, Mar 14, 2013 at 5:24 PM, Jonathan Coveney <jcove...@gmail.com > > >wrote: > > > > > >> Can you perhaps share more of your implementation? I can imagine a > > couple > > >> of things which would cause errors like this. Are you making sure that > > >> each > > >> instance of EvalFunc is dealing with a different connection? > > >> > > >> That's what I'd take a look at first...if that isn't the issue, I can > > look > > >> into how final() is implemented. > > >> > > >> > > >> 2013/3/14 Mike Sukmanowsky <m...@parsely.com> > > >> > > >> > Hi there, > > >> > > > >> > I have an EvalFunc which uses an internal class that opens up > > >> connections > > >> > to a Redis and MongoDB server. This class has a close() method > which > > >> > closes connections to both Redis and MongoDB. > > >> > > > >> > In my EvalFunc implementation, I placed the close() method > > >> > > > >> > @Override > > >> > public void finish() { > > >> > meta.close(); > > >> > } > > >> > > > >> > But this is leading to a > > >> > > > >> > java.lang.IllegalStateException: this Mongo has been closed > > >> > > > >> > indicating that Pig attempted to read from Mongo after a call to > > >> finish(). > > >> > Is this an error specific to running a Pig script locally? > > >> > > > >> > Mike > > >> > > > >> > -- > > >> > Mike Sukmanowsky > > >> > > > >> > Product Lead, http://parse.ly > > >> > 989 Avenue of the Americas, 3rd Floor > > >> > New York, NY 10018 > > >> > p: +1 (416) 953-4248 > > >> > e: m...@parsely.com > > >> > > > >> > > > > > > > > > > > > -- > > > Mike Sukmanowsky > > > > > > Product Lead, http://parse.ly > > > 989 Avenue of the Americas, 3rd Floor > > > New York, NY 10018 > > > p: +1 (416) 953-4248 > > > e: m...@parsely.com > > > > > > > > > > > -- > > Mike Sukmanowsky > > > > Product Lead, http://parse.ly > > 989 Avenue of the Americas, 3rd Floor > > New York, NY 10018 > > p: +1 (416) 953-4248 > > e: m...@parsely.com > > >
