Re: GEODE-2714 Proposal for new api on Function interface (Please read)
Hey guys, take the conversation over to the ticket so context is maintained with the ticket. On Fri, Mar 24, 2017 at 3:16 PM Swapnil Bawaskar wrote: > For step 2 above, won't we end up recomputing the list of buckets? > > I agree, however, that we do have the problem of moving primary, > especially in the middle of iterating over data. We already have an > optimizeForWrite() method on Function that makes the function execute on > the primary. Do we want to make locking part of this method, rather than > introducing a new method? > > On Fri, Mar 24, 2017 at 2:10 PM Hitesh Khamesra > wrote: > > > Hi Swapnil: > > > > Right, and then we execute that function on that node. Now assume during > > that function execution time(first user thread) two things happens > > simultaneously.. > > > > 1. we move that bucket to other node. > > 2. And other user thread execute function with same set of keys on other > > node, which first thread is still executing. > > > > Basically we need to make sure while function execution we don't move the > > primary bucket. > > > > Thanks. > > Hitesh > > -- > > *From:* Swapnil Bawaskar > > *To:* dev@geode.apache.org; Hitesh Khamesra > > *Sent:* Friday, March 24, 2017 1:55 PM > > *Subject:* Re: GEODE-2714 Proposal for new api on Function interface > > (Please read) > > > > Here is the link: https://issues.apache.org/jira/browse/GEODE-2714 > > > > Hi Hitesh, > > > > Before executing a function, we build a list of buckets that the function > > needs to execute on and then we build the function context using only > those > > buckets. So, as long as the function is using the context to get the > local > > data set, it should not see the data twice, even if rebalance was in > > progress. > > > > When you saw this behavior, was the function using the function context? > > > > Thanks! > > > > > > On Fri, Mar 24, 2017 at 1:21 PM Jared Stewart > wrote: > > > > Can you give a link to the proposal? > > > > Thanks, > > Jared > > > On Mar 24, 2017, at 12:57 PM, Hitesh Khamesra > > wrote: > > > > > > Please let us know your opinion on that. > > > Thanks.Hitesh > > > > > > > > >
Re: GEODE-2714 Proposal for new api on Function interface (Please read)
For step 2 above, won't we end up recomputing the list of buckets? I agree, however, that we do have the problem of moving primary, especially in the middle of iterating over data. We already have an optimizeForWrite() method on Function that makes the function execute on the primary. Do we want to make locking part of this method, rather than introducing a new method? On Fri, Mar 24, 2017 at 2:10 PM Hitesh Khamesra wrote: > Hi Swapnil: > > Right, and then we execute that function on that node. Now assume during > that function execution time(first user thread) two things happens > simultaneously.. > > 1. we move that bucket to other node. > 2. And other user thread execute function with same set of keys on other > node, which first thread is still executing. > > Basically we need to make sure while function execution we don't move the > primary bucket. > > Thanks. > Hitesh > -- > *From:* Swapnil Bawaskar > *To:* dev@geode.apache.org; Hitesh Khamesra > *Sent:* Friday, March 24, 2017 1:55 PM > *Subject:* Re: GEODE-2714 Proposal for new api on Function interface > (Please read) > > Here is the link: https://issues.apache.org/jira/browse/GEODE-2714 > > Hi Hitesh, > > Before executing a function, we build a list of buckets that the function > needs to execute on and then we build the function context using only those > buckets. So, as long as the function is using the context to get the local > data set, it should not see the data twice, even if rebalance was in > progress. > > When you saw this behavior, was the function using the function context? > > Thanks! > > > On Fri, Mar 24, 2017 at 1:21 PM Jared Stewart wrote: > > Can you give a link to the proposal? > > Thanks, > Jared > > On Mar 24, 2017, at 12:57 PM, Hitesh Khamesra > wrote: > > > > Please let us know your opinion on that. > > Thanks.Hitesh > > > >
Re: GEODE-2714 Proposal for new api on Function interface (Please read)
Hi Swapnil: Right, and then we execute that function on that node. Now assume during that function execution time(first user thread) two things happens simultaneously.. 1. we move that bucket to other node. 2. And other user thread execute function with same set of keys on other node, which first thread is still executing. Basically we need to make sure while function execution we don't move the primary bucket. Thanks.Hitesh From: Swapnil Bawaskar To: dev@geode.apache.org; Hitesh Khamesra Sent: Friday, March 24, 2017 1:55 PM Subject: Re: GEODE-2714 Proposal for new api on Function interface (Please read) Here is the link: https://issues.apache.org/jira/browse/GEODE-2714 Hi Hitesh, Before executing a function, we build a list of buckets that the function needs to execute on and then we build the function context using only those buckets. So, as long as the function is using the context to get the local data set, it should not see the data twice, even if rebalance was in progress. When you saw this behavior, was the function using the function context? Thanks! On Fri, Mar 24, 2017 at 1:21 PM Jared Stewart wrote: Can you give a link to the proposal? Thanks, Jared > On Mar 24, 2017, at 12:57 PM, Hitesh Khamesra > wrote: > > Please let us know your opinion on that. > Thanks.Hitesh
Re: GEODE-2714 Proposal for new api on Function interface (Please read)
Here is the link: https://issues.apache.org/jira/browse/GEODE-2714 Hi Hitesh, Before executing a function, we build a list of buckets that the function needs to execute on and then we build the function context using only those buckets. So, as long as the function is using the context to get the local data set, it should not see the data twice, even if rebalance was in progress. When you saw this behavior, was the function using the function context? Thanks! On Fri, Mar 24, 2017 at 1:21 PM Jared Stewart wrote: > Can you give a link to the proposal? > > Thanks, > Jared > > On Mar 24, 2017, at 12:57 PM, Hitesh Khamesra > wrote: > > > > Please let us know your opinion on that. > > Thanks.Hitesh > >
Re: GEODE-2714 Proposal for new api on Function interface (Please read)
Can you give a link to the proposal? Thanks, Jared > On Mar 24, 2017, at 12:57 PM, Hitesh Khamesra > wrote: > > Please let us know your opinion on that. > Thanks.Hitesh