Re: 8.0 Java API to 7.6.4 server - update records issue
John, The only problem with that logic is that the server has always been in control of the create/modify date/time. To verify your scenario, you could quickly swap out the api that your java program is running for a 7.6.04 SP4, or even an 8.1 jar file for testing On Wed, Jun 12, 2013 at 9:50 AM, John Sundberg john.sundb...@kineticdata.com wrote: ** We have recently seen a sporadic issue with the 8.0 Java API against a 7.6.4 server… The issue is -- a record is created at 2013-06-10T12:00:00 … (call this 123456789) However when the record gets updated… The log shows update T1 where c4 = 123456788 …… (Notice - the one second less than the create time) So I think what is happening is that the client library is somehow losing a second (off by one error???) on the update. So - the update fails with This record has been updated since you last touched it… However -- if you drop in the 7.6.4 client library -- it works fine. So … 8.0 client bug ??? or something else? -John -- *John Sundberg* Kinetic Data, Inc. Your Business. Your Process. 651-556-0930 I john.sundb...@kineticdata.com www.kineticdata.com I community.kineticdata.com _ARSlist: Where the Answers Are and have been for 20 years_ ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Where the Answers Are, and have been for 20 years
Re: 8.0 Java API to 7.6.4 server - update records issue
LJ -- agreed. And - since I posted -- the problem showed up with client 7.6.4 libraries too. (Just took awhile) But -- on close inspection -- the time stamp used on update is one second older than the timestamp on create. So -- definitely funky. (I would call it a server bug -- but I can't be sure) So -- 7.6.4 patch 2 server… reading the 3,4 update logs -- nothing jumps out as changing on the server that would fix this. May just update anyway -- maybe a change happened in the code - but did not make it into the release notes. -John On Wed, Jun 12, 2013 at 1:29 PM, Longwing, Lj llongw...@usgs.gov wrote: ** John, The only problem with that logic is that the server has always been in control of the create/modify date/time. To verify your scenario, you could quickly swap out the api that your java program is running for a 7.6.04 SP4, or even an 8.1 jar file for testing On Wed, Jun 12, 2013 at 9:50 AM, John Sundberg john.sundb...@kineticdata.com wrote: ** We have recently seen a sporadic issue with the 8.0 Java API against a 7.6.4 server… The issue is -- a record is created at 2013-06-10T12:00:00 … (call this 123456789) However when the record gets updated… The log shows update T1 where c4 = 123456788 …… (Notice - the one second less than the create time) So I think what is happening is that the client library is somehow losing a second (off by one error???) on the update. So - the update fails with This record has been updated since you last touched it… However -- if you drop in the 7.6.4 client library -- it works fine. So … 8.0 client bug ??? or something else? -John -- *John Sundberg* Kinetic Data, Inc. Your Business. Your Process. 651-556-0930 I john.sundb...@kineticdata.com www.kineticdata.com I community.kineticdata.com _ARSlist: Where the Answers Are and have been for 20 years_ _ARSlist: Where the Answers Are and have been for 20 years_ -- *John Sundberg* Kinetic Data, Inc. Your Business. Your Process. 651-556-0930 I john.sundb...@kineticdata.com www.kineticdata.com I community.kineticdata.com ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Where the Answers Are, and have been for 20 years
Re: 8.0 Java API to 7.6.4 server - update records issue
NO, they would NEVER put something into the code base that they don't document and put in the release notes, would they? Yes...I would certainly recommend getting to SP4...it is by all means the most stable of all of the 7.6 releases. On Wed, Jun 12, 2013 at 12:36 PM, John Sundberg john.sundb...@kineticdata.com wrote: ** LJ -- agreed. And - since I posted -- the problem showed up with client 7.6.4 libraries too. (Just took awhile) But -- on close inspection -- the time stamp used on update is one second older than the timestamp on create. So -- definitely funky. (I would call it a server bug -- but I can't be sure) So -- 7.6.4 patch 2 server… reading the 3,4 update logs -- nothing jumps out as changing on the server that would fix this. May just update anyway -- maybe a change happened in the code - but did not make it into the release notes. -John On Wed, Jun 12, 2013 at 1:29 PM, Longwing, Lj llongw...@usgs.gov wrote: ** John, The only problem with that logic is that the server has always been in control of the create/modify date/time. To verify your scenario, you could quickly swap out the api that your java program is running for a 7.6.04 SP4, or even an 8.1 jar file for testing On Wed, Jun 12, 2013 at 9:50 AM, John Sundberg john.sundb...@kineticdata.com wrote: ** We have recently seen a sporadic issue with the 8.0 Java API against a 7.6.4 server… The issue is -- a record is created at 2013-06-10T12:00:00 … (call this 123456789) However when the record gets updated… The log shows update T1 where c4 = 123456788 …… (Notice - the one second less than the create time) So I think what is happening is that the client library is somehow losing a second (off by one error???) on the update. So - the update fails with This record has been updated since you last touched it… However -- if you drop in the 7.6.4 client library -- it works fine. So … 8.0 client bug ??? or something else? -John -- *John Sundberg* Kinetic Data, Inc. Your Business. Your Process. 651-556-0930 I john.sundb...@kineticdata.com www.kineticdata.com I community.kineticdata.com _ARSlist: Where the Answers Are and have been for 20 years_ _ARSlist: Where the Answers Are and have been for 20 years_ -- *John Sundberg* Kinetic Data, Inc. Your Business. Your Process. 651-556-0930 I john.sundb...@kineticdata.com www.kineticdata.com I community.kineticdata.com _ARSlist: Where the Answers Are and have been for 20 years_ ___ UNSUBSCRIBE or access ARSlist Archives at www.arslist.org Where the Answers Are, and have been for 20 years
Re: 8.0 Java API to 7.6.4 server - update records issue
First, let's review what the logic of the system is. Create-date is completely uninvolved in the discussion here. Modified-date is the date that is significant. What occurs is the following: A record is created. The create-date and modified-date should be identical because the time of create and the time of last modify is the same at this point. A record may be modified, in that case, the create-date is unchanged, but the modified-date is updated to reflect the new date. This defines the records we may be interacting with. There is no difference in handling regardless of which scenario above was used. Now, we get to the logic around updates. The server gets an update request. It has no clue about when you may have last touched it or when you retrieved it because there is no state information in the server. There is a parameter to the API that the client program sets to indicate when did I retrieve this record. THAT value is the one the server uses in its check. It tests the value that the client gave it to see if the record was changed since that date. It checks the Modified-date (C4 is the Modified-date, C3 is the Create-date) in the test. Now, in the current version, it also tests the Last-modified-by field and if it is the same user, it will not return the error (because YOU are the same user so it is not modified by a different user). There is also ignoring of things like AR_ESCALATOR or maybe configurable ignoring of that user I think so you don't get warnings about things escalations are the one who changed but I am not sure about that and that is not the focus of this explanation. Now, I thought that the user test was also in 7.6.04 but I cannot remember for sure what version it was added in. Again, just clarification, not critical for this explanation. You can specify a time of 0 for when it was retrieved to indicate you don't want the test for whether changed run at all and just to modify the record without testing. In the clients supplied by BMC, we either put 0 for that time if we didn't retrieve them or we put the retrieve time that we got from the GetEntry call (there is a time of retrieval in that call that we use as that ensures we are using SERVER timestamp and not client timestamp to eliminate issues with clock drift between client and server). So, this is how everything is designed to work. Now, I have not worked directly with the API for a while and am more familiar with the C API where this time value is an explicit parameter to the call. I am not sure how it is set in the Java API. Scenario 1 - Is your code creating and then modifying the entry? Where do you get the timestamp you are using to pass to the modify call for when the entry was retrieved? Since you are not doing a get of the entry, the server cannot be giving you a server timestamp, are you using a client timestamp of when you created it and if the server clock is a second or two different (ahead) or the entry is created over the boundary of a second since you saved the timestamp You can see the problem you are having. Scenario 2 - Is your code getting the entry that was created by someone else and then updating it? If so, what timestamp are you using for the retrieved time? Are you using the client time or a time returned from the get API call? see above for issues if it is the client time. Scenario 3 - Are you worried about whether the entry has been changed by someone else since the time you retrieved it? If not, why aren't you setting the retrieved time to 0 to eliminate the test that is done on the modify as you don't need the test run, you are just modifying without checking if someone else has changed the record. We have seen absolutely no issue with any program that is using the Java API correctly in this area. We have hundreds of customers using 8.0 and 8.1 programs and mid-tiers against 7.6.04 (and 7.6.03 and 7.6 and 7.5 and ...) servers and none have reported an issue in this area. And, as you have found, it isn't about the 8.1 API as the issue occurs when using 7.6.04 as well. Now, there is always a chance that no other customer has encountered an issue with any of their programs and there is something wrong in the code. It is software and you can never discount things. But, the logic around this area has been stable for many releases and there have been no reports of problems. Do you fall into one of the 3 scenarios I noted above? Should you be a scenario 3? But, regardless, it is modified-date that is involved so create-date is a distraction. And, one second off is no kind of an off by one error within the API/server as there is never any manipulation of the date, just what date and from where is used. There is no processing of the data value itself. I hope this helps explain the process that the system uses and points to some ideas for solution within your code. Doug Mueller From: Action Request System discussion
Re: 8.0 Java API to 7.6.4 server - update records issue
Good info. I referenced create_date -- since that is what we see in the log -- and just assumed last_mod would be the same. We are not really setting/dealing with the last_modified date/time -- however I suspect the Java API holds that data internally in the EntryObject… so when we update the EntryObject with our new field name/value pairs I suspect the last_modified date/time is contained in there and is passed along. The fundamental thing we are looking at is -- is our problem client or server side? With the info you gave - I suspect client side -- since no changes or manipulation or state info is stored server side. (We were sort of wondering -- since we never explicitly set the last_mod date in our update call. So - did not even know where that was coming from) We have had lots of transactions happening with this code -- however - have only seen this issue at this one customer and the odd thing was the 8.0 client api and 7.6.4 sp2 server. So -- sort of jumped to that area. Also -- replaced with 7.6.4 client -- and it did not come up again (but eventually did). So -- throws a wrench into that thought :) However -- the fact that nobody else said me too … is also helpful. Will do some more digging - and return the findings… -John On Wed, Jun 12, 2013 at 2:21 PM, Mueller, Doug doug_muel...@bmc.com wrote: ** First, let's review what the logic of the system is. ** ** Create-date is completely uninvolved in the discussion here. Modified-date is the date that is significant. ** ** What occurs is the following: ** ** A record is created. The create-date and modified-date should be identical because the time of create and the time of last modify is the same at this point. ** ** A record may be modified, in that case, the create-date is unchanged, but the modified-date is updated to reflect the new date. ** ** This defines the records we may be interacting with. There is no difference in handling regardless of which scenario above was used. ** ** Now, we get to the logic around updates. ** ** The server gets an update request. It has no clue about when you may have last touched it or when you retrieved it because there is no state information in the server. There is a parameter to the API that the client program sets to indicate when did I retrieve this record. THAT value is the one the server uses in its check. It tests the value that the client gave it to see if the record was changed since that date. It checks the Modified-date (C4 is the Modified-date, C3 is the Create-date) in the test. ** ** Now, in the current version, it also tests the Last-modified-by field and if it is the same user, it will not return the error (because YOU are the same user so it is not modified by a different user). There is also ignoring of things like AR_ESCALATOR or maybe configurable ignoring of that user I think so you don't get warnings about things escalations are the one who changed but I am not sure about that and that is not the focus of this explanation. Now, I thought that the user test was also in 7.6.04 but I cannot remember for sure what version it was added in. Again, just clarification, not critical for this explanation. ** ** You can specify a time of 0 for when it was retrieved to indicate you don't want the test for whether changed run at all and just to modify the record without testing. ** ** In the clients supplied by BMC, we either put 0 for that time if we didn't retrieve them or we put the retrieve time that we got from the GetEntry call (there is a time of retrieval in that call that we use as that ensures we are using SERVER timestamp and not client timestamp to eliminate issues with clock drift between client and server). ** ** So, this is how everything is designed to work. ** ** ** ** Now, I have not worked directly with the API for a while and am more familiar with the C API where this time value is an explicit parameter to the call. I am not sure how it is set in the Java API. ** ** ** ** Scenario 1 – Is your code creating and then modifying the entry? Where do you get the timestamp you are using to pass to the modify call for when the entry was retrieved? Since you are not doing a get of the entry, the server cannot be giving you a server timestamp, are you using a client timestamp of when you created it and if the server clock is a second or two different (ahead) or the entry is created over the boundary of a second since you saved the timestamp…. You can see the problem you are having. ** ** Scenario 2 – Is your code getting the entry that was created by someone else and then updating it? If so, what timestamp are you using for the retrieved time? Are you using the client time or a time
Re: 8.0 Java API to 7.6.4 server - update records issue
John, You may try creating a new 'Entry' object for the update instead of using the same one from the Create? On Wed, Jun 12, 2013 at 1:53 PM, John Sundberg john.sundb...@kineticdata.com wrote: ** Good info. I referenced create_date -- since that is what we see in the log -- and just assumed last_mod would be the same. We are not really setting/dealing with the last_modified date/time -- however I suspect the Java API holds that data internally in the EntryObject… so when we update the EntryObject with our new field name/value pairs I suspect the last_modified date/time is contained in there and is passed along. The fundamental thing we are looking at is -- is our problem client or server side? With the info you gave - I suspect client side -- since no changes or manipulation or state info is stored server side. (We were sort of wondering -- since we never explicitly set the last_mod date in our update call. So - did not even know where that was coming from) We have had lots of transactions happening with this code -- however - have only seen this issue at this one customer and the odd thing was the 8.0 client api and 7.6.4 sp2 server. So -- sort of jumped to that area. Also -- replaced with 7.6.4 client -- and it did not come up again (but eventually did). So -- throws a wrench into that thought :) However -- the fact that nobody else said me too … is also helpful. Will do some more digging - and return the findings… -John On Wed, Jun 12, 2013 at 2:21 PM, Mueller, Doug doug_muel...@bmc.comwrote: ** First, let's review what the logic of the system is. ** ** Create-date is completely uninvolved in the discussion here. Modified-date is the date that is significant. ** ** What occurs is the following: ** ** A record is created. The create-date and modified-date should be identical because the time of create and the time of last modify is the same at this point. ** ** A record may be modified, in that case, the create-date is unchanged, but the modified-date is updated to reflect the new date. ** ** This defines the records we may be interacting with. There is no difference in handling regardless of which scenario above was used. ** ** Now, we get to the logic around updates. ** ** The server gets an update request. It has no clue about when you may have last touched it or when you retrieved it because there is no state information in the server. There is a parameter to the API that the client program sets to indicate when did I retrieve this record. THAT value is the one the server uses in its check. It tests the value that the client gave it to see if the record was changed since that date. It checks the Modified-date (C4 is the Modified-date, C3 is the Create-date) in the test. ** ** Now, in the current version, it also tests the Last-modified-by field and if it is the same user, it will not return the error (because YOU are the same user so it is not modified by a different user). There is also ignoring of things like AR_ESCALATOR or maybe configurable ignoring of that user I think so you don't get warnings about things escalations are the one who changed but I am not sure about that and that is not the focus of this explanation. Now, I thought that the user test was also in 7.6.04 but I cannot remember for sure what version it was added in. Again, just clarification, not critical for this explanation. ** ** You can specify a time of 0 for when it was retrieved to indicate you don't want the test for whether changed run at all and just to modify the record without testing. ** ** In the clients supplied by BMC, we either put 0 for that time if we didn't retrieve them or we put the retrieve time that we got from the GetEntry call (there is a time of retrieval in that call that we use as that ensures we are using SERVER timestamp and not client timestamp to eliminate issues with clock drift between client and server). ** ** So, this is how everything is designed to work. ** ** ** ** Now, I have not worked directly with the API for a while and am more familiar with the C API where this time value is an explicit parameter to the call. I am not sure how it is set in the Java API. ** ** ** ** Scenario 1 – Is your code creating and then modifying the entry? Where do you get the timestamp you are using to pass to the modify call for when the entry was retrieved? Since you are not doing a get of the entry, the server cannot be giving you a server timestamp, are you using a client timestamp of when you created it and if the server clock is a second or two different (ahead) or the entry is created over the boundary of a second since you saved the timestamp…. You can see the problem you are having.
Re: 8.0 Java API to 7.6.4 server - update records issue
Yes - that is a good idea -- probably a few ways to work around it… However - been using this code for years -- all the sudden pops up -- weird. I will write back with the findings. Thanks for the help/comments. -John On Wed, Jun 12, 2013 at 2:59 PM, Longwing, Lj llongw...@usgs.gov wrote: ** John, You may try creating a new 'Entry' object for the update instead of using the same one from the Create? On Wed, Jun 12, 2013 at 1:53 PM, John Sundberg john.sundb...@kineticdata.com wrote: ** Good info. I referenced create_date -- since that is what we see in the log -- and just assumed last_mod would be the same. We are not really setting/dealing with the last_modified date/time -- however I suspect the Java API holds that data internally in the EntryObject… so when we update the EntryObject with our new field name/value pairs I suspect the last_modified date/time is contained in there and is passed along. The fundamental thing we are looking at is -- is our problem client or server side? With the info you gave - I suspect client side -- since no changes or manipulation or state info is stored server side. (We were sort of wondering -- since we never explicitly set the last_mod date in our update call. So - did not even know where that was coming from) We have had lots of transactions happening with this code -- however - have only seen this issue at this one customer and the odd thing was the 8.0 client api and 7.6.4 sp2 server. So -- sort of jumped to that area. Also -- replaced with 7.6.4 client -- and it did not come up again (but eventually did). So -- throws a wrench into that thought :) However -- the fact that nobody else said me too … is also helpful. Will do some more digging - and return the findings… -John On Wed, Jun 12, 2013 at 2:21 PM, Mueller, Doug doug_muel...@bmc.comwrote: ** First, let's review what the logic of the system is. ** ** Create-date is completely uninvolved in the discussion here. Modified-date is the date that is significant. ** ** What occurs is the following: ** ** A record is created. The create-date and modified-date should be identical because the time of create and the time of last modify is the same at this point. ** ** A record may be modified, in that case, the create-date is unchanged, but the modified-date is updated to reflect the new date. ** ** This defines the records we may be interacting with. There is no difference in handling regardless of which scenario above was used. ** ** Now, we get to the logic around updates. ** ** The server gets an update request. It has no clue about when you may have last touched it or when you retrieved it because there is no state information in the server. There is a parameter to the API that the client program sets to indicate when did I retrieve this record. THAT value is the one the server uses in its check. It tests the value that the client gave it to see if the record was changed since that date. It checks the Modified-date (C4 is the Modified-date, C3 is the Create-date) in the test. ** ** Now, in the current version, it also tests the Last-modified-by field and if it is the same user, it will not return the error (because YOU are the same user so it is not modified by a different user). There is also ignoring of things like AR_ESCALATOR or maybe configurable ignoring of that user I think so you don't get warnings about things escalations are the one who changed but I am not sure about that and that is not the focus of this explanation. Now, I thought that the user test was also in 7.6.04 but I cannot remember for sure what version it was added in. Again, just clarification, not critical for this explanation. ** ** You can specify a time of 0 for when it was retrieved to indicate you don't want the test for whether changed run at all and just to modify the record without testing. ** ** In the clients supplied by BMC, we either put 0 for that time if we didn't retrieve them or we put the retrieve time that we got from the GetEntry call (there is a time of retrieval in that call that we use as that ensures we are using SERVER timestamp and not client timestamp to eliminate issues with clock drift between client and server). ** ** So, this is how everything is designed to work. ** ** ** ** Now, I have not worked directly with the API for a while and am more familiar with the C API where this time value is an explicit parameter to the call. I am not sure how it is set in the Java API. ** ** ** ** Scenario 1 – Is your code creating and then modifying the entry? Where do you get the timestamp you are using to pass to the modify call for when the entry was retrieved? Since you are not doing a get of the entry, the
Re: 8.0 Java API to 7.6.4 server - update records issue
Update -- solved. We changed the api code to use the timestamp from the last_modified_date field and all is well. We were not taking advantage of the api call where you can set the last_modified_date field. Doug - your pointers were great -- thanks. -John On Wed, Jun 12, 2013 at 3:12 PM, John Sundberg john.sundb...@kineticdata.com wrote: Yes - that is a good idea -- probably a few ways to work around it… However - been using this code for years -- all the sudden pops up -- weird. I will write back with the findings. Thanks for the help/comments. -John On Wed, Jun 12, 2013 at 2:59 PM, Longwing, Lj llongw...@usgs.gov wrote: ** John, You may try creating a new 'Entry' object for the update instead of using the same one from the Create? On Wed, Jun 12, 2013 at 1:53 PM, John Sundberg john.sundb...@kineticdata.com wrote: ** Good info. I referenced create_date -- since that is what we see in the log -- and just assumed last_mod would be the same. We are not really setting/dealing with the last_modified date/time -- however I suspect the Java API holds that data internally in the EntryObject… so when we update the EntryObject with our new field name/value pairs I suspect the last_modified date/time is contained in there and is passed along. The fundamental thing we are looking at is -- is our problem client or server side? With the info you gave - I suspect client side -- since no changes or manipulation or state info is stored server side. (We were sort of wondering -- since we never explicitly set the last_mod date in our update call. So - did not even know where that was coming from) We have had lots of transactions happening with this code -- however - have only seen this issue at this one customer and the odd thing was the 8.0 client api and 7.6.4 sp2 server. So -- sort of jumped to that area. Also -- replaced with 7.6.4 client -- and it did not come up again (but eventually did). So -- throws a wrench into that thought :) However -- the fact that nobody else said me too … is also helpful. Will do some more digging - and return the findings… -John On Wed, Jun 12, 2013 at 2:21 PM, Mueller, Doug doug_muel...@bmc.comwrote: ** First, let's review what the logic of the system is. ** ** Create-date is completely uninvolved in the discussion here. Modified-date is the date that is significant. ** ** What occurs is the following: ** ** A record is created. The create-date and modified-date should be identical because the time of create and the time of last modify is the same at this point. ** ** A record may be modified, in that case, the create-date is unchanged, but the modified-date is updated to reflect the new date. ** ** This defines the records we may be interacting with. There is no difference in handling regardless of which scenario above was used. ** ** Now, we get to the logic around updates. ** ** The server gets an update request. It has no clue about when you may have last touched it or when you retrieved it because there is no state information in the server. There is a parameter to the API that the client program sets to indicate when did I retrieve this record. THAT value is the one the server uses in its check. It tests the value that the client gave it to see if the record was changed since that date. It checks the Modified-date (C4 is the Modified-date, C3 is the Create-date) in the test. ** ** Now, in the current version, it also tests the Last-modified-by field and if it is the same user, it will not return the error (because YOU are the same user so it is not modified by a different user). There is also ignoring of things like AR_ESCALATOR or maybe configurable ignoring of that user I think so you don't get warnings about things escalations are the one who changed but I am not sure about that and that is not the focus of this explanation. Now, I thought that the user test was also in 7.6.04 but I cannot remember for sure what version it was added in. Again, just clarification, not critical for this explanation. ** ** You can specify a time of 0 for when it was retrieved to indicate you don't want the test for whether changed run at all and just to modify the record without testing. ** ** In the clients supplied by BMC, we either put 0 for that time if we didn't retrieve them or we put the retrieve time that we got from the GetEntry call (there is a time of retrieval in that call that we use as that ensures we are using SERVER timestamp and not client timestamp to eliminate issues with clock drift between client and server). ** ** So, this is how everything is designed to work. ** ** ** ** Now, I have not worked directly with the API for a while and am more familiar with the C API where this
