Re: Does it matter if you lock an IP object before updating?
it (simple assignment) can be sliced and that's what make preemptive multi-threading in v15r5/16 at the 4D language level fundamentally different to "protecting IP arrays using semaphores the classic way". you might recall the slide presented at the last Summit by Laurent Esnault http://www.slideshare.net/kmiyako/inside-multithreading Japanese version, but you get the picture > 2016/11/02 8:52、Tim Nevels のメール： > It does not contain any 4D commands. Can that line of code be time sliced, or > will this line of code run atomically? 宮古 啓介 セールス・エンジニア 株式会社フォーディー・ジャパン 〒150-0043 東京都渋谷区道玄坂1-10-2 渋谷THビル6F Tel: 03-6427-8441 Fax: 03-6427-8449 keisuke.miy...@4d.com www.4D.com/JP ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
I’m just guessing, but if you think about how that line of code would look in assembly wouldn’t it be a few different instructions? Moving a value into a memory, incrementing it, and moving it back? If so, that give the opportunity for it to be time sliced by the OS. -- Cannon Smith Synergy Farm Solutions Inc. Hill Spring, AB Canada 403-626-3236 > On Nov 1, 2016, at 5:52 PM, Tim Nevels wrote: > > But what about this: > > <>counter_l := <>counter_l + 1 > > It does not contain any 4D commands. Can that line of code be time sliced, or > will this line of code run atomically? ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
On Nov 1, 2016, at 6:00 PM, Douglas von Roeder wrote: > On Tue, Nov 1, 2016 at 2:57 PM, David Adams wrote: > >> For what it's worth, I was working at 4D in Cupertino when 4D Server >> shipped. We had big arguments in the US office about IP communications and >> semaphores. (Most people really don't love semaphores.) Semaphores turned >> out to be the one and only reliable mechanism available. Unless something >> has changed, that's still going to be true. > > LR addressed that question directly at a Summit - don't know the year but > it was post-Schaumburg. The only command not susceptible to timeslicing is > Semaphore. OK I can accept that. But what about this: <>counter_l := <>counter_l + 1 It does not contain any 4D commands. Can that line of code be time sliced, or will this line of code run atomically? Tim Tim Nevels Innovative Solutions 785-749-3444 timnev...@mac.com ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
De nada. ;-) -- Douglas von Roeder 949-336-2902 On Tue, Nov 1, 2016 at 4:28 PM, David Adams wrote: > On Wed, Nov 2, 2016 at 10:12 AM, Douglas von Roeder > wrote: > > > > Insofar a not Semaphoring shared resources, that'll work fine until it > > doesn't. Unfortunately, when it doesn't, tracking it could be, > essentially, > > impossible. > > > > Thanks to Doug for saying in a few words what I said in a few pages ;-) > ** > 4D Internet Users Group (4D iNUG) > FAQ: http://lists.4d.com/faqnug.html > Archive: http://lists.4d.com/archives.html > Options: http://lists.4d.com/mailman/options/4d_tech > Unsub: mailto:4d_tech-unsubscr...@lists.4d.com > ** > ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
On Wed, Nov 2, 2016 at 10:12 AM, Douglas von Roeder wrote: > Insofar a not Semaphoring shared resources, that'll work fine until it > doesn't. Unfortunately, when it doesn't, tracking it could be, essentially, > impossible. > Thanks to Doug for saying in a few words what I said in a few pages ;-) ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
> Unless the name of the variable means what it says. :-7 D'oh! Well, in my code I always want to leave the youngster's guessing ;-) On Wed, Nov 2, 2016 at 9:43 AM, Tom Dillon wrote: > David Adams wrote: > > >Say you have an IP variable like > ><>TAB and you assign in Char(13) more than once. No harm. > > Unless the name of the variable means what it says. :-7 > > -- >-- >Tom Dillon 825 N. 500 W. >DataCraft Moab, UT 84532 >tomdil...@datacraft-inc.com 720/209-6502 >-- > Maybe we only think we have consciousness. --- Sunastar >-- > > > ** > 4D Internet Users Group (4D iNUG) > FAQ: http://lists.4d.com/faqnug.html > Archive: http://lists.4d.com/archives.html > Options: http://lists.4d.com/mailman/options/4d_tech > Unsub: mailto:4d_tech-unsubscr...@lists.4d.com > ** > ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
Jeffrey: That was how I took it. There was a NUG thread on atomicity some time back. Perhaps there's some insight in that thread? Insofar a not Semaphoring shared resources, that'll work fine until it doesn't. Unfortunately, when it doesn't, tracking it could be, essentially, impossible. -- Douglas von Roeder 949-336-2902 On Tue, Nov 1, 2016 at 3:19 PM, Jeffrey Kain wrote: > Well, probably the only command GUARANTEED to be atomic is Semaphore. > Other commands may happen to be in the current version, or were in previous > versions... but Semaphore always has been atomic and always will be. > > > On Nov 1, 2016, at 6:07 PM, Douglas von Roeder > wrote: > > > > LR addressed that question directly at a Summit - don't know the year but > > it was post-Schaumburg. The only command not susceptible to timeslicing > is > > Semaphore. > > ** > 4D Internet Users Group (4D iNUG) > FAQ: http://lists.4d.com/faqnug.html > Archive: http://lists.4d.com/archives.html > Options: http://lists.4d.com/mailman/options/4d_tech > Unsub: mailto:4d_tech-unsubscr...@lists.4d.com > ** > ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
David Adams wrote: >Say you have an IP variable like ><>TAB and you assign in Char(13) more than once. No harm. Unless the name of the variable means what it says. :-7 -- -- Tom Dillon 825 N. 500 W. DataCraft Moab, UT 84532 tomdil...@datacraft-inc.com 720/209-6502 -- Maybe we only think we have consciousness. --- Sunastar -- ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
Well, probably the only command GUARANTEED to be atomic is Semaphore. Other commands may happen to be in the current version, or were in previous versions... but Semaphore always has been atomic and always will be. > On Nov 1, 2016, at 6:07 PM, Douglas von Roeder wrote: > > LR addressed that question directly at a Summit - don't know the year but > it was post-Schaumburg. The only command not susceptible to timeslicing is > Semaphore. ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
On Tue, Nov 1, 2016 at 2:57 PM, David Adams wrote: > For what it's worth, I was working at 4D in Cupertino when 4D Server > shipped. We had big arguments in the US office about IP communications and > semaphores. (Most people really don't love semaphores.) Semaphores turned > out to be the one and only reliable mechanism available. Unless something > has changed, that's still going to be true. > LR addressed that question directly at a Summit - don't know the year but it was post-Schaumburg. The only command not susceptible to timeslicing is Semaphore. -- Douglas von Roeder 949-336-2902 ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
> I’m not sure I agree that the ONLY 4D command in the language that is atomic is “Semaphore”. > I’m guessing there are others. Gonna need a 4D engineer like Laurent Esnault that knows the > deep internals of 4D’s code to know for sure. For what it's worth, I was working at 4D in Cupertino when 4D Server shipped. We had big arguments in the US office about IP communications and semaphores. (Most people really don't love semaphores.) Semaphores turned out to be the one and only reliable mechanism available. Unless something has changed, that's still going to be true. I'm sure there are a narrow set of cases where not using semaphores to lock a shared read-write resource won't cause any harm. It's easy to come up with cases and I'd bet we all have some of each. Say you have an IP variable like <>TAB and you assign in Char(13) more than once. No harm. But for groups of arrays (or even single arrays), it's pretty much not save to read or write without a lock unless the arrays are static. And it's *totally* normal to have a static array. Initialize it once at startup (or on first use) and that's it. No modifications, just lookups. No elements change position or value, the array never changes size. (Some languages have a construct for this type of list, we just have arrays and how we use them.) No need for a semaphore there either. I guess the rule is: If you think that you might need a semaphore lock, you do ;-) ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
On Nov 1, 2016, at 2:00 PM, Bernd Fröhlich wrote: > Kirk Brooks wrote: > >> True, but this illustrates that without multi-threading it's still a >> sequential processor. Only one process at a time can be executing. > > Yes, BUT... > > The thing about semaphores is, that they are atomic commands (the command is > guranteed to execute without any other process interrupting it). > And semaphore is the ONLY atomic command, all the other commands can be split > in the middle by the systems scheduler. I’m not sure I agree that the ONLY 4D command in the language that is atomic is “Semaphore”. I’m guessing there are others. Gonna need a 4D engineer like Laurent Esnault that knows the deep internals of 4D’s code to know for sure. > Your example might work, as you are only adding to the array but it will > break, when you also delete array elements. > Consider this: > > Process A adds array elements to an IP array and Process B deletes elements. > Works fine, as long as the scheduler does not interrupt one of those commands. > > If the array has zero elements and process A tries to create a new one, the > sheduler might switch processes when the element is created, but before a > value is assigned. > Then process B deletes an element. > Now if it´s processes A time to continue, it tries to write the value to an > array element that does not exist anymore. BAD! > > That´s why you better protect access to IP arrays/variables with semaphores. > > It is very hard to simulate the above since lots of conditions come into play > and you MIGHT get lucky that it never happens to you, but if it DOES happen - > good luck with finding the error. The rule that I have always followed is there is no need to protect simple IP variables like real, longint, text, etc. I can’t see how reading or writing to a single, simple IP variable could cause any issues with multiple processes. I treat IP arrays differently because usually when you are manipulating an array there are multiple step involved in getting and setting values in the array. You may have to check the array size first, then you APPEND TO ARRAY or INSERT IN ARRAY. Or you do a "Find in array” and then you read or write an element in the array. The point is the read or write to the array requires more than 1 line of 4D code, and I am working under the assumption that a single line of 4D code will run atomically. Just think of the problems if “<>counter_l := <>counter_l + 1” were to be interrupted in the middle of execution. First you get the value of <>counter_l, then there is a context switch and another process changes <>counter_l, then your process resumes and it takes its own <>counter_l value it had before the context switch, adds 1, and then stores the result in <>counter_l. I might be wrong about “one line of 4D code runs atomically”. But and again you need somebody like Laurent Esnault — maybe Miyako or Thomas Maul knows — to tell you how the interpreter works. And also how the 4D compiler works. Could be compiled code acts a bit differently that interpreted at this super low level. So if I am going to work on an IP array, I protect that block of code with a semaphore. And of course if I are dealing with a group of related IP arrays it’s absolutely critical to protect them with a semaphore. You want all the operations on all the IP arrays in the group to take place as a single action. There are too many chances for a context switch with so many lines of code. I think Kirk was asking about the “<>counter_l := <>counter_l + 1” example not IP array updates. I see no reason to protect simple IP variables that are just getting a value set or retrieving a value. IP arrays are another story and the code block does need to be protected with a semaphore. Tim Tim Nevels Innovative Solutions 785-749-3444 timnev...@mac.com ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
Kirk Brooks wrote: > True, but this illustrates that without multi-threading it's still a > sequential processor. Only one process at a time can be executing. Yes, BUT... The thing about semaphores is, that they are atomic commands (the command is guranteed to execute without any other process interrupting it). And semaphore is the ONLY atomic command, all the other commands can be split in the middle by the systems scheduler. Your example might work, as you are only adding to the array but it will break, when you also delete array elements. Consider this: Process A adds array elements to an IP array and Process B deletes elements. Works fine, as long as the scheduler does not interrupt one of those commands. If the array has zero elements and process A tries to create a new one, the sheduler might switch processes when the element is created, but before a value is assigned. Then process B deletes an element. Now if it´s processes A time to continue, it tries to write the value to an array element that does not exist anymore. BAD! That´s why you better protect access to IP arrays/variables with semaphores. It is very hard to simulate the above since lots of conditions come into play and you MIGHT get lucky that it never happens to you, but if it DOES happen - good luck with finding the error. Greetings from Germany, Bernd Fröhlich ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
Just to add my 2 cents worth to this. When one writes a large system with many things happening (potentially) at the same time one needs to code extremely defensively. This means protecting against all kinds of potential problems that have the opportunity to arise. As well, trapping if they do arise and writing a log or other ways that you as the programmer (because users will not tell you) are informed of the issues that you never thought would happen (because they will). Hopefully your logs (or other means) give you enough information so you can track it down and write the code better. The alternative I lived through many years ago with my own code. As the application grows the strange behaviour grows. Tracking it down becomes near impossible. Therefore we had to go back in and write the code the way we should have written to begin with. Lots of back work, but all of us learned - hopefully. Therefore creating standards of code, standards of methods, documentation, etc that are ALWAYS done. This is so that the ‘not likely to happen’ just cannot happen as far as you know because of the defensive code you have written. Still though - being notified so you can look back at your code and the data at that site and get that Ahh moment and write the code even better. We implemented a feature set with an OEM product we had installed at many sites >100 with many 100+ concurrent users. We would collect various ‘problems’ at each site, and each night all the sites ‘called home’ and we recorded them in our local server for this feature. I could then look at these and be proactive in understanding what was happening that was a surprise. It might mean there was no problem, but we could write things better so that the issue didn’t occur. It could mean that there was a problem and we wrote our code to protect that. It could mean speed, convince, better user interface. All in all a great tool, and learned a lot about our code, coding, and user interaction with our system. I may be paranoid, but I find writing protective code a better way to rest at night, weekends, and holidays knowing that the code is written well, though it takes much longer to get it that way (why I like our shell). Version 15 and version 16 are giving us a great set of commands so that we can write more defensively than we could before. It is giving us new features that let us write things we had to jump through many hoops to accomplish and ensure got done. Part of it for us is that our shell was based on a version of 4D with version 11/12. Our new shell is taking advantage of the new versions of 4D. An exciting time for us. Jody ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
> Le 1 nov. 2016 à 06:43, Kirk Brooks a écrit : > > David, > Great points - but, respectfully, does it ever actually happen? Can you > write an example db that does this? In v5 we had the multiprocess for the 1st time. To manage my processes, I used a "process descriptor" made of 5 or 6 arrays, let's say something like this: process number | parent process number | form name | table number | record PK Each time a process was created, ended, modified, these arrays were updated. 4D was much sloower than now, and I was quite fast in clicking. I soon realized that my arrays became unsynchronized because I did not use a semaphore. -- Arnaud de Montard ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
Richard, That's essentially the type of variance I added to the test by changing the delay periods. And it's not the question I was looking at. I may not have stated it clearly enough to begin with: I wanted to know if competing processes would create collisions attempting to write the same variable at the same time with the result of lost data because of the collisions. You and David are talking about protecting the sequence of the data writes which I'm not concerned with in this case. I'm not saying that's not an important point, it is, - just not the point I'm looking at. On Tue, Nov 1, 2016 at 7:49 AM, Richard Wright < rwri...@datadomainsoftware.com> wrote: > Yes, I’ve seen it happen. 4D language is multi-threaded already - it’s > cooperative instead of pre-emptive prior to v16. A method looping through > an interprocess array can easily be paused while another method updates > that same array. > > > Richard Wright > DataDomain > rwri...@datadomainsoftware.com > > > > Date: Mon, 31 Oct 2016 22:43:48 -0700 > > From: Kirk Brooks > > > > > > David, > > Great points - but, respectfully, does it ever actually happen? Can you > > write an example db that does this? > > > > And bear in mind I'm talking about conditions on a client - not the > server. > > And I'm not talking about situations where an external client might be > > attempting to set variables either. Just plain old accessing IP vars from > > multiple processes. > > > > On Mon, Oct 31, 2016 at 10:12 PM, David Adams wrote: > > > >> Sure, it's absolutely possible to encounter a race condition if you > write > >> to a shared resource (IP arrays in your case) without a locking > mechanism. > >> It's likelihood increases in compiled mode and as you add more > >> participants. Even if you can't predict how often you'll encounter a > race > >> condition, you already know that the chance is greater than zero. > >> ... > >> > >> > >> * Hoping that it won't happen isn't a sound strategy. Maybe it won't, > maybe > >> it will. It definitely might. > >> > > -- Kirk Brooks San Francisco, CA === ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
Yes, I’ve seen it happen. 4D language is multi-threaded already - it’s cooperative instead of pre-emptive prior to v16. A method looping through an interprocess array can easily be paused while another method updates that same array. Richard Wright DataDomain rwri...@datadomainsoftware.com > Date: Mon, 31 Oct 2016 22:43:48 -0700 > From: Kirk Brooks > > > David, > Great points - but, respectfully, does it ever actually happen? Can you > write an example db that does this? > > And bear in mind I'm talking about conditions on a client - not the server. > And I'm not talking about situations where an external client might be > attempting to set variables either. Just plain old accessing IP vars from > multiple processes. > > On Mon, Oct 31, 2016 at 10:12 PM, David Adams wrote: > >> Sure, it's absolutely possible to encounter a race condition if you write >> to a shared resource (IP arrays in your case) without a locking mechanism. >> It's likelihood increases in compiled mode and as you add more >> participants. Even if you can't predict how often you'll encounter a race >> condition, you already know that the chance is greater than zero. >> ... >> >> >> * Hoping that it won't happen isn't a sound strategy. Maybe it won't, maybe >> it will. It definitely might. >> > > -- > Kirk Brooks > San Francisco, CA > === ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
David, On Mon, Oct 31, 2016 at 11:34 PM, David Adams wrote: > > Can you write an example db that does this? > Probably, I've seen it done in the past. Would I now? No, I wouldn't bother > trying. Unless 4D *guarantees* the non-standard behavior you're hoping to > exploit, it's risky at best. They don't offer any such reassurance. Also, I > pretty rarely used IP arrays for shared objects for much of anything...I > just don't have a lot of situations where that's the best solution. > I think the example I provided establishes the sort of situation you describe. I mean - 400 processes simultaneously writing to the same variable for 2 seconds. As I understand it a race condition develops when there is a sequencing issue - the value Process A will write depends on a value Process B already wrote. I'm not talking about that either. In this test I don't care what order the elements are written in (I had to add inconsistency to the delay periods to force it out of sequence in the first place) or what the value is - just whether it would be written correctly. Adding the lock wouldn't prevent that in this case either. If sequence were important you would need to address it but that wasn't the question I was looking at. As I mentioned before I didn't look at changing multiple vars at the same time. So if I have 4 arrays that needed to be updated at the same time the situation could be different. There's more variability in this case and setting the lock makes sense. And I'm not sure 4D guarantees anything. I'm loath too. Buried within pretty much everyone's EULA is something to the effect of, "we think this works pretty well but it's up to you to decide if you trust it or not because we can't guarantee it actually does anything." I may be paraphrasing. I'm old enough to avoid saying 'never' any more but I'm still a sucker for strong evidence. > > And bear in mind I'm talking about conditions on a client - not the > > server. > Doesn't matter. It's a multi-process machine, either way. > True, but this illustrates that without multi-threading it's still a sequential processor. Only one process at a time can be executing. > > And I'm not talking about situations where an external client might be > > attempting to set variables either. > You mean SET PROCESS VARIABLE, etc.? I forgot about those. I never use > > them. Me either. I prefer writing records to a messaging table instead. An idea I believe I got from you. But in this case I suspect the result would be the same - that 4D doesn't actually execute multiple processes simultaneously. The buried lede in this story may actually be how easily 4D handles 400 simultaneous processes. -- Kirk Brooks San Francisco, CA === ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
> What would be the difference between adding and deleting an element? Because you might be writing to an element that doesn't exist. Some versions/settings, 4D crashes. Other times, it doesn't crash but your array isn't right. If you've got a group of arrays working together to implement some kind of data structure, you might end up with 'rows' where the various columns are about a mixture of different things because of bad luck with timing. So, you're risking crashing, having bad data, or having meaningless data. And, again, all very hard to detect and correct after the fact. ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
> David, Great points - but, respectfully, does it ever actually happen? Unless 4D prevents it from happening, then it can happen. So, unless 4D groups multiple lines of code together in a block, you can get different processes interleaving in overall execution order. Last I heard, can split execution across processes between lines. (I'm use lines a bit loosely here as what we write of 1 or 10 lines of code may translate into something else once compiled.) That's precisely why you would need a lock. Particularly compiled. > Can you write an example db that does this? Probably, I've seen it done in the past. Would I now? No, I wouldn't bother trying. Unless 4D *guarantees* the non-standard behavior you're hoping to exploit, it's risky at best. They don't offer any such reassurance. Also, I pretty rarely used IP arrays for shared objects for much of anything...I just don't have a lot of situations where that's the best solution. > And bear in mind I'm talking about conditions on a client - not the server. Doesn't matter. It's a multi-process machine, either way. > And I'm not talking about situations where an external client might be > attempting to set variables either. You mean SET PROCESS VARIABLE, etc.? I forgot about those. I never use them. While they can be used without trouble if used correctly (and very carefully), they demonstrate a pretty profound violation of any sensible concept of scope. I mean, it's bad enough that we don't have private vars and functions without making it *worse* by giving people a stick to poke inside of other processes with. Left up to mean, you wouldn't even allow for public variables, everything would be set through functions. (Can you tell that I'm reading Bertrand Meyer again?) Those commands just plain suck. They're like bad globals on steroids. Thank goodness V16 is offering a very nice alternative for what people (mis)used SET PROCESS VARIABLE for in the past. I understand the temptation to think that race conditions won't happen to you. It's pretty easy to think that way. But since they can, chances are that they eventually will. That brings you back to what to do about it. Again, sometimes the situation is harmless (you lose so low-quality/low-value data), other times it's a big deal (you crash writing to an element that doesn't exist and/or you scramble key tracking data.) Like any risk, the first question is "what harm is done if things go wrong?" It's not worth much effort to prevent a problem that causes no harm. On the other hand, if the outcome is going to be bad, is it worth a risk? And if The Bad Thing happens, how will you even know? How will you recover? Can you even detect it or recover? Again, it depends on what you're working with. And, for the record, I've seen tons of 4D code that loads records for writing without checking that their writable, either because of record locks or the table being in read-write. That's easy to understand and, in some ways, a less dire problem...but would anyone suggest it's a strong design or a reliable approach? I hope not. (Again, unless it's data that doesn't really need to be 100% correct or complete. There's plenty of that out there in the world.) ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
Miyako, On Mon, Oct 31, 2016 at 10:16 PM, Keisuke Miyako wrote: > what if another process removed or changed elements while that process is > working its way though the IP array? > would that not become a problem? > What would be the difference between adding and deleting an element? Granted I didn't test that because I wanted an easy metric to measure but it seems like the same case - the actions appear to be queued by 4D. > > but the problem with multi-threaded access in compiled mode is more > serious; > you don't want another thread to use the same register while a thread is > in the middle of a read/write operation. > Absolutely, as I noted this is only in respect to a single client without multi-threading. > The "while" loop in your "Lock" method can be an over kill; > That's actually what got me thinking about it. I see it in a number of databases though. And if you were to carry this line of thought to it's logical conclusion such a lock is to ensure the variable will be written and accept nothing less - which you wouldn't want to do normally. I was surprised how little overhead that added. On my macBookPro the un-protected loops took about 2800ms and the protected ones about 3100ms, interpreted. It's not a significant performance issue in my view. -- Kirk Brooks San Francisco, CA === ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
David, Great points - but, respectfully, does it ever actually happen? Can you write an example db that does this? And bear in mind I'm talking about conditions on a client - not the server. And I'm not talking about situations where an external client might be attempting to set variables either. Just plain old accessing IP vars from multiple processes. On Mon, Oct 31, 2016 at 10:12 PM, David Adams wrote: > Sure, it's absolutely possible to encounter a race condition if you write > to a shared resource (IP arrays in your case) without a locking mechanism. > It's likelihood increases in compiled mode and as you add more > participants. Even if you can't predict how often you'll encounter a race > condition, you already know that the chance is greater than zero. > ... > > > * Hoping that it won't happen isn't a sound strategy. Maybe it won't, maybe > it will. It definitely might. > -- Kirk Brooks San Francisco, CA === ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
I tend to agree, in general, semaphores should be used to protect "methods" rather than "objects". the method may read/write certain documents or access IP variables, but that is not the most important issue. some singleton methods may do neither, but yet require protection. > 2016/11/01 14:03、Kirk Brooks のメール： > So I'm concluding the whole variable locking fetish is unnecessary. 宮古 啓介 セールス・エンジニア 株式会社フォーディー・ジャパン 〒150-0043 東京都渋谷区道玄坂1-10-2 渋谷THビル6F Tel: 03-6427-8441 Fax: 03-6427-8449 keisuke.miy...@4d.com www.4D.com/JP ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
Re: Does it matter if you lock an IP object before updating?
what if another process removed or changed elements while that process is
working its way though the IP array?
would that not become a problem?
but the problem with multi-threaded access in compiled mode is more serious;
you don't want another thread to use the same register while a thread is in the
middle of a read/write operation.
p.s.
The "while" loop in your "Lock" method can be an over kill;
why not simply
If (Not (Semaphore ("<>aTestArray";$timeout) ) )
//..do something..
CLEAR SEMAPHORE("<>aTestArray")
End if
> 2016/11/01 14:03、Kirk Brooks のメール：
> So I'm concluding the whole variable locking fetish is unnecessary.
宮古 啓介
セールス・エンジニア
株式会社フォーディー・ジャパン
〒150-0043
東京都渋谷区道玄坂1-10-2 渋谷THビル6F
Tel: 03-6427-8441
Fax: 03-6427-8449
keisuke.miy...@4d.com
www.4D.com/JP
**
4D Internet Users Group (4D iNUG)
FAQ: http://lists.4d.com/faqnug.html
Archive: http://lists.4d.com/archives.html
Options: http://lists.4d.com/mailman/options/4d_tech
Unsub: mailto:4d_tech-unsubscr...@lists.4d.com
**
Re: Does it matter if you lock an IP object before updating?
Sure, it's absolutely possible to encounter a race condition if you write to a shared resource (IP arrays in your case) without a locking mechanism. It's likelihood increases in compiled mode and as you add more participants. Even if you can't predict how often you'll encounter a race condition, you already know that the chance is greater than zero. A few more points: * What are you risking? If your data is written incorrectly because of out-of-expectation sequencing, what happens? Do you crash? Do you get screwed up calculations? If you don't care about the errors, and they don't harm you...perhaps it doesn't matter. For example, if you're writing to an event log and lose a few entries every 10,000, perhaps it doesn't make any real difference. On the other hand, if an error will cost you, why would you risk those 2 out of 10,000 (or whatever) errors? Keeping in mind *they will be very hard or impossible to detect or correct after the fact.* I don't see the upside. * Race conditions are, by nature, a bit unpredictable and *very* hard to diagnose after the fact. * Hoping that it won't happen isn't a sound strategy. Maybe it won't, maybe it will. It definitely might. * If you don't feel like writing your own locking scheme for IP variables, think about using records instead of arrays. 4D already has an easy to use locking system for records. * IP arrays are often used for tasks that would be better served by a queue. If so, write a real queue using NTK IPC channels, or some other scheme. (There are several options.) * If locking your IP arrays is a performance bottleneck, you probably need an alternative design, not a low-quality implementation of shared resource management. ** 4D Internet Users Group (4D iNUG) FAQ: http://lists.4d.com/faqnug.html Archive: http://lists.4d.com/archives.html Options: http://lists.4d.com/mailman/options/4d_tech Unsub: mailto:4d_tech-unsubscr...@lists.4d.com **
