Hi Bob, I just need to make one pass at the data, so building an SQL database for it doesn’t make sense.
I don’t have any definitive line for the boundary between good and bad data. It’s more of a consistent first guess used just to cut down on the amount of data to process. I start looking for the good data after about a third of it has passed by. Thanks, Rick > On May 16, 2022, at 6:00 PM, Bob Sneidar via use-livecode > <use-livecode@lists.runrev.com> wrote: > > Do you know exactly which lines you need to toss, or do you need to searc the > data to find out where the beginning and end of the useful data is? > If the former, then just put line x to y of your data into a new variable. If > the latter, then a divide and conquer approach might be the answer. Get the > line 30% in, test for valid, get the line 40% in, test, then 35% then 32.5% > or 37.5% depending on your test. > > You may only have to do this a dozen or so times to find the exact line where > your valid data begins. > > The other way of course is to get it all into a SQL database (how did you all > know I was going to say that??) The downside is that you have to iterate > through all your data once. The upside is a good one liner query statement > may be all you need to process your data. And if you need to make multiple > passes at your data, all the better. > > Bob S _______________________________________________ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode
