Re: Divide Large Data Blob?

2022-05-16 Thread Richard Gaskin via use-livecode
There may be other approaches, but without knowing what the data looks like, what the resulting index should look like, and how big the data is we'd just be guessing. -- Richard Gaskin Fourth World Systems Rick Harrison wrote: Hi Richard, I was looking at the offset function and thinking

Re: Divide Large Data Blob?

2022-05-16 Thread Rick Harrison via use-livecode
Hi Bob, Nice to know this for the future. Thanks for your research! Rick > On May 16, 2022, at 7:53 PM, Bob Sneidar via use-livecode > wrote: > > OK so it appears there is a log2 property. The log2 of 1000 yields 9.965784 > so I suppsoe if you round up, that would give you the maximum numbe

Re: Divide Large Data Blob?

2022-05-16 Thread Rick Harrison via use-livecode
Hi Bob, Yes, I was looking into the binary sort idea. It’s too bad that isn’t just a built-in function for LC. I’m not sure it’s worth that kind of effort for my particular case. Thanks, Rick > On May 16, 2022, at 6:44 PM, Bob Sneidar via use-livecode > wrote: > > So this has got me thin

Re: Divide Large Data Blob?

2022-05-16 Thread Rick Harrison via use-livecode
Hi Bob, I just need to make one pass at the data, so building an SQL database for it doesn’t make sense. I don’t have any definitive line for the boundary between good and bad data. It’s more of a consistent first guess used just to cut down on the amount of data to process. I start looking

Re: Divide Large Data Blob?

2022-05-16 Thread Bob Sneidar via use-livecode
OK so it appears there is a log2 property. The log2 of 1000 yields 9.965784 so I suppsoe if you round up, that would give you the maximum number of iterations to isolate a single line in a 1000 line sorted list. Bob S > On May 16, 2022, at 15:44 , Bob Sneidar via use-livecode > wrote: > >

Re: Divide Large Data Blob?

2022-05-16 Thread Bob Sneidar via use-livecode
So this has got me thinking. Apparently what I am calling Divide and Conquer is really called a binary sort. I have looked up on the interwebs to calculate the maximum number of iterations for a given number of values, but it seems that all the formulas offered up use functions for C. I am tryin

Re: Divide Large Data Blob?

2022-05-16 Thread Bob Sneidar via use-livecode
A maximum of 7 recursions are necessary to isolate a single instance of 100 possible values. 1000 requires a maximum of 10. 1 values requires 14. The idea is that for every factor of 10, you need roughly 3 more recursions. This of course assumes the data is sorted, which in your case is sort

Re: Divide Large Data Blob?

2022-05-16 Thread Bob Sneidar via use-livecode
Do you know exactly which lines you need to toss, or do you need to searc the data to find out where the beginning and end of the useful data is? If the former, then just put line x to y of your data into a new variable. If the latter, then a divide and conquer approach might be the answer. Get

Re: Divide Large Data Blob?

2022-05-16 Thread Rick Harrison via use-livecode
ecode > Sent: Monday, May 16, 2022 1:46 PM > To: How to use LiveCode > Cc: Rick Harrison > Subject: Divide Large Data Blob? > > I have a large chunk of data that I want to search as quickly as possible. > > Unfortunately the part I want to search is the middle third of

Re: Divide Large Data Blob?

2022-05-16 Thread Rick Harrison via use-livecode
> rdim...@evergreeninfo.net > > -Original Message- > From: use-livecode [mailto:use-livecode-boun...@lists.runrev.com] On Behalf > Of Rick Harrison via use-livecode > Sent: Monday, May 16, 2022 1:46 PM > To: How to use LiveCode > Cc: Rick Harrison > Subject: Divide Large Da

RE: Divide Large Data Blob?

2022-05-16 Thread Ralph DiMola via use-livecode
...@lists.runrev.com] On Behalf Of Rick Harrison via use-livecode Sent: Monday, May 16, 2022 1:46 PM To: How to use LiveCode Cc: Rick Harrison Subject: Divide Large Data Blob? I have a large chunk of data that I want to search as quickly as possible. Unfortunately the part I want to search is the middle third of

Re: Divide Large Data Blob?

2022-05-16 Thread Rick Harrison via use-livecode
Hi Richmond, Doesn’t the following still require a loop? LC doesn’t like the syntax you provided. Thanks, Rick > On May 16, 2022, at 2:23 PM, Richmond via use-livecode > wrote: > > 3 put chars 1-SLICER of MyDATA into firstTHIRD ___ use-livecode m

Re: Divide Large Data Blob?

2022-05-16 Thread Rick Harrison via use-livecode
Hi Richard, I was looking at the offset function and thinking about the starting points. It still presents a looping problem for me that I’m trying to avoid. If other methods aren’t more efficient I will play with it more. Thanks, Rick > On May 16, 2022, at 2:32 PM, Richard Gaskin via use-liv

Re: Divide Large Data Blob?

2022-05-16 Thread Rick Harrison via use-livecode
Hi Richmond, An interesting approach. I’ll give it try Thanks! Rick > On May 16, 2022, at 2:23 PM, Richmond via use-livecode > wrote: > > Well one of the things you could do is this: > > slightly pseudo > > 1 put the number of chars in MyDATA into MyNUM > 2 put MyNUM / 3 into SLICER > 3 p

Re: Divide Large Data Blob?

2022-05-16 Thread Richard Gaskin via use-livecode
Rick Harrison wrote: I have a large chunk of data that I want to search as quickly as possible. Unfortunately the part I want to search is the middle third of the data. The other thirds at the beginning and at the end are just junk and slow down my search so I want to get rid of them. I

Re: Divide Large Data Blob?

2022-05-16 Thread Richmond via use-livecode
Great Behinds Stink Alike.  :) On 16.05.22 21:19, Craig Newman via use-livecode wrote: Hi. Can you get the number of lines of the whole blob, if lines are pertinent, divide that number by 3, and search from there? Another words, if you had 1000 lines, divide by 3 and search from line 333 to 6

Re: Divide Large Data Blob?

2022-05-16 Thread Richmond via use-livecode
Well one of the things you could do is this: slightly pseudo 1 put the number of chars in MyDATA into MyNUM 2 put MyNUM / 3 into SLICER 3 put chars 1-SLICER of MyDATA into firstTHIRD 4 put chars SLICER-(SLICER * 2) into secondTHIRD 5 put chars (SLICER * 2)-(SLICER*3) into thirdTHIRD at which po

Re: Divide Large Data Blob?

2022-05-16 Thread Craig Newman via use-livecode
Hi. Can you get the number of lines of the whole blob, if lines are pertinent, divide that number by 3, and search from there? Another words, if you had 1000 lines, divide by 3 and search from line 333 to 666. Craig > On May 16, 2022, at 1:46 PM, Rick Harrison via use-livecode > wrote: > >

Divide Large Data Blob?

2022-05-16 Thread Rick Harrison via use-livecode
I have a large chunk of data that I want to search as quickly as possible. Unfortunately the part I want to search is the middle third of the data. The other thirds at the beginning and at the end are just junk and slow down my search so I want to get rid of them. I don’t want to search lin