Couldn't you read the file partially you would open the file for read
open file tFile for read then in a repeat loop you could read e.g. 10000 lines and process the data read from file tFile for 10000 lines Just a thought. - Matthias Rebbe Life Is Too Short For Boring Code > Am 03.11.2020 um 13:14 schrieb Keith Clarke via use-livecode > <use-livecode@lists.runrev.com>: > > Hi folks, > I’ve a 3GB CSV file with 18M rows from which I need to create a subset CSV > file for the rows containing tString. The following script results in > LiveCode becoming unresponsive, so I’m wondering if I’m missing an obvious > trick or what alternative approach I might take... > on mouseUp pButtonNumber > > -- Select CSV File > > answer file prompt as sheet > > put it into tFile > > put tFile into field "File" > > -- Process CSV extract > > put URL ("file:" & tFile) into tText > > put line 1 of tText into field "List" > > filter lines of tText with "*tString*" > > put tText into field tList > > end mouseUp > > I don’t normally deal with large data sets, so maybe LiveCode isn’t the best > tool for this but Excel 365 seems to only offer CSV file import, not query. > > Any thoughts greatly appreciated. > > Thanks & regards, > Keith > _______________________________________________ > use-livecode mailing list > use-livecode@lists.runrev.com > Please visit this url to subscribe, unsubscribe and manage your subscription > preferences: > http://lists.runrev.com/mailman/listinfo/use-livecode _______________________________________________ use-livecode mailing list use-livecode@lists.runrev.com Please visit this url to subscribe, unsubscribe and manage your subscription preferences: http://lists.runrev.com/mailman/listinfo/use-livecode