Thank you so much for the tip.
It took 12 hours to split 75 Gb into 1278 files of 200000 lines each.
I had to add a -a5 parameter otherwise Terminal would output a "too
many files" error.
Thanks again.
Le 2023-04-01 10:19, kee nethery via use-livecode a écrit :
I had to do this once upon a time and I used some of the unix commands
in Terminal on a mac. The command is “split” and it was wicked fast.
Just be sure to work on a copy of the file. Many examples on the
interwebs. If you were going to do this day in a day out, yes, a GUI
app would be a good idea. But for an infrequent thing … use “split”.
https://kb.iu.edu/d/afar
https://www.geeksforgeeks.org/split-command-in-linux-with-examples/
I think I split mine into 5Gb files and used BBEdit to open and
examine them. Not sure what maximum size file LiveCode can parse so
you might want to adjust. But for sure BBEdit was a champ in opening
them and scrolling through them easily.
Kee Nethery
On Apr 1, 2023, at 5:17 AM, jbv via use-livecode
<use-livecode@lists.runrev.com> wrote:
Hi list,
I have a 75Go xml file that I need to process, and therefore
need to split it into smaller files.
I tried to do that with LC, reading successive chunks of 50000
lines with an endless loop and save them in successive files,
but it crashes LC (9.6.9 on OSX 10.15) every time before
finishing.
Of course, I am interested in any working solution using LC.
On the other hand, I found EasySplitMerge that runs on Mac
and the free trial version works fine on files under 30Mb.
Does anyone have any experience with it on larger files ?
I'd be interested in any feedback before purchasing a
license.
https://www.321soft.com/easysplitmerge/
Thank you in advance.
jbv
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your
subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your
subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
