On Fri, Mar 2, 2012 at 7:02 AM, Brian J Dumont <brian.j.dum...@gmail.com> wrote: > Hi all, > > I've struggled with large commentaries before. It always seemed that once > commentary sections get too big, then random sections of text start > disappearing. > > I'm currently almost done repackaging a part of the EarlyFathers module to > be the homilies of John Chrysostom in commentary form. Thing is, his > homilies are long and I'm hitting the same problems again. Previously, I've > been able to split sections between verses to awkwardly get things to work. > This time I can't; some of these large homilies are on a single verse, and > have another homily on the next. > > So this time I've had no choice but to give up or to buckle down and get > something that the developers can really bite into. I've created an example > OSIS module. One section: Matt.1.1. No special characters, no footnotes, > no headers, almost no markup at all really. > > I'm not compiling it into a compressed format. My command/output is: > [bjdasc@ascpc5] osis2mod mod debug.osis.xml > You are running osis2mod: $Rev: 2671 $ > SUCCESS: osis2mod: has finished its work and will now rest > > I can't always predict *exactly* at what size it will fail (it seems to > depend on things like number of paragraph markers, etc). But it always > fails when the section gets to be roughly 64,000 - 65,000 characters. This > is a pretty suspicious number, being just below 2^16. I assume that I can't > exactly predict the size because there's some overhead, some amount for > paragraph markers, header, etc. > > The attached example file fails (I've also included the conf file that I'm > using to make life easier for any developer that wishes to look at this). > The output is drastically shorter than the input. Please delete one > character from the text in Matt.1.1. It all works nicely. > > This is not a front end problem. I can reproduce the problem in either > Xiphos or diatheke. My diatheke command has been: > diatheke -b Chrysostom -k "Mt 1:1" > > But you can even see when it has failed by the size of the "nt" file created > by osis2mod. This osis file, as attached, creates an nt file that is 70 > bytes. Delete one character and it goes up to 65607 bytes. Note that 70 > bytes + 2^16 bytes = 65606 bytes. > > PLEASE PLEASE PLEASE PLEASE ... I beg you ... I'm stuck without a fix to > this ... it seems like it's an array size limit or a compiler size limit for > some data type or something like that. If you need some legwork, then let > me know; I'll do it. > > Thanks in advance, > Brian > > > -- > ----------------------------------------------------------------------- > The question is not "Have you accepted Jesus into your heart", but "Has > Jesus accepted you into His heart?"; not "have you given your whole > life for God", but "has God given His whole life for you" > - Pr Bryan Wolfmueller, 2010 > > > _______________________________________________ > sword-devel mailing list: sword-devel@crosswire.org > http://www.crosswire.org/mailman/listinfo/sword-devel > Instructions to unsubscribe/change your settings at above page
Brian, Try adding the switch '-s 4' to your import. Also change your driver in the conf file to RawCom4. This will increase the per-entry size limit from 2^16 bytes (the default) to 2^32 bytes giving you an effective limit of 2GB for each verse or other entry. --Greg _______________________________________________ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
