x-talk@mailman.uni-konstanz.de
Subject: Re: [basex-talk] Out of Main Memory
Thanks, Christian. Distributing documents across many databases sounds fine, as
long as XPath expressions and full-text searching remain reasonably efficient.
In the documentation, the example of addressing multiple databases uses
ubject: Re: [basex-talk] Out of Main Memory
Hi Greg,
I would have guessed that 12 GB is enough for 4.7 GB; but it sometimes depends
on the input. If you like, you can share a single typical document with us, and
we can have a look at it. 61 GB will be too large for a complete full-text
index, thoug
quantities of text.
>
>
>
> Thanks,
>
> Greg
>
>
>
> *From: *Christian Grün
> *Date: *Thursday, March 14, 2024 at 8:48 AM
> *To: *Murray, Gregory
> *Cc: *basex-talk@mailman.uni-konstanz.de <
> basex-talk@mailman.uni-konstanz.de>
> *Subject: *Re: [basex
at 5:55 PM
To: Christian Grün
Cc: basex-talk@mailman.uni-konstanz.de
Subject: Re: [basex-talk] Out of Main Memory
Thanks, Christian. I don’t think selective indexing is applicable in my use
case, because I need to perform full-text searches on the entirety of each
document. Each XML document
, Gregory
Cc: Christian Grün ,
basex-talk@mailman.uni-konstanz.de
Subject: Re: [basex-talk] Out of Main Memory
You don't often get email from bdysonsm...@gmail.com. Learn why this is
important<https://aka.ms/LearnAboutSenderIdentification>
Hi Greg,
Have you tried experimenting with the
*Date: *Thursday, March 14, 2024 at 8:48 AM
> *To: *Murray, Gregory
> *Cc: *basex-talk@mailman.uni-konstanz.de <
> basex-talk@mailman.uni-konstanz.de>
> *Subject: *Re: [basex-talk] Out of Main Memory
>
> Hi Greg,
>
>
>
> A quick reply: If only parts of your documents
optimizing such quantities of text.
Thanks,
Greg
From: Christian Grün
Date: Thursday, March 14, 2024 at 8:48 AM
To: Murray, Gregory
Cc: basex-talk@mailman.uni-konstanz.de
Subject: Re: [basex-talk] Out of Main Memory
Hi Greg,
A quick reply: If only parts of your documents are relevant for full-text
Hi Greg,
A quick reply: If only parts of your documents are relevant for full-text
queries, you can restrict the selection with the FTINDEX option (see [1]
for more information).
How large is the total size of your input documents?
Best,
Christian
[1] https://docs.basex.org/wiki/Indexes#Selecti
Hello,
I’m working with a database that has a full-text index. I have found that if I
iteratively add XML documents, then optimize, add more documents, optimize
again, and so on, eventually the “optimize” command will fail with “Out of Main
Memory.” I edited the basex startup script to change t
tine ELS-AMS" <c.hond...@elsevier.com>, "BaseX"
<basex-talk@mailman.uni-konstanz.de>
Oggetto: Re: [basex-talk] out of main memory
Data: 03/03/16 14:38
> I Christian, i tried and this is my error:
> Error:Could not create the Java Virtual Machine
Did you
tine ELS-AMS" <c.hond...@elsevier.com>, "BaseX"
<basex-talk@mailman.uni-konstanz.de>
Oggetto: Re: [basex-talk] out of main memory
Data: 03/03/16 11:57
> I tried to do what i suggested Costantine with the value 1024m, but
my
> memory is alwais 495m.
Plea
"
> To:
> Cc: "Hondros Constantine ELS-AMS" , "BaseX"
>
> Oggetto: Re: [basex-talk] out of main memory
> Data: 03/03/16 11:57
>
>> I tried to do what i suggested Costantine with the value 1024m, but my
>> memory is alwais 495m.
>
> Please t
---
> Da: "Christian Grün"
> To:
> Cc: "Hondros Constantine ELS-AMS" , "BaseX"
>
> Oggetto: Re: [basex-talk] out of main memory
> Data: 03/03/16 11:03
>
>
>
>> queries work. The Query on the extraction of text from files. I 495m
>> a
"
<basex-talk@mailman.uni-konstanz.de>
Oggetto: Re: [basex-talk] out of main memory
Data: 03/03/16 11:03
> queries work. The Query on the extraction of text from files. I 495m
> available and would like increase it to at least 1024m.
So which value did yo
tian Grün"
> To:
> Cc: "Hondros Constantine ELS-AMS" , "BaseX"
>
> Oggetto: Re: [basex-talk] out of main memory
> Data: 03/03/16 10:42
>
>
>
> Hi Michele,
>
> How much memory do you have available on your system, and how much di
- Original Message
Da: "Christian Grün" <christian.gr...@gmail.com>
To:
Cc: "Hondros Constantine ELS-AMS" <c.hond...@elsevier.com>, "BaseX"
<basex-talk@mailman.uni-konstanz.de>
Oggetto: Re: [basex-talk] out of main memory
Data:
nstanz.de
> [mailto:basex-talk-boun...@mailman.uni-konstanz.de] On Behalf Of
> michele.gre...@email.it
> Sent: 01 March 2016 10:48
> To: basex-talk@mailman.uni-konstanz.de
> Subject: [basex-talk] out of main memory
>
> Hi, i get the error "out of main memory", when i run
ne ELS-AMS" <c.hond...@elsevier.com>
To: "basex-talk@mailman.uni-konstanz.de"
<basex-talk@mailman.uni-konstanz.de>
Oggetto: RE: [basex-talk] out of main memory
Data: 01/03/16 10:07
Hi Michele,
1. Go to <BASEX_INSTALL>/bin
2. Open ba
: [basex-talk] out of main memory
Hi, i get the error "out of main memory", when i run my query on a collection
in basex gui.
How can i fix?
How can i increase my memory?
Regards
Michele
ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per tutti i
dettagli clicca
Hi, i get the error "out of main memory", when i run my query on a
collection in basex gui.
How can i fix?
How can i increase my memory?
Regards
Michele
--
ZE-Light e ZE-Pro: servizi zimbra per caselle con dominio email.it, per tutti
i dettagli
Clicca qui
http://posta.email.it
Hi Martín,
> I think one of our DB seems to have gotten corrupt. I wonder if there's
> a command that will recover it.
Unfortunately, that's generally a difficult task. However, it may be
important to find out how the problem was caused: Can you tell if
there was any update that was manually
Hi all,I think one of our DB seems to have gotten corrupt. I wonder if
there's a command that will recover it. Issuing REPLACE or OPTIMIZE on it will
result in "Out of Main Memory" errors. We have DBs that occupy more space and
have more documents, so I assume the problem must be that this D
Hi Geoff,
have you assigned enough memory to the JVM on the 48GB machine (via
-Xmx)? You may need to edit the BaseX start scripts [1] for that (but
it depends on how you starting BaseX).
Hope this helps,
Christian
[1] http://docs.basex.org/wiki/Start_Scripts
Hi Elangovan,
> I am using BaseX to evaluate Xpaths (Xqueries) for larger size of XML files.
> I have created BaseX database initially using CreateDB.execute(ctx) API.
What do you mean with "CreateDB.execute(ctx) API"? Could you provide
us with a minimized example?
Best,
Christian
>
> And ever
Hi All,
I am using BaseX to evaluate Xpaths (Xqueries) for larger size of XML
files. I have created BaseX database initially using *CreateDB*.execute(ctx)
API.
And every time when I need to evaluate Xpath I open database using
Open("database name") API and get a context. Using this context I eval
Hi Christian,
after further reasearch I found one way it works:
> let $prepared := sql:prepare($conn, "INSERT INTO imagetest VALUES (?,
> decode(?, 'base64'))")
> let $params :=
> {$id}
> type='string'>{$image}
>
Hi Alex,
the mapping of types is defined in the BaseX FNSql class [1]. "bytea"
seems to be a PostgreSQL-specific data type, so I’m not sure which
mapping would be appropriate here. Could you do some research for us
and try to find out which SQL types may give satifying results (see
[2] for the exi
Hi Christian,
thank you very much, I am going to test this as soon as I am at home. I think
this is going to fix my issue.
However, do you happen to know which sql:parameter type I have to use when I
try to insert xs:base64binary into my bytea-column in postgres?
Thank you again,
Alex
--
| Al
Hi Alexander,
how does your XQuery/BaseX script look like? If you use the XQuery
doc() function, you could try to replace it with
parse-xml(fetch:text(...)), because the latter approach will close
your documents and free memory if the processed document is not
required anymore.
Best,
Christian
__
Hello all,
my basex-script should fetch 10.000something XML-files automatically from a
website and insert their content into a external PostgreSQL-database. After
about 8.000 files my script stops and I get "Out of Main Memory".
I found your discussion with "kgfhjjgrn" [1] regarding this issue,
2013
Betreff: RE: [basex-talk] Out of Main Memory
Dear Hans-Juergen,
Did it happen during db creation, or during index creation (in that case, the
collection is available and can be opened) ?
With big collections like this, I usually disable indices before creation, and
create them later
Dear BaseX team,
trying to create a database from a document with a size of 6.5 GB I get this
error:
"Out of Main Memory".
Using the latest snapshot.
Could this be expected? I suppose that data loading is done in a streaming
fashion? Is there something I can do to upload a doc of that size?
32 matches
Mail list logo
