To follow up:
I got use of a system with 512GB of RAM, which is not nearly as helpful as
I'd like. The JVM is still perfectly happy to eat all of it and go commit
seppuku.
The first thing I had to do was split the Lucene index into chunks. Trying
to parse 4700 files of 10,000 documents each
On Thursday, October 10, 2019 at 7:25:50 AM UTC-4, Noel Grandin wrote:
>
>
>
> On 2019/10/10 1:21 PM, Tim Fielder wrote:
> >
> > Rebuilt using the latest code from Tuesday, still doesn't work. Going
> to try the manual sharding thing next.
> >
>
>
On Wednesday, October 9, 2019 at 8:45:50 AM UTC-4, Noel Grandin wrote:
>
>
>
> On 2019/10/09 2:35 PM, Tim Fielder wrote:
> >
> > Is it just the case that any conditional SELECT will fail once a table
> reaches a certain size?
> >
>
> Quite possibly
On Wednesday, October 9, 2019 at 6:23:09 AM UTC-4, Tim Fielder wrote:
>
> My suspicion is that nesting a SELECT inside any other function is going
> to cause these kinds of errors, and I will need to instead retrieve a
> ResultSet and use a PreparedStatement to batch-INSERT into tabl
On Monday, October 7, 2019 at 3:55:17 PM UTC-4, Noel Grandin wrote:
>
> Also, looking at your schema, and your saying that you are indexing the
> tables, you may want to consider doing
> CREATE TABLE reference (docid varchar NOT NULL,
> refid varchar NOT NULL );
> CREATE
an actual MySQL instance on this machine and
connect to it. Would I reasonably expect to see a performance improvement
by doing this?
On Monday, October 7, 2019 at 8:47:53 AM UTC-4, Tim Fielder wrote:
>
> O(n log n) sounds about right. Based on the most recent results I get
> this
f 0.21 that's just above linear, but once you're up to
over 4,600 files it's still way too damn long.
I will try the CREATE TABLE ... AS SELECT DISTINCT thing and get back to
you. Thanks.
On Monday, October 7, 2019 at 7:40:01 AM UTC-4, Noel Grandin wrote:
>
>
>
> On 2019/10/07 1:32 PM,
I'm using version 1.4.199 for this. That's good to know though.
On Monday, October 7, 2019 at 7:51:29 AM UTC-4, Evgenij Ryazanov wrote:
>
> BTW, what version of H2 do you use?
>
> IN (query) condition needs a lot of memory in 1.4.197 and older versions.
>
--
You received this message because
I'm trying to develop an application that parses scientific journal
publications from a few sources and collects them in a single database.
The problem is that there's just too damn many of them and it creates a
catch-22 situation where I either need literally years to build the
database, or
On Friday, July 19, 2019 at 5:13:09 PM UTC-4, Tim Fielder wrote:
> On Friday, July 19, 2019 at 3:49:24 PM UTC-4, Noel Grandin wrote:
>>
>>
>> The weird stack trace you are seeing in .197 is because you have a
>> varchar column that you are querying against, bu
On Friday, July 19, 2019 at 5:13:09 PM UTC-4, Tim Fielder wrote:
>
>
> On Friday, July 19, 2019 at 3:49:24 PM UTC-4, Noel Grandin wrote:
>>
>>
>> The weird stack trace you are seeing in .197 is because you have a
>> varchar column that you are querying again
On Friday, July 19, 2019 at 3:49:24 PM UTC-4, Noel Grandin wrote:
>
>
> The weird stack trace you are seeing in .197 is because you have a varchar
> column that you are querying against, but you are passing in a very large
> integer value, which tends to hit a rather unfortunate fallback path
On Friday, July 19, 2019 at 12:47:00 PM UTC-4, Noel Grandin wrote:
>
> Use the SCRIPT coomand or tool to dump and restore the db, that is more
> reliable.
>
I'm not trying to load a database built in v1.4.197 in .199 - I rebuilt it
entirely from the raw xml that I'm parsing. Version 1.4.199
More reliable when doing what?
On Friday, July 19, 2019 at 12:47:00 PM UTC-4, Noel Grandin wrote:
>
> Use the SCRIPT coomand or tool to dump and restore the db, that is more
> reliable.
>
--
You received this message because you are subscribed to the Google Groups "H2
Database" group.
To
I'm working with tables that are keyed on varchar columns. The values are
strictly numeric, but leading zeroes are significant and as a result
converting the values to bigint results in collisions. H2, however, seems
to want them to be ints because it's attempting to convert them and
15 matches
Mail list logo
