On 2024-04-17 23:06, jack wrote:
As a result of this I will be checking the RAM on all my machines once
a month or the moment a machine starts to act strange.
Once a month is overkill, and unlikely to be useful. :)
With server or enterprise grade hardware, it'll support "ECC" memory.
That
That kind of support for “damaged ram” you have it with ECC memory on CPU’s
that support it.
XEON cpus for example.
> On 17 Apr 2024, at 15:06, jack wrote:
>
> uld advise me if there was ever an issue with me
I discovered that one of the memory sticks in the machine was damaged.
Running memtest86 on the machine generated many RAM errors.
This was causing the strange bi-polar errors in postgresql.
The hardware technician explained that he sees this often and that there is no
one cause for such
It seems that the hardware may in fact be the issue.
I ran the exact same process during the night on a much slower machine (4 core
Xeon E5-1620 v4 32G Ram) and it worked fine. It did not crash and completed all
the updates on 113 million records successfully.
I will be bringing the i9 in for
Jack:
On Sun, 14 Apr 2024 at 22:18, jack wrote:
> The CSV files are being produced by another system, a WIndows app on a
> Windows machine. I then copy them to a USB key and copy them onto the ubuntu
> machine. The data is then imported via the COPY command.
>
> COPY master
On Sun, Apr 14, 2024 at 6:58 PM jack wrote:
> But the current i9 machine is a machine from only 4 years ago which should
> have no issues.
>
>
That is a non sequitur. Besides, if the current code is a representative
example you've probably given it the effective usage of 8 years.
Whether
On Mon, 15 Apr 2024 at 02:25, jack wrote:
> Then I tried to update the data in blocks of 100,000 records and it crashed 4
> times on 4 different blocks.
> So I updated the first crashed block down to the a block of 10 records, until
> it crashed.
> Then I updated each of the 10 records
I wrote the windows app.
I export all data to simple ASCII text where fields are delimited with a tab
and then run the file through a UTF8 converter (convertcp_v8.3_x86).
I will try the entire process on a Xeon E5-1620 and let it run during the night
to see what happens. But the current i9
Adrian Klaver writes:
> Unless it does. That is the point of the questions, getting to what is
> actually causing the issue. Until the problem can be boiled down to a
> reproducible test case there really is not much hope of anything more
> then the the 'yes you have a problem' answer.
IIUC,
On 4/14/24 14:50, jack wrote:
Reply to list also
Ccing list
Hello,
I am not sure what "locale" means.
Go to the settings App for whatever version of Windows you are on and
search for locale.
The Windows app is an inhouse application which uses Actian-Zen SQL.
The data is exported to
On 4/14/24 13:18, jack wrote:
The CSV files are being produced by another system, a WIndows app on a
Windows machine. I then copy them to a USB key and copy them onto the
ubuntu machine. The data is then imported via the COPY command.
The app?
The locale in use on the Windows machine?
The
The CSV files are being produced by another system, a WIndows app on a Windows
machine. I then copy them to a USB key and copy them onto the ubuntu machine.
The data is then imported via the COPY command.
COPY master (field01,field02..fieldX) FROM '/data/file.text' DELIMITER E'\t'
The fields
To show you how bi-polar this is really becoming, I tried a work-around...
I took the table called us113 with 113 million records and tried to break it
down into 10 smaller tables each having about 10 million records, using the
following code:
\set ECHO all
\set ON_ERROR_STOP on
-- Create 10
On 4/14/24 12:22, jack wrote:
Here is an excerpt of /var/log/postgresql/postgresql-16-main.log
Where and how are the CSV files being produced?
What is the database locale?
--
Adrian Klaver
adrian.kla...@aklaver.com
Here is the table structure:
The fields being updated are the ones that are NOT named field##.
Except for "3fc" which I left as is to show that it is named differently in the
unlikely chance that this would be causing problems.
CREATE TABLE main (
field01 character(10) COLLATE
jack writes:
> Here is an excerpt of /var/log/postgresql/postgresql-16-main.log
> 2024-04-14 12:17:43.153 EDT [1227] LOG: server process (PID 7289) was
> terminated by signal 11: Segmentation fault
That's pretty interesting; it suggests an actual PG bug.
Can you get a stack trace from that?
Ignore my thread, I guess there might be a bug given it segfaulted.
On Mon, Apr 15, 2024, 12:48 AM Vijaykumar Jain <
vijaykumarjain.git...@gmail.com> wrote:
>
>
> On Sun, 14 Apr 2024 at 21:50, jack wrote:
>
>> The full error reads:
>> server closed the connection expectantly
>> This probably
Here is an excerpt of /var/log/postgresql/postgresql-16-main.log
2024-04-14 12:17:42.321 EDT [7124] LOG: checkpoint starting: wal
2024-04-14 12:17:43.153 EDT [1227] LOG: server process (PID 7289) was
terminated by signal 11: Segmentation fault
2024-04-14 12:17:43.153 EDT [1227] DETAIL: Failed
On Sun, 14 Apr 2024 at 21:50, jack wrote:
> The full error reads:
> server closed the connection expectantly
> This probably means the server terminated abnormally
> before or while processing the request.
> error: connection to server was lost
>
> PostgreSQL 16.2
>
> I also believe it is a
On Sun, Apr 14, 2024 at 10:20 AM Francisco Olarte
wrote:
>
> If postgres does not supress redundant updates ( lots of people can
> answer that )
It does not. My other usual piece of advice along these lines, if doing
the transform outside the database is not desirable, is to at least ensure
Jack:
On Sun, 14 Apr 2024 at 18:20, jack wrote:
> The full error reads:
> server closed the connection expectantly
> This probably means the server terminated abnormally
> before or while processing the request.
> error: connection to server was lost
This is a CLIENT-side error. What people are
Adrian Klaver writes:
> On 4/14/24 09:20, jack wrote:
>> The full error reads:
>> server closed the connection expectantly
>> This probably means the server terminated abnormally
>> before or while processing the request.
>> error: connection to server was lost
> Look at the OS system log.
On 4/14/24 09:20, jack wrote:
The full error reads:
server closed the connection expectantly
This probably means the server terminated abnormally
before or while processing the request.
error: connection to server was lost
Look at the OS system log.
PostgreSQL 16.2
I also believe it is a
The full error reads:
server closed the connection expectantly
This probably means the server terminated abnormally
before or while processing the request.
error: connection to server was lost
PostgreSQL 16.2
I also believe it is a resource issue which can be rectified with a setting,
but which
jack writes:
> I am trying to load about 1.4 billion records into a postgresql table.
> The data is in about 100 individual CSV files which I load individually into
> the table using the COPY command.
> After the data is loaded I perform updates to the data, and this is when I
> get errors.
On 4/14/24 07:24, jack wrote:
Hello,
I am trying to load about 1.4 billion records into a postgresql table.
The data is in about 100 individual CSV files which I load individually
into the table using the COPY command.
After the data is loaded I perform updates to the data, and this is when
I
Hello,
I am trying to load about 1.4 billion records into a postgresql table.
The data is in about 100 individual CSV files which I load individually into
the table using the COPY command.
After the data is loaded I perform updates to the data, and this is when I get
errors. Some updates work,
27 matches
Mail list logo
