Nagayasu-san,
> Do you have any chance to build the PostgreSQL binary from the source code by
> yourself, and try it?
Yes. I'll try it later soon.
I also found other 3 facts about the conditions that issue happens.
1. Compiling MADLib by using gcc/g++ 4.8 works as I expect (SEGFAULT doesn't
happen)
[OK] MADLib 1.10(gcc 4.8) + PG 9.6(Ubuntu pkg) + Ubuntu 16.04
[NG] MADLib 1.10(gcc 5.4) + PG 9.6(Ubuntu pkg) + Ubuntu 16.04
2. I obtained backtrace through gdb when SEGFAULT happens.
*:I followed the direction below to attach gdb to postgre backend process
https://wiki.postgresql.org/wiki/Getting_a_stack_trace_of_a_running_PostgreSQL_backend_on_Linux/BSD#Attaching_gdb_to_the_backend
It seems someone put invalid address in pfree().
3. Debug build MADLib works as I expect (SEGFAULT doesn't happen)
To get more information when SEGFAULT happens,
I compiled MADLib with CMAKE_BUILD_TYPE=Debug.
# select madlib.version();
version
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
MADlib version: 1.10.0, git revision: unknown, cmake configuration time: Mon
Apr 10 03:02:41 UTC 2017, build type: Debug, build system:
Linux-4.4.0-72-generic, C compiler: gcc 5.4.0, C++ compiler: g++ 5.4.0
(1 row)
Is there any possibility that code optimization of gcc 5.4 cause any bad
effect for MADLib ?
Regards,
Atsushi
-----Original Message-----
From: Satoshi Nagayasu [mailto:[email protected]]
Sent: Sunday, April 9, 2017 3:48 PM
To: [email protected]
Subject: Re: Multinomial Regression: Failed with a msg "Hessian or gradient is
not finite."
Neki-san,
> > 1) [NG] MADLib 1.10(Compiling From Source) + PG 9.6 (+ Ubuntu
> 16.04)
I'm not familiar with Ubuntu, but now I doubt that the PostgreSQL binary you
used here might have some issues, because SEGFAULT often happens in such
situation.
Do you have any chance to build the PostgreSQL binary from the source code by
yourself, and try it?
Regards,
2017-04-06 17:28 GMT+09:00 Neki, Atsushi <[email protected]>:
> Nagayasu-san,
>
>
>
>> Could you find something in your PostgreSQL server log?
>
> I attached log files that collected under the below situation.
>
> > 1) [NG] MADLib 1.10(Compiling From Source) + PG 9.6 (+ Ubuntu 16.04)
> > : Failed. Same result with before.
> > As an additional information,
> > madlib install-check was failed on this environment.
> > It looks similar with the below.
> > https://issues.apache.org/jira/browse/MADLIB-1068
>
> log_install-check_fail.txt:
> Log when madlib install check failed at check_elastic_net.
>
> log_multinom_fail.txt:
> Log when madlib.multinorm failed.
>
> In both of them, I can see the following message.
>
> > 2017-04-06 07:53:03 UTC [12162-4] LOG: server process (PID 12483) was
> terminated by signal 11: Segmentation fault
> > 2017-04-06 07:53:03 UTC [12162-6] LOG: terminating any other active
> server processes
> > 2017-04-06 07:53:03 UTC [12168-2] WARNING: terminating connection
> because of crash of another server process
> > 2017-04-06 07:53:03 UTC [12168-3] DETAIL: The postmaster has commanded
> this server process to roll back the current transaction and exit, because
> another server process exited abnormally and possibly corrupted shared memory.
> > 2017-04-06 07:53:03 UTC [12168-4] HINT: In a moment you should be able
> to reconnect to the database and repeat your command.
>
> According to HINT in log message,
> I confirmed that other server had already been stopped.
> And then I reconnected. But the situation didn't get better.
>
> $ pg_lsclusters
> Ver Cluster Port Status Owner Data directory Log file
> 9.4 main 5434 down postgres /var/lib/postgresql/9.4/main
> /var/log/postgresql/postgresql-9.4-main.log
> 9.5 main 5432 down postgres /var/lib/postgresql/9.5/main
> /var/log/postgresql/postgresql-9.5-main.log
> 9.6 main 5433 online postgres /var/lib/postgresql/9.6/main
> pg_log/postgresql-%Y-%m-%d_%H%M%S.log
>
>
>> And how did you install your PostgreSQL 9.6 on Ubuntu?
>> Built from the source by yourself, or used some packages?
>
> I used packages that provided from PostgreSQL Apt Repository.
>
> http://askubuntu.com/questions/831292/how-to-install-postgresql-9-6-on
> -any-ubuntu-version
>
>
> Thank you,
> Atsushi
>
> -----Original Message-----
> From: Satoshi Nagayasu [mailto:[email protected]]
> Sent: Thursday, April 6, 2017 2:57 PM
> To: [email protected]
> Subject: Re: Multinomial Regression: Failed with a msg "Hessian or gradient
> is not finite."
>
> Hi Neki-san,
>
> I have tried to reproduce the situation with PG9.6 and MADlib 1.10.0 on
> CentOS 6.
> Unfortunately, I couldn't reproduce it (see below), and it looked working
> well on my box.
>
> Could you find something in your PostgreSQL server log?
>
> And how did you install your PostgreSQL 9.6 on Ubuntu?
> Built from the source by yourself, or used some packages?
>
> Regards,
>
> ------------------------------------
> testdb=# select version();
> version
> ----------------------------------------------------------------------
> ------------------------------------
> PostgreSQL 9.6.0 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.4.7
> 20120313 (Red Hat 4.4.7-17), 64-bit
> (1 row)
>
> testdb=# select madlib.version();
>
> version
>
> ----------------------------------------------------------------------
> ----------------------------------------------------------------------
> --------------
> ----------------------------------------------------------------------
> ------ MADlib version: 1.10.0, git revision: unknown, cmake
> configuration
> time: 2017年 4月 6日 木曜日 05:36:25 UTC, build type: RelWithDebInfo, build
> system:
> Linux-2.6.32-504.el6.x86_64, C compiler: gcc 4.4.7, C++ compiler: g++
> 4.4.7
> (1 row)
>
> testdb=# create table mactbl_mini(
> testdb(# ap1 integer,
> testdb(# ap2 integer,
> testdb(# ap3 integer,
> testdb(# floor integer,
> testdb(# id integer
> testdb(# );
> CREATE TABLE
> testdb=# INSERT INTO mactbl_mini VALUES
> testdb-# (-90,-86,0,601,1),
> testdb-# (-84,0,0,601,2),
> testdb-# (-83,0,0,601,3),
> testdb-# (0,-72,-84,601,6),
> testdb-# (0,0,-89,602,7),
> testdb-# (0,0,0,602,8),
> testdb-# (0,-85,0,603,43);
> INSERT 0 7
> testdb=# SELECT madlib.multinom('mactbl_mini',
> testdb(# 'mactbl_output',
> testdb(# 'floor',
> testdb(# 'ARRAY[1,
> testdb'# ap1,
> testdb'# ap2,
> testdb'# ap3]',
> testdb(# '601',
> testdb(# 'logit');
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> WARNING: Hessian or gradient is not finite.
> multinom
> ----------
>
> (1 row)
>
> testdb=# \x
> Expanded display is on.
> testdb=# select * from mactbl_output;
> -[ RECORD 1
> ]------+----------------------------------------------------------------------
> category | 602
> coef |
> {98.501202727275,1.66424014822047,1.23992159121658,0.719054608268981}
> log_likelihood | NaN
> std_err | {NaN,NaN,NaN,NaN}
> z_stats | {NaN,NaN,NaN,NaN}
> p_values | {NaN,NaN,NaN,NaN}
> num_rows_processed | 7
> num_rows_skipped | 0
> num_iterations | 100
> -[ RECORD 2
> ]------+----------------------------------------------------------------------
> category | 603
> coef |
> {6715539712038.1,-52386230744.054,-483412078248451,-563980757956539}
> log_likelihood | NaN
> std_err | {NaN,NaN,NaN,NaN}
> z_stats | {NaN,NaN,NaN,NaN}
> p_values | {NaN,NaN,NaN,NaN}
> num_rows_processed | 7
> num_rows_skipped | 0
> num_iterations | 100
>
> testdb=#
> ------------------------------------
>
> 2017-04-06 13:50 GMT+09:00 Neki, Atsushi <[email protected]>:
>> Frank,
>>
>> I tried again the following environments:
>>
>> 1) [NG] MADLib 1.10(Compiling From Source) + PG 9.6 (+ Ubuntu 16.04)
>> : Failed. Same result with before.
>> As an additional information,
>> madlib install-check was failed on this environment.
>> It looks similar with the below.
>> https://issues.apache.org/jira/browse/MADLIB-1068
>>
>> 2) [OK] MADLib 1.9.1(Binary) + PG 9.5 (+ Ubuntu 16.04)
>> : Successed.
>> Still remain the msg 'WARNING: Hessian or gradient is not finite.'
>>
>> 3) [NT] MADLib 1.10(Compiling From Source) + PG 9.4 (+ Ubuntu 16.04)
>> : MADLib Build error (so far, I will not report more about that in here)
>> > [ 48%] Building CXX object
>> src/ports/postgres/9.6/CMakeFiles/madlib_postgresql_9_6.dir/__/__/__/modules/linalg/matrix_decomp.cpp.o
>> > g++: internal compiler error: Killed (program cc1plus)
>>
>> 4) [OK] MADLib 1.10(Compiling From Source) + PG 9.6 (+ CentOS 7)
>> : Successed.
>> Still remain the msg 'WARNING: Hessian or gradient is not finite.'
>>
>>
>> I will move my workspace to the 4th environment (1.10 + PG9.6 + CentOS).
>> Thank you very much for your quick response and advice.
>>
>>
>> Does the msg 'WARNING: Hessian or gradient is not finite.' mean that
>> something is wrong?
>> Should I care something about it?
>> In case I see the message, some NaN values are included in output.
>> This model can predict category well.
>>
>>
>> # select * from mactbl_output;
>> -[ RECORD 1
>> ]------+-----------------------------------------------------------------------
>> category | 602
>> coef |
>> {104.464636547264,1.72398814787594,1.29895329386266,0.688758979636415}
>> log_likelihood | NaN
>> std_err | {NaN,NaN,NaN,NaN}
>> z_stats | {NaN,NaN,NaN,NaN}
>> p_values | {NaN,NaN,NaN,NaN}
>> num_rows_processed | 7
>> num_rows_skipped | 0
>> num_iterations | 100
>> -[ RECORD 2
>> ]------+-----------------------------------------------------------------------
>> category | 603
>> coef |
>> {-572252710812.094,4826990013.37584,41195201231745.7,48061068103698.5}
>> log_likelihood | NaN
>> std_err | {NaN,NaN,NaN,NaN}
>> z_stats | {NaN,NaN,NaN,NaN}
>> p_values | {NaN,NaN,NaN,NaN}
>> num_rows_processed | 7
>> num_rows_skipped | 0
>> num_iterations | 100
>>
>>
>>
>> Thank you,
>> Atsushi
>>
>>
>>
>> From: Frank McQuillan [mailto:[email protected]]
>> Sent: Thursday, April 6, 2017 2:21 AM
>> To: [email protected]
>> Subject: Re: Multinomial Regression: Failed with a msg "Hessian or gradient
>> is not finite."
>>
>> Atsushi-san,
>>
>> The error that you see makes me think that you lost connection to the
>> database in the middle of the query:
>>
>> "server closed the connection unexpectedly
>> This probably means the server terminated abnormally
>> before or while processing the request.
>> The connection to the server was lost. Attempting reset: Failed."
>>
>> I ran the same query with MADlib 1.10 on PG 9.4 and it seemed to work fine
>> for me:
>>
>> madlib=# SELECT * FROM mactbl_mini;
>> ap1 | ap2 | ap3 | floor | id
>> -----+-----+-----+-------+----
>> -90 | -86 | 0 | 601 | 1
>> -84 | 0 | 0 | 601 | 2
>> -83 | 0 | 0 | 601 | 3
>> 0 | -72 | -84 | 601 | 6
>> 0 | 0 | -89 | 602 | 7
>> 0 | 0 | 0 | 602 | 8
>> 0 | -85 | 0 | 603 | 43
>> (7 rows)
>>
>>
>> madlib=# SELECT madlib.multinom('mactbl_mini',
>> madlib(# 'mactbl_output',
>> madlib(# 'floor',
>> madlib(# 'ARRAY[1,
>> madlib'# ap1,
>> madlib'# ap2,
>> madlib'# ap3]',
>> madlib(# '601',
>> madlib(# 'logit');
>> multinom
>> ----------
>>
>> (1 row)
>>
>>
>> madlib=# \x on
>> Expanded display is on.
>> madlib=# SELECT * FROM mactbl_output; -[ RECORD 1
>> ]------+--------------------------------------------------------------------------------------
>> category | 602
>> coef |
>> {112.327782768804,2.47679877229691,1.69069062458081,0.841889190837338}
>> log_likelihood | NaN
>> std_err |
>> {5.81382676773028e+16,8.77427991785089e+17,1.06802178555571e+15,1.67358709762743e+15}
>> z_stats |
>> {1.93207997514273e-15,2.82279434379335e-18,1.58301136497989e-15,5.0304474265537e-16}
>> p_values | {0.999999999999998,1,0.999999999999999,1}
>> num_rows_processed | 7
>> num_rows_skipped | 0
>> num_iterations | 100
>> -[ RECORD 2
>> ]------+--------------------------------------------------------------------------------------
>> category | 603
>> coef |
>> {32.633774536969,1.61185753977669,-0.177075877885652,1.8769521118073}
>> log_likelihood | NaN
>> std_err |
>> {9.40692076915866e+16,5.45753786923624e+18,1.30468621462021e+15,1.1421554354824e+22}
>> z_stats |
>> {3.46912399262056e-16,2.95345186491992e-19,-1.35722962273498e-16,1.6433420999433e-22}
>> p_values | {1,1,1,1}
>> num_rows_processed | 7
>> num_rows_skipped | 0
>> num_iterations | 100
>>
>>
>>
>> So please try again.
>>
>> Frank
>>
>> On Tue, Apr 4, 2017 at 7:28 PM, Neki, Atsushi <[email protected]>
>> wrote:
>> Hello,
>>
>>
>> I detected a failure of computation when using madlib.multinom function.
>> Is there anything wrong with my procedure?
>>
>> ====================================
>>
>> - Issue description
>>
>> madlib.multinom failed with the following message:
>>
>> > WARNING: Hessian or gradient is not finite.
>> > server closed the connection unexpectedly
>> > This probably means the server terminated abnormally
>> > before or while processing the request.
>> > The connection to the server was lost. Attempting reset: Failed.
>>
>>
>> - Library
>>
>> This issue happens in the following environment:
>> 1. madlib 1.9.1 + PostgreSQL 9.5
>> 2. madlib 1.10 + PostgreSQL 9.6
>>
>>
>> - Expectation
>>
>> The result should be:
>>
>> 1. Success computing the model
>> 2. Success building a model that can predict output precisely.
>> (Because this is quite simple use case)
>>
>>
>> - Procedure to reproduce this issue
>>
>> 1. Prepare the following table:
>>
>> testdb=# select * from mactbl_mini;
>> ap1 | ap2 | ap3 | floor | id
>> -----+-----+-----+-------+----
>> -90 | -86 | 0 | 601 | 1
>> -84 | 0 | 0 | 601 | 2
>> -83 | 0 | 0 | 601 | 3
>> 0 | -72 | -84 | 601 | 6
>> 0 | 0 | -89 | 602 | 7
>> 0 | 0 | 0 | 602 | 8
>> 0 | -85 | 0 | 603 | 43
>> (7 rows)
>>
>>
>> 2. Compute a model to predict floor with ap by using madlib.multinom
>>
>> testdb=# drop table mactbl_output;
>> DROP TABLE
>> testdb=# drop table mactbl_output_summary; DROP TABLE testdb=#
>> testdb=# SELECT madlib.multinom('mactbl_mini',
>> testdb(# 'mactbl_output',
>> testdb(# 'floor',
>> testdb(# 'ARRAY[1,
>> testdb'# ap1,
>> testdb'# ap2,
>> testdb'# ap3]',
>> testdb(# '601',
>> testdb(# 'logit');
>> WARNING: Hessian or gradient is not finite.
>> server closed the connection unexpectedly
>> This probably means the server terminated abnormally
>> before or while processing the request.
>> The connection to the server was lost. Attempting reset: Failed.
>> !>
>>
>>
>> When I changed some parameter, it successed.
>>
>> - Set '602' or '603' to ref_category. [Success]
>> - Change max_iter from 100(default) to 10. [Success]
>>
>> ====================================
>>
>>
>> Thanks in advance.
>>
>> --
>> Atsushi Neki
>>
>
>
>
> --
> Satoshi Nagayasu <[email protected]>
--
Satoshi Nagayasu <[email protected]>
4103
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 4103
Reading symbols from /usr/lib/postgresql/9.6/bin/postgres...Reading symbols
from
/usr/lib/debug/.build-id/4c/9da516a8f5decbeb5c6e5368144c6332f753b2.debug...done.
done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libxml2.so.2...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libpam.so.0...(no debugging symbols
found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libssl.so.1.0.0...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/librt.so.1...Reading symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/librt-2.23.so...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libdl.so.2...Reading symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/libdl-2.23.so...done.
done.
Reading symbols from /lib/x86_64-linux-gnu/libm.so.6...Reading symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/libm-2.23.so...done.
done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libsystemd.so.0...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libc.so.6...Reading symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/libc-2.23.so...done.
done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libicuuc.so.55...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libz.so.1...(no debugging symbols
found)...done.
Reading symbols from /lib/x86_64-linux-gnu/liblzma.so.5...(no debugging symbols
found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libaudit.so.1...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libkrb5.so.3...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libk5crypto.so.3...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libcom_err.so.2...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libkrb5support.so.0...(no
debugging symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libpthread.so.0...Reading symbols
from
/usr/lib/debug/.build-id/84/538e3c6cfcd5d4e3c0d2b6c3373f802915a498.debug...done.
done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Reading symbols from /lib64/ld-linux-x86-64.so.2...Reading symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/ld-2.23.so...done.
done.
Reading symbols from /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libresolv.so.2...Reading symbols
from /usr/lib/debug//lib/x86_64-linux-gnu/libresolv-2.23.so...done.
done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libsasl2.so.2...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libgssapi.so.3...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libgnutls.so.30...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libselinux.so.1...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgcrypt.so.20...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libicudata.so.55...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libstdc++.so.6...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgcc_s.so.1...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libkeyutils.so.1...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libheimntlm.so.0...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libkrb5.so.26...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libasn1.so.8...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libhcrypto.so.4...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libroken.so.18...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libp11-kit.so.0...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libidn.so.11...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libtasn1.so.6...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libnettle.so.6...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libhogweed.so.4...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libgmp.so.10...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libpcre.so.3...(no debugging symbols
found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libgpg-error.so.0...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libwind.so.0...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libheimbase.so.1...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libhx509.so.5...(no debugging
symbols found)...done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libsqlite3.so.0...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libcrypt.so.1...Reading symbols from
/usr/lib/debug//lib/x86_64-linux-gnu/libcrypt-2.23.so...done.
done.
Reading symbols from /usr/lib/x86_64-linux-gnu/libffi.so.6...(no debugging
symbols found)...done.
Reading symbols from /lib/x86_64-linux-gnu/libnss_files.so.2...Reading symbols
from /usr/lib/debug//lib/x86_64-linux-gnu/libnss_files-2.23.so...done.
done.
0x00007f7ba07e8e03 in __epoll_wait_nocancel () at
../sysdeps/unix/syscall-template.S:84
84 ../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) c
Continuing.
Program received signal SIGSEGV, Segmentation fault.
pfree (pointer=0x31) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/utils/mmgr/mcxt.c:1007
1007
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/utils/mmgr/mcxt.c:
No such file or directory.
(gdb) cbt
#0 pfree (pointer=0x31) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/utils/mmgr/mcxt.c:1007
#1 0x00007f7b8e9a503d in
madlib::dbconnector::postgres::Allocator::free<(madlib::dbal::MemoryContext)0>
(inPtr=inPtr@entry=0x55f1635f6850, this=<optimized out>)
at
/home/ubuntu/madlib_ws/apache-madlib-src-1.10-incubating/src/ports/postgres/dbconnector/Allocator_impl.hpp:189
#2 0x00007f7b8e9a537d in operator delete[] (ptr=0x55f1635f6850) at
/home/ubuntu/madlib_ws/apache-madlib-src-1.10-incubating/src/ports/postgres/dbconnector/NewDelete.cpp:67
#3 0x00007f7b8e7cef82 in __gnu_cxx::new_allocator<char>::deallocate
(this=0x7ffd7a383e30, __p=<optimized out>) at
/usr/include/c++/5/ext/new_allocator.h:110
#4 __gnu_cxx::__alloc_traits<std::allocator<char> >::deallocate (__a=...,
__n=<optimized out>, __p=<optimized out>) at
/usr/include/c++/5/ext/alloc_traits.h:185
#5 std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> >::_M_destroy (__size=<optimized out>,
this=0x7ffd7a383e30) at /usr/include/c++/5/bits/basic_string.h:185
#6 std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> >::_M_dispose (this=0x7ffd7a383e30) at
/usr/include/c++/5/bits/basic_string.h:180
#7 std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> >::~basic_string (this=0x7ffd7a383e30,
__in_chrg=<optimized out>)
at /usr/include/c++/5/bits/basic_string.h:543
#8
madlib::modules::glm::MultiResponseGLMAccumulator<madlib::dbal::DynamicStructRootContainer<madlib::dbconnector::postgres::MutableByteString,
madlib::dbconnector::postgres::TypeTraits>, madlib::modules::glm::Multinomial,
madlib::modules::glm::MultiLogit>::apply (this=<optimized out>)
at
/home/ubuntu/madlib_ws/apache-madlib-src-1.10-incubating/src/modules/glm/MultiResponseGLM_impl.hpp:206
#9 madlib::modules::glm::multi_response_glm_final::run
(this=this@entry=0x7ffd7a383e90, args=...)
at
/home/ubuntu/madlib_ws/apache-madlib-src-1.10-incubating/src/modules/glm/multi_response_glm.cpp:90
#10 0x00007f7b8e97a6c3 in
madlib::dbconnector::postgres::UDF::invoke<madlib::modules::glm::multi_response_glm_final>
(args=...)
at
/home/ubuntu/madlib_ws/apache-madlib-src-1.10-incubating/src/ports/postgres/dbconnector/UDF_impl.hpp:82
#11
madlib::dbconnector::postgres::UDF::call<madlib::modules::glm::multi_response_glm_final>
(fcinfo=0x7ffd7a384820)
at
/home/ubuntu/madlib_ws/apache-madlib-src-1.10-incubating/src/ports/postgres/dbconnector/UDF_impl.hpp:161
#12 0x000055f1625a4c81 in finalize_aggregate
(aggstate=aggstate@entry=0x55f1637efc20, peragg=peragg@entry=0x55f1637f7120,
pergroupstate=0x55f1637f81c0,
resultVal=resultVal@entry=0x55f1637f70f0,
resultIsNull=resultIsNull@entry=0x55f1637f7108 "")
at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/nodeAgg.c:1406
#13 0x000055f1625a53e8 in finalize_aggregates
(aggstate=aggstate@entry=0x55f1637efc20, peraggs=peraggs@entry=0x55f1637f7120,
pergroup=pergroup@entry=0x55f1637f81c0,
currentSet=currentSet@entry=0) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/nodeAgg.c:1616
#14 0x000055f1625a5afe in agg_retrieve_direct (aggstate=0x55f1637efc20) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/nodeAgg.c:2212
#15 ExecAgg (node=node@entry=0x55f1637efc20) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/nodeAgg.c:1892
#16 0x000055f162597fc8 in ExecProcNode (node=node@entry=0x55f1637efc20) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/execProcnode.c:503
#17 0x000055f16259420e in ExecutePlan (dest=0x55f162bf5340 <spi_printtupDR>,
direction=<optimized out>, numberTuples=0, sendTuples=<optimized out>,
operation=CMD_SELECT,
use_parallel_mode=<optimized out>, planstate=0x55f1637efc20,
estate=0x55f1637efb10) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/execMain.c:1569
#18 standard_ExecutorRun (queryDesc=0x55f1637e1170, direction=<optimized out>,
count=0) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/execMain.c:338
#19 0x000055f1625c1af9 in _SPI_pquery (tcount=0, fire_triggers=1 '\001',
queryDesc=0x55f1637e1170) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/spi.c:2412
#20 _SPI_execute_plan (plan=plan@entry=0x55f163718970, paramLI=<optimized out>,
snapshot=snapshot@entry=0x0, crosscheck_snapshot=crosscheck_snapshot@entry=0x0,
read_only=read_only@entry=0 '\000', fire_triggers=fire_triggers@entry=1
'\001', tcount=0) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/spi.c:2188
#21 0x000055f1625c1dc1 in SPI_execute_plan (plan=0x55f163718970,
Values=0x55f1637498b0, Nulls=0x7f7b8e3fba60 " ", read_only=<optimized out>,
tcount=tcount@entry=0)
at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/spi.c:415
#22 0x00007f7b90035342 in PLy_spi_execute_plan (ob=0x7f7b8ed7ea70,
list=0x7f7b8f18dea8, limit=0) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/pl/plpython/plpy_spi.c:291
#23 0x00007f7b9003572e in PLy_spi_execute (self=<optimized out>,
args=0x7f7b8f0e1680) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/pl/plpython/plpy_spi.c:190
#24 0x00007f7b8fb55751 in PyEval_EvalFrameEx () from
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#25 0x00007f7b8fc7e01c in PyEval_EvalCodeEx () from
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#26 0x00007f7b8fb53cfd in PyEval_EvalFrameEx () from
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#27 0x00007f7b8fb53e24 in PyEval_EvalFrameEx () from
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#28 0x00007f7b8fb53e24 in PyEval_EvalFrameEx () from
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#29 0x00007f7b8fc7e01c in PyEval_EvalCodeEx () from
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#30 0x00007f7b8fbd43dd in ?? () from
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#31 0x00007f7b8fba71e3 in PyObject_Call () from
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#32 0x00007f7b8fb4df2c in PyEval_EvalFrameEx () from
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#33 0x00007f7b8fb53e24 in PyEval_EvalFrameEx () from
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#34 0x00007f7b8fc7e01c in PyEval_EvalCodeEx () from
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#35 0x00007f7b8fb4cb89 in PyEval_EvalCode () from
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#36 0x00007f7b9002f411 in PLy_procedure_call (kargs=kargs@entry=0x7f7b90038d41
"args", vargs=<optimized out>, proc=0x55f163634b50, proc=0x55f163634b50)
at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/pl/plpython/plpy_exec.c:1062
#37 0x00007f7b900309ae in PLy_exec_function
(fcinfo=fcinfo@entry=0x55f163550340, proc=0x55f163634b50)
at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/pl/plpython/plpy_exec.c:105
#38 0x00007f7b900317e4 in plpython_call_handler (fcinfo=0x55f163550340) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/pl/plpython/plpy_main.c:266
#39 0x000055f1625991e3 in ExecMakeFunctionResultNoSets (fcache=0x55f1635502d0,
econtext=0x55f1635500e0, isNull=0x55f163550f58 "", isDone=<optimized out>)
at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/execQual.c:2041
#40 0x000055f16259f471 in ExecTargetList (tupdesc=<optimized out>,
isDone=0x7ffd7a386174, itemIsDone=0x55f163551070, isnull=0x55f163550f58 "",
values=0x55f163550f40, econtext=0x55f1635500e0,
targetlist=0x55f163551040) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/execQual.c:5423
#41 ExecProject (projInfo=<optimized out>, isDone=isDone@entry=0x7ffd7a386174)
at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/execQual.c:5647
#42 0x000055f1625b5388 in ExecResult (node=node@entry=0x55f16354ffd0) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/nodeResult.c:155
#43 0x000055f162598148 in ExecProcNode (node=node@entry=0x55f16354ffd0) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/execProcnode.c:392
#44 0x000055f16259420e in ExecutePlan (dest=0x55f16354b830,
direction=<optimized out>, numberTuples=0, sendTuples=<optimized out>,
operation=CMD_SELECT, use_parallel_mode=<optimized out>,
planstate=0x55f16354ffd0, estate=0x55f16354fec0) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/execMain.c:1569
#45 standard_ExecutorRun (queryDesc=0x55f1634c8a60, direction=<optimized out>,
count=0) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/executor/execMain.c:338
---Type <return> to continue, or q <return> to quit---
#46 0x000055f1626b4207 in PortalRunSelect (portal=portal@entry=0x55f16354deb0,
forward=forward@entry=1 '\001', count=0, count@entry=9223372036854775807,
dest=dest@entry=0x55f16354b830)
at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/tcop/pquery.c:948
#47 0x000055f1626b5818 in PortalRun (portal=portal@entry=0x55f16354deb0,
count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=1 '\001',
dest=dest@entry=0x55f16354b830,
altdest=altdest@entry=0x55f16354b830,
completionTag=completionTag@entry=0x7ffd7a386720 "") at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/tcop/pquery.c:789
#48 0x000055f1626b23f6 in exec_simple_query (
query_string=0x55f163523500 "SELECT madlib.multinom('mactbl_mini',\n", ' '
<repeats 22 times>, "'mactbl_output',\n", ' ' <repeats 22 times>, "'floor',\n",
' ' <repeats 22 times>, "'ARRAY[1,\n", ' ' <repeats 22 times>, "ap1,\n", ' '
<repeats 22 times>, "ap2,\n "...) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/tcop/postgres.c:1094
#49 PostgresMain (argc=<optimized out>, argv=argv@entry=0x55f1634cf9e0,
dbname=0x55f1634cf8c0 "docodoco", username=<optimized out>)
at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/tcop/postgres.c:4076
#50 0x000055f16242d21c in BackendRun (port=0x55f1634c7c70) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/postmaster/postmaster.c:4271
#51 BackendStartup (port=0x55f1634c7c70) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/postmaster/postmaster.c:3945
#52 ServerLoop () at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/postmaster/postmaster.c:1701
#53 0x000055f16264fd12 in PostmasterMain (argc=5, argv=<optimized out>) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/postmaster/postmaster.c:1309
#54 0x000055f16242e3f1 in main (argc=5, argv=0x55f1634a1820) at
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/main/main.c:228
(gdb)
(gdb) pl
1002 in
/build/postgresql-9.6-ZHxyhz/postgresql-9.6-9.6.2/build/../src/backend/utils/mmgr/mcxt.c
(gdb)
