[
https://issues.apache.org/jira/browse/MADLIB-1136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16094110#comment-16094110
]
David Chen commented on MADLIB-1136:
------------------------------------
hi Frank,
The sample data is being uploaded and the following is the table schemas for
the test data and regression result:
CREATE TABLE xinos_plus_case_dlinterference_v2.temp_neighbor_pair_cqi_prb_nonull
(
"time" character varying(18),
main_lnbts_id character varying(20),
main_lncel_id character varying(20),
lnbts_id character varying(20),
lncel_id character varying(20),
average_cqi double precision,
prb_utilization double precision
)
WITH (
OIDS=TRUE
)
DISTRIBUTED BY ("time");
ALTER TABLE xinos_plus_case_dlinterference_v2.temp_neighbor_pair_cqi_prb_nonull
OWNER TO gpadmin;
CREATE TABLE xinos_plus_case_dlinterference_v2.taipei_lm_result_temp
(
main_lnbts_id character varying(20),
main_lncel_id character varying(20),
lnbts_id character varying(20),
lncel_id character varying(20),
coef double precision[],
r2 double precision,
std_err double precision[],
t_stats double precision[],
p_values double precision[],
condition_no double precision,
num_rows_processed bigint,
num_missing_rows_skipped bigint,
variance_covariance double precision[]
)
WITH (
OIDS=FALSE
)
DISTRIBUTED BY (main_lnbts_id, main_lncel_id, lnbts_id, lncel_id);
ALTER TABLE xinos_plus_case_dlinterference_v2.taipei_lm_result_temp
OWNER TO gpadmin;
> Getting "ERROR: plpy.SPIError: Function" when calling linregr_train function
> with big data
> -------------------------------------------------------------------------------------------
>
> Key: MADLIB-1136
> URL: https://issues.apache.org/jira/browse/MADLIB-1136
> Project: Apache MADlib
> Issue Type: Bug
> Components: Module: Linear Regression
> Reporter: David Chen
>
> hi MADLib developers,
> we have been trying to use MADlib on Greenplum to in-database perform linear
> regression calculation on a large amount of data (789,626,243 rows of data,
> segmented in ~475,000 groups). However, after running the following SQL
> statement for a little bit more than ten minutes, the following error message
> occurs:
> SQL statement:
> SELECT madlib.linregr_train(
> 'xinos_plus_case_dlinterference_v2.temp_neighbor_pair_cqi_prb_nonull',
> 'xinos_plus_case_dlinterference_v2.taipei_lm_result_temp',
> 'average_cqi', 'array[1, prb_utilization]',
> 'main_lnbts_id,main_lncel_id,lnbts_id,lncel_id');
> Error message:
> ERROR: plpy.SPIError: Function
> "madlib.linregr_merge_states(madlib.bytea8,madlib.bytea8)": ByteString
> improperly aligned for alignment request in seek(). (UDF_impl.hpp:210) (seg2
> 59-120-199-107.HINET-IP.hinet.net:50002 pid=9137) (plpython.c:4648)
> If we downsize the input data to 269837688 rows, then the same SQL statement
> can run with successful result.
> We are not sure if what we encountered here is a bug or an issue with how we
> use this MADLib linear regression function and we will appreciate it a lot if
> you could give us some pointers.
> We are willing to provide more information about input data (e.g. data
> schema) for further investigation if needed.
> thank you very much for taking care of this issue.
> David
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
