Blize created MADLIB-1292:
-----------------------------
Summary: Elastic Net error: The final target lambda value is not
reached in warm-up iterations. You need more iterations!
Key: MADLIB-1292
URL: https://issues.apache.org/jira/browse/MADLIB-1292
Project: Apache MADlib
Issue Type: Question
Reporter: Blize
running MADLIB 1.10 on hawq 2.4, the test module is elastic_net.
The test command is:
madpack -s madlib -p hawq -c gpadmin@localhost:5432/postgres -t elastic_net
install-check
It failed with output:
{color:#333333}ERROR: plpy.Error: (plpython.c:4663){color}
{color:#333333}DETAIL: {color}
{color:#f691b2} *{color:#205081}Elastic Net error: The
final target lambda value is not{color}*{color}
*{color:#205081}reached in warm-up iterations. You need
more iterations!{color}*
then I run the commands in
{color:#205081}elastic_net_install_check.sql_in.tmp{color} step by step, and
finally find it is caused by {color:#205081}elastic_net_train{color} with
optimizer {color:#205081}*igd*{color} (fista is OK). See the below command
output:**
postgres=# \d
List of relations
Schema | Name | Type | Owner | Storage
--------+-------------------------+----------+---------+-------------
madlib | elastic_type_src | table | gpadmin | append only
madlib | house_en | table | gpadmin | append only
madlib | house_en_summary | table | gpadmin | append only
madlib | housing_test | table | gpadmin | append only
madlib | housing_test_id_seq | sequence | gpadmin | heap
madlib | lin_housing_wi | table | gpadmin | append only
madlib | lin_housing_wi_id_seq | sequence | gpadmin | heap
madlib | migrationhistory | table | gpadmin | append only
madlib | migrationhistory_id_seq | sequence | gpadmin | heap
(9 rows)
postgres=# drop table if exists house_en, house_en_summary;
DROP TABLE
postgres=# *{color:#205081}select elastic_net_train{color}*(
postgres(# 'lin_housing_wi',
postgres(# 'house_en',
postgres(# 'y',
postgres(# 'x',
postgres(# 'gaussian',
postgres(# 1,
postgres(# 0.12,
postgres(# True,
postgres(# 'grp_by_col',
postgres(# '{color:#205081}*igd*{color}',
postgres(# '{stepsize = 2
postgres'# } 1',
postgres(# NULL,
postgres(# 2000,
postgres(# 1e-6
postgres(# );
ERROR: plpy.Error: (plpython.c:4663)
DETAIL:
{color:#f691b2} *{color:#205081}Elastic Net error: The
final target lambda value is not{color}*{color}
*{color:#205081}reached in warm-up iterations. You need
more iterations!{color}*
CONTEXT: Traceback (most recent call last):
PL/Python function "elastic_net_train", line 27, in <module>
excluded, max_iter, tolerance)
PL/Python function "elastic_net_train", line 467, in elastic_net_train
PL/Python function "elastic_net_train", line 495, in
_internal_elastic_net_train
PL/Python function "elastic_net_train", line 46, in
_elastic_net_gaussian_igd_train
PL/Python function "elastic_net_train", line 167, in _elastic_net_igd_train
PL/Python function "elastic_net_train", line 296, in
_elastic_net_igd_train_compute
PL/Python function "elastic_net_train", line 379, in _compute_igd
PL/Python function "elastic_net_train"
postgres=#
by inserting debug lines in{color:#205081} elastic_net_optimizer_igd.py{color},
the {color:#205081}elastic_net_train{color}{color:#333333} with optimizer
{color}{color:#205081}*igd*{color} failed due to it can't pass the judgement
it.test in {color:#205081}_compute_igd{color}
{color:#205081}_if it.test("""_{color}
{color:#205081} _\{iteration} > \{max_iter} or_{color}
{color:#205081}
_\{schema_madlib}.\{func_state_diff}(_{color}
{color:#205081} __state_previous, _state_current) <
\{use_tolerance}_{color}
{color:#205081} _"""):_{color}
def _compute_igd(schema_madlib, func_step_aggregate, func_state_diff,
tbl_args, tbl_state, tbl_source,
col_ind_var, col_dep_var, grouping_str, grouping_col,
start_iter, **kwargs):
...
*{color:#14892c}#For 2000 times, it never pass it.test{color}*
if it.test("""
\{iteration} > \{max_iter} or
\{schema_madlib}.\{func_state_diff}(
_state_previous, _state_current) < \{use_tolerance}
"""):
if (it.iteration < it.kwargs["max_iter"] and
it.kwargs["lambda_count"] < it.kwargs["warm_no"]):
it.kwargs["lambda_count"] += 1
else:
break
it.final()
*{color:#14892c}#according to the debug, 2000 times later,
it.kwargs["lambda_count"]=0, it.kwargs["warm_no"]=1{color}*
if it.kwargs["lambda_count"] < it.kwargs["warm_no"]:
plpy.error("""
Elastic Net error: The final target lambda value is not
reached in warm-up iterations. You need more iterations!
""")
return iterationCtrl.iteration
why does the two values keep no change:
*{color:#14892c}it.kwargs["lambda_count"]=0, it.kwargs["warm_no"]=1{color}*?
what is the expected value?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
