[
https://issues.apache.org/jira/browse/MADLIB-1292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16739916#comment-16739916
]
Rahul Iyer commented on MADLIB-1292:
------------------------------------
Hi [~Blize]
I tried multiple combinations but was unable to reproduce this exact issue on
MADLIB 1.15 with Postgres and Greenplum. I also noticed that the install-check
(and the SQL query included) has a bug in the input (after {{\{stepsize =
2\}}}). This does not affect the latest version but might be an issue for
v1.10.
Since in this call warmup is not being used, I expect the {{lambda_count}} and
{{warm_no}} to be 1.
> Elastic Net error: The final target lambda value is not reached in warm-up
> iterations. You need more iterations!
> ----------------------------------------------------------------------------------------------------------------
>
> Key: MADLIB-1292
> URL: https://issues.apache.org/jira/browse/MADLIB-1292
> Project: Apache MADlib
> Issue Type: Question
> Reporter: Blize
> Assignee: Rahul Iyer
> Priority: Major
>
> running MADLIB 1.10 on hawq 2.4, the test module is elastic_net.
> The test command is:
> madpack -s madlib -p hawq -c gpadmin@localhost:5432/postgres -t elastic_net
> install-check
> It failed with output:
> {color:#333333}ERROR: plpy.Error: (plpython.c:4663){color}
> {color:#333333}DETAIL: {color}
> {color:#f691b2} *{color:#205081}Elastic Net error: The
> final target lambda value is not{color}*{color}
> *{color:#205081}reached in warm-up iterations. You
> need more iterations!{color}*
> then I run the commands in
> {color:#205081}elastic_net_install_check.sql_in.tmp{color} step by step, and
> finally find it is caused by {color:#205081}elastic_net_train{color} with
> optimizer {color:#205081}*igd*{color} (fista is OK). See the below command
> output:**
> postgres=# \d
> List of relations
> Schema | Name | Type | Owner | Storage
> --------+-------------------------+----------+---------+-------------
> madlib | elastic_type_src | table | gpadmin | append only
> madlib | house_en | table | gpadmin | append only
> madlib | house_en_summary | table | gpadmin | append only
> madlib | housing_test | table | gpadmin | append only
> madlib | housing_test_id_seq | sequence | gpadmin | heap
> madlib | lin_housing_wi | table | gpadmin | append only
> madlib | lin_housing_wi_id_seq | sequence | gpadmin | heap
> madlib | migrationhistory | table | gpadmin | append only
> madlib | migrationhistory_id_seq | sequence | gpadmin | heap
> (9 rows)
> postgres=# drop table if exists house_en, house_en_summary;
> DROP TABLE
> postgres=# *{color:#205081}select elastic_net_train{color}*(
> postgres(# 'lin_housing_wi',
> postgres(# 'house_en',
> postgres(# 'y',
> postgres(# 'x',
> postgres(# 'gaussian',
> postgres(# 1,
> postgres(# 0.12,
> postgres(# True,
> postgres(# 'grp_by_col',
> postgres(# '{color:#205081}*igd*{color}',
> postgres(# '{stepsize = 2
> postgres'# } 1',
> postgres(# NULL,
> postgres(# 2000,
> postgres(# 1e-6
> postgres(# );
> ERROR: plpy.Error: (plpython.c:4663)
> DETAIL:
> {color:#f691b2} *{color:#205081}Elastic Net error: The
> final target lambda value is not{color}*{color}
> *{color:#205081}reached in warm-up iterations. You
> need more iterations!{color}*
> CONTEXT: Traceback (most recent call last):
> PL/Python function "elastic_net_train", line 27, in <module>
> excluded, max_iter, tolerance)
> PL/Python function "elastic_net_train", line 467, in elastic_net_train
> PL/Python function "elastic_net_train", line 495, in
> _internal_elastic_net_train
> PL/Python function "elastic_net_train", line 46, in
> _elastic_net_gaussian_igd_train
> PL/Python function "elastic_net_train", line 167, in _elastic_net_igd_train
> PL/Python function "elastic_net_train", line 296, in
> _elastic_net_igd_train_compute
> PL/Python function "elastic_net_train", line 379, in _compute_igd
> PL/Python function "elastic_net_train"
> postgres=#
>
> by inserting debug lines in{color:#205081}
> elastic_net_optimizer_igd.py{color}, the
> {color:#205081}elastic_net_train{color}{color:#333333} with optimizer
> {color}{color:#205081}*igd*{color} failed due to it can't pass the judgement
> it.test in {color:#205081}_compute_igd{color}
> {color:#205081}_if it.test("""_{color}
> {color:#205081} _\{iteration} > \{max_iter} or_{color}
> {color:#205081}
> _\{schema_madlib}.\{func_state_diff}(_{color}
> {color:#205081} __state_previous, _state_current)
> < \{use_tolerance}_{color}
> {color:#205081} _"""):_{color}
>
> def _compute_igd(schema_madlib, func_step_aggregate, func_state_diff,
> tbl_args, tbl_state, tbl_source,
> col_ind_var, col_dep_var, grouping_str, grouping_col,
> start_iter, **kwargs):
> ...
> *{color:#14892c}#For 2000 times, it never pass it.test{color}*
> if it.test("""
> \{iteration} > \{max_iter} or
> \{schema_madlib}.\{func_state_diff}(
> _state_previous, _state_current) <
> \{use_tolerance}
> """):
> if (it.iteration < it.kwargs["max_iter"] and
> it.kwargs["lambda_count"] < it.kwargs["warm_no"]):
> it.kwargs["lambda_count"] += 1
> else:
> break
> it.final()
> *{color:#14892c}#according to the debug, 2000 times later,
> it.kwargs["lambda_count"]=0, it.kwargs["warm_no"]=1{color}*
> if it.kwargs["lambda_count"] < it.kwargs["warm_no"]:
> plpy.error("""
> Elastic Net error: The final target lambda value is not
> reached in warm-up iterations. You need more
> iterations!
> """)
> return iterationCtrl.iteration
>
> why does the two values keep no change:
> *{color:#14892c}it.kwargs["lambda_count"]=0, it.kwargs["warm_no"]=1{color}*?
> what is the expected value?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
