[ https://issues.apache.org/jira/browse/MADLIB-413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095586#comment-16095586 ]
Cooper Sloan commented on MADLIB-413: ------------------------------------- Proposed interface: optimizer_params, -- optional, default NULL parameters for optimization in a comma-separated string of key-value pairs. learning_rate_init DOUBLE PRECISION, -- Default: 0.001 Initial learning rate learning_rate_policy VARCHAR, -- Default: 'constant' One of 'constant','exp','inv','step' For 'constant': learning_rate = learning_rate_init For 'exp': learning_rate = learning_rate_init * gamma^(iter) For 'inv': learning_rate = learning_rate_init * (1 + gamma*iter)^(-power) For 'step': learning_rate = learning_rate_init * gamma^(floor(iter/step_iterations)) (Learning rate is multiplied by gamma every step_iterations iterations) Where iter is the current iteration of SGD. Learning rate gamma DOUBLE PRECISION, -- Default: '0.1' Decay rate for learning rate. Valid for learning_rate_policy = 'exp','inv', or 'step' power DOUBLE PRECISION, -- Default: '0.5' Exponent for learning_rate_policy = 'inv' iterations_per_step INTEGER, -- Default: '1000' Number of iterations to run before decreasing the learning rate by a factor of gamma. Valid for learning rate policy = 'step' -- Rest unchanged n_iterations INTEGER, -- Default: 100 Number of iterations per try n_tries INTEGER, -- Default: 1 Total number of training cycles, with random initializations to avoid local minima. tolerance DOUBLE PRECISION, -- Default: 0.001 If the distance in loss between two iterations is less than the tolerance training will stop, even if n_iterations has not been reached > Neural Networks - MLP - Phase 1 > ------------------------------- > > Key: MADLIB-413 > URL: https://issues.apache.org/jira/browse/MADLIB-413 > Project: Apache MADlib > Issue Type: New Feature > Components: Module: Neural Networks > Reporter: Caleb Welton > Assignee: Cooper Sloan > Fix For: v1.12 > > > Multilayer perceptron with backpropagation > Modules: > * mlp_classification > * mlp_regression > Interface > {code} > source_table VARCHAR > output_table VARCHAR > independent_varname VARCHAR -- Column name for input features, should be a > Real Valued array > dependent_varname VARCHAR, -- Column name for target values, should be Real > Valued array of size 1 or greater > hidden_layer_sizes INTEGER[], -- Number of units per hidden layer (can be > empty or null, in which case, no hidden layers) > optimizer_params VARCHAR, -- Specified below > weights VARCHAR, -- Column name for weights. Weights the loss for each input > vector. Column should contain positive real value > activation_function VARCHAR, -- One of 'sigmoid' (default), 'tanh', 'relu', > or any prefix (eg. 't', 's') > grouping_cols > ) > {code} > where > {code} > optimizer_params: -- eg "step_size=0.5, n_tries=5" > { > step_size DOUBLE PRECISION, -- Learning rate > n_iterations INTEGER, -- Number of iterations per try > n_tries INTEGER, -- Total number of training cycles, with random > initializations to avoid local minima. > tolerance DOUBLE PRECISION, -- Maximum distance between weights before > training stops (or until it reaches n_iterations) > } > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)