[ https://issues.apache.org/jira/browse/SYSTEMML-2299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540715#comment-16540715 ]
LI Guobao commented on SYSTEMML-2299: ------------------------------------- [~mboehm7], in fact, until now, it seems that the arguments "val_features" and "val_labels" have not been used inside paramserv function. For me, they are leveraged to calculate the model precision and we use a UDF to do it but not inside our paramserv function. So do they have some other utilities? > API design of the paramserv function > ------------------------------------ > > Key: SYSTEMML-2299 > URL: https://issues.apache.org/jira/browse/SYSTEMML-2299 > Project: SystemML > Issue Type: Sub-task > Reporter: LI Guobao > Assignee: LI Guobao > Priority: Major > Fix For: SystemML 1.2 > > > The objective of “paramserv” built-in function is to update an initial or > existing model with configuration. An initial function signature would be: > {code:java} > model'=paramserv(model=paramsList, features=X, labels=Y, val_features=X_val, > val_labels=Y_val, upd="fun1", agg="fun2", mode="LOCAL", utype="BSP", > freq="BATCH", epochs=100, batchsize=64, k=7, scheme="disjoint_contiguous", > hyperparams=params, checkpointing="NONE"){code} > We are interested in providing the model (which will be a struct-like data > structure consisting of the weights, the biases and the hyperparameters), the > training features and labels, the validation features and labels, the batch > update function (i.e., gradient calculation func), the update strategy (e.g. > sync, async, hogwild!, stale-synchronous), the update frequency (e.g. epoch > or mini-batch), the gradient aggregation function, the number of epoch, the > batch size, the degree of parallelism, the data partition scheme, a list of > additional hyper parameters, as well as the checkpointing strategy. And the > function will return a trained model in struct format. > *Inputs*: > * model <list>: a list consisting of the weight and bias matrices > * features <matrix>: training features matrix > * labels <matrix>: training label matrix > * val_features <matrix>: validation features matrix > * val_labels <matrix>: validation label matrix > * upd <string>: the name of gradient calculation function > * agg <string>: the name of gradient aggregation function > * mode <string> (options: LOCAL, REMOTE_SPARK): the execution backend where > the parameter is executed > * utype <string> (options: BSP, ASP, SSP): the updating mode > * freq <string> [optional] (default: BATCH) (options: EPOCH, BATCH) : the > frequence of updates > * epochs <integer>: the number of epoch > * batchsize <integer> [optional] (default: 64): the size of batch, if the > update frequence is "EPOCH", this argument will be ignored > * k <integer> [optional] (default: number of vcores, otherwise vcores / 2 if > using openblas): the degree of parallelism > * scheme <string> [optional] (default: disjoint_contiguous) (options: > disjoint_contiguous, disjoint_round_robin, disjoint_random, > overlap_reshuffle): the scheme of data partition, i.e., how the data is > distributed across workers > * hyperparams <list> [optional]: a list consisting of the additional hyper > parameters, e.g., learning rate, momentum > * checkpointing <string>[optional] (default: NONE) (options: NONE, EPOCH, > EPOCH10) : the checkpoint strategy, we could set a checkpoint for each epoch > or each 10 epochs > *Output*: > * model' <list>: a list consisting of the updated weight and bias matrices -- This message was sent by Atlassian JIRA (v7.6.3#76005)