[
https://issues.apache.org/jira/browse/SINGA-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangwei updated SINGA-20:
-------------------------
Description:
Job Launching. Currently users have to set the model and cluster configuration
by writing the google protocol buffer files (e.g., model.conf and
cluster.conf). However it is not easy for beginning users as it requires full
understanding of all configuration fields. To make it easier, a web interface
can be provided, which assist users to configure the cluster and model by
providing hints and selections. For example, if a user wants to train a MLP
model, then we only need to show him/her the fields necessary for layers of MLP
model.
To implement this feature, we need to do
1. write a daemon program to bridge SINGA system and the web interface. It
receives start request (i.e., start training) from web browsers and runs the
script (i.e., /bin/singa-run) to launch the training job. This program would
later be implemented to handle other requests like performance report which is
to visualize the training performance in real-time in the browsers.
2. decide the interface/format of communication message between the daemon
program and the web browser. To launch a training job, two configuration files
are necessary, i.e., cluster configuration and model configuration. Hence the
start request should provide the corresponding configurations. Currently the
configuration fields are defined using google protocol buffer format (e.g.,
model.proto and cluster.proto). However, the web pages are written in
javascript which is more familiar with json format. Current solution is to
write the configuration definition in json format and convert it into protocol
buffer format using a simple script. The json format is like
[
{
"type": "message",
"key": "ModelProto",
"value":
[
{ "id": 2, "qualifier": "required", "type": "string", "key":
"name", "comment":"model name"} ,
{ "id": 3, "qualifier": "repeated", "type": "LayerProto",
"key": "layer", "selection": "LayerType", "comment":"layers" },
{}...
]
},
{
"type": "message",
"key": "LayerProto",
"value":
[
{ "id": 0, "qualifier": "optional", "type": "int", "key":
"locationid", "default": 0, "comment":"location of the layer"},
{}...
]
},
{
"type":"enum",
"key": "LayerType",
"value":
[
{"id": 2, "key":"kInnerProduct" },
...
]
},
...
]
If the qualifier is "optional", then a default field must be provided. The
value of "selection" is the key of a enum object, which indicates the value of
that variable should be selected from the numeration list.
The web browser parses this json format file to determine the content of the
web page, i.e., which configuration fields should be hidden or shown. The
webpage should update automatically upon user's input. For example, if the user
adds an inner product layer, then the configuration fields of that layer should
be expanded. Once the user submit the job, the web browser should send all
configurations to the daemon. The communication message is similar to the above
snippet except that a value field is added for each key field. If the qualifier
for one key is "repeated", then an array of values are added.
The daemon program then converts the json code to the google protocol buffer
messages like those in model.conf and cluster.conf.
was:
Job Launching. Currently users have to set the model and cluster configuration
by writing the google protocol buffer files (e.g., model.conf and
cluster.conf). However it is not easy for beginning users as it requires full
understanding of all configuration fields. To make it easier, a web interface
can be provided, which assist users to configure the cluster and model by
providing hints and selections. For example, if a user wants to train a MLP
model, then we only need to show him/her the fields necessary for layers of MLP
model.
To implement this feature, we need to do
1. write a daemon program to bridge SINGA system and the web interface. It
receives start request (i.e., start training) from web browsers and runs the
script (i.e., /bin/singa-run) to launch the training job. This program would
later be implemented to handle other requests like performance report which is
to visualize the training performance in real-time in the browsers.
2. decide the interface/format of communication message between the daemon
program and the web browser. To launch a training job, two configuration files
are necessary, i.e., cluster configuration and model configuration. Hence the
start request should provide the corresponding configurations. Currently the
configuration fields are defined using google protocol buffer format (e.g.,
model.proto and cluster.proto). However, the web pages are written in
javascript which is more familiar with json format. Current solution is to
write the configuration definition in json format and convert it into protocol
buffer format using a simple script. The json format is like
[
{
"type": "message",
"key": "ModelProto",
"value":
[
{ "id": 2,
"qualifier": "required",
"type": "string",
"key": "name",
"comment":"model name"} ,
{ "id": 3,
"qualifier": "repeated",
"type": "LayerProto",
"key": "layer",
"selection": "LayerType",
"comment":"layers" },
{}...
]
},
{
"type": "message",
"key": "LayerProto",
"value":
[
{ "id": 0,
"qualifier": "optional",
"type": "int",
"key": "locationid",
"default": 0,
"comment":"location of the layer"},
{}...
]
},
{
"type":"enum",
"key": "LayerType",
value:
[
{"id": 2,
"key":"kInnerProduct" },
...
]
},
...
]
If the qualifier is "optional", then a default field must be provided. The
value of "selection" is the key of a enum object, which indicates the value of
that variable should be selected from the numeration list.
The web browser parses this json format file to determine the content of the
web page, i.e., which configuration fields should be hidden or shown. The
webpage should update automatically upon user's input. For example, if the user
adds an inner product layer, then the configuration fields of that layer should
be expanded. Once the user submit the job, the web browser should send all
configurations to the daemon. The communication message is similar to the above
snippet except that a value field is added for each key field. If the qualifier
for one key is "repeated", then an array of values are added.
The daemon program then converts the json code to the google protocol buffer
messages like those in model.conf and cluster.conf.
> Web interface for launching training jobs
> -----------------------------------------
>
> Key: SINGA-20
> URL: https://issues.apache.org/jira/browse/SINGA-20
> Project: Singa
> Issue Type: New Feature
> Reporter: wangwei
> Assignee: Jinyang Gao
>
> Job Launching. Currently users have to set the model and cluster
> configuration by writing the google protocol buffer files (e.g., model.conf
> and cluster.conf). However it is not easy for beginning users as it requires
> full understanding of all configuration fields. To make it easier, a web
> interface can be provided, which assist users to configure the cluster and
> model by providing hints and selections. For example, if a user wants to
> train a MLP model, then we only need to show him/her the fields necessary for
> layers of MLP model.
> To implement this feature, we need to do
> 1. write a daemon program to bridge SINGA system and the web interface. It
> receives start request (i.e., start training) from web browsers and runs the
> script (i.e., /bin/singa-run) to launch the training job. This program would
> later be implemented to handle other requests like performance report which
> is to visualize the training performance in real-time in the browsers.
> 2. decide the interface/format of communication message between the daemon
> program and the web browser. To launch a training job, two configuration
> files are necessary, i.e., cluster configuration and model configuration.
> Hence the start request should provide the corresponding configurations.
> Currently the configuration fields are defined using google protocol buffer
> format (e.g., model.proto and cluster.proto). However, the web pages are
> written in javascript which is more familiar with json format. Current
> solution is to write the configuration definition in json format and convert
> it into protocol buffer format using a simple script. The json format is like
> [
> {
> "type": "message",
> "key": "ModelProto",
> "value":
> [
> { "id": 2, "qualifier": "required", "type": "string",
> "key": "name", "comment":"model name"} ,
> { "id": 3, "qualifier": "repeated", "type": "LayerProto",
> "key": "layer", "selection": "LayerType", "comment":"layers" },
> {}...
> ]
> },
> {
> "type": "message",
> "key": "LayerProto",
> "value":
> [
> { "id": 0, "qualifier": "optional", "type": "int",
> "key": "locationid", "default": 0, "comment":"location of the layer"},
> {}...
> ]
> },
> {
> "type":"enum",
> "key": "LayerType",
> "value":
> [
> {"id": 2, "key":"kInnerProduct" },
> ...
> ]
> },
> ...
> ]
> If the qualifier is "optional", then a default field must be provided. The
> value of "selection" is the key of a enum object, which indicates the value
> of that variable should be selected from the numeration list.
> The web browser parses this json format file to determine the content of the
> web page, i.e., which configuration fields should be hidden or shown. The
> webpage should update automatically upon user's input. For example, if the
> user adds an inner product layer, then the configuration fields of that layer
> should be expanded. Once the user submit the job, the web browser should send
> all configurations to the daemon. The communication message is similar to the
> above snippet except that a value field is added for each key field. If the
> qualifier for one key is "repeated", then an array of values are added.
> The daemon program then converts the json code to the google protocol buffer
> messages like those in model.conf and cluster.conf.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
