[
https://issues.apache.org/jira/browse/SAMZA-416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14147057#comment-14147057
]
Roger Hoover commented on SAMZA-416:
------------------------------------
FWIW, I strongly advocate keeping logic out of config for the reasons Jay
mentioned. JSON is OK but I find YAML very simple to use and extremely
succinct. Here's an example of a Java project using YAML for config and it
looks pretty reasonable to me
(https://brooklyn.incubator.apache.org/quickstart/index.html).
The CLI sounds handy but you also want a way to version-control your config.
In that case, it's better to have a file that can be managed in SCM rather than
a series of CLI commands.
Perhaps the CLI tool should be more like:
./configure-job.sh foo.yaml
and would change whatever settings are necessary to get to the new state
instead of sending a series of individual changes. I haven't thought through
it that much this would definitely be idempotent and atomic.
> Samza Configuration DSL
> -----------------------
>
> Key: SAMZA-416
> URL: https://issues.apache.org/jira/browse/SAMZA-416
> Project: Samza
> Issue Type: New Feature
> Reporter: David Chen
>
> The user-facing language for Samza configurations is currently Java
> Properties. While this works for the time being and simple to implement, it
> is verbose and cumbersome for the end user.
> Since SAMZA-40 is opened to refactor the way Samza configuration is wired and
> SAMZA-348 is opened to allow Samza jobs to be configured through a stream, it
> is thus natural to consider a scripting model for configuring Samza jobs.
> One approach would be to implement a DSL in a scripting language like Python.
> Implementing a Python DSL is actually very easy to do, and there are many
> examples of Python DSLs in non-trivial use cases, most notably, [Google's
> build system,
> Blaze|http://google-engtools.blogspot.com/2011/08/build-in-cloud-how-build-system-works.html].
> I found [a GitHub Gist containing a more detailed example of a Blaze BUILD
> file|https://gist.github.com/wiseman/3834928]. Blaze also inspired a number
> of open source clones, most notably [Twitter
> Pants|http://pantsbuild.github.io/].
> Another idea, by [~jonbringhurst] from SAMZA-348:
> {quote}
> Regarding the client commands to create and modify Samza jobs (such as
> configure-job and run-job), it may be useful to review existing commands that
> perform a similar role:
> My personal favorite is Slurm's set of client commands, of which sbatch is
> probably the most relevant (http://www.schedmd.com/slurmdocs/sbatch.html).
> To go back a bit further in history, it might be a good idea to take a look
> at the POSIX qsub style command from PBS/Torque
> (http://docs.adaptivecomputing.com/torque/4-1-4/Content/topics/commands/qsub.htm).
> Moab's msub also follows this design.
> Regarding a possible DSL for building configuration, it may be useful to look
> at Slurm's lua callback for job configuration (warning, GPLv2 code)
> https://github.com/SchedMD/slurm/blob/master/contribs/lua/job_submit.lua
> {quote}
> We could, of course, have both a command line tool and a DSL, and I am sure
> once Samza takes off, there will be people implementing DSLs for other
> language clients as well. The key would be to implement a standard interface
> for these different implementations to talk to.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
