[
https://issues.apache.org/jira/browse/SAMZA-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169341#comment-14169341
]
Martin Kleppmann commented on SAMZA-40:
---------------------------------------
bq. Those 3 jobs would always need to be packaged and deployed together. Do you
guys see this as a common pattern?
I think that pattern makes sense, but I agree with Chris that it should be
implemented as an external workflow scheduler like Oozie or Azkaban. It doesn't
need to be part of the core of Samza. It could be developed as an optional
module within the Samza project, or as a completely separate project.
I would suggest that topology support is out of scope for this issue (job
configuration), and that further discussion about topology support is moved to
SAMZA-73.
> Refactor Samza configuration
> ----------------------------
>
> Key: SAMZA-40
> URL: https://issues.apache.org/jira/browse/SAMZA-40
> Project: Samza
> Issue Type: Bug
> Components: container
> Affects Versions: 0.6.0
> Reporter: Chris Riccomini
> Labels: project
>
> Samza's configuration system has several problems that we need to resolved.
> * Want to auto-generate documentation based off of configuration.
> * Should support global defaults for a config property. Right now, we do
> config.getFoo.getOrElse() everywhere.
> * Should validate config up front, rather than thrown runtime exceptions
> randomly throughout the code.
> * We are mixing wiring and configuration together. How do other systems
> handle this?
> * We have fragmented configuration (anybody can define configuration). How do
> other systems handle this?
> * How to handle undefined configuration? How to make this interoperable with
> both Java and Scala (i.e. should we support Option in Scala)?
> * Should remain immutable.
> * Should remove implicits. It's just confusing.
> * Do we want to support complex types (list, map) for values, not just String?
> We need a design proposal for this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
