[
https://issues.apache.org/jira/browse/SAMZA-40?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14128707#comment-14128707
]
Chris Riccomini commented on SAMZA-40:
--------------------------------------
bq. (In YARN terms I think they should still be separate jobs, so this would
just be a shortcut for launching several jobs in one go, perhaps with some
wiring.)
Yes, this is how I'd been thinking about it as well.
bq. However, this could also be a feature outside of core Samza, analogous to
Oozie or Azkaban for MapReduce. It could potentially grow into a web dashboard
with visualisation of the data flow in the cluster (cf. SAMZA-300).
Agreed. Again, this is how I'd been thinking about topologies and Samza. IMO,
Samza should see deployment at the job-level. Other systems can then build on
this to handle topologies as they see fit. Very similar to Ooozie/Azkaban.
> Refactor Samza configuration
> ----------------------------
>
> Key: SAMZA-40
> URL: https://issues.apache.org/jira/browse/SAMZA-40
> Project: Samza
> Issue Type: Bug
> Components: container
> Affects Versions: 0.6.0
> Reporter: Chris Riccomini
> Labels: project
>
> Samza's configuration system has several problems that we need to resolved.
> * Want to auto-generate documentation based off of configuration.
> * Should support global defaults for a config property. Right now, we do
> config.getFoo.getOrElse() everywhere.
> * Should validate config up front, rather than thrown runtime exceptions
> randomly throughout the code.
> * We are mixing wiring and configuration together. How do other systems
> handle this?
> * We have fragmented configuration (anybody can define configuration). How do
> other systems handle this?
> * How to handle undefined configuration? How to make this interoperable with
> both Java and Scala (i.e. should we support Option in Scala)?
> * Should remain immutable.
> * Should remove implicits. It's just confusing.
> * Do we want to support complex types (list, map) for values, not just String?
> We need a design proposal for this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
