Hi folks,
Currently we're using Chef in our organization to deploy a lot of
infrastructure services around Hadoop. Of course it makes a lot of sense to
offer these as self-services on YARN using slider, but i'm looking at a
number of challenges. So please forgive the broad range of questions :)
I
Hi Jean,
Please see answers inline.
-Gour
On 5/6/15, 6:16 AM, "Jean-Baptiste Note"
mailto:jbn...@gmail.com>> wrote:
Hi folks,
Currently we're using Chef in our organization to deploy a lot of
infrastructure services around Hadoop. Of course it makes a lot of sense to
offer these as self-servi
Jean,
You will see updates in the KOYA repository soon. As part of that we will
move up to the latest release of Slider and also document the configuration
process.
Thanks,
Thomas
On Thu, May 7, 2015 at 5:52 PM, Gour Saha wrote:
> Hi Jean,
>
> Please see answers inline.
>
> -Gour
>
> On 5/6
> On 8 May 2015, at 01:52, Gour Saha wrote:
>
> Last but not least, I'm wondering if there would already be a plan to
> expose somehow (through an internal or an external service) the registry
> through DNS (that's what we really use for service location for HTTPFS &
> OpenTSDB). A bash polling
Hi Gour,
Thanks a lot for the detailed answer, and the pointer to tomcat packaging,
which does half the work for httpfs.
I'll try to wrap properly unpacking of the RPM & extraction of the relevant
parts for slider packaging. That was my gripe; other than that, i can
launch httpfs services and flex
Hi Thomas,
Thanks a lot for the updates you brought to the main Koya repository.
I saw and can see you're still declaring a resource for each broker. This
is painful as it means modifying your metainfo & possibly resource.json in
case you want to grow your cluster, say beyond 10 machines :)
Woul
Hi Steve,
Thanks a lot for your reply from your very busy schedule.
Actually we'll get away with a python daemon watching zookeeper and doing
dynamic DNS updates.
This seems easy enough and probably more palatable than duplicating a full
DNS server (i'm on the operations side ;)).
I'll keep you p
Hi Jean,
Indeed we would like to use component instances as you outline. So far, I
have not found a way to derive the Kafka server id from the Slider
configuration. I checked on my cluster and I find 2 containers using the
same app_container_tag in the logs:
u'componentName': u'BROKER1',
u'confi
Hi Thomas,
This is because the app_container_tag is unique under each resource.
Given your two brokers are on separate resources BROKER0 and BROKER1, they
get identical (1) container_tag.
You should set them in the same resource (BROKER), and the numbering will
be sequential. No idea how it behav
Excellent, will look the pull request shortly. Any thoughts on merging the
server properties defined into the slider config into the server.properties
that came with the Kafka archive?
Thomas
On Mon, May 11, 2015 at 8:10 AM, Jean-Baptiste Note
wrote:
> Hi Thomas,
>
> This is because the app_con
There's a remark on the pull request about this, with more details than in
this mail, but basically:
* Other apps seem to regenerate the config files directly through a
template rather than try to do a merge (you seem to be doing a SED on
defined properties, however it does not work here, maybe a
In order to work for different Kafka versions, it would be nice to pick
whatever server.properties the archive comes with and apply all the
properties that are defined in server.xml on top of it. Does that work for
you? We can look into making that merge work then.
Everything else looks great, tha
Hi Thomas,
According to kafka's documentation:
http://kafka.apache.org/07/configuration.html there should be a default
value for any added property; I would expect the provided server.properties
file to actually reflect those default values.
Therefore, I'd look twice before overconstraining the pr
Hi Jean,
Thanks for the change, using instance tag(is it a new feature in the latest
version? I didn't see it in the older slider versions) is a really good
idea. it might be good for other's to have a template but not for kafka.
Kafka is evolving in quite fast pace. I've seen many property key/v
Jean,
We pulled in your changes and added modifications on top of it. It appears
we agree that we should not force the user to redefine the default values
that ship with server.properties. Please see whether the properties merge
as implemented works on your environment or not. If not, what is the
I might be wrong, but I sense there is a requirement here, where Slider
needs to accept custom application specific config files in it¹s original
raw format (like properties, xml, json, yaml, etc.) in addition to
appConfig.json. Then it is expected to merge them with appConfig.json and
send the com
16 matches
Mail list logo
