geo_polygon filter with hole?

[Andrej Rosenheinrich]

Short question about geo_polygon filter as I haven't found anything about 
it in the reference: can the geo_polygon filter handle polygons with holes? 
If yes, how are they defined?

Thanks in advance!
Andrej


-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/e1f71c29-4d1a-46fb-aa48-5cbf9a839e12%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

puppet directory options

[Andrej Rosenheinrich]

Hi,

I once again stumpled over a little problem with the puppet script. I am 
trying to install elasticsearch using puppet module 0.9.3. First, I am 
creating a data directory like this:

  file { "/data/elasticsearch":
ensure  => "directory",
owner   => "elasticsearch",
group   => "elasticsearch",
require => [Class ['common'], Class ['elasticsearch::package']]
  }

Class common is taking care of mounting and formatting the drive, 
elasticsearch package is needed for owner and group, right?

Then I am setting up elasticsearch, I am configuring the datadir in the 
class, not in the instance. Using 

  datadir => '/data/elasticsearch/data'

a directory /data/elasticsearch/data/es-01 is created when setting up the 
instance. Interesting here is that directory /data/elasticsearch/data 
belongs to the group elasticsearch, but /data/elasticsearch/data/es-01 to 
the group root. Why ist that so?

Another thing I tried is to use the hash writeup to configure directories :
config {
path => {
  data => '/data/elasticsearch/data',
  work => '/data/elasticsearch/work',
  logs => '/var/log/elasticsearch'
}
}

It seems to be a valid way of configuration for me, problem here is that 
directories for data and work are not created. I can create datadir using 
the line of code provided first, but what about work directory? Will it be 
created when needed only? Is this way of configuration supposed to work at 
all?

Would really appreciate some help on these questions!

Thanks in advance,
Andrej




-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/fe483e44-e62a-483c-99b9-79a9d4cf336b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Number of shards in 4 node Cluster

[Andrej Rosenheinrich]

Hi Mark,

may I ask what the reason for this recommendation is?

Thanks,
Andrej

Am Mittwoch, 18. März 2015 17:50:09 UTC+1 schrieb Mark Walkom:
>
> We recommend shards no larger than 50GB, but as you mention there is no 
> exact limit.
>
> On 18 March 2015 at 04:09, Georgi Ivanov  > wrote:
>
>> My rules is : 1 primary shard per server.
>>
>> Also make some estimation how big will be the single index/shard 
>>
>> I think it is not good if single shard exceed 10 GB, although there is no 
>> exact limit.
>>
>>
>> Georgi 
>>
>> On Tuesday, March 17, 2015 at 7:00:23 PM UTC+1, John S wrote:
>>>
>>> Hi All,
>>>
>>> Is there any best practices of having on the number of shards for a 
>>> cluster? I have a 4 node cluster and used shards of 20.
>>>
>>> During any node failure or other events i doubts since the shards number 
>>> is high, replication to new node is taking more time...
>>>
>>> Is there any metrics or formula to be done for number or shards?
>>>
>>> Regards
>>> John
>>>
>>  -- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/36ef3ed0-870f-41a5-915b-fb3ad919f7a0%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/385c959b-0132-4d9d-95cd-b4b08fca9c94%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Search for new documents only

[Andrej Rosenheinrich]

I do have the following use case: I want to search for documents with a 
query (e.g. a time range), but only for new documents, meaning I only want 
to return documents where _version is 1. Is there an elegant way to do this 
(_version is not searchable, I think) or do I have to return all documents 
and check version afterwards?

Thanks!
Andrej

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/7f0fa873-e4a8-4aca-a564-e6096fe0077f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: char_filter for German

[Andrej Rosenheinrich]

Hello Jörg,

could you maybe share the configuration for the german_normalize analyzer 
without stemming? I actually only need the umlaut expansion. And what do 
you mean by "at the right places in words" for snowball?

Thanks!
Andrej

Am Sonntag, 30. November 2014 17:20:16 UTC+1 schrieb Jörg Prante:
>
> Do not use regex, this will give wrong results.
>
> Elasticsearch comes with full support for german umlaut handling.
>
> If you install ICU plugin, you can use something like this analysis setting
>
> {
> "index" : {
> "analysis" : {
> "filter" : {
> "german_normalize_stem" : {
>   "type" : "snowball",
>   "name" : "German2"
> }
> },
> "analyzer" : {
> "stemmed" : {
> "type" : "custom",
> "tokenizer" : "standard",
> "filter" : [
> "lowercase",
> "icu_normalizer",
> "icu_folding",
> "german_normalize_stem"
> ]
> },
> "unstemmed" : {
> "type" : "custom",
> "tokenizer" : "standard",
> "filter" : [
> "lowercase",
> "icu_normalizer",
> "icu_folding",
> "german_normalize"
> ]
> }
> }
> }
> }
> }
>
> ICU handles german umlauts, and also case folding like "ss" and "ß".
>
> Snowball handles umlaut expansions (ae, oe, ue) at the right places in 
> words.
>
> You can choose between stemmed and unstemmed analysis. Snowball tends to 
> overstem words. The "german_normalize" token filter is copied from Snowball 
> but works without stemming.
>
> The effect of the combination is that all german words like Jörg,  Joerg, 
> Jorg are reduced to jorg in the index.
>
> Best,
>
> Jörg
>
>
> On Sun, Nov 30, 2014 at 11:37 AM, Krešimir Slugan  > wrote:
>
>> Hi Jürgen,
>>
>> Currently we don't have big volumes of data to index so we would like to 
>> yield more results in hope that proper ones would still be shown in the 
>> top. In future, when we have more data, we'll have to sacrifice some use 
>> cases in order to provide more precise results for the rest of users. 
>>
>> I think I will try regexp token approach to replace umlauts with "e" 
>> forms to solve this double expansion problem. 
>>
>> Best,
>>
>> Krešimir
>>
>> On Saturday, November 29, 2014 11:23:47 PM UTC+1, Jürgen Wagner (DVT) 
>> wrote:
>>>
>>>  Hi Krešimir,
>>>   the correct term is "über" (over, above) or "hören" (hear) or "ändern" 
>>> (change). When you cannot write umlauts, the correct alternative spelling 
>>> in print is "ueber", "hoeren", "aendern". Everybody can write this in 
>>> ASCII. However, those who are possibly non-speakers of German who still 
>>> want to search for German terms are usually not aware of this and believe 
>>> it's like with accents in French, where "á" is lexically treated like "a". 
>>> Those users are wrong in spelling "uber", "horen", "andern" because "u" and 
>>> "ü" are in fact different letters. It's like "ll" in Spanish. "ll" is ONE 
>>> letter :-)
>>>
>>> However, in order to provide a convenience to those users as well,  you 
>>> could decide that - to yield at least some meaningful results - you will 
>>> also consider the versions without the umlaut dots equivalent. In that 
>>> case, you want to map any token containing an umlaut (ä, ö, ü) to three 
>>> alternatives: umlaut, without umlaut marker, alternative spelling with 'e'. 
>>> This won't let you distinguish between the "Bar" (bar, the place to get a 
>>> drink) and "Bär" (bear, the one giving you a great, dangerous hug). 
>>> "Forderung" (demand) and "Förderung" (encouragement, facilitation, 
>>> promotion, extraction [geol.]) are also quite different, just to give a few 
>>> examples.
>>>
>>> For the proper recognition of those terms, you would normally use a 
>>> dictionary of German, including some frequent proper names as well. So, if 
>>> you look for "clown boll", you would not only get "Der Clown im Advent - 
>>> Evangelische Akademie Bad Boll", but also "Heinrich Böll, Ansichten eines 
>>> Clowns", because the query would be transformed into "clown AND (boll OR 
>>> boell OR böll)" as "boll" matches an umlaut candidate in your dictionary. 
>>> If you dare to normalize your indexed texts, so "Boell" would already have 
>>> been turned into "Böll", you could even do with a disjunction of only the 
>>> one correct form and the misspelling. Again, however, you would make use of 
>>> a dictionary to perform such normalization. Ideally, you would even have a 
>>> POS tagger in place, so you would only make such replacements where the 
>>> name Böll is referred to, not the city of Bad Boll.
>>>
>>> It's a question of 

Re: How to query with JSON when use Java API in ElasticSearch?

[Andrej Rosenheinrich]

Hi,

there are QueryBuilder that can parse JSON, e.g. WrapperQueryBuilder 

 wrapperQuery(java.lang.String source). 
This should do the job.

Greets,
Andrej

PS: Still wondering why javadocs disappeared a while ago. This would be a 
great help if javadocs would be available for each version for download.

Am Montag, 15. September 2014 09:22:45 UTC+2 schrieb 张新星:
>
> Hi all,
>  I'm using ElasticSearch as a search service in Spring Web project. 
> Now I'm using Transport Client to communicate with ES. 
> I'm wondering if there exists a method which can construct a QueryBuilder 
> from a JSON DSL. for example, convert this bool query DSL JSON 
> {
>"query" : {
>"bool" : {
>"must" : { "match" : {"content" : "quick"},
>"should": { "match": {"content" : "lazy"}
> }
>}
> }
>
> to a QueryBuilder. I need this function because I have to receive 
> user's bool string input from web front-side, and parse this bool string to 
> a QueryBuilder.
> However it not suit to use 
> QueryBuilders.boolQuery().must(matchQB).should(shouldQB).must_not(mustNotQB). 
> Because we may need several must or non must keywords.
>
>Thanks for your time! I'm really need some help.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/780122c7-16b2-4f17-9c83-b77a6401e85f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elasticsearch cluster on AWS. Article.

[Andrej Rosenheinrich]

David, you are of course right with 2), but one thing to concider is that 
you pay for incoming and outgoing traffic between different availability 
zones.

Am Freitag, 15. August 2014 14:32:54 UTC+2 schrieb David Severski:
>
> Thanks for collecting this information together! A couple points for 
> tweaking:
>
> 1) Instead of hard coding the IAM credentials into the file, associate the 
> instances with an IAM role. cloud-aws will use those automatically and AWS 
> will handle key rotation for you.
> 2) You are launching all the instances into the same availability zone. 
> That greatly reduces the ability of the cluster to tolerate an AWS outage. 
> Stick each of your three nodes in a different availability zone and you'll 
> be much better off.
> 3) EC2-Classic is deprecated. Demonstrating use of VPC would be helpful.
> 4) I encourage AWS hosts _not_ to be named. Users should plan for hosts to 
> come and go. This means no-unique host names and hard coded IPs. AWS is 
> ephemeral infrastructure and ES, as a cluster app, is very happy playing in 
> this space.
>
> and the big one...
>
> 5) Your security group looks to open ES to the world. DON'T DO THIS! 
> There's been a tremendous amount of angst recently from ES clusters getting 
> owned via open tcp/9200 and these security groups look to open your cluster 
> to the entire internet. There's no need for that. cloud-aws will work with 
> private IPs just fine.
>
> David
>
> On Thursday, August 14, 2014 10:13:34 AM UTC-7, Pavel P wrote:
>>
>> Hi everyone,
>>
>> Below you can find one big article, summing up all my experience of 
>> building the cluster on AWS.
>> When I started I had no information at all, but I found the needed pieces 
>> in different places, including this user group.
>>
>> With your help I succeeded, and want to share the knowledge, that 
>> newcomers would find everything in one place.
>>
>> Elasticsearch cluster on AWS. Part 1 - preparing the environment. 
>> 
>> Elasticsearch cluster on AWS. Part 2 - configuring the elasticsearch. 
>> 
>>
>> Hope it would help someone!
>>
>> Regards,
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/aa245f74-5ae1-46ad-b01d-415735345356%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elasticsearch cluster on AWS. Article.

[Andrej Rosenheinrich]

Nice collection, well presented, thanks!

One note, you can probably restrict your permissions even further, instead 
of ReadOnly I use just the following 5 rules (may be even that is too much) 
:

{
  "Statement": [
{
  "Effect": "Allow",
  "Action": [
"ec2:DescribeAvailabilityZones",
"ec2:DescribeInstances", 
"ec2:DescribeRegions",
"ec2:DescribeSecurityGroups",
"ec2:DescribeTags"
  ],
  "Resource": "*"
}
  ]
}

Greets,
Andrej



Am Donnerstag, 14. August 2014 19:13:34 UTC+2 schrieb Pavel P:
>
> Hi everyone,
>
> Below you can find one big article, summing up all my experience of 
> building the cluster on AWS.
> When I started I had no information at all, but I found the needed pieces 
> in different places, including this user group.
>
> With your help I succeeded, and want to share the knowledge, that 
> newcomers would find everything in one place.
>
> Elasticsearch cluster on AWS. Part 1 - preparing the environment. 
> 
> Elasticsearch cluster on AWS. Part 2 - configuring the elasticsearch. 
> 
>
> Hope it would help someone!
>
> Regards,
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/3a76bdfb-1d95-4b32-8769-116641011c1b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: leave content in mySQL and use ElasticSearch only for Index

[Andrej Rosenheinrich]

What I don't understand is why you generate an index and want to store it 
in elasticsearch. You could use the plugin as Jörg suggested, transfer you 
data to elasticsearch, set index:true for the fields you want and set 
store:false in the mapping. This way you get an index build by 
elasticsearch, can search on it, get the id as result and the data is not 
stored (except metadata, if you set it to be stored). See 
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#mapping-core-types.

Cheers,
Andrej

Am Mittwoch, 6. August 2014 15:34:11 UTC+2 schrieb asekn...@gmail.com:
>
> Using this plugin would lead to a migration from mysql data into 
> Elasticsearch.
>
> So let me reformulate my question:
>
> My infrastructure is like this:
>
> client>Elasticsearch
>|
>|
> >mySQL
>
> So I have a client which generates an index and some metadata for a 
> mail(header and body). The mail is stored in mySQL. And the client-side 
> generated index and metadata is stored in Elasticsearch.
>
> The reason is because I have > 1 TB of mail content every day. This 
> content shall still be written to mySQL. Elasticsearch shall keep only the 
> index. Is that possible? And how?
>
> Regards
> Michael
>
>
> Am Mittwoch, 6. August 2014 13:21:09 UTC+2 schrieb asekn...@gmail.com:
>>
>> Hello,
>>
>> I want to use Elasticsearch or only indexing and searching E-Mails. We 
>> want to store the meta-info within Elasticsearch, keeping the content/body 
>> of every Mail in an mySQL database. So Elasticsearch shall have a reference 
>> to the mail body.
>>
>> Is that possible and how?
>>
>> Regards
>> Michael
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/d1ffc059-abad-4b11-8179-35ed3c077cbf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

rss river for 1.3

[Andrej Rosenheinrich]

Hi David,

a probably easy to answer question: is the current stable release 
(rssriver-1.1.0) 
supposed to work with elasticsearch 1.3.1? Will there be a new release any 
time soon? Or a logstash input? ;)

Thanks!
Andrej


-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/5a0dac1b-83e2-438e-9407-d24f00978c2e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] Elasticsearch Twitter River plugin 2.2.0 released

[Andrej Rosenheinrich]

Hi David,

does that mean in conclusion that there will be a rss input for logstash 
soon as replacement of the river? Same goes for wikipedia?

Greets,
Andrej

Am Mittwoch, 23. Juli 2014 14:13:51 UTC+2 schrieb David Pilato:
>
> Rivers will be deprecated in favor of logstash inputs.
> Deprecated does not mean removed yet.
> So in the meantime we still try to keep up to date official plugins.
>
> But yes, you should prefer using if possible logstash twitter input (
> http://logstash.net/docs/1.4.2/inputs/twitter)
>
>
> -- 
> *David Pilato* | *Technical Advocate* | *Elasticsearch.com*
> @dadoonet  | @elasticsearchfr 
> 
>
>
> Le 23 juillet 2014 à 14:08:49, James Green (james.m...@gmail.com 
> ) a écrit:
>
> I was told a week or two ago on IRC that rivers were deprecated in favour 
> of external data sources like Logstash. 
>
> Is this not correct?
>  
>
> On 23 July 2014 13:00, Elasticsearch Team > 
> wrote:
>
>>  Heya,
>>
>> We are pleased to announce the release of the *Elasticsearch Twitter 
>> River plugin*, *version 2.2.0*
>>
>> The Twitter River plugin allows index twitter stream using elasticsearch 
>> rivers feature.
>>
>> Release Notes - Version 2.2.0 Fix 
>>
>>- [62 
>>] 
>>- Generate default mapping even if index already exists 
>>- [60 
>>] 
>>- ignore_retweets does not ignore RT 
>>
>> Update 
>>
>>- [61 
>>] 
>>- Update to Twitter4J 4.0.2 
>>- [58 
>>] 
>>- Remove old deprecated user and password properties 
>>- [57 
>>] 
>>- Deprecate usage of camelCase settings 
>>- [54 
>>] 
>>- Added not_analyzed to retweet user_screen_name 
>>- [52 
>>] 
>>- Update to elasticsearch 1.2.0 
>>
>> New 
>>
>>- [56 
>>] 
>>- Add oauth credentials in `elasticsearch.yml` file 
>>- [51 
>>] 
>>- Support geo location array format 
>>- [50 
>>] 
>>- Add user stream support 
>>- [37 
>>] 
>>- Move tests to elasticsearch test framework 
>>
>> Doc 
>>
>>- [49 
>>] 
>>- [DOC] Link to rivers documentation is incorrect 
>>
>> Issues, Pull requests, Feature requests are warmly welcome on 
>> elasticsearch-river-twitter 
>>  project 
>> repository!
>>
>> For questions or comments around this plugin, feel free to use 
>> elasticsearch mailing list 
>> !
>>
>> Enjoy,
>>
>> - The Elasticsearch team 
>>  --
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/53cfa3ef.8124b40a.2b81.3571SMTPIN_ADDED_MISSING%40gmr-mx.google.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>  
> --
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/CAMH6%2Bawb9LYHPnm_5yxVJBr6c6xNkuz200mjES52uyCOPTfK%3Dg%40mail.gmail.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https:/

Re: puppet-elasticsearch options

[Andrej Rosenheinrich]

Hi Richard,

another question: you are creating the elasticsearch user and group 
somewhere in the module (havent found exactly where yet). My problem is 
that I have to create a directory for data_dir (on a different device) that 
is needed by the class (or instance, not sure), but I need the owner and 
the group to be able to set it otherwise the service won't start. Can I set 
a requirement in my file declaration to make sure that the user and the 
group already exist? Something like

  file { "/data/elasticsearch":
ensure  => "directory",
owner   => "elasticsearch",
group   => "elasticsearch",
require => ???
  }


Once again, thanks!
Andrej

Am Dienstag, 1. Juli 2014 14:37:55 UTC+2 schrieb Richard Pijnenburg:
>
> Hi Andrej,
>
> Sorry for the late response. Didn't get an update email about it.
>
> As long as you don't setup an instance with the 'elasticsearch::instance' 
> define it will only install the package but do nothing afterwards.
> I recently fixed that the default files from the packages are being 
> removed now.
> The memory can be set via the init_defaults hash by setting the ES_HEAP 
> option.
>
> The issue with 0.90.x versions is that it automatically starts up after 
> package installation.
> Since i don't stop it, it keeps running. Its advised to run a newer 
> version of ES since 0.90.x will be EOL'd at some point.
>
>
> On Thursday, June 26, 2014 2:24:47 PM UTC+1, Andrej Rosenheinrich wrote:
>>
>> Hi Richard,
>>
>> thanks for your answer, it for sure helped! Still, I am puzzling with a 
>> few effects and questions:
>>
>> 1.) I am a bit confused by your class/instance idea. I can do something 
>> pretty simple like class { 'elasticsearch' :  version => '0.90.7' } and it 
>> will install elasticsearch in the correct version using the default 
>> settings you defined. Repeating this (I tested every step on a fresh debian 
>> instance in a VM, no different puppet installation steps in between) with a 
>> config added in class like 
>>
>> class { 'elasticsearch' :
>> version => '0.90.7',
>> config => {
>>   'cluster'=> {
>> 'name' => 'andrejtest'
>>   },
>>   'http.port' => '9210'
>> }
>> }
>>   
>> I still get elasticsearch installed, but it completely ignores everything 
>> in the config. (I should be able to curl localhost:9210, but its up and 
>> running on the old default port, using the old cluster name). You explained 
>> overwriting for instances and classes a bit, so I tried the following thing 
>> (again, blank image, no previous installation) :
>>
>>   class { 'elasticsearch' :
>> version => '0.90.7',
>> config => {
>>   'cluster'=> {
>> 'name' => 'andrejtest'
>>   },
>>   'http.port' => '9210'
>> }
>>   }
>>
>>   elasticsearch::instance { 'es-01':
>>   }
>>
>> What happened is that I have two elasticsearch instances running, one 
>> with the default value and another one (es-01) that uses the provided 
>> configuration. Even freakier, I install java7 in my script before the 
>> snippet posted , the first (default based) elasticsearch version uses the 
>> standard openjdk-6 java, the second instance (es-01) uses java7. 
>> So, where is my mistake or what am I doing wrong? What would be the way 
>> to install and start only one service using provided configuration? And 
>> does elasticsearch::instance require an instance name? I would really miss 
>> the funny comic node names ;)
>>
>> 2. As you pointed out I can define all values from elasticsearch.yml in 
>> the config hash. But what about memory settings (I usually modify the 
>> init.d script for that), can I configure Xms and Xmx settings in the puppet 
>> module somehow?
>>
>> Logging configuration would be a nice-to-have (no must-have), just in 
>> case you were wondering ;)
>>
>> I hope my questions don't sound too confusing, if you could give me a 
>> hint on what I am doing wrong I would really appreciate it.
>>
>> Thanks in advance!
>> Andrej
>>
>>
>> Am Freitag, 20. Juni 2014 09:44:49 UTC+2 schrieb Richard Pijnenburg:
>>>
>>> Hi Andrej,
>>>
>>> Thank you for using the puppet module :-)
>>>
>>

Re: puppet-elasticsearch options

[Andrej Rosenheinrich]

Hi Richard,

getting back to this after a while. Thanks for pointing me to the fact, 
that the class itself actually does nothing than install and only instances 
merge configs. This was something completely not aware to me (may be you 
could add a line in the documentation?) but looking through the module I 
could figure it out.
What I am wondering, what was the reason for this design decision? When I 
want to install elasticsarch once on a machine I have to configure the 
class (things like manage_repo can not be configured in the instance, 
right?) and an instance. This will install two services, one for the class, 
not caring about config, and one for the instance, what might be confusing 
for some users. 

Again, thanks for your help that finally got me on the right track!
Andrej

Am Dienstag, 1. Juli 2014 14:37:55 UTC+2 schrieb Richard Pijnenburg:
>
> Hi Andrej,
>
> Sorry for the late response. Didn't get an update email about it.
>
> As long as you don't setup an instance with the 'elasticsearch::instance' 
> define it will only install the package but do nothing afterwards.
> I recently fixed that the default files from the packages are being 
> removed now.
> The memory can be set via the init_defaults hash by setting the ES_HEAP 
> option.
>
> The issue with 0.90.x versions is that it automatically starts up after 
> package installation.
> Since i don't stop it, it keeps running. Its advised to run a newer 
> version of ES since 0.90.x will be EOL'd at some point.
>
>
> On Thursday, June 26, 2014 2:24:47 PM UTC+1, Andrej Rosenheinrich wrote:
>>
>> Hi Richard,
>>
>> thanks for your answer, it for sure helped! Still, I am puzzling with a 
>> few effects and questions:
>>
>> 1.) I am a bit confused by your class/instance idea. I can do something 
>> pretty simple like class { 'elasticsearch' :  version => '0.90.7' } and it 
>> will install elasticsearch in the correct version using the default 
>> settings you defined. Repeating this (I tested every step on a fresh debian 
>> instance in a VM, no different puppet installation steps in between) with a 
>> config added in class like 
>>
>> class { 'elasticsearch' :
>> version => '0.90.7',
>> config => {
>>   'cluster'=> {
>> 'name' => 'andrejtest'
>>   },
>>   'http.port' => '9210'
>> }
>> }
>>   
>> I still get elasticsearch installed, but it completely ignores everything 
>> in the config. (I should be able to curl localhost:9210, but its up and 
>> running on the old default port, using the old cluster name). You explained 
>> overwriting for instances and classes a bit, so I tried the following thing 
>> (again, blank image, no previous installation) :
>>
>>   class { 'elasticsearch' :
>> version => '0.90.7',
>> config => {
>>   'cluster'=> {
>> 'name' => 'andrejtest'
>>   },
>>   'http.port' => '9210'
>> }
>>   }
>>
>>   elasticsearch::instance { 'es-01':
>>   }
>>
>> What happened is that I have two elasticsearch instances running, one 
>> with the default value and another one (es-01) that uses the provided 
>> configuration. Even freakier, I install java7 in my script before the 
>> snippet posted , the first (default based) elasticsearch version uses the 
>> standard openjdk-6 java, the second instance (es-01) uses java7. 
>> So, where is my mistake or what am I doing wrong? What would be the way 
>> to install and start only one service using provided configuration? And 
>> does elasticsearch::instance require an instance name? I would really miss 
>> the funny comic node names ;)
>>
>> 2. As you pointed out I can define all values from elasticsearch.yml in 
>> the config hash. But what about memory settings (I usually modify the 
>> init.d script for that), can I configure Xms and Xmx settings in the puppet 
>> module somehow?
>>
>> Logging configuration would be a nice-to-have (no must-have), just in 
>> case you were wondering ;)
>>
>> I hope my questions don't sound too confusing, if you could give me a 
>> hint on what I am doing wrong I would really appreciate it.
>>
>> Thanks in advance!
>> Andrej
>>
>>
>> Am Freitag, 20. Juni 2014 09:44:49 UTC+2 schrieb Richard Pijnenburg:
>>>
>>> Hi Andrej,
>>>
>>&

Re: puppet-elasticsearch options

[Andrej Rosenheinrich]

Hi Richard,

thanks for your answer, it for sure helped! Still, I am puzzling with a few 
effects and questions:

1.) I am a bit confused by your class/instance idea. I can do something 
pretty simple like class { 'elasticsearch' :  version => '0.90.7' } and it 
will install elasticsearch in the correct version using the default 
settings you defined. Repeating this (I tested every step on a fresh debian 
instance in a VM, no different puppet installation steps in between) with a 
config added in class like 

class { 'elasticsearch' :
version => '0.90.7',
config => {
  'cluster'=> {
'name' => 'andrejtest'
  },
  'http.port' => '9210'
}
}
  
I still get elasticsearch installed, but it completely ignores everything 
in the config. (I should be able to curl localhost:9210, but its up and 
running on the old default port, using the old cluster name). You explained 
overwriting for instances and classes a bit, so I tried the following thing 
(again, blank image, no previous installation) :

  class { 'elasticsearch' :
version => '0.90.7',
config => {
  'cluster'=> {
'name' => 'andrejtest'
  },
  'http.port' => '9210'
}
  }

  elasticsearch::instance { 'es-01':
  }

What happened is that I have two elasticsearch instances running, one with 
the default value and another one (es-01) that uses the provided 
configuration. Even freakier, I install java7 in my script before the 
snippet posted , the first (default based) elasticsearch version uses the 
standard openjdk-6 java, the second instance (es-01) uses java7. 
So, where is my mistake or what am I doing wrong? What would be the way to 
install and start only one service using provided configuration? And does 
elasticsearch::instance require an instance name? I would really miss the 
funny comic node names ;)

2. As you pointed out I can define all values from elasticsearch.yml in the 
config hash. But what about memory settings (I usually modify the init.d 
script for that), can I configure Xms and Xmx settings in the puppet module 
somehow?

Logging configuration would be a nice-to-have (no must-have), just in case 
you were wondering ;)

I hope my questions don't sound too confusing, if you could give me a hint 
on what I am doing wrong I would really appreciate it.

Thanks in advance!
Andrej


Am Freitag, 20. Juni 2014 09:44:49 UTC+2 schrieb Richard Pijnenburg:
>
> Hi Andrej,
>
> Thank you for using the puppet module :-)
>
> The 'port' and 'discovery minimum' settings are both configuration 
> settings for the elasticsearch.yml file.
> You can set those in the 'config' option variable, for example:
>
> elasticsearch::instance { 'instancename':
>   config => { 'http.port' => '9210', 'discovery.zen.minimum_master_nodes' 
> => 3 }
> }
>
>
> For the logging part, management of the logging.yml file is very limited 
> at the moment but i hope to get some feedback on extending that.
> The thresholds for the slowlogs can be set in the same config option 
> variable.
> See 
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-slowlog.html#index-slow-log
>  
> for more information.
>
> If you have any further questions, let me know.
>
> Cheers
>
> On Thursday, June 19, 2014 9:53:10 AM UTC+1, Andrej Rosenheinrich wrote:
>>
>> Hi,
>>
>> i am playing around with puppet-easticsearch 0.4.0, works wells so far 
>> (thanks!), but I am missing a few options I havent seen in the 
>> documentation. As I couldnt figure it out immediately by reading the 
>> scripts, may be someone can help me fast on this:
>>
>> - there is an option to change the port (9200), but this is only the http 
>> port. Is there an option to change the tcp transport port as well?
>> - how can I configure logging? I think about logfile names and loglevel, 
>> may be even thresholds for slowlog. May be this is interesting enough to 
>> add it to the documentation?
>> - is there an option in the module to easily configure memory usage?
>> - how can I configure the discovery minimum?
>>
>> I am aware that I could go ahead and manipulate the elasticsearch.yml 
>> file with puppet, I am just curious if there are options for my questions 
>> already implemented in the module I have missed. So if someone could give 
>> me a hint or an example it would be really helpful!
>>
>> Thanks in advance!
>> Andrej
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/dc849f26-3d9f-4f47-9a55-2b2476029b55%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

puppet-elasticsearch options

[Andrej Rosenheinrich]

Hi,

i am playing around with puppet-easticsearch 0.4.0, works wells so far 
(thanks!), but I am missing a few options I havent seen in the 
documentation. As I couldnt figure it out immediately by reading the 
scripts, may be someone can help me fast on this:

- there is an option to change the port (9200), but this is only the http 
port. Is there an option to change the tcp transport port as well?
- how can I configure logging? I think about logfile names and loglevel, 
may be even thresholds for slowlog. May be this is interesting enough to 
add it to the documentation?
- is there an option in the module to easily configure memory usage?
- how can I configure the discovery minimum?

I am aware that I could go ahead and manipulate the elasticsearch.yml file 
with puppet, I am just curious if there are options for my questions 
already implemented in the module I have missed. So if someone could give 
me a hint or an example it would be really helpful!

Thanks in advance!
Andrej

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/01bd5d3c-f00b-4a5d-b341-054af734462d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.