Hi,
The ES version is "2.2.0".
This is the HTTP response. Look for the status 503 in the payload:
T 127.0.0.1:9200 -> 127.0.0.1:34497 [AP]
HTTP/1.1 200 OK.
Content-Type: application/json; charset=UTF-8.
Content-Length: 3770.
{
- "took": 39911,
- "errors": true,
- "items": [
- {
- "create": {
- "_index": "vdps-log-wf-2016.04.26",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qa",
- "status": 503,
- "error": {
- "type": "process_cluster_event_timeout_exception",
- "reason": "failed to process cluster event (put-mapping
[WAF]) within 30s"
}
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.04.26",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qb",
- "status": 503,
- "error": {
- "type": "process_cluster_event_timeout_exception",
- "reason": "failed to process cluster event (put-mapping
[WAF]) within 30s"
}
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.04.26",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qc",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.03.28",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qd",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.03.28",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qe",
- "status": 503,
- "error": {
- "type": "process_cluster_event_timeout_exception",
- "reason": "failed to process cluster event (put-mapping
[WAF]) within 30s"
}
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.03.28",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qf",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.04.26",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qg",
- "status": 503,
- "error": {
- "type": "process_cluster_event_timeout_exception",
- "reason": "failed to process cluster event (put-mapping
[WAF]) within 30s"
}
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.04.26",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qh",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.04.26",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qi",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.04.26",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qj",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.04.26",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qk",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.04.26",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4ql",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.03.28",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qm",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.03.28",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qn",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.03.28",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qo",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.04.26",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qp",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.04.26",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qq",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.04.26",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qr",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.04.26",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qs",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.04.26",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qt",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
},
- {
- "create": {
- "_index": "vdps-log-wf-2016.04.26",
- "_type": "WAF",
- "_id": "AVRTguzBSCHabxnyv4qu",
- "_version": 1,
- "_shards": {
- "total": 3,
- "successful": 3,
- "failed": 0
},
- "status": 201
}
}
]
}
On Wed, Apr 27, 2016 at 3:41 AM, Swann Croiset <swan...@gmail.com> wrote:
> Hi,
>
> IIRC, according to the code you're right, heka doesn't handle such case.
>
> That said, I've never seen a such ES response ... I'm interested about it.
>
> Could you share these informations: the ES response, the ES version and ES
> logs (when the thing happens)
> also, what is your configuration on ES side? index template, field mapping
> ?
>
> --
> Swann
>
>
>
> 2016-04-26 22:28 GMT+02:00 Ramin Ali Dousti <dou...@gmail.com>:
>
>> Hi,
>>
>> I have an ES output that bulk uploads to a cluster. The HTTP status code
>> is 200 OK but the reply payload says that it failed the upload for a few of
>> the items. But heka doesn't seem to care about the failed items. I looked
>> at the code and it says:
>>
>>
>> https://github.com/mozilla-services/heka/blob/dev/plugins/elasticsearch/elasticsearch.go#L429
>>
>> if response != nil {
>>
>> defer response.Body.Close()
>>
>> if response_body, err = ioutil.ReadAll(response.Body); err != nil
>> {
>>
>> return fmt.Errorf("Can't read HTTP response body. Status:
>> %s. Error: %s",
>>
>> response.Status, err.Error()), true
>>
>> }
>>
>> err = json.Unmarshal(response_body, &response_body_json)
>>
>> if err != nil {
>>
>> return fmt.Errorf("HTTP response didn't contain valid
>> JSON. Status: %s. Body: %s",
>>
>> response.Status, string(response_body)), true
>>
>> }
>>
>> json_errors, ok := response_body_json["errors"].(bool)
>>
>> if ok && json_errors && response.StatusCode != 200 {
>>
>> return fmt.Errorf(
>>
>> "ElasticSearch server reported error within JSON.
>> Status: %s. Body: %s",
>>
>> response.Status, string(response_body)), false
>>
>> }
>>
>> if response.StatusCode > 304 {
>>
>> return fmt.Errorf("HTTP response error. Status: %s. Body:
>> %s", response.Status,
>>
>> string(response_body)), false
>>
>> }
>>
>> }
>>
>>
>> 1- In my case I see a 200 OK with "errors = true" which does not seem to
>> be caught, according to the code.
>> 2- I don't see any logic for recovery based on individual items. Am I
>> missing anything here?
>>
>>
>> --
>> Ramin
>>
>> _______________________________________________
>> Heka mailing list
>> Heka@mozilla.org
>> https://mail.mozilla.org/listinfo/heka
>>
>>
>
--
Ramin
_______________________________________________
Heka mailing list
Heka@mozilla.org
https://mail.mozilla.org/listinfo/heka
