For example this is the message we get
HTTP response body:
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "rpc error: code = ResourceExhausted desc = grpc: trying to send
message larger than max (11652891 vs. 2097152)",
"code": 500
}
I know that this indicates we have exceeded data volume however I am still
curious to hear which architectural approach is 'better'.
Thanks for any assistance.
lewismc
On 2022/03/04 18:01:30 lewis john mcgibbney wrote:
> Hi users,
> We are using the KubernetesPodOperator to isolate some code which writes
> data into a VERY OLD Elasticsearch 2.X cluster. Please don't make fun of me
> for this!!!
> We are wondering, does a recommended practice exists for processing (JSON)
> data within the KubernetesPodOperator?
> Currently, we've experimented with passing various volumes of JSON string
> data to the KubernetesPodOperator 'argument' parameter. This works for
> reasonably small record batches such as 100's but fails for >10k's records.
> Should we be using a custom XCom backend to pull data into the container
> rather than push it via 'arguments'?
> Thank you
> lewismc
>
> --
> http://home.apache.org/~lewismc/
> http://people.apache.org/keys/committer/lewismc
>
