[
https://issues.apache.org/jira/browse/STREAMPIPES-388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17366657#comment-17366657
]
Patrick Wiener commented on STREAMPIPES-388:
--------------------------------------------
Ok - we already did some clean up on the latest dev version that will make it
to the next release. The colocation of the backend and the pipeline-element-all
pod results from the shared PVC, where we only use a local file volume mount.
Depending on your setup you could use a network storage driver, e.g. GlusterFS
that should resolve this issue.
However, I'm currently not quite sure why there seems to be no connection to
the kafka pod via its service dns all of the sudden. I haven't experienced this
problem before. Is there any way we could reproduce this behavior? Just seems
like that for some reason the service name, here "kafka" could not be resolved
anymore. But what strikes me more is the fact that a pipeline restart resolved
the problem. Because technically, this does not change the setup within your
k8s cluster.
> Pipeline stops writing data to mysql inside kubernetes
> -------------------------------------------------------
>
> Key: STREAMPIPES-388
> URL: https://issues.apache.org/jira/browse/STREAMPIPES-388
> Project: StreamPipes
> Issue Type: Bug
> Reporter: Simon Bosse
> Priority: Major
>
> Hi,
> we are using streampipes in kubernetes where we have a mqtt broker in the
> same namespace which is used in a pipeline together with a mysql sink.
> The pipeline works and data is written into the database but in the morning
> around the same time there is no data written anymore into the database. If
> the pipeline is manually restarted it works again. I looked in the logs of
> the connect worker pod and I can see these kind of entries around the time
> when the pipeline stops.
> 04:08:32.561 SP [kafka-admin-client-thread | adminclient-12] WARN
> o.apache.kafka.clients.NetworkClient - [AdminClient clientId=adminclient-12]
> Error connecting to node kafka:9092 (id: 1001 rack: null)
> java.net.UnknownHostException: kafka
> at java.net.InetAddress.getAllByName0(InetAddress.java:1281)
> at java.net.InetAddress.getAllByName(InetAddress.java:1193)
> at java.net.InetAddress.getAllByName(InetAddress.java:1127)
> at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:117)
> at
> org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.moveToNextAddress(ClusterConnectionStates.java:387)
> at
> org.apache.kafka.clients.ClusterConnectionStates.connecting(ClusterConnectionStates.java:121)
> at
> org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:917)
> at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:287)
> at
> org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.sendEligibleCalls(KafkaAdminClient.java:904)
> at
> org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1119)
> at java.lang.Thread.run(Thread.java:823)
>
> I also looked in the logs of the kafka container but there are no entries at
> this time.
> Do you know what could be the reason for this or how I can further debug it?
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
