Hi Roxanne, It seems like you are dealing with message flow control issue [1], where the subscriber client might be slower in processing and acknowledging messages than Cloud Pub/Sub is in sending them. In your case, this might have led to a backlog of messages to one client because it doesn’t have the capacity to handle the influx of messages, thereby, being unable to send its messages quickly enough to the second client it gets stuck.
To resolve this, I suggest that you use the flow control feature of the subscriber to control the rate at which it receives messages. Follow the example provided in this documentation to try out the flow control feature [1]. You can also reference this troubleshooting guide for future reference [2][3]. Hope that helps. Let me know how that works for you. [1] https://cloud.google.com/pubsub/docs/pull#message-flow-control [2] https://cloud.google.com/pubsub/docs/troubleshooting#create [3] https://cloud.google.com/pubsub/docs/push#app-engine-standard-endpoints On Thursday, August 2, 2018 at 3:47:44 AM UTC-4, Roxana Ioana Mirel wrote: > > Hello, > > I have a GKE cluster with three nodes, which are being subscribers to the > same PubSub subscription > and polling continuously for new messages. > I am using the Asynchronous Pull mechanism for my subscribers. These are > running in a NodeJS Express application. > > subscription.on("message", message => messageHandler(message)); > > async function messageHandler(message) { > try { > await doSomeProcessing(); > message.ack(); > } catch (error) { > if (error instanceof DuplicateRowError) { > message.ack(); > } else { > console.log(`Message ${message.id} was not acknowledged. Waiting to > be sent again.`); > } > } > } > > Things go well most of the time, but from time to time the subscribers > stop receiving messages. Redeploying the application doesn't help and they > recover by themselves after aprox 30min- 1 hour, when the pending > messages are being sent. The quota has not been exceeded and > subscription.on("error", > error ..) does not get triggered, the StackDriver logs also don't show > any errors. > > Any idea what might cause this pb? > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/21a5c6b9-51bf-4734-b77c-2501a263dbbb%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
