On 16/10/2019 08:35, Dweep Sharma wrote:
From the docs, it seems every time a producer needs to be created for
publishing a message, which involves a connection
Is there a REST API which can be used to simply POST a message to a
topic instead ?
You do realise that making a HTTP POST also involves making a new
connection each time? (Unless the client uses HTTP keepalive to hold the
connection open for a while).
What's the problem you're trying to solve?
If not, does it make sense to use a client library, hosted as a Lambda
function + API gateway
It might make sense if the client which wants to talk to Pulsar doesn't
have a Pulsar client library available. However, there are some things
about Pulsar architecture you need to beware of.
Producers are responsible for generating sequential message IDs
<https://pulsar.apache.org/docs/en/concepts-messaging/>, and each
producer needs a unique name to distinguish its message IDs from any
other producer. You can't have multiple connections open with the same
producer name.
Lambda functions scale up and down on demand, so the number of producers
you have will vary. By default, each time the lambda function starts it
will generate a random producer name and start its own message ID
sequence from scratch, until it dies. This will work for simple message
delivery, as long as you don't care about messages being delivered
in-order or deduplication. The way deduplication works is that you need
to present the same producer name and message ID. Each separate HTTP
client POST will land on an arbitrary member of your pool of lambda
functions / producers; and the original producer may no longer be
running anyway.
If the problem is that you want Javascript running in a web browser to
talk to Pulsar, you would probably be better off using the WebSocket API
<https://pulsar.apache.org/docs/en/client-libraries-websocket/>.
To support languages which don't have a Pulsar client API available,
you'd still likely be better off finding a WebSocket client library for
that language, rather than try to bolt a REST frontend onto Pulsar.
Regards,
Brian.
