Re: Problem with kafka with key=None using pyhton-kafka module

[Igal Shilman]

Hello,
The default kafka ingress for remote functions, does require a key
component. The key is being translated to the 'id' part of the
receiving function address.
If your functions are stateless, or the id doesn't have a meaning for you,
you can simply provide a random id.

I hope that helps,
Igal.

On Thu, Feb 10, 2022 at 3:08 PM mrAlexTFB  wrote:

> Hello,
>
> I am following the example in Python Walkthrough
> ,
> I downloaded the zip file with the project skeleton. I'm having a problem
> when changing the key attribute in the function producer.send to none.
> From:
>
> def produce():
> if len(sys.argv) == 2:
> delay_seconds = int(sys.argv[1])
> else:
> delay_seconds = 1
> producer = KafkaProducer(bootstrap_servers=[KAFKA_BROKER])
> for request in random_requests():
> key = request.name.encode('utf-8')
> val = request.SerializeToString()
> producer.send(topic='names', key=key, value=val)
> producer.flush()
> time.sleep(delay_seconds)
>
> To:
>
> def produce():
> if len(sys.argv) == 2:
> delay_seconds = int(sys.argv[1])
> else:
> delay_seconds = 1
> producer = KafkaProducer(bootstrap_servers=[KAFKA_BROKER])
> for request in random_requests():
> key = request.name.encode('utf-8')
> val = request.SerializeToString()
> producer.send(topic='names', key=None, value=val)
> producer.flush()
> time.sleep(delay_seconds)
>
> After doing this the consumer is not displaying anything.
>
> I modified python code so the message arrived is printed and it is not
> being printed here, I suppose that the problem could be a bad configuration
> in module.yaml file?
>
> I understand that by putting key=None the topic partition will be chosen
> randomly, that was the behaviour that I was aiming for as I do not need any
>  ordering in the messages.
>
> Do I need any additional configuration in this walkthrough to achieve this?
>
> Thank you very much in advance.
>
>
>

Problem with kafka with key=None using pyhton-kafka module

[mrAlexTFB]

Hello,

I am following the example in Python Walkthrough
,
I downloaded the zip file with the project skeleton. I'm having a problem
when changing the key attribute in the function producer.send to none.
From:

def produce():
if len(sys.argv) == 2:
delay_seconds = int(sys.argv[1])
else:
delay_seconds = 1
producer = KafkaProducer(bootstrap_servers=[KAFKA_BROKER])
for request in random_requests():
key = request.name.encode('utf-8')
val = request.SerializeToString()
producer.send(topic='names', key=key, value=val)
producer.flush()
time.sleep(delay_seconds)

To:

def produce():
if len(sys.argv) == 2:
delay_seconds = int(sys.argv[1])
else:
delay_seconds = 1
producer = KafkaProducer(bootstrap_servers=[KAFKA_BROKER])
for request in random_requests():
key = request.name.encode('utf-8')
val = request.SerializeToString()
producer.send(topic='names', key=None, value=val)
producer.flush()
time.sleep(delay_seconds)

After doing this the consumer is not displaying anything.

I modified python code so the message arrived is printed and it is not
being printed here, I suppose that the problem could be a bad configuration
in module.yaml file?

I understand that by putting key=None the topic partition will be chosen
randomly, that was the behaviour that I was aiming for as I do not need any
 ordering in the messages.

Do I need any additional configuration in this walkthrough to achieve this?

Thank you very much in advance.