hello Claus,
Thank you for responding. i tried with camel 3.17.0 and kafka clients 3.2.0
, but I still see the CPU busy, The process is busy at 99% even i am not
sending my messages to the topics.
Looks like Kafka clients calls(polling) are doing CPU-intensive operations
repeatedly. Any configuration which can reduce the number of Kafka Clients
calls?
Attached below is the Real Time process Monitor output, $sr10 & $sr11
running on CPUs 2 & 3 respectively, are my processes running Kafka
endpoint.
Cpus Cp hh:mm ss Busy Sec QLen Disp Disk Chit Swap MLock Pcb PcbX
-------- -- -------- ---% --- ---- ------ ----- ------ ---- ----% --- ----
0 18:57:29 10 998 1 12.88
29 359
1 18:57:29 10 843 12.43
21 237
* 2 *18:57:29 *18 * 10 1 1191 326
11.27 12 276
* 3 *18:57:29 * 17 *10 1 1168
11.11 10 465
4 18:57:29 10 821 6.15 12
149
5 18:57:29 10 832 6.24
9 153
6 18:57:29 10 805 7.79 9
141
Process Cpu,Pin Pri Busy% Name RPM T0877(16SEP16) ET=10.1 Top User
-------- ------- --- ----- -------- -------------------------- --- -------
3,612 170 *99.62* *$SR11 * /jdk110_l11/bin/java 1
201,4
2,1073 170 * 98.59 $SR10* /jdk110_l11/bin/java 2
201,4
2,1076 170 .99 /jdk110_l11/bin/javahelper 3 201,4
2,1077 170 .99 /jdk110_l11/bin/javahelper 4 201,4
2,1074 170 .98 /jdk110_l11/bin/javahelper 5 201,4
The route looks like :
<route id="kafka_endpoint-kafka">
<from
uri="kafka:kavi-reqs?brokers=<IP>:9092&fetchMinBytes=20480&reconnectBackoffMaxMs=10000&reconnectBackoffMs=2000&groupId=APIG"/>
<doTry>
<process ref="api.gway.kafka_endpoint.kafka.request"/>
<to uri="custom_component"/>
<doCatch >
<exception>java.lang.Exception</exception>
<!--setBody-->
<transform>
<simple>Error reported: ${exception.message} - cannot process this message.
</simple>
</transform>
<!--/setBody-->
<log message="${exception.stacktrace}" />
</doCatch>
</doTry>
On Sat, Jun 25, 2022 at 6:40 PM Claus Ibsen <[email protected]> wrote:
> Hi
>
> Are you able to try with a newer Camel version as we have improved
> camel-kafka a lot over the last couple of years.
>
>
>
>
> On Fri, Jun 24, 2022 at 4:06 PM Rohan Emmanuel <[email protected]
> >
> wrote:
>
> > hi,
> > i can using camel 3.7.0 with kafka consumer endpoint with default
> > configurations. when I start my process, even though i am not sending any
> > messages to the topic, i see high CPU utilization by the java process. by
> > looking at JMeter for the utilization, the KafkaConsumer poll is
> resulting
> > in a lot number of calls internally.
> > Tried different configurations of fetchMinBytes ,reconnectBackoffMs &
> > reconnectBackoffMaxMs but these did not alter the HIGH CPU utilization.
> >
> > Any help on what configuration can be done to avoid high CPU utilization?
> > any pointers would be appreciated.
> >
> > --
> > Regards,
> > Rohan Emmanuel
> >
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>
--
Regards,
Rohan Emmanuel
