RE: lambda not returning?

[John Hawkins]

Hi,
The connector is up and running - it's reading in files (I'm using custenborder 
file adapter as a simple test adapter but it fails with JDBC too )
.
I have seen issues when I've tried to use transformers that weren't configured 
correctly- sometimes they throw and I get an internal error thrown back to the 
REST client.
I've added instrumentation - not sure how to add the screen shots here !
It clearly shows that it's a single thread all the way down the stack. I've 
added in notation when I'm just about to call the callback and when I've done 
it - the originating thread never takes up the challenge and times out after 90 
seconds.
As I say - I've also put this under a debugger and when I try to go into the 
oncompletion method it just disappears. I have put instrumentation and extra 
trace in the futureCallback#onCompletion method and it never shows.

Regards,
John.


John Hawkins
Senior Architect
   
p: +44 (0) 7879 992532
Follow Coliance:  LinkedIn  | YouTube  |  Facebook  |  Instagram

-Original Message-
From: Chris Egerton  
Sent: Tuesday, April 30, 2024 6:59 PM
To: dev@kafka.apache.org
Subject: Re: lambda not returning?

[EXTERNAL EMAIL : Use caution when opening attachments, clicking links or 
responding to requests]

Hi John,

Do you have logs demonstrating that the connector was able to start up 
successfully? If not, it's possible that instead of the callback being dropped, 
it was properly retained but never invoked because the connector was blocked.

Cheers,

Chris

On Tue, Apr 30, 2024 at 1:57 PM John Hawkins  
wrote:

> Hi folks,
>
> I’ve been working with the standalone kafka for quite a while now and 
> been debugging a problem I’m having when deploying new connectors 
> (using the PUT method but it happens in the POST as well).
>
>
>
> I get a 500 timeout whenever I try to create a new connector. It looks 
> to me like the lambda callback is being called but disappears into a 
> blackhole. So, the original caller – ConnectorsResources.java never 
> gets notified and so times-out.
>
> The herder never calls back…
>
>
>
> herder.putConnectorConfig(connector, connectorConfig, *true*, cb);
>
>
>
> or rather the callback does get #onComplete() called in the 
> doTransition method of the connectorworker but the callback disappears 
> – it never seems to work.
>
>
>
> I’m left wondering if the nested lambda calling is losing the memory 
> somehow and this could be a classloading issue somewhere that is 
> somehow losing the context of the original lambda?
>
>
>
> Has this been seen before?
>
>
>
> I’ve seen a few posts (not in kakfa) where lambda calls fails to 
> callback if there are still threads running in the call itself. From 
> what I can see there are plenty of threads still running in the call
>
> I’m using a 21 microsoft OpenJDK JRE but it also fails with IBMs version.
>
>
>
> Our project is dependent on using the kakfa standalone so I need to 
> get this to work.
>
>
>
> Thanks for your thoughts,
>
> John.
>
>
>
>
>
> John Hawkins
>
> *Senior Architect*
>
>
>
>
>
>
>
>
>
>
>
> *john.hawk...@coliance.co * | 
> www.coliance.co
>
> p: +44 (0) 7879 992532
>
> Follow Coliance:  LinkedIn
> <https://www.linkedin.com/company/colianceco/about/>  | YouTube 
> <https://www.youtube.com/channel/UCfE4Nm5XJMcY0215uTRs4bQ>  |  
> Facebook <http://facebook/>  |  Instagram 
> <https://www.instagram.com/coliance.co/>
>
>
>
>
>

Re: lambda not returning?

[Chris Egerton]

Hi John,

Do you have logs demonstrating that the connector was able to start up
successfully? If not, it's possible that instead of the callback being
dropped, it was properly retained but never invoked because the connector
was blocked.

Cheers,

Chris

On Tue, Apr 30, 2024 at 1:57 PM John Hawkins
 wrote:

> Hi folks,
>
> I’ve been working with the standalone kafka for quite a while now and been
> debugging a problem I’m having when deploying new connectors (using the PUT
> method but it happens in the POST as well).
>
>
>
> I get a 500 timeout whenever I try to create a new connector. It looks to
> me like the lambda callback is being called but disappears into a
> blackhole. So, the original caller – ConnectorsResources.java never gets
> notified and so times-out.
>
> The herder never calls back…
>
>
>
> herder.putConnectorConfig(connector, connectorConfig, *true*, cb);
>
>
>
> or rather the callback does get #onComplete() called in the doTransition
> method of the connectorworker but the callback disappears – it never seems
> to work.
>
>
>
> I’m left wondering if the nested lambda calling is losing the memory
> somehow and this could be a classloading issue somewhere that is somehow
> losing the context of the original lambda?
>
>
>
> Has this been seen before?
>
>
>
> I’ve seen a few posts (not in kakfa) where lambda calls fails to callback
> if there are still threads running in the call itself. From what I can see
> there are plenty of threads still running in the call
>
> I’m using a 21 microsoft OpenJDK JRE but it also fails with IBMs version.
>
>
>
> Our project is dependent on using the kakfa standalone so I need to get
> this to work.
>
>
>
> Thanks for your thoughts,
>
> John.
>
>
>
>
>
> John Hawkins
>
> *Senior Architect*
>
>
>
>
>
>
>
>
>
>
>
> *john.hawk...@coliance.co * | www.coliance.co
>
> p: +44 (0) 7879 992532
>
> Follow Coliance:  LinkedIn
>   | YouTube
>   |  Facebook
>   |  Instagram 
>
>
>
>
>

lambda not returning?

[John Hawkins]

Hi folks,
I've been working with the standalone kafka for quite a while now and been 
debugging a problem I'm having when deploying new connectors (using the PUT 
method but it happens in the POST as well).

I get a 500 timeout whenever I try to create a new connector. It looks to me 
like the lambda callback is being called but disappears into a blackhole. So, 
the original caller - ConnectorsResources.java never gets notified and so 
times-out.

The herder never calls back...

herder.putConnectorConfig(connector, connectorConfig, true, cb);

or rather the callback does get #onComplete() called in the doTransition method 
of the connectorworker but the callback disappears - it never seems to work.

I'm left wondering if the nested lambda calling is losing the memory somehow 
and this could be a classloading issue somewhere that is somehow losing the 
context of the original lambda?

Has this been seen before?

I've seen a few posts (not in kakfa) where lambda calls fails to callback if 
there are still threads running in the call itself. From what I can see there 
are plenty of threads still running in the call
I'm using a 21 microsoft OpenJDK JRE but it also fails with IBMs version.

Our project is dependent on using the kakfa standalone so I need to get this to 
work.

Thanks for your thoughts,
John.


John Hawkins
Senior Architect


[cid:image001.png@01DA9B2E.049E3910]


john.hawk...@coliance.co | www.coliance.co
p: +44 (0) 7879 992532
Follow Coliance:  LinkedIn  
| YouTube  |  
Facebook  |  Instagram