Re: Camel-k operator fails on building

[Roberto Camelk]

Yes, I found the problem. Thanks and sorry


On Thu, Dec 9, 2021 at 4:38 PM Antonin Stefanutti
 wrote:
>
> The build fails because it cannot find the specified registry Secret.
>
> It seems you created the Secret in the `camel` namespace, while the operator 
> and the build run in the `default` namespace.
>
> > On 9 Dec 2021, at 20:14, Roberto Camelk  
> > wrote:
> >
> > I installed camel-k in my "default" namespace in minikube. This is my 
> > scripts:
> >
> > read -p 'Dockerhub organization: ' dockerorganization
> > read -p 'Dockerhub user name: ' dockeruser
> > read -sp 'Dockerhub user password: ' dockerpassword
> >
> > kubectl \
> >  -n camel \
> >  create secret docker-registry camel-k-registry-secret \
> >  --docker-username $dockeruser \
> >  --docker-password "$dockerpassword"
> >
> > kamel install \
> >  --force \
> >  --build-publish-strategy=Spectrum \
> >  --olm=false \
> >  -n default \
> >  --operator-resources requests.cpu=100m \
> >  --operator-resources limits.cpu=200m \
> >  --operator-resources requests.memory=256Mi \
> >  --operator-resources limits.memory=1024Mi \
> >  --registry docker.io \
> >  --organization $dockerorganization \
> >  --registry-secret camel-k-registry-secret \
> >  --wait
> >
> > Bu when I try to run my integration, the build fails with:
> >
> > {"level":"info","ts":1639077040.119277,"logger":"camel-k.builder","msg":"base
> > image: adoptopenjdk/openjdk11:slim"}
> > {"level":"info","ts":1639077040.1192799,"logger":"camel-k.builder","msg":"resolved
> > base image: adoptopenjdk/openjdk11:slim"}
> > {"level":"error","ts":1639077040.1429865,"logger":"camel-k.controller.build","msg":"Build
> > kit-c6p503epuevc73829220 failed: secrets \"camel-k-registry-secret\"
> > not 
> > found","request-namespace":"default","request-name":"kit-c6p503epuevc73829220","api-version":"camel.apache.org/v1","kind":"Build","ns":"default","name":"kit-c6p503epuevc73829220"}
> > {"level":"info","ts":1639077040.2197232,"logger":"camel-k.controller.build","msg":"state
> > transition","request-namespace":"default","request-name":"kit-c6p503epuevc73829220","api-version":"camel.apache.org/v1","kind":"Build","ns":"default","name":"kit-c6p503epuevc73829220","phase-from":"Running","phase-to":"Failed"}
> > {"level":"info","ts":1639077040.2216291,"logger":"camel-k.controller.build","msg":"Reconciling
> > Build","request-namespace":"default","request-name":"kit-c6p503epuevc73829220"}
> > {"level":"info","ts":1639077040.2217255,"logger":"camel-k.controller.build","msg":"Invoking
> > action 
> > error-recovery","request-namespace":"default","request-name":"kit-c6p503epuevc73829220","api-version":"camel.apache.org/v1","kind":"Build","ns":"default","name":"kit-c6p503epuevc73829220"}
> > {"level":"info","ts":1639077040.2217386,"logger":"camel-k.controller.build","msg":"Recovery
> > attempt 
> > (5/5)","request-namespace":"default","request-name":"kit-c6p503epuevc73829220","api-version":"camel.apache.org/v1","kind":"Build","ns":"default","name":"kit-c6p503epuevc73829220"}
>

Re: Camel-k operator fails on building

[Antonin Stefanutti]

The build fails because it cannot find the specified registry Secret.

It seems you created the Secret in the `camel` namespace, while the operator 
and the build run in the `default` namespace.

> On 9 Dec 2021, at 20:14, Roberto Camelk  wrote:
> 
> I installed camel-k in my "default" namespace in minikube. This is my scripts:
> 
> read -p 'Dockerhub organization: ' dockerorganization
> read -p 'Dockerhub user name: ' dockeruser
> read -sp 'Dockerhub user password: ' dockerpassword
> 
> kubectl \
>  -n camel \
>  create secret docker-registry camel-k-registry-secret \
>  --docker-username $dockeruser \
>  --docker-password "$dockerpassword"
> 
> kamel install \
>  --force \
>  --build-publish-strategy=Spectrum \
>  --olm=false \
>  -n default \
>  --operator-resources requests.cpu=100m \
>  --operator-resources limits.cpu=200m \
>  --operator-resources requests.memory=256Mi \
>  --operator-resources limits.memory=1024Mi \
>  --registry docker.io \
>  --organization $dockerorganization \
>  --registry-secret camel-k-registry-secret \
>  --wait
> 
> Bu when I try to run my integration, the build fails with:
> 
> {"level":"info","ts":1639077040.119277,"logger":"camel-k.builder","msg":"base
> image: adoptopenjdk/openjdk11:slim"}
> {"level":"info","ts":1639077040.1192799,"logger":"camel-k.builder","msg":"resolved
> base image: adoptopenjdk/openjdk11:slim"}
> {"level":"error","ts":1639077040.1429865,"logger":"camel-k.controller.build","msg":"Build
> kit-c6p503epuevc73829220 failed: secrets \"camel-k-registry-secret\"
> not 
> found","request-namespace":"default","request-name":"kit-c6p503epuevc73829220","api-version":"camel.apache.org/v1","kind":"Build","ns":"default","name":"kit-c6p503epuevc73829220"}
> {"level":"info","ts":1639077040.2197232,"logger":"camel-k.controller.build","msg":"state
> transition","request-namespace":"default","request-name":"kit-c6p503epuevc73829220","api-version":"camel.apache.org/v1","kind":"Build","ns":"default","name":"kit-c6p503epuevc73829220","phase-from":"Running","phase-to":"Failed"}
> {"level":"info","ts":1639077040.2216291,"logger":"camel-k.controller.build","msg":"Reconciling
> Build","request-namespace":"default","request-name":"kit-c6p503epuevc73829220"}
> {"level":"info","ts":1639077040.2217255,"logger":"camel-k.controller.build","msg":"Invoking
> action 
> error-recovery","request-namespace":"default","request-name":"kit-c6p503epuevc73829220","api-version":"camel.apache.org/v1","kind":"Build","ns":"default","name":"kit-c6p503epuevc73829220"}
> {"level":"info","ts":1639077040.2217386,"logger":"camel-k.controller.build","msg":"Recovery
> attempt 
> (5/5)","request-namespace":"default","request-name":"kit-c6p503epuevc73829220","api-version":"camel.apache.org/v1","kind":"Build","ns":"default","name":"kit-c6p503epuevc73829220"}

Camel-k operator fails on building

[Roberto Camelk]

I installed camel-k in my "default" namespace in minikube. This is my scripts:

read -p 'Dockerhub organization: ' dockerorganization
read -p 'Dockerhub user name: ' dockeruser
read -sp 'Dockerhub user password: ' dockerpassword

kubectl \
  -n camel \
  create secret docker-registry camel-k-registry-secret \
  --docker-username $dockeruser \
  --docker-password "$dockerpassword"

kamel install \
  --force \
  --build-publish-strategy=Spectrum \
  --olm=false \
  -n default \
  --operator-resources requests.cpu=100m \
  --operator-resources limits.cpu=200m \
  --operator-resources requests.memory=256Mi \
  --operator-resources limits.memory=1024Mi \
  --registry docker.io \
  --organization $dockerorganization \
  --registry-secret camel-k-registry-secret \
  --wait

Bu when I try to run my integration, the build fails with:

{"level":"info","ts":1639077040.119277,"logger":"camel-k.builder","msg":"base
image: adoptopenjdk/openjdk11:slim"}
{"level":"info","ts":1639077040.1192799,"logger":"camel-k.builder","msg":"resolved
base image: adoptopenjdk/openjdk11:slim"}
{"level":"error","ts":1639077040.1429865,"logger":"camel-k.controller.build","msg":"Build
kit-c6p503epuevc73829220 failed: secrets \"camel-k-registry-secret\"
not 
found","request-namespace":"default","request-name":"kit-c6p503epuevc73829220","api-version":"camel.apache.org/v1","kind":"Build","ns":"default","name":"kit-c6p503epuevc73829220"}
{"level":"info","ts":1639077040.2197232,"logger":"camel-k.controller.build","msg":"state
transition","request-namespace":"default","request-name":"kit-c6p503epuevc73829220","api-version":"camel.apache.org/v1","kind":"Build","ns":"default","name":"kit-c6p503epuevc73829220","phase-from":"Running","phase-to":"Failed"}
{"level":"info","ts":1639077040.2216291,"logger":"camel-k.controller.build","msg":"Reconciling
Build","request-namespace":"default","request-name":"kit-c6p503epuevc73829220"}
{"level":"info","ts":1639077040.2217255,"logger":"camel-k.controller.build","msg":"Invoking
action 
error-recovery","request-namespace":"default","request-name":"kit-c6p503epuevc73829220","api-version":"camel.apache.org/v1","kind":"Build","ns":"default","name":"kit-c6p503epuevc73829220"}
{"level":"info","ts":1639077040.2217386,"logger":"camel-k.controller.build","msg":"Recovery
attempt 
(5/5)","request-namespace":"default","request-name":"kit-c6p503epuevc73829220","api-version":"camel.apache.org/v1","kind":"Build","ns":"default","name":"kit-c6p503epuevc73829220"}

Re: Adding something new into Camel DSL -- but how?

[Claus Ibsen]

Hi Steven

Good to know. Sorry I am too focused on 3.14 LTS release, a few last
things to clear up.

About ascii doc links, then they can be tricky. There has been some
improvements/updates to the website build and whatnot,
so its a bit in "flux" at the moment. So if you need to continue then
do not add links with xref yet, but add a TODO etc in the doc then we
can look at that later.

On Sat, Dec 4, 2021 at 4:51 PM Steve973  wrote:
>
> Hi, Claus, and thanks for the reply.  I created the ticket on 31 Oct:
> https://issues.apache.org/jira/projects/CAMEL/issues/CAMEL-17154 and I am
> working on it in the evenings as time allows.  I have finished the initial
> implementation, and I am working on tests and the asciidoc stuff.  So far,
> I have gotten through at least half of the happy-path unit tests, and I
> have some questions about aspects of the asciidoc format and content,
> mostly related to the links that I see in some of the other adoc files that
> I am using as examples.  I just added my notes about camel-core, and I
> added a comment with some of my design and implementation ideas.  Any of my
> questions can wait until the 3.14 LTS release is complete.
>
> Thanks again,
> Steve
>
> On Tue, Nov 30, 2021 at 3:28 PM Claus Ibsen  wrote:
>
> > Hi Steven
> >
> >
> > On Sun, Nov 28, 2021 at 5:01 PM Steve973  wrote:
> > >
> > > Hi, Claus.  I have done what I hope was a fairly full tracing through the
> > > code of camel-core, and I can now say that I appreciate your concern
> > about
> > > adding things to core!  I also took a closer look at the Saga and Kamelet
> > > components, and that helped to correct a misconception that I had --
> > > components are not solely for implementing things like transport
> > > implementations into Camel, and that it is (arguably) a more suitable
> > > mechanism for adding anything that does not have a strict and solid
> > reason
> > > for existing in core.  So I will implement this EIP as a component, and
> > not
> > > inside of core.  I created a markdown file with my notes when I traced
> > > through core.  If you, or anyone else, might be interested, I can attach
> > it
> > > to a reply here.
> > >
> >
> > Thanks for your patience and impressive to hear that you dived into
> > the deep source code and
> > found the good, bad, "and ugly" of our huge code base.
> >
> > One "bad" about EIPs is that the model is defined tightly into the
> > same JAR which means
> > that adding new EIPs is harder - or that users can't develop custom
> > EIPs and plugin as they need.
> >
> > But on the other side then everybody of Camel has the same DSL, and
> > therefore its "standard" and familiar.
> >
> > About your new dynamic eip, then did you create a JIRA for it? If not
> > you are welcome to do so.
> > And then attach the notes in the JIRA so its not lost, and we have it
> > tracked for everybody to see,
> > and also easier find in years to come.
> >
> > I am a bit busy this week with Red Hat virtual company event the
> > entire week. And on the same time I must finish some work on health
> > checks.
> > However I should get some "clear timer" later in December.
> >
> > And we need Camel 3.14 LTS released first also. So have some patience,
> > and then we have better clear road for 3.15 onwards to add new EIPs.
> >
> >
> > > Thanks,
> > > Steve
> > >
> > > On Wed, Nov 24, 2021 at 10:20 AM Claus Ibsen 
> > wrote:
> > >
> > > > Hi
> > > >
> > > > Adding EIPs or changes to the DSL is not a Camel end user task, and is
> > > > not documented thoroughly.
> > > > Adding new EIPs happens more rarely and there is maybe a little
> > > > hesitation to add because when its in the DSL it stays for a long
> > > > time.
> > > > Also the DSL can become bloated - there are some mistakes from Camel
> > > > v1 and v2 that we have today.
> > > >
> > > > That said to add a new EIP there are 2 ways
> > > >
> > > > - The EIP is implemented fully in the core, so all its code are in
> > core.
> > > > - The EIP is implemeted in a component and are located in camel-xxx in
> > > > the components folder. Examples are circuit breakers, saga, kamelet
> > > > and maybe others I forgot.
> > > >
> > > > So first you need to find out if its the former or the latter. However
> > > > for both cases, the the DSL model are in the core and there is a fair
> > > > bit of code to write to add a new model.
> > > > Therefore its a good idea to find a simple EIP and see where its used
> > > > in the code - can be ConvertBodyDefinition (are in core). Or if its
> > > > outside then maybe KameletDefinition which is the latest EIP added.
> > > >
> > > > The reifer is the "guy" that sits between the model (DSL) and the
> > > > processor (implementation), so all 3 are separated in their own JARs.
> > > >
> > > > And on top of that then there is also the new YAML DSL, so adding a
> > > > new EIP can require some coding there too, but that can be done
> > > > afterwards.
> > > > At first is to get the regular Java and XML DSLs to work. 

Re: Liveness probe or reconnect to Kafka when using of camel-kafka

[Vyacheslav Boyko]

I agree that Camel 3.13 reconnects to Kafka as expected.

Logs are:


2021-12-09 11:50:36.267  WARN [fast-rms,,] 1 --- [t-kyc-response]] 
org.apache.kafka.clients.NetworkClient   : [Consumer 
clientId=consumer-fast-rms-4, groupId=fast-rms] Connection to node 2 
(dev-pf-kafka02.local.site.com/10.6.43.30:9093) could not be 
established. Broker may not be available.
2021-12-09 11:50:36.267  INFO [fast-rms,,] 1 --- [t-kyc-response]] 
o.a.kafka.clients.FetchSessionHandler    : [Consumer 
clientId=consumer-fast-rms-4, groupId=fast-rms] Error sending fetch 
request (sessionId=1835519236, epoch=INITIAL) to node 2:


org.apache.kafka.common.errors.DisconnectException: null

2021-12-09 11:50:48.881  WARN [fast-rms,,] 1 --- [t-kyc-response]] 
o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer 
clientId=consumer-fast-rms-4, groupId=fast-rms] Offset commit failed on 
partition wallet-kyc-response-0 at offset 14: The request timed out.
2021-12-09 11:50:48.881  INFO [fast-rms,,] 1 --- [t-kyc-response]] 
o.a.k.c.c.internals.AbstractCoordinator  : [Consumer 
clientId=consumer-fast-rms-4, groupId=fast-rms] Group coordinator 
dev-pf-kafka01.local.site.com:9093 (id: 2147483646 rack: null) is 
unavailable or invalid due to cause: error response 
REQUEST_TIMED_OUT.isDisconnected: false. Rediscovery will be attempted.
2021-12-09 11:50:48.999  INFO [fast-rms,,] 1 --- [t-kyc-response]] 
o.a.k.c.c.internals.AbstractCoordinator  : [Consumer 
clientId=consumer-fast-rms-4, groupId=fast-rms] Discovered group 
coordinator dev-pf-kafka01.local.site.com:9093 (id: 2147483646 rack: null)




On 11/18/21 14:10, Andrea Cosentino wrote:
For work around, I meant to say work in the area of health check, not 
workaround.


So there are some improvements in 3.13.0, but not something specific 
for your case.


Please keep the users ML through the addresses, it would be helpful 
for the community.


Il giorno gio 18 nov 2021 alle ore 12:01 Vyacheslav Boyko 
 ha scritto:


Where to read about the work around?

On 11/18/21 12:09, Andrea Cosentino wrote:

There are some work around the health check and other stuff.

But first you need to provide what version of Camel you're using.

Il giorno gio 18 nov 2021 alle ore 10:07 Vyacheslav Boyko
 ha scritto:

Hello!

I have just faced an issue.

My microservice (camel-kafka component) got disconnected from
kafka
(consumer and producer both) after Kafka restarted.

Last logs were:

2021-11-16 17:43:02.703  INFO 1 --- [-card-requests]]
o.a.k.c.c.internals.AbstractCoordinator  : [Consumer
clientId=consumer-card-products-2, groupId=card-products] Group
coordinator dev-pf-kafka02.local.site.com:9093
 (id: 2147483645
rack:
null) is unavailable or invalid due to cause: coordinator
unavailable.isDisconnected: true. Rediscovery will be attempted.
2021-11-16 17:43:02.981  INFO 1 --- [-card-requests]]
o.a.k.c.c.internals.AbstractCoordinator  : [Consumer
clientId=consumer-card-products-2, groupId=card-products]
Discovered
group coordinator dev-pf-kafka01.local.site.com:9093
 (id: 2147483646
rack: null)
2021-11-16 17:43:03.225  INFO 1 --- [-card-requests]]
o.a.kafka.clients.FetchSessionHandler    : [Consumer
clientId=consumer-card-products-2, groupId=card-products]
Node 1 was
unable to process the fetch request with (sessionId=1076866435,
epoch=1028386): FETCH_SESSION_ID_NOT_FOUND.
2021-11-16 17:44:21.172  INFO 1 --- [-card-requests]]
o.a.kafka.clients.FetchSessionHandler    : [Consumer
clientId=consumer-card-products-2, groupId=card-products]
Node 2 was
unable to process the fetch request with (sessionId=688855896,
epoch=1027353): FETCH_SESSION_ID_NOT_FOUND.
2021-11-16 17:44:23.373  INFO 1 --- [-card-requests]]
o.a.k.c.c.internals.AbstractCoordinator  : [Consumer
clientId=consumer-card-products-2, groupId=card-products]
Attempt to
heartbeat failed since coordinator
dev-pf-kafka01.local.site.com:9093

(id: 2147483646 rack: null) is either not started or not valid
2021-11-16 17:44:23.373  INFO 1 --- [-card-requests]]
o.a.k.c.c.internals.AbstractCoordinator  : [Consumer
clientId=consumer-card-products-2, groupId=card-products] Group
coordinator dev-pf-kafka01.local.site.com:9093
 (id: 2147483646
rack:
null) is unavailable or invalid due to cause: error response
NOT_COORDINATOR.isDisconnected: false. Rediscovery will be
attempted.
2021-11-16 17:44:23.774  INFO 1 --- [-card-requests]]