Request to remove credential from TRACE level logging

[akabhishek1]

Hi Team,

We are using qpid-jms-client-57.0 to receive message from ServiceBus. We
need to enable TRACE level logging for issue identification.

While enabling TRACE level logging, we realized that credentials are getting
logged at TRACE level which is a security breach.

Can I request you to remove/mask credential while getting logged?

Sample TRACE log - 
2021-04-01 11:56:37,963 [AmqpProvider
:(1):[amqps://.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - [476759868:0] SENT:
SaslInit{mechanism=PLAIN,
initialResponse=\x00**USER_NAME\x00CREDENTIAL,
hostname='***.servicebus.windows.net'}

Please let me know for any concern.

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-56.0 - Prod issue - consumer stopped receiving message without any connection failure and detached receiver error

[akabhishek1]

Hi Robbie,

One more details on my last post. We have added JmsConnectionListener in
Qpid-56 application. We got three log entries from JmsConnectionListener.
Please find below details.

1. 3/18/2021, 3:48:18.176 AM - "onConsumerClosed" method -  Consumer closed
of '**-test-**/subscriptions/56-sub1'   
ExceptionStackTrace -
https://github.com/abhikt48/servicebus/blob/master/18-March/onConsumerClosed_03_48_18_AM.txt
 
2. 3/18/2021, 3:48:18.245 AM - "onConnectionEstablished" method - Connection
established URI
'amqps://.servicebus.windows.net?transport.tcpKeepAlive=true&amqp.traceFrames=true'
3. 3/18/2021, 3:48:18.381 AM - "onConnectionFailure" method - Connection
failure
ExceptionStackTrace -
https://github.com/abhikt48/servicebus/blob/master/18-March/onConnectionFailure_03_48_18_AM.txt
 


Surprisingly - 
   1. 3:48:18 AM - when my method checks for "isConnectionActive" then
connection was active(true) which should be false
   2. 3:48:18 AM - when my method "isConsumerActive" checks for
'**-test-**/subs..s/56-sub1' then consumer was active
   3. Looks like connection got established automatically without reconnect
attempt and we are not using failover protocol as well.

Questions -  
1) As we are not using failover protocol, so connection should not be
recover automatically. Can you please suggest why connection got reconnect
automatically?
2) Looks like consumer also recover automatically without reconnect attempt.
We don't have any “consumerEstablished” notification in
JmsConnectionListener, but based on testing I can confirm that consumer
still receiving messages.
   Can you please suggest why consumer got reconnected automatically without
trying to reconnect consumer?

Please take a look on this issue and let me know for any information.

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-56.0 - Prod issue - consumer stopped receiving message without any connection failure and detached receiver error

[akabhishek1]

Hi Robbie,

We got 'Unknown error from remote peer' today in both applications QPid-56
and Qpid-57, surprisingly both applications are working fine and consuming
messages as expected.

However I would like to share interesting details of "QPid-56" application.
Note - In this application, I am not considering connection is closed for
JmsConnectionRemotelyClosedException/JmsConnectionFailedException. I am
relying on “isConnectionActive” method to check connection status– details
are in the first post of this issue.

## JMS Infra ##
JMS connection - 1
Exception listener - 1
2 listener - (8(2*4) consumers) - Every listener have 4 consumers and
sessions
 (a)topic:A/subscriptions/a1
 (b)topic:B/subscriptions/b1
1 publisher – (session + producer)
qpid-jms-client - 56.0

Steps - 
1)
3/18/2021, 3:48:18.380 AM - Received error on "onException" - Connection
exception from 'servicebus_2', isConnectionActive 'true', Reason 'Unknown
error from remote peer' - 'class
org.apache.qpid.jms.JmsConnectionRemotelyClosedException'   
StackTrace -
https://github.com/abhikt48/servicebus/blob/master/18-March/ConnectionActive_true_JmsConnectionRemotelyClosedException.txt
 
2)Checked "isConsumerActive" for first receiver(A/a1) - It was active, so no
need to reconnect
3)Checked "isConsumerActive" for first receiver(B/b1) - it was active, so no
need to reconnect

4) Published message at 3/18/2021, 10 AM - Both receiver consumed message
successfully

We have enable proton TRACES logs. I attach TRACES of last 12 hour, so we
can't miss anything, you can check after "3/18/2021, 3:45:34.157 AM".
Please refer -
https://github.com/abhikt48/servicebus/blob/master/18-March/Proton_Traces.txt 

As we have received "JmsConnectionRemotelyClosedException", so we are
assuming that connection should be broken, but connection and receiver
wasn't broken. Both receivers are consuming message perfectly.

I am not sure this is expected behaviour or not. Could you please take a
look on this and suggest your suggestion.

P.S. - I haven't shared TRACES and log details on Qpid-57 application for
comparison, as expected it is working fine. But if you need TRACES and log
details of Qpid-57 application as well, then please let me know. I can
extract and share with you.

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-56.0 - Prod issue - consumer stopped receiving message without any connection failure and detached receiver error

[akabhishek1]

Hi Robbie,

Thanks a lot for quick reply. 

Can i assume that JMS Connection will be closed for
"JmsConnectionRemotelyClosedException" or
"ProviderConnectionRemotelyClosedException" exception.

So whenever I got "JmsConnectionRemotelyClosedException" or
"ProviderConnectionRemotelyClosedException"  on "onException()" method then
I will recreate JMS connection.

Please let me know if there are any other JMS connection closed Exception
classes, so i will handle them as well.

Could you please confirm on this or correct me if I am wrong?


Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-56.0 - Prod issue - consumer stopped receiving message without any connection failure and detached receiver error

[akabhishek1]

Hi Robbie, 

Thanks a lot for quick update. I looked on our codebase, logs and
stacktrace, and I can confirm that 

1. "javax.jms.ExceptionListener#onException()" is not getting called from
any of our defined classes/methods.
2. ExceptionListener is used only by connection object

We got this error in simple application which have - 
JMS connection - 1
Exception listener - 1
2 listener - (8(2*4) consumers) - Every listener have 4 consumers and
sessions
1 publisher – (session + producer)
qpid-jms-client - 56.0

So Application has only one JMS connection and one exception listener which
is only operated by JMS connection object.

I shared exact stacktrace with you without any customization and we can see
all call hierarchy from Qpid classes.

We are logging from “JmsConnectionListener” as well on
"onConnectionFailure()" method. Please find below stacktrace details, but
there is no difference between them
https://github.com/abhikt48/servicebus/blob/master/9_march_2021/ConnectionListener_isConnectionActive_False.txt
 
https://github.com/abhikt48/servicebus/blob/master/9_march_2021/ConnectionListener_onConnectionFailure_True.txt
 

With above details, I must request you to please look on this issue again. I
got this issue in other application as well, where we have multiple JMS
connections and listeners, but I shared details of simple application with
only one JMS connection to avoid confusion.

Please feel free to let me know for any information. 

Regards,
Abhishek Kumar





--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-56.0 - Prod issue - consumer stopped receiving message without any connection failure and detached receiver error

[akabhishek1]

Hi Robbie,

Thanks a lot for your quick reply and confirmation on this issue. 
I referred your test case "testRemotelyEndSessionWithMessageListener"  and
trying to replicate same test case with Azure ServiceBus, So I can replicate
this issue and provide confirmation for fixes.

Main challenge for me to perform action based on "TestAmqpPeer" class such
as "remotelyEndLastOpenedSession()", "expectBegin()",
"waitForAllHandlersToComplete()" operations.
I think all these operations are performing under Proton library and don't
have direct access to us. 

Please find sample test class -
https://github.com/abhikt48/servicebus/blob/master/Qpid-57/TestSessionClosedException.java
It is not working fine due to missing equivalent operation of "TestAmqpPeer"
Class.

Can you please suggest any hacking through which I can perform same
functionality of "TestAmqpPeer" with ServiceBus and qpid SDK. 
It will be really helpful if you can provide us any clue to perform
equivalent operation of "TestAmqpPeer", So I can execute test case and
confidently share test result with you.

Regarding reproducing of 'unknown error from peer' and complete logger - We
don't have any standard way to reproduce this issue, infact we are dependent
on unknown ServiceBus Broker activity and  we don't have any control on
this. 

Going forward, I will enable all logs including "JmsConnectionListener" and
Traces logs, and deploy below two applications with 56 and 57-SNAPSHOT. I
will keep these applications running for next 7-10days, and will update you
if we got same issue again with all traces
1. Deploy application with 56.0 version 
2. Deploy application with 57.0-SNAPSHOT version

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-56.0 - Prod issue - consumer stopped receiving message without any connection failure and detached receiver error

[akabhishek1]

Hi Robbie,

I would like to provide you one more input. We got 'Unknown error from
remote peer' error today morning due to
JmsConnectionRemotelyClosedException. 
In first scenario connection was active and in second scenario connection
was active. I compared both stacktrace and both are similar.

Expected behaviour - As we got “JmsConnectionRemotelyClosedException”, then
in both scenario connection should be inactive. Please correct me if I am
wrong.

Please find below exception details - 

First Exception - 
Connection exception from 'servicebusConnector_2', isConnectionActive
'false', Reason 'Unknown error from remote peer'
Refer here for Stacktrace -
https://github.com/abhikt48/servicebus/blob/master/9_march_2021/ConnectionRemoteClose_isConnectionActive_False.txt
 

Second Exception - 
Connection exception from 'servicebusConnector_2', isConnectionActive
'true', Reason 'Unknown error from remote peer' 
Refer here for Stacktrace -
https://github.com/abhikt48/servicebus/blob/master/9_march_2021/ConnectionRemoteClose_isConnectionActive_True.txt
 

Please suggest your opinion.

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-56.0 - Prod issue - consumer stopped receiving message without any connection failure and detached receiver error

[akabhishek1]

Hi Robbie,

We got this issue again in multiple instances. We got exception stack trace
this time on "onException" block on one of instances.

Overall Issue - Consumer is not consuming message from endpoint without
throwing any exception

##JMS Infrastructure##
qpid-jms-client - 56.0
JMS connection - 1
Exception listener - 1
2 listener - (8(2*4) consumers) - Every listener have 4 consumers and
sessions
(a)topic:A/subscriptions/a1
(b)topic:B/subscriptions/b1
1 publisher -

Note - every consumer/publisher have separate JMS session, MessageConsumer
and
producer


Impact/Action/Observations -

1. Depolyed Application on -3/5/2021, 1:14:00.550 PM  -- Application was
idle not receiving single messages at all
2. Got only one "Unknown error from remote peer" error on "onException" 
block, Please refer error stacktrace for more details -
https://github.com/abhikt48/servicebus/blob/master/5_march_error_for_qpid/Exception_Stack_Trace.txt
 
3. Checked connection status with "isConnectionActive()" method - Connection
was active, so didn't reconnected. 
4. Checked "isConsumerActive" for first receiver(A/a1) - It was broken, so
reconnected on this receiver
5. Checked "isConsumerActive" for first receiver(B/b1) - it was active, so
didn't reconnected
6. Application was idle over weekend and did tested today. I published
message on both receiver and found that 
 (a) receiver(A/a1) - consuming msg perfectly, this was broken and
reconnected after "Unknown error from remote peer"
 (b) receiver(B/b1) - not consuming messages, this was active at time of
"Unknown error from remote peer"
 
Please refer first post for implementation of "isConsumerActive"  and
"isConnectionActive()"

Application is very light and simple, which i created for testing of
ServiceBus only. Please refer below thread dump if this can help -
https://github.com/abhikt48/servicebus/blob/master/5_march_error_for_qpid/threadDump.tdump
 


Note - We have more than 100 instances running on Qpid-45, we haven't
received this type of issue in all 100+ instances. But we are facing
continuously this issue on some applications which are using Qpid-56.

Main Issues - Whenever we got "Unknown error from remote peer" then only one
consumer got broken and other consumer status shows active for this error.
We should receive - 
1. Multiple connection exception for every receiver
2. Or all consumer should be inactive for "Unknown error from remote peer"

Could you please take a look on this issue and let us know for any
information.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-56.0 - Prod issue - consumer stopped receiving message without any connection failure and detached receiver error

[akabhishek1]

Hi Robbie,

We are trying to find out root cause for this issue. Can you please provide
details on below query which can help us to investigate for future issue

1) What is the idle way to check link status between JMS consumer and broker
at runtime?
  - We can get to know via TRACE level logs, but we can't turn on in PROD
server.
  - Is there anything can we do at runtime to check link status between
consumer and broker?
  
2) To check consumer status - I disabled endpoint and waited for 6-7m, but
we didn't receive any error for 6-7minutes on "onException" method neither
any System.Error, so we concluded that consumer was already detached from
long time, that's why it was not receiving message from more than 12hrs.  Is
my understanding correct? Or I should do something else to verify?
  
3) We have added JmsConnectionListener and added logger on all operations
except "onInboundMessage" operation, because "onInboundMessage" operation
are getting called for receive message.
I have not enabled logger on session class, because JmsConnectionListener
will give us enough information. 
Do you think, I need to enabled any other logger which will give us detached
link information or this should be enough?  (I think, this should be enough,
just cross verifying for safer side)

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-56.0 - Prod issue - consumer stopped receiving message without any connection failure and detached receiver error

[akabhishek1]

Hi Robbie,

Thanks a lot for quick suggestion. We have implemented JmsConnectionListener
and planned to roll out in lower environmnet. This will give us more
granular details for any connection failure, session/producer closer.

We will let you know if we found any root cause.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-56.0 - Prod issue - consumer stopped receiving message without any connection failure and detached receiver error

[akabhishek1]

Hi Robbie,

Thanks a lot for quick reply.  We received only one below exception. There
are 7 destinations connected with single serviceBus connection. If links are
detached on “unknown peer error”, then we should receive at least 7
exceptions at "onException" block, but received only one.

I checked the log and find out that Application has not requested to close
JMS connection and does not initiate to close session/consumer of 6
destinations, but initiate to close/create session/consumer of only one
destination.

As Application received one exception, and found issue in only one
destination, So application re-established communication with only one
receiver (sub1). I am hoping closer of one destination (session/consumer)
shouldn't impact other session/consumer of 6 destinations.

Big trouble points for us that we haven't received any exception for 6
destinations and links were detached as well (tested with disabling the
queue which didn’t bubbled up error on onException block).

Just FYI - All Seven destinations(sub1, sub2, sub3..) are subscriptions of
single topic ('TestTopic')

Unfortunately I don't have any stacktrace, so i can't provide much details.
Please take a look below thread dumps, if it helps -  First_App_ThreadDump

  

Going forward, we will log stacktrace in onException block.

---Details of another Application which have same issue like
above---

We are using latest jar 3 applications and we received this type of error in
two applications. I looked error on other application and that application
is very light, not heavier like above one. Please find below findings

##JMS Infrastructure##
JMS connection - 1
Exception listener - 1
3 listener - (12 (3*4) consumers)
7 publisher - 
every consumer/publisher have separate JMS session, MessageConsumer and
producer

Listener Details -
1. QueueA
2. QueueA/$DeadLetterQueue
3. QueueB

Actions after exception ###
1. Exception on "onException" block - 
Connection exception from 'SBus_Connector_1', isConnectionActive 'true',
Reason 'Unknown error from remote peer' - 'class javax.jms.JMSException',
cause 'org.apache.qpid.jms.provider.ProviderException: Unknown error from
remote peer'

2. Checked isConsumerActive for 'QueueA' - result “true” - (We have problem
here, not consuming messages)
3. Checked isConsumerActive for 'QueueA/$DeadLetterQueue' - result “false”, 
Re-established consumer
 - I have doubt here, As 'QueueA/$DeadLetterQueue' consumer is 
deactive, so
'QueueA' consumer should be deactive as well. Please suggest your opinion.
4. Checked isConsumerActive for 'QueueB' - result “true” - (Did nothing)

FYI - We re-establish consumer only if consumer is deactive.

Same Application have another JMS connection(Separate ServiceBus broker
endpoint), on which we got same exception,
Error - Connection exception from 'SBus_Connector_2', isConnectionActive
'true', Reason 'Unknown error from remote peer' - 'class
javax.jms.JMSException', cause
'org.apache.qpid.jms.provider.ProviderException: Unknown error from remote
peer'

This JMS connection has one publisher which is trying to publish message,
while publishing it also throws "ProviderException - Unknown error from
remote peer".

Please find exception StackTrace -   StackTrace_Unknown_Remote_Error.json

  


ThreadDump - Application_Thread_Dump

  

##Overall Obesrvation ##
1. Exception listener does not receiver multiple notification for receiver
detachment. At least one notification should trigger for one destination.
2. JmsMessageConsumer.getMessageListener()->checkClosed() --> does not
provide correct closed status. It could be sync up issue.

Suggestion/Question - We can't enable TRACE in PROD environment because it
emits lot of logs. Can you please tell us any qpid class which logs TRACE
event only on failure like "Unknown error from remote peer". So we can log
enable Qpid log atleast for failure.


Please suggest your opinion on above query, So we can proceed further. I
will keep on working parallelly to replicate this issue if possible and
update you if found any.

Regards,
Abhishek Kumar














--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-56.0 - Prod issue - consumer stopped receiving message without any connection failure and detached receiver error

[akabhishek1]

Hi Team,

After observing this issue for 12 hours , we performed one activity 

- We disabled the one subscription which should throw detached error on
"onException" block, But we haven't  received any error.

It means link got detached from broker without informing to connection
exception listener.

We haven't observed this type of issue in 45 from more than years. 

Could you please take a look on this issue. This looks like a defect.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

qpid-jms-client-56.0 - Prod issue - consumer stopped receiving message without any connection failure and detached receiver error

[akabhishek1]

Hi Team,

We recently upgraded qpid-jms-client to 56.0. We are using qpid-jms-client
ti receive message from ServiceBus. We need your urgent support on this
issue, because we are facing this issue in PROD environment.

Issue - Some consumers are not receiving message from queue without any
connection failure and detached receiver error

JMS Infrastructure -
JMS connection - 1
Exception listener - 1
7 destination (Queue/topic) - every destination has 4 consumers
28 consumers - every consumer have separate JMS session and MessageConsumerr

Scenario -
1. Received only one below error on "onException" block -

Connection exception from 'ServiceBus_Connector', isConnectionActive 'true',
Reason 'Unknown error from remote peer' - 'class javax.jms.JMSException',
cause 'org.apache.qpid.jms.provider.ProviderException: Unknown error from
remote peer'

2. As connection is active, so we didn't establish re-connection. We checked
connection status with below method
3. Checked receiver status with below isConsumerActive method and found that
only reciever is closed, so re-established receiver and this consumer is
consuming messages perfectly.

Impact - 1. One consumer receiving message perfectly
 2. Six consumer does not throw any connection error, or detached
error
 3. These six consumers stopped receiving message from queue

Could you please urgently suggest your opinion on this issue ? 


public boolean isConnectionActive(Connection connection) {
boolean connectionStatus = false;
try {
String clientID = connection.getClientID();
connectionStatus = true;
} catch (Exception e) {
connectionStatus = false;
}
return connectionStatus;
}

public boolean isConsumerActive(MessageConsumer consumer) {
boolean isConsumerActive = false;
try {
consumer.getMessageListener();
isConsumerActive = true;
} catch (Exception e) {
isConsumerActive = false;
}
return isConsumerActive;
}


Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Getting jms.provider.ProviderException with closed session - Qpid-56

[akabhishek1]

Hey Robbie,

Thank you so much update. As always, your input helps alot to fix/understand
issue.

We don't have ServiceBus test server and Microsoft also can't help us to
close session remotely.

So, we  have only one option reproduce this issue - forcefully break
existing JMS session either in debug or test mode.

I am able to genearate slightly same error with setting below value in Debug
mode while creating JMS Destination
1.  closed.set(true) 
2. failureCause.set(new ProviderException("Unknown error from remote
peer"))

 protected void checkClosed() throws IllegalStateException {
if (closed.get()) {
IllegalStateException jmsEx = null;
if (failureCause.get() == null) {
jmsEx = new IllegalStateException("The 
Session is closed");
} else {
jmsEx = new IllegalStateException("The 
Session was closed due to an
unrecoverable error.");
jmsEx.initCause(failureCause.get());
}
throw jmsEx;
}
}


In actual scenario, JmsSession.setFailureCause() and
JmsSession.closed.set(true) getting called from proton/Netty library. Refer
-  Netty_Call_Hierarchy.PNG
  

We want to generate same error forcefully with Qpid-Proton library. I
checked the call hierarchy of JmsSession.setFailureCause() which is getting
set by AmqpAbstractResource.processRemoteClose method - refer 
SetFailureCause_Hierarchy.PNG
  

Can you please guide us source location of Netty/proton class which
derives/triggers to close JMS session and .setFailureCause(). 

It will really help us to forcefully generate actual error. Please let me
know for any input.

Thanks again for help!!

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Getting jms.provider.ProviderException with closed session - Qpid-56

[akabhishek1]

Hi Team,

We recently upgraded Qpid library to "qpid-jms-client-0.56.0" and started
getting ProviderException with session closed error while communicating with
ServiceBus broker.

We are trying to reproduce ProviderException with closed session. 

Scenario :
1. Publish message
2. Got illegalStateException - The MessageProducer is closed/The
MessageProducer was closed due to an unrecoverable error.
3. ResetPoducer -- (Assuming session is not closed) --> Which throws "The
Session was closed due to an unrecoverable error."
4. ProviderException -- "Unknown error from remote peer" --> why are we
getting this exception as well in error call stack?

Please find attached "ProviderException_SessionClosed_Qpid.json" and
SessionClosed_ProviderException.PNG.

We are cleared upto ResetPoducer issue but not cleared with
ProviderException. 

If you check attached stack trace there is no directly link between
IllegalStateException and ProviderException.

We have fixed ResetPoducer but not able to fix ProviderException issue,
because we are not able to generate this issue.

Could you please check attached stack trace and help with below queris - 
1. Is ProviderException being generated by
"org.apache.qpid.jms.JmsSession.checkClosed" ? Please refer error
Array[1]-parsedStack[0]
2. How can we generate ProviderException while creating destination ? Please
refer error Array[1]-parsedStack[1]

Could you please let us know your thought, it will help us to close this
issue. Let me know for any query.

ProviderException_SessionClosed_Qpid.json

  

SessionClosed_ProviderException.PNG

  

 
Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

What can cause to forcefully close the JMS Session?

[akabhishek1]

Hi Team,

We are using Qpid-JMS-Client-56.0 to publish/consume from ServiceBus.

We are getting "javax.jms.IllegalStateException - The MessageProducer is
closed"  error.

We reset the Producer whenever we got  "The MessageProducer is closed"
error. At this time, Producer and Session, both was closed. I am trying to
find out the reason for closed JMS Session.

Can you please guide us, What are the criteria can trigger to close the JMS
Session? So I can replicate with real enironment.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: JMS connection is active on the event of org.apache.qpid.jms.JmsConnectionRemotelyClosedException

[akabhishek1]

Hi Team,

Could you please update us on this issue, so we can handle it accordingly.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

JMS connection is active on the event of org.apache.qpid.jms.JmsConnectionRemotelyClosedException

[akabhishek1]

Hi Team,

We are using "qpid-jms-client-0.56.0" to publish message to Azure
ServiceBus.

We recently started getting below error on JMS exception listener -
"onException" block

Error Message - 'Unknown error from remote peer'
Error class - org.apache.qpid.jms.JmsConnectionRemotelyClosedException',
Error cause -
'org.apache.qpid.jms.provider.exceptions.ProviderConnectionRemotelyClosedException:
Unknown error from remote peer'
JMS Connection Status - Active (sometimes inactive)

We got this error multiple times, but JMS connection status is inconsistent,
sometimes JMS connection is Active, and sometimes not. We are recreating JMS
connection only for - inactive JMS connection.

We are using below code to check JMS connection status -
protected boolean isConnectionActive() {
boolean connectionStatus = false;
try {
@SuppressWarnings("unused")
String clientID = connection.getClientID();  // connection is
javax.jms.Connection
connectionStatus = true;
} catch (Exception e) {
connectionStatus = false;
}
return connectionStatus;
}


Could you please suggest that - Is this a expected behaviour ?
Can JMS connection could be active on the event of
JmsConnectionRemotelyClosedException ?

Please let me know for any information.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

Hey Robbie,

Thanks a lot for details !!

Yes messages are sending back to ServiceBus and Delivery Count is also
increasing at first time. I was worried that why client is not reciving
message again and how we will identify them. You pointed correctlly about
deposition, I looked again bad messages in ServiceBus explorer and realized
that all the bad messages are in Defferred state, So Client will not receive
those messages due to Defferred state and will stay idle on broker. 


We need your input on below open questions to handle defer scenarion - 

1. We are getting exception directly on 'onException' method where we don't
have any details of JMS Message.
   
   Can you please guide us - how can we extract Message details of bad
messages like - Message ID, Sequence NO, queue name ?
   Can we get hold of JMSMessage object? It will be so great if we can get
hold it, so we can reject/abandon message.

2. As per microsoft
(https://docs.microsoft.com/en-us/azure/service-bus-messaging/message-deferral),
we can receive deferred message with the use of only Sequence Number.
Do we have any option to receive deferred message with Qpid Client? It 
will
be great for reconciliation.  
3. Do we have any option to control Disposition? So for example we can
specify deadletter rather than defer.
4. What is the disposition/delivery state set by qpid, that causes it to
appear as deferred in Azure Service Bus? 


Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

I waited upto 30m still, but still application does not recived bad messages. 
After that I published one good message and application received it
immediately.

I can see that message delivery count is not increasing, it means QPID
client is not touching to bad messages at all. 

Please find attached log.  Latest_Logs.txt
  

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

Thanks for confirmation. It means QPID client will receive bad message at
only one time.

Can you please suggest - How can we identify that Queue have bad messages ?

In Production environment - If we have 1000 messages, and 100 messages are
bad messages. Support team needs to identify bad message, so they can take
action accordingly.

We need to have some mechanism to know bad messages otherwise it will be
very difficult for support person. 

Could you please suggest your opinion to handle this scenario.

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

Hey Robbie,

Thanks a lot for quick reply. I did testing based on your suggestion.
Application is not able to consume messages even after recreation of NEW JMS
connection and consumer as well.

Please find below comparison between "Qpid 0.56.0" and "Qpid 0.45.0" jar

Qpid 0.56.0
1. Started Application
2. Published 10 bad messages
3. Application got exception on 'onException' method while consuming 10
messages.  (Only first time – total 10 exception)
4. Application doesn't received any message/exception after that, even after
exhaustion of lock duration, waited for couple of minutes
5. Restarted Application - NEW JMS Connection with New Consumer
6. Doesn't received any message/exception after restarting the application
as well
7. I even retried to consume these messages with Old Qpid library 0.45.O as
well, but application doesn't received that message as well.

Qpid 0.45.0
1. Started Application
2. Published 10 bad messages
3. Application got exception on 'onException' method while consuming 10
messages. 
4. Continuously application receiving message/exception unless reached to
max delivery count. Working as expected

Please suggest your opinion on this issue.

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

Hey Robbie,

I want to provide you one more input regarding above issue. Just now, I
tested with old Qpid Library 0.45.0 and it is working as expected.

Old Qpid library (0.45.0) continuously receiving bad message. It seems like
a defect in latest library(0.56.0).

Please take a look and let me know for any query.

Regards,
Abhishek Kumar





--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

Hey Robbie,

I was doing testing with 0.56.0 jar and I found one issue while consuming
bad messages from ServiceBus. 

Issue - Application is not consuming bad message continuously.

Steps - 
1. Started Application (ServiceBus Message lock duration is 30seconds)
2. Published 10 bad messages
3. Application got exception on 'onException' method while consuming 10
messages. Application logged exception and doesn't reconnected to JMS
connection and receiver as both are connected.
4. Published 1 good message
5. Application successfully received good message. Which means receiver is
connected and consuming messages.
6. Issue - As message lock duration is 30seconds, so application should
receive bad message continuously (30 seconds) until reached to max
DeliveryCount(10). But application hasn't received bad message after first
attempt.

Problem - As application doesn't receiving bad message continuously, so
message is not moving in dead letter queue. 

I am not sure above issue belongs to Qpid or Microsoft. Could you please
suggest your opinion on this issue.

Please find attached log file and screenShot of message.
Error_Logs_Not_Consuming_Continuously.txt

  
Message_ScreenShot.PNG
  

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

Hi Robbie,

I applied the changes in exception listener which will make sure to cleanup
once instead of multiple times. After changes, I tested multiple times and
it is working fine so far.

As you mentioned before - "By pairing the invalid message causing failure
before
onMessage (which is really quite unexpected) with disconnecting the actual
connection in full"

Do we need to worry about this? 

I saw your release notification for 0.56, I am hoping you will release this
within 1-2 days. So i will use released version.

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

HI Robbie,

Thanks a lot for quick feedback. I will raise this with Microsoft.

Regards,
Abhi



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

Hey Robbie,

Thanks a lot for your suggestion. I am still working on exception listener,
will let you know for any issue. 

While testing with 0.56.0-SNAPSHOT, I found one distinct issue - consumer
closed for wrong receiver. Please find below details

JMS Infra details -
a) JMS Connection - 1
   receiver endpoint - 2
   Consumer on every receiver - 4 (Total 8)
   Separate session for every consumer - Total 8
b) JMS Connection - 1
   receiver endpoint - 2
   Consumer on every receiver - 4 (Total 8)
   Separate session for every consumer - Total 8
   
Steps to reproduce this issue - 
1. Start attached MultipleConsumerTest.java class
2. disable only one receiver endpoint (only one subscription - sub1)
3. Wait for connection exception
4. We can see that two receivers are failing to consume messages which was
registered on same JMS connection. 
As we have disabled only one subscription, so consumer should be close for
only one receiver instead of two.
   
Please find attached sample test class  MultipleConsumerTest.java
  
and error log  consumer_closed_for_wrong_receiver.txt

 
. Please take a look on this issue and let us know your 
opinion.


Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

Hi Robbie,

Thanks a lot for details insight and correction. Test case is working fine
now with 0.56-SNAPSHOT.
I am able to generate invalid message which continously throw 'Cannot decode
String in UTF-8' error.

The good thing is - I am able to generate "Suspected deadlocked threads"
error repeatedly with Old(0.45.o) jar. I used invalid message .

Steps to generate error - I need to do enable/disable queue and WiFI ON/OFF
couple of times to get this error. I have not found consistent pattern yet,
but couple of retries helps to generate this error. 

I did similar testing with New "0.56.0-SNAPSHOT" jar. Unfortunatelly I am
getting same "Suspected deadlocked threads" error with new jar.

Can you please check attached thread dump
"ThreadDumps_Deadlock_56-SNAPSHOT.txt", if you find anything from your side. 

ThreadDumps_Deadlock_56-SNAPSHOT.txt

  

I am also trying to find root cause from my side. 

Regrads,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

Hi Robbie,

Thanks for quick reply. I referenced your test case and tried to generate
"Illegal invocation from MessageListener callback" error. 
At this moment, I am not able to generate this error neither "Suspected
deadlocked threads" error.

So, I can't confirm that this fix is working or not.

Please find attached sample test class  ConnectionTest.java
  . I
tested with both library "45.0" and "0.56.0-SNAPSHOT", And test cases are
failing for both jar versions. 
So definitely, I am missing something .

Is that possible for you to suggest correction in attached
"ConnectionTest.java" ? So I can generate error and proceed further for
testing.

Regarding last reported 3 issues - I will try to replicate these issues in
independent project and share details with you.
As we are upgrading library, so we want to make sure that existing(0.45.0)
positive functionalities should match with latest jar (0.56.SNAPSHOT). 


Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

Hi Robbie,

Sure, I will be happy to provide you testing feedback. 

I was doing testing with "qpid-jms-client-0.56.0-SNAPSHOT" and
"proton-j-0.33.8". So far I managed to found three issues. 
These issues are intermittent, however we can regenerate this issue in
couple of retries. I manage to capture error with Qpid Traces. Please find
below details.

Infrastructure Details --> Created 2 JMS connection instance for testing.
First JMS connection. 
 * 1 endpoint topic-subscription
 * 4 consumer/listener
 
Second JMS connection. 
 * 1 endpoint Queue
 * 4 consumer/listener 
 
Steps to reproduce this issue -
1. Start the application, which should start listening on both endpoints
with separate JMS connections
2. DO NOT publish any message for consumption
3. Keep Application idle for 5m or sometimes 20m. If you didn't get this
issue then please restart the application couple of times

 Issue Details *
1. Both JMS connection have one active listener, but still getting this
error - 'The connection was inactive for more than the allowed 12
milliseconds and is closed by container
'7A2E0A0515A9427F82D1ACF7F25A024D_G12'. [condition =
amqp:connection:forced]' - 'class javax.jms.JMSException'
   
 We shouldn't get this error as we have 1 active listener on both JMS
connection. Please refer "Receiver_Timeout_12_ms.txt" log file for more
details.

2. Inconsistent JMS connection status while getting timeout error.
a) Connection Active for Receiver TimeOut -  If we are getting timeout
error due to receiver then we can see that connection is active at time of
exception. Refer "Receiver_Timeout_12_ms.txt"

   b) Connection InActive for No Session/Producer/Consumer -  Create a JMS
connection which does not have any session/producer/consumer and keep
application running.  We will get 6ms idle timeout error after some
time. Please refer below logs.
We are using "JmsConnection.getClientID" to check connection status. Please
refer below sample method 'isConnectionActive'.

Problem - At receiver time out (a) connection is active, But at second
scenario (b) connection is inactive.
I think at both scenario connection should be inactive.

***Logs**
TRACE 2020-12-03 12:40:05,354 [AmqpProvider
:(3):[amqps://***.servicebus.windows.net:-1]]
org.apache.qpid.jms.provider.amqp.FRAMES: [1314740929:0] RECV:
Close{error=Error{condition=amqp:connection:forced, description='The
connection was inactive for more than the allowed 6 milliseconds and is
closed by container 'LinkTracker'.
TrackingId:8ac03af7d6794026ab3948b199e58a35_G5, SystemTracker:gateway5,
Timestamp:2020-12-03T12:40:05', info=null}}
TRACE 2020-12-03 12:40:05,364 [AmqpProvider
:(3):[amqps://***.servicebus.windows.net:-1]]
org.apache.qpid.jms.provider.amqp.FRAMES: [1314740929:0] SENT:
Close{error=null}
ERROR 2020-12-03 12:40:05,370 [QpidJMS Connection Executor:
ID:a6cf63d9-4608-4643-8955-eee32e604674:1]
org.mule.servicebus.SbusConnector: Connection exception, Reason 'The
connection was inactive for more than the allowed 6 milliseconds and is
closed by container 'LinkTracker'.
TrackingId:8ac03af7d6794026ab3948b199e58a35_G5, SystemTracker:gateway5,
Timestamp:2020-12-03T12:40:05 [condition = amqp:connection:forced]' - 'class
org.apache.qpid.jms.JmsConnectionRemotelyClosedException',
isConnectionActive 'false'

Sample Code ***
protected boolean isConnectionActive() {
boolean connectionStatus = false;
try {
String clientID = getConnection().getClientID();
connectionStatus = true;
} catch (Exception e) {
connectionStatus = false;
}
return connectionStatus;
}

3. Use Case - We have 1 JMS connection, which will be used for publishing
message. 
We know that ServiceBus detach active connection link, if we don't have 
any
active Producer/Consumer. 
For avoiding this issue, we registered a timer which will periodically
create/reset temporary Session+Producer instead of JMS connection. This
solution help us to keep JMS connection live, so we don't need to reset JMS
connection again and again. We haven't faced any issue so far with old
client 0.45.0 jar.

After upgrading the JAR, we are facing intermittent issue while 
resetting
Producer. Error Log - "Failed to reset temp producer -
javax.jms.JMSException: The service was unable to process the request;"
Please refer "3rd_issue_Failed_To_Reset_Producer_Error.txt" for more
details.


NOTE - We haven't faced above 3 issues with Old client 0.45.0 jar.

Could you please advise us on above 3 issues. Please feel free to let me
know for any other information.

Receiver_Timeout_12_ms.txt
 
 
3rd_issue_Failed_To_Reset_Producer_Error.txt

  



--
Sent from: http://qpid.2158936.n2.n

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

Hi Robbie, 

Thanks a lot for detail reply. We have started testing with
"0.56.0-SNAPSHOT" . 

Do we have any high level timeline to release "0.56.0" ? So we can plan
accordigly.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

Hi Robbie,

We are trying to replicate this issue from our end, so we can confidently
apply fix and roll out to all. We need some your input/help to replicate
this issue.  It could be be some issue from my end, So I am trying to find
trouble code as well. Sorry for many questions.

In May, we had this issue due to bad Message which was published with .Net,
but now we got issue due to normal messages as well. 

We are having this issue while restarting of ServiceBus connector restart
(due to ServiceBus connectivity loss). I follow below steps for restart -
---
1. Closing and Nullifying JMS Objects

a. Connection.close(), 
Connection = null,

b. MessageConsumer.setMessageListener(null); --> I think i am doing wrong
here. I don't need to set null here, as i am closing on next step. What's
your opinion?
MessageConsumer.close();
MessageConsumer=null;

c. Session.close();
   Session = null
--
2. Creating JMS Objects
a. Create Connection
b. Create Session
c. Create Consumer
--
Q1. ) Could you please suggest if I am doing anything wrong while restarting
of JMS Objects ?

Q2. ) If we got any exception while closing Connection, Session, Consumer
and Producer, then how can we make sure that Connection, Session, Consumer
and Producer closed and clear successfully from system? 
Is it good practise set Null to JMS Objects after closing to make sure that
JMS Objects are cleaned up properly?
I want to make sure that JMS object and threads are clear from system
properly before creating new connection, so it will not cause any issue
while starting of JMS connection, Session, etc.
Overall I feel that, this could be an issue with clearance/disposing of
existing JMS Objects.

Could you please suggest your opinion to handle gracefully for exception
scenario while closing JMS Objects?


Q3. ) We got  "Illegal invocation from MessageListener callback" error while
closing Connection & Session, and then we started having "Suspected
deadlocked threads" error.

Could you please suggest any idea to generate "Illegal invocation from
MessageListener callback" error ? It could be one of reason for deadlocked.
If I am able to generate this error then I could generate deadlock error.

It will be great help if you provide your opinion on above queries.

Please find below error logs. 

Error Pattern 1 -
"Failed to close connection - 'Illegal invocation from MessageListener
callback'"
"Stopping: 'null'. Object is: ServicebusMessageReceiver"
"Failed to reset message listener on endpoint
'servicebus://topic:***/subscriptions/***' - 'The MessageConsumer was closed
due to an unrecoverable error.' - 'class javax.jms.IllegalStateException'"
"Suspected deadlocked threads: JmsSession
[ID:ge300959-71f8-8fff-9d61-0a3e6dc02b98:9:10] delivery dispatcher
Id=41620240 is in BLOCKED on
lock=org.mule.transport.servicebus.SbusConnector@58e4b68c is owned by
JmsSession [ID:ge300959-71f8-8fff-9d61-0a3e6dc02b98:9:7] delivery dispatcher
Id=51620937 | JmsSession [ID:ge300959-71f8-8fff-9d61-0a3e6dc02b98:9:7]
delivery dispatcher Id=51620937 is in WAITING on
lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@26cc0a98 is
owned by JmsSession [ID:ge300959-71f8-8fff-9d61-0a3e6dc02b98:9:5] delivery
dispatcher Id=41620235 | JmsSession
[ID:ge300959-71f8-8fff-9d61-0a3e6dc02b98:9:5] delivery dispatcher
Id=41620235 is in BLOCKED on
lock=org.mule.transport.servicebus.SbusConnector@58e4b68c is owned by
JmsSession [ID:ge300959-71f8-8fff-9d61-0a3e6dc02b98:9:7] delivery dispatcher
Id=51620937 | "
"Suspected deadlocked threads: JmsSession
[ID:ge300959-71f8-8fff-9d61-0a3e6dc02b98:9:10] delivery dispatcher
Id=41620240 is in BLOCKED on
lock=org.mule.transport.servicebus.SbusConnector@58e4b68c is owned by
JmsSession [ID:ge300959-71f8-8fff-9d61-0a3e6dc02b98:9:7] delivery dispatcher
Id=51620937 | JmsSession [ID:ge300959-71f8-8fff-9d61-0a3e6dc02b98:9:7]
delivery dispatcher Id=51620937 is in WAITING on
lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@26cc0a98 is
owned by JmsSession [ID:ge300959-71f8-8fff-9d61-0a3e6dc02b98:9:5] delivery
dispatcher Id=41620235 | JmsSession
[ID:ge300959-71f8-8fff-9d61-0a3e6dc02b98:9:5] delivery dispatcher
Id=41620235 is in BLOCKED on
lock=org.mule.transport.servicebus.SbusConnector@58e4b68c is owned by
JmsSession [ID:ge300959-71f8-8fff-9d61-0a3e6dc02b98:9:7] delivery dispatcher
Id=51620937 | "
"Suspected deadlocked threads: JmsSession
[ID:ge300959-71f8-8fff-9d61-0a3e6dc02b98:9:10] delivery dispatcher
Id=41620240 is in BLOCKED on
lock=org.mule.transport.servicebus.SbusConnector@58e4b68c is owned by
JmsSession [ID:ge300959-71f8-8fff-9d61-0a3e6dc02b98:9:7] delivery dispatcher
Id=51620937 | JmsSession [ID:ge300959-71f8-8fff-9d61-0a3e6dc02b98:9:7]
delivery dispatcher Id=51620937 is in WAITING on
lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@26cc0a98 is
owned by JmsSe

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

Thanks a lot for details reply.

We have added story in our backlog to upgrade project with 0.55.0 client. As
we have to do a lot of performance testing, so we will use latest version
for testing which will save our time. 

As you are targeting to release this week, so we will wait for release and
then plan to upgrade accordingly.

Thanks again.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

Hey Robbie,

Thanks a lot for quick reply.

We are using JmsSession for below purposes
1. Creating JMS messages
2. Creating producers for sending message
3. Creating consumers for receiving message

Thread name "delivery dispatcher" suggests that there is an issue in
consumer JmsSession. Could you please confirm this ? 

If you think there could be an issue in other processor like JmsMessage
creation or publishing then please let me know. So I can try to find
blocking thread.

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession delivery dispatcher

[akabhishek1]

Hi Team,

We are getting "Suspected deadlocked threads" error in PROD environment. We
are trying to find out root cause of this issue.

Could you please give us any clue regarding below error, so we can proceed
further.

Logs - 
Suspected deadlocked threads: JmsSession
[ID:de300758-71f8-4fff-9d51-0a3e6dc01b88:9:10] delivery dispatcher
Id=41620240 is in BLOCKED on lock=org.mule.servicebus.SbusConnector@58e4b68c
is owned by JmsSession [ID:de300758-71f8-4fff-9d51-0a3e6dc01b88:9:7]
delivery dispatcher Id=41620237 | JmsSession
[ID:de300758-71f8-4fff-9d51-0a3e6dc01b88:9:7] delivery dispatcher
Id=41620237 is in WAITING on
lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@26cc0a98 is
owned by JmsSession [ID:de300758-71f8-4fff-9d51-0a3e6dc01b88:9:5] delivery
dispatcher Id=41620235 | JmsSession
[ID:de300758-71f8-4fff-9d51-0a3e6dc01b88:9:5] delivery dispatcher
Id=41620235 is in BLOCKED on lock=org.mule.servicebus.SbusConnector@58e4b68c
is owned by JmsSession [ID:de300758-71f8-4fff-9d51-0a3e6dc01b88:9:7]
delivery dispatcher Id=41620237 | 


Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Error while creating message - The Session was closed due to an unrecoverable error

[akabhishek1]

Hi Robbie,

Thanks a lot for your reply. In our application there are multiple
ServiceBus connection which serves for different endpoints. However we
didn't oberserve this issue for other endpoints but we observed this issue
for two endpoints. Yes you are are right multiple threads are envolved.

However i did testing with 0.52.0 (single thread) and i can see same
behaviour. Session.createTextMessage not throwing exception in the absence
of connection. I think, it should throw exception in the absence of
connection

Could you please check for this issue? 
I am not sure this issue is related with our issue or not, but it could be
related, because session is not able to idetinfy correct connection status.

Please find below steps to reproduce
this issue and attached "TestQpidSendSessionRecoverableError.java".

Actual Behavior - Session successfully created text message in absence of
connection
Expected Behavior -  session.createTextMessage("Hello World") - should throw
exception in the absence of connection.

Steps to reproduce this issue -
1. Provide correct ServiceBus endpoint details
2. Add debug point at Line no 55  (before creating text message)
3. Start class in debug mode
5. Application should stop at debug point, and disable network
6. Click on resume button

We can see that, class successfully created text message instead of
connection exception. Please let us know your opinon.

TestQpidSendSessionRecoverableError.java

  

Regards,
Abhi



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Set Custom MessageID for duplicate message detection

[akabhishek1]

Hi Robbie,

Thanks a lot for giving idea for ThreadLocal. I tested and it is working
fine at this moment.

Regards,
ABhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Error while creating message - The Session was closed due to an unrecoverable error

[akabhishek1]

Hi Tim,

I was doing testing and found that Session.createTextMessage not throwing
exception in the absence of connection. Please find below steps to reproduce
this issue and attached "TestQpidSendSessionRecoverableError.java".

Actual Behavior - Session successfully created text message in absence of
connection
Expected Behavior -  session.createTextMessage("Hello World") - should throw
exception in the absence of connection.

Steps to reproduce this issue -
1. Provide correct ServiceBus endpoint details
2. Add debug point at Line no 55  (before creating text message)
3. Start class in debug mode
5. Application should stop at debug point, and disable network
6. Click on resume button

We can see that, class successfully created text message instead of
connection exception. Please find attached log "ApplicationLog.txt".

TestQpidSendSessionRecoverableError.java

  
ApplicationLog.txt
  


I think this issue is related with our issue. Could you please take a look
on this issue and suggest some solution.

Regards,
Abhishek Kumar  



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Error while creating message - The Session was closed due to an unrecoverable error

[akabhishek1]

Hi Tim,

Thanks a lot for quick response. I investigate about this issue and we found
that, we got the same issue in two environments but both having different
behaviors.

First Environment - Application throws this error (unrecoverable error) but
at the same time application received call at
ExceptionListener-onException() method, so application JMS connection
restarts and everything works fine as usual.

Second Environment(issue one) - Application throws this error (unrecoverable
error) but at the same time or after 1.5 days as well, application does not
receive call on JmsConnection - onException() method, so application stuck
here.
We have to forcefully restart the application to fix this issue.

I think, this could be a defect. The application should receive call on
ExceptionListener-onException() method at the event of connection failure.
But it does not happen.

After restart of this application, we haven't received this issue, so we
can't provide Trace level log.

I found some qpid logs which could helps us for more information. Please
take a look on attached "qpid-app-log.txt" and "ApplicationError.txt"

Please let us know your opinion to fix this issue.

qpid-app-log.txt
  
ApplicationError.txt
  



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Error while creating message - The Session was closed due to an unrecoverable error

[akabhishek1]

Hi Tim,

I was doing testing and found that Session.createTextMessage not throwing
exception in the absence of connection. Please find below steps to reproduce
this issue and attached "TestQpidSendSessionRecoverableError.java".

Actual Behavior - Session successfully created text message in absence of
connection
Expected Behavior -  session.createTextMessage("Hello World") - should throw
exception in the absence of connection.

Steps to reproduce this issue -
1. Provide correct ServiceBus endpoint details
2. Add debug point at Line no 55  (before creating text message)
3. Start class in debug mode
5. Application should stop at debug point, and disable network
6. Click on resume button

We can see that, class successfully created text message instead of
connection exception. Please find attached log "ApplicationLog.txt".


TestQpidSendSessionRecoverableError.java

  
ApplicationLog.txt
  

I think this issue is related with our issue. Could you please take a look
on this issue and suggest some solution.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Error while creating message - The Session was closed due to an unrecoverable error

[akabhishek1]

Hi Tim,

Thanks a lot for quick response. I investigate about this issue and we found
that, we got the same issue in two environments but both having different
behaviors.

First Environment - Application throws this error (unrecoverable error) but
at the same time application received call at
ExceptionListener-onException() method, so application JMS connection
restarts and everything works fine as usual.

Second Environment(issue one) - Application throws this error (unrecoverable
error) but at the same time or after 1.5 days as well, application does not
receive call on JmsConnection - onException() method, so application stuck
here. 
We have to forcefully restart the application to fix this issue.

I think, this could be a defect. The application should receive call on
ExceptionListener-onException() method at the event of connection failure.
But it does not happen.

After restart of this application, we haven't received this issue, so we
can't provide Trace level log. 

I found some qpid logs which could helps us for more information. Please
take a look on attached "qpid-app-log.txt" and "ApplicationError.txt"

Please let us know your opinion to fix this issue.

qpid-app-log.txt
  
ApplicationError.txt
  




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Error while creating message - The Session was closed due to an unrecoverable error

[akabhishek1]

HI Team,

We are using qpid-jms-client-0.45.0 to publish/receive messages from
ServiceBus.

We are getting "The Session was closed due to an unrecoverable error" in our
several environments while creating JMS message with the help of Session.

We haven't received this type of error while creating JMS message from more
than 2 years. We are trying to find out the root cause of this issue. 

We haven't got any clue for this issue. Please find below exception
stacktrace. Could you please take a look on stacktrace and suggest - what
could be the root cause for this issue?  

Any suggestion will be great help. I tried to reproduce this issue but i
haven't got anyway to reproduce this issue. Please suggest if you have any
idea to reproduce this type of issue.


Exception StackTrace - 

The Session was closed due to an unrecoverable error.
(javax.jms.IllegalStateException)
(org.mule.api.transformer.TransformerException)

org.apache.qpid.jms.provider.amqp.AmqpSupport.convertToNonFatalException(AmqpSupport.java:184)

org.apache.qpid.jms.provider.amqp.AmqpAbstractResource.processRemoteClose(AmqpAbstractResource.java:266)

org.apache.qpid.jms.provider.amqp.AmqpProvider.processUpdates(AmqpProvider.java:952)

org.apache.qpid.jms.provider.amqp.AmqpProvider.onData(AmqpProvider.java:850)

org.apache.qpid.jms.transports.netty.NettyTcpTransport$NettyTcpTransportHandler.channelRead0(NettyTcpTransport.java:542)

org.apache.qpid.jms.transports.netty.NettyTcpTransport$NettyTcpTransportHandler.channelRead0(NettyTcpTransport.java:535)

io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)

io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)

io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)

io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
 io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475)
 io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
 io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)

io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)

io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)

io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)

io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)

io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)

io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)

io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1421)

io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)

io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)

io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:930)

io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:794)
 io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:424)
 io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:326)

io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918)
 io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
 java.lang.Thread.run(Thread.java:745)
 

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Feature request - increase visibility of JmsSession.checkClosed() method

[akabhishek1]

Hi Team,

We are using qpid library to publish/receive messages from ServiceBus.
Microsoft ServiceBus forcefully detach the connected session if session is
idle, same behavior for JMS connection as well.

Microsoft ServiceBus also has limitation that we can not create too much JMS
sessions and producers as well.

Due to this limitation, we have to use some sessions and producer to perform
all activities.

I looked on QPid class org.apache.qpid.jms.JmsSession, checkClosed()
internally used to check session status but this method is protected, so it
will be used in internal classes only.

I would like to request you - Can you please increase visibility(protected
to public) of checkClosed() method, so we can check session status as well
before performing any action.

This will be a great facility for ServiceBus use case. Please suggest your
opinion.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Set Custom MessageID for duplicate message detection

[akabhishek1]

Hey Tim,

Thanks a lot for quick reply. I tested with your suggestion, and I am able
to set custom message ID with the help of CustomJmsMessageIDPolicy,
But we have to apply this policy at the connection level, which is a big
problem for us because we can't have a new connection for every new message.

Every message will have different custom MessgeID and we need to set/send
application defined messageID.

Can you please suggest anyway to give precedence of setting messageID on
JMSMessage? 
I tried to use "textMessage.setJMSMessageID("custom-ID");", but this doesn't
help. 

Please suggest any approach to define custom messageID with the help of
JmsMessage object.

Example -

import java.util.Hashtable;
import javax.jms.Connection;
import javax.jms.Destination;
import javax.jms.ExceptionListener;
import javax.jms.JMSException;
import javax.jms.MessageProducer;
import javax.jms.Session;
import javax.jms.TextMessage;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import org.apache.qpid.jms.JmsConnectionFactory;
import org.apache.qpid.jms.JmsDestination;
import org.apache.qpid.jms.JmsSession;
import org.apache.qpid.jms.message.JmsMessage;
import org.apache.qpid.jms.message.JmsMessageIDBuilder;
import org.apache.qpid.jms.policy.JmsMessageIDPolicy;
import org.apache.qpid.jms.provider.amqp.message.AmqpJmsMessageFacade;

public class TestQpidSendCustomMessageID implements ExceptionListener{

private static final String QUEUE_NAME = "test-msg-queue";
private static final String SBUS_NAME = "#";
private static final String USERNAME = "##";
private static final String PASSWORD = "";
private static final String QPID_CONNECTION_FACTORY_CLASS =
"org.apache.qpid.jms.jndi.JmsInitialContextFactory";

public static void main(String[] args) throws Exception {
TestQpidSendCustomMessageID test = new 
TestQpidSendCustomMessageID();
test.send();
}

private void send() throws NamingException, JMSException,
InterruptedException {
Hashtable hashtable = new Hashtable<>();

hashtable.put("connectionfactory.SBCF", "amqps://"+ SBUS_NAME
+".servicebus.windows.net?transport.tcpKeepAlive=true&amqp.traceFrames=true&jms.prefetchPolicy.all=1000&jms.forceAsyncSend=true");

hashtable.put(Context.INITIAL_CONTEXT_FACTORY,
QPID_CONNECTION_FACTORY_CLASS);

Context context = new InitialContext(hashtable);

JmsConnectionFactory connectionFactory = (JmsConnectionFactory)
context.lookup("SBCF");

//Set Custom Jms MessageID Policy
CustomJmsMessageIDPolicy customJmsMessageIDPolicy = new
CustomJmsMessageIDPolicy();
connectionFactory.setMessageIDPolicy(customJmsMessageIDPolicy);

Connection connection = 
connectionFactory.createConnection(USERNAME,
PASSWORD);
connection.setExceptionListener(this);

connection.start();

Session session = connection.createSession(false,
Session.CLIENT_ACKNOWLEDGE);
Destination destination = session.createQueue(QUEUE_NAME);
MessageProducer messageProducer = 
session.createProducer(destination);

TextMessage textMessage = session.createTextMessage("Hello -QA 
ASD");
textMessage.setJMSMessageID("Hello-SetJMSID"); //First Way

((AmqpJmsMessageFacade) ((JmsMessage)
textMessage).getFacade()).setMessageId("AKQWS:9d678sdfsdsds"); //Second Way

messageProducer.send(textMessage);

System.out.println(" Published successfully ");

}

@Override
public void onException(JMSException exception) {
exception.printStackTrace();
}

class CustomJmsMessageIDPolicy implements JmsMessageIDPolicy{

@Override
public JmsMessageIDPolicy copy() {
return new CustomJmsMessageIDPolicy();
}

@Override
public JmsMessageIDBuilder getMessageIDBuilder(JmsSession 
session,
JmsDestination destination) {

return new JmsMessageIDBuilder() {

@Override
public Object createMessageID(String 
producerId, long messageSequence) {
return "test-custom-msg-id"; // How can 
we set message from JmsMessage
Object?
}
};

}

}


}




--

Set Custom MessageID for duplicate message detection

[akabhishek1]

HI Team,

We are using ServiceBus Broker, and using qpid-jms-client-0.45.0 to publish
message. ServiceBus provides facility to 'ignore duplicate messages' with
the help of MessageID. 

For achieving this requirement - We need to set JMS message ID while
publishing the message. At this moment, i have not found any way to set
custom messageID via qpid JMS client. I tried with below options but that
doesn't help.

JmsMessageProducer messageProducer = (JmsMessageProducer)
session.createProducer(destination);
messageProducer.setDisableMessageID(true);
TextMessage textMessage = session.createTextMessage("Hello World");
((AmqpJmsMessageFacade) ((JmsMessage)
textMessage).getFacade()).setMessageId("AKQWS:9d678sdfsdsds");


I did testing with Microsoft JAVA SDK
(com.microsoft.azure:azure-servicebus:1.2.11), and i am able to achieve this
requirement via setting MessageID.

Could you please provide any way to set MessageID.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession

[akabhishek1]

Hi Robbie,

Thanks a lot for quick reply. Yes you are right - there was issue in message
content. We got this issue while processing failed message with ServiceBus
explorer. 

I raised this issue with serviceBus explorer team.  Issue URL -
https://github.com/paolosalvatori/ServiceBusExplorer/issues/451

Thanks again for quick help.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession

[akabhishek1]

Hi Rob/Team,

Could you please take a look on this issue? 

At this moment, i am not able to replicate this issue. 

Can you please provide any clue to generate  'Cannot decode String in UTF-8'
error?

So i can generate 'Suspected deadlocked threads' error and share with you
details.

Regards,
Abhishek



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession

[akabhishek1]

Hi Robbie,

Thanks a lot for quick reply. I was researching on this issue and found
below root cause for deadlock.

Everytime application trying to reconnect receiver but getting exception
'Cannot decode String in UTF-8'. Looks like it could be issue due to message
content. But how can we resolve this?
Could you please take a look on this issue and suggest your opinion. Please
let me for any question and find below excpetion stack trace.

*Stacktrace*

"Connection exception, Reason 'Cannot decode String in UTF-8' - 'class
javax.jms.JMSException'"
Message : Cannot decode String in UTF-8 (javax.jms.JMSException)
Exception stack is: Cannot decode String in UTF-8 (javax.jms.JMSException)
(org.mule.transport.ConnectException)
  java.nio.charset.CoderResult.throwException(CoderResult.java:281)
  java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
 
org.apache.qpid.jms.provider.amqp.message.AmqpJmsTextMessageFacade.getText(AmqpJmsTextMessageFacade.java:84)
 
org.apache.qpid.jms.provider.amqp.message.AmqpJmsTextMessageFacade.copy(AmqpJmsTextMessageFacade.java:65)
 
org.apache.qpid.jms.provider.amqp.message.AmqpJmsTextMessageFacade.copy(AmqpJmsTextMessageFacade.java:41)
  org.apache.qpid.jms.message.JmsTextMessage.copy(JmsTextMessage.java:37)
  org.apache.qpid.jms.message.JmsTextMessage.copy(JmsTextMessage.java:25)
  org.apache.qpid.jms.JmsMessageConsumer.copy(JmsMessageConsumer.java:419)
 
org.apache.qpid.jms.JmsMessageConsumer.deliverNextPending(JmsMessageConsumer.java:769)
 
org.apache.qpid.jms.JmsMessageConsumer.access$100(JmsMessageConsumer.java:61)
 
org.apache.qpid.jms.JmsMessageConsumer$MessageDeliverTask.run(JmsMessageConsumer.java:840)
 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 
org.apache.qpid.jms.util.QpidJMSThreadFactory$1.run(QpidJMSThreadFactory.java:86)
  java.lang.Thread.run(Thread.java:745)
  (set debug level logging or '-Dmule.verbose.exceptions=true' for
everything)
 
\n"
  
Exception caught is a ConnectException, attempting to reconnect..."
Stopping connector: Sbus-Connector"
Failed to close connection, Reason 'Illegal invocation from MessageListener
callback'"
Failed to close session, Reason 'Illegal invocation from MessageListener
callback'"
Failed to close connection, Reason 'Illegal invocation from MessageListener
callback'"
Starting connector: Sbus-Connector"
Starting: ServicebusConnector
Connected: ServicebusConnector
Starting message receiver"
Connection exception, Reason 'Cannot decode String in UTF-8' - 'class
javax.jms.JMSException'"
Requesting to reconnect broken receiver, Reason 'Cannot decode String in
UTF-8'"
Sending exception to SystemException handler, Reason 'Cannot decode String
in UTF-8' - 'class javax.jms.JMSException'"
Message   : Cannot decode String in UTF-8
(javax.jms.JMSException)
Exception stack trace :Cannot decode String in UTF-8
(javax.jms.JMSException) (org.mule.transport.ConnectException)
  java.nio.charset.CoderResult.throwException(CoderResult.java:281)
  java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:816)
 
org.apache.qpid.jms.provider.amqp.message.AmqpJmsTextMessageFacade.getText(AmqpJmsTextMessageFacade.java:84)
 
org.apache.qpid.jms.provider.amqp.message.AmqpJmsTextMessageFacade.copy(AmqpJmsTextMessageFacade.java:65)
 
org.apache.qpid.jms.provider.amqp.message.AmqpJmsTextMessageFacade.copy(AmqpJmsTextMessageFacade.java:41)
  org.apache.qpid.jms.message.JmsTextMessage.copy(JmsTextMessage.java:37)
  org.apache.qpid.jms.message.JmsTextMessage.copy(JmsTextMessage.java:25)
  org.apache.qpid.jms.JmsMessageConsumer.copy(JmsMessageConsumer.java:419)
 
org.apache.qpid.jms.JmsMessageConsumer.deliverNextPending(JmsMessageConsumer.java:769)
 
org.apache.qpid.jms.JmsMessageConsumer.access$100(JmsMessageConsumer.java:61)
 
org.apache.qpid.jms.JmsMessageConsumer$MessageDeliverTask.run(JmsMessageConsumer.java:840)
 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 
org.apache.qpid.jms.util.QpidJMSThreadFactory$1.run(QpidJMSThreadFactory.java:86)
  java.lang.Thread.run(Thread.java:745)
  (set debug level logging or '-Dmule.verbose.exceptions=true' for
everything)
 
\n"



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

qpid-jms-client-0.45 --> Suspected deadlocked threads - owned by JmsSession

[akabhishek1]

Hi All,

We are having deadlock error at prod system. We are using
"qpid-jms-client-0.45.0" to connect Azure ServiceBus.  At this moment, we
have not found any root cause of this problem. As this is prod system, so we
can't enable qpid trace level login.

Could you please take a look on this issue and suggest if this is caused by
qpid client or something else?

Error Details - First error
"2020-04-06T10:47:20.17Z","Suspected deadlocked threads: QpidJMS Connection
Executor: ID:9d006669-51f1-4fdb-9529-fe4c72e2481d:62 Id=1383 is in BLOCKED
on lock=org.mule.servicebus.Connector@511db1d is owned by JmsSession
[ID:951ecf9f-4991-4eq1-9c5e-85c001fwwe00:64:1] delivery dispatcher Id=1309 |
JmsSession [ID:951ecf9f-4991-4eq1-9c5e-85c001fwwe00:64:1] delivery
dispatcher Id=1309 is in WAITING on
lock=java.util.concurrent.locks.ReentrantLock$NonfairSync@4d039032 is
owned by JmsSession [ID:951ecf9f-4991-4eq1-9c5e-85c001fwwe00:64:3] delivery
dispatcher Id=1311 | JmsSession
[ID:951ecf9f-4991-4eq1-9c5e-85c001fwwe00:64:3] delivery dispatcher Id=1311
is in BLOCKED on lock=org.mule.servicebus.Connector@511db1d is owned by
JmsSession [ID:951ecf9f-4991-4eq1-
9c5e-85c001fwwe00:64:1] delivery dispatcher Id=1309 | "

Complete error - Please refer attached file. ServiceBus_DeadLock_Errors.zip
 
 

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Out Of Memory error - Qpid not releasing memory in failed to acknowledge scenario

[akabhishek1]

Hi Tim,

Thanks a lot for quick reply. Please find attached updated code. I am
intentionally throwing exception after receiving the message. We could face
this issue in production. 

We have implemented reliability pattern, message should be acknowledge only
after successful transformation. Bussiness transformation/process could be
fail continously due to unavailability of service. 

We are using *connection.createSession(false, 101)* , so only one message
will be acknowledged at a time not prior messages.

The given example is just to replicate Out of Memory error based on
bussiness scenario.

Please suggest, if i can do anything to avoid this error.  


  TestQpidRcvOnly.txt
  



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Out Of Memory error - Qpid not releasing memory in failed to acknowledge scenario

[akabhishek1]

Hi Team,

We were doing testing of Qpid JMS client for failed scenario. We are
receiving message as client acknowledgment mode.

Scenario - Receive message but do not acknowledge the message.

Issue - Memory is not getting released by Qpid on failed Scenario

We could have this scenario in prodcution. We received the message, but
failed to process that message, so there will no aknowledgment. As we are
acknowledging only one success scenario.

Please find attached sample TestQpidRcvOnly.java, Total_Memory.PNG,
Sampler_Per_Thread_Allocation.PNG, Out_Of_Memory_Error.log .

Steps to reproduce this issue -->

1. Provide Correct details of QUEUE_NAME, SBUS_NAME, USERNAME, USERNAME in
TestQpidRcvOnly class
2. Publish 20 message on QUEUE_NAME, from any utility
3. Start the TestQpidRcvOnly class and keep it running
4. Open JVisualVM and have a look on Heap memory
5. Wait for 20m, you can see memory consumption continously increasing
6. You can see Out Of Memory Error on exhaustion of defined Max heap Memory.
Could be 1hr depends on Max Size
7. You can have a look on Sampler--> Memory -->Per Thread Allocation, you
can observe that Qpid thread is consuming more that 98% memory.

Consumption is OK but it is not releasing the memory, which is the cause for
Out of Memory Error.

Using -- Java 8, Azure ServiceBus, Qpid-jms-client-0.48


Could you please take a look on this issue.

Regards,
Abhishek Kumar

Out_Of_Memory_Error.log
  
Sampler_Per_Thread_Allocation.PNG

  
TestQpidRcvOnly.txt
  
Total_Memory.PNG
  



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: How to enable serialization of QPid JMSMessage object?

[akabhishek1]

Hi Gordon And Robbie,

Thanks a lot for quick reply. It really helps to understand this scenario.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: How to enable serialization of QPid JMSMessage object?

[akabhishek1]

Hi Robbie,

Thanks a lot for quick response. We got a situation, where we are in bottle
neck at this moment.

Problem/Situation - We are using ServiceBus to receive the message and
acknowledge the message after completing all the process/transformation.

As one of the processor(Mule Batch) needs every object should be
serializable. We can't acknowledge the message before batch execution which
could be reason for Message loss.

As acknowledgement needs JMSMessage which is not serializable, so we are
trying to find out alternative to acknowledge message without using
JMSMessage object. 

Is there anyway to acknowledge message via Message ID?

Could you please suggest or provide any hint to acknowledge message without
using JMSMessage object?

Regards,
Abhi




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

How to enable serialization of QPid JMSMessage object?

[akabhishek1]

Hi Team,

We have requirement that, all the qpid JMS messages should be serializable.
I received below object type and which is not serializable.

Received Object Type - JmsBytesMessage,
org.apache.qpid.jms.provider.amqp.message.AmqpJmsBytesMessageFacade

I tested with adding below connection property, but that doesn't help.

jms.deserializationPolicy.whiteList=org.apache.qpid.jms.provider.amqp.message.AmqpJmsBytesMessageFacade

Could you please suggest that, how can we make received Qpid JMS object as
serializable?

Thanks in advance !!

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: receiveNoWait() - not receiving messages

[akabhishek1]

Thanks a lot for confirmation Robbie. I raised an issue with ServiceBus. I
will update you if i got any information from them.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

"consumer.receive(1)" operation not receiving message continuous

[akabhishek1]

Hi Team,

I am using "qpid-jms-client-0.43.0" to receive message from ServiceBus
broker. 

We have requirement to receive message based on polling duration, so i am
using "messageConsumer.receive(1)" operation to receive message from
ServiceBus.

Precondition - Queue has already 10K messages. PrefetechCount = 0;
Issue - Consumer stopped receiving message continuously. As message already
available on queue but still it doesn't consume the message.

Please find below short logs. I found only one difference that is
"linkCredit=0". Could you please suggest why consumer not able to receive
message continuously. 

Please find attached log file for complete reference and let me know for any
other details.

//*** Logs ***//
2019-06-21 12:04:49,862 [[ipt-connector-send-test].recieve-flow.stage1.03]
INFO  org.mule.api.processor.LoggerMessageProcessor - *** Received Message
**
2019-06-21 12:04:49,862 [AmqpProvider
:(1):[amqps://XX.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - [629790599:1] SENT:
Flow{nextIncomingId=4935, incomingWindow=2047, nextOutgoingId=1,
outgoingWindow=2147483647, handle=0, deliveryCount=4934, linkCredit=1,
available=null, drain=false, echo=false, properties=null}
2019-06-21 12:04:49,862 [AmqpProvider
:(1):[amqps://XX.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - [629790599:1] SENT:
Disposition{role=RECEIVER, first=4933, last=4933, settled=true,
state=Accepted{}, batchable=false}
2019-06-21 12:04:49,877 [AmqpProvider
:(1):[amqps://XX.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - [629790599:1] SENT:
Flow{nextIncomingId=4935, incomingWindow=2047, nextOutgoingId=1,
outgoingWindow=2147483647, handle=0, deliveryCount=4934, linkCredit=1,
available=null, drain=true, echo=false, properties=null}
2019-06-21 12:04:49,877 [AmqpProvider
:(1):[amqps://XX.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - [629790599:1] RECV:
Transfer{handle=0, deliveryId=4934,
deliveryTag=\xcbBRv~\xf7NB\xac"\x1c\x1c\xdex\x0f\xc3, messageFormat=0,
settled=null, more=false, rcvSettleMode=null, state=null, resume=false,
aborted=false, batchable=true} (652)
"\x00Sp\xc0\x06\x05C\x00Sq\xc1$\x02\xa3\x10x-opt-lock-token\x98vRB\xcb\xf7~BN\xac"\x1c\x1c\xdex\x0f\xc3\x00Sr\xc1}\x08\xa3\x13x-opt-enqueued-time\x83\x00\x00\x01ky\xaf\xc8\xaf\xa3\x15x-opt-sequence-number\x81\x00\x00\x00\x00\x00\x00\x13G\xa3\x1dx-opt-enqueue-sequence-numberU\x00\xa3\x12x-opt-locked-until\x83\x00\x00\x01ky\xb6\xc6\x1e\x00Ss\xc03\x0d\xa1$a56009e5-5f37-4509-9f84-650a46e4af9a\x00St\xc1\x01\x00\x00Su\xb0\x00\x00\x01\x90ABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDasdABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDasdABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABaqwCDABCDABCDABCDABCDABCDABCDasdABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDABCDasda"
2019-06-21 12:04:49,877 [[ipt-connector-send-test].recieve-flow.stage1.03]
INFO  org.mule.api.processor.LoggerMessageProcessor - *** Received Message
**
2019-06-21 12:04:49,877 [AmqpProvider
:(1):[amqps://XX.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - [629790599:1] SENT:
Flow{nextIncomingId=4936, incomingWindow=2047, nextOutgoingId=1,
outgoingWindow=2147483647, handle=0, deliveryCount=4935, linkCredit=1,
available=null, drain=false, echo=false, properties=null}
2019-06-21 12:04:49,877 [AmqpProvider
:(1):[amqps://XX.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - [629790599:1] SENT:
Disposition{role=RECEIVER, first=4934, last=4934, settled=true,
state=Accepted{}, batchable=false}
2019-06-21 12:04:49,893 [AmqpProvider
:(1):[amqps://XX.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - [629790599:1] SENT:
Flow{nextIncomingId=4936, incomingWindow=2047, nextOutgoingId=1,
outgoingWindow=2147483647, handle=0, deliveryCount=4935, linkCredit=1,
available=null, drain=true, echo=false, properties=null}
2019-06-21 12:04:49,956 [AmqpProvider
:(1):[amqps://XX.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - [629790599:1] RECV:
Flow{nextIncomingId=1, incomingWindow=5000, nextOutgoingId=4936,
outgoingWindow=2047, handle=0, deliveryCount=4935, linkCredit=0,
available=0, drain=true, echo=false, properties=null}
2019-06-21 12:04:49,956 [AmqpProvider
:(1):[amqps://XX.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - [629790599:1] RECV:
Flow{nextIncomingId=1, incomingWindow=5000, nextOutgoingId=4936,
outgoingWindow=2047, handle=0, deliveryCount=4936, linkCredit=0,
available=0, drain=true, echo=false, properties=null}
2019-06-21 12:04:49,956
[[ipt-connector-send-test].ServiceBus_Connector.receiver.01] INFO 
org.mule.transport.servicebus.ServicebusPollingMessageReceiver 

Re: receiveNoWait() - not receiving messages

[akabhishek1]

Hi Robbie,

Thanks a lot for quick reply. I replaced only "consumer.receiveNoWait()"
operation to "consumer.receive(100)" is source code and kept everything
same.

Please find the attached updated log. As you can see that, I am able to
receive messages continuously.

It will be very difficult for me to justify ServiceBus team for this issue,
as One operation "receive(100)" is working fine, but other operation
"receiveNoWait()" is not working fine.

Please take a look on attached log, if you still feel that there is no issue
from Qpid side then please let me know, so I can raise ticket against
ServiceBus team.

Regards,
Abhishek Kumar
ReceiveWaitDurationLogs.txt
  



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: receiveNoWait() - not receiving messages

[akabhishek1]

Hi Robbie,

Please find the attached log. I kept the application idle for 9m, but
consumer doesn't receive any message. 

As i am consuming message on demand, so client should receive message if
message available on broker. I checked broker and i can confirm that,
messages are avalaible on queue. 

 Please suggest me if i am missing anything or is this a defect?

Regards,
Abhishek Kumar

ReceiveNoWaitLogs.txt
  



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

receiveNoWait() - not receiving messages

[akabhishek1]

Hi Team,

We need to consume message based on polling duration. So i did testing for
"receive(long timeout)" and "receiveNoWait()".

So far "receive(long timeout)" is working perfectly but there is an issue in
"receiveNoWait()" operation. 

Issue - "receiveNoWait()" operation not consuming message

Please find attached class. I am publishing message continuously and
consuming as well. Publishing is working perfectly but i am not able to
receive message. 

If you change operation name from "receiveNoWait()" to "receive(long
timeout)", then we will start receiving message continuously.

Am i doing any mistake? Could you please take a look on this issue.

Broker - Azure ServiceBus
Qpid - qpid-jms-client-0.43.0
Java - JDK8+

Regards,
Abhishek Kumar

TestQpidRcvrNoWait.java
  



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: remove prefix "ID:" and "ID:AMQP_NO_PREFIX:" while receiving messages

[akabhishek1]

Hi Robbie,

Thanks a lot for confirmation.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

remove prefix "ID:" and "ID:AMQP_NO_PREFIX:" while receiving messages

[akabhishek1]

Hi Team,

We need to remove prefix "ID:" and "ID:AMQP_NO_PREFIX:" while receiving the
message.
I published message on ServiceBus broker via two ways,  
1. Qpid JMS Client
2. Microsoft JAVA SDK.

Please find the attached screenshot “Published_Messages.png” of service bus
explorer. 

When I received the message via "qpid-jms-client-0.43.0" then i am getting
two prefix "ID:" and "ID:AMQP_NO_PREFIX:"

Published Via QPID JMS client -
ID:9f54c235f790-4f81-b74d-843bfe4c59f8:1:1:1-1
Published Via Microsoft JAVA SDK  -  ID:AMQP_NO_PREFIX:TestMessageID


I tried to remove prefix "ID:" and "ID:AMQP_NO_PREFIX:" with the use of
"jms.clientIDPrefix" and "jms.clientID" at connection URL, BUT this setting
doesn't work.

Could you please provide some guidance to remove "ID:" and
"ID:AMQP_NO_PREFIX:" while receiving messages via Qpid JMS Client? 

I know, String manipulation is one option but i don't want to use this,
because this is not idle.

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: how to maintain reliability with ASYNC operation?

[akabhishek1]

Hi Robbie,

Based on your input, i did some testing and got different test result in
terms of TPS(transaction per second) for below conditions

1. jms.forceSyncSend=true + send(jmsMsg, PERSISTENT, DEFAULT_PRIORITY,
DEFAULT_TIME_TO_LIVE)  -- > I got TPS around 1200.

2. NO Setting @Connection Level + send(jmsMsg, PERSISTENT, DEFAULT_PRIORITY,
DEFAULT_TIME_TO_LIVE)  -- > I got TPS around 1800.


As both conditions tend to SYNCHRONOUS. I wondered to see different test
result with variance of 600TPS. My sourceCode is completely same for both
conditions except setting at connection level.

Expectation - As both conditions tend to SYNCHRONOUS, so i am expecting
approx. same TPS for both conditions. 

Could you please provide some insights, why i am getting different Test
throughput as both are synchronous? Am i missing something?

Client - qpid-jms-client-0.43.0
Java8
Broker - Azure ServiceBus(Premium)

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: how to maintain reliability with ASYNC operation?

[akabhishek1]

Hi Robbie,

Thank you so much for your all inputs. I really appreciate your fast
response. 

We are doing testing based on your inputs. We will come back again, if we
have any other questions.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: how to maintain reliability with ASYNC operation?

[akabhishek1]

Hi Robbie,

Thank you so much for quick response. However, we have couple of open
questions related with Qpid. If you provide us input that will be too great.
We are using "qpid-jms-client-0.40.0" and sending message to Azure Service
Bus.

1. Condition - if "jms.forceSyncSend=true" + send(Message message,
CompletionListener completionListener);
   Q- Is message will process in SYNC or ASYNC? As i am using
"jms.forceSyncSend=true", so could we get response in CompletionListener or
in same thread? 
   
   I have done testing locally with "qpid-jms-client-0.40.0" and i am
getting response in CompletionListener-onCompletion/onException. 
   
   As per Qpid documentation, "jms.forceSyncSend" should override all
asynchronous send conditions, 
   But it is not doing at this moment because i am getting response back
with CompletionListener . Is this BUG??
   

2. Condition - if "jms.forceAsyncSend=false" + Send with NO
CompletionListener,
   Q -Could you please confirm, in this scenario QPID will send message in
SYNC or ASYNC?

2. Condition - if "jms.forceSyncSend=false" + Send with NO
CompletionListener,
   Q -Could you please confirm, in this scenario QPID will send message in
SYNC or ASYNC? 


3. Can we make assumption that, If we are send message with
CompletionListener, then Qpid always send message to broker in ASYNC way or
it can be SYNC also with "jms.forceSyncSend=true"?

4. SYNCHRONOUS - What is the internal behaviour of Qpid in SYNC messaging?
Is qpid waiting for successful response back from broker or it is working
like NON blocking IO concept?

5. Is there anyway to identify to nature of SEND call, our SEND call is
ASYNC or SYNC? Proton logs?

6. Enforce ASYNC - Can you please tell us best way to enforce ASYNC?

7. Enforce SYNC - Can you please tell us best way to enforce SYNC?


Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

how to maintain reliability with ASYNC operation?

[akabhishek1]

Hi Team,

We are using "qpid-jms-client-0.40.0" to send messages to Azure ServiceBus.
At this moment, we are using "jms.forceSyncSend=true" to avoid message loss.

We got the requirement to publish message asynchronously for better
throughtput.

I have few questions around ASYNC operation to use it effectively in our
estate -
- How async operation are handling the messages internally (documentation is
very thin) 
- Is this making network call to broker and waiting for responses?
- How the message reliability is maintained?  
- what are the downsides of using ASYNC from a library usage perspective?

Any insights, you can offer here.

Regards,
Abhishek Kumar





--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Better way to handle 'MessageNotWriteableException: Message is currently read-only'

[akabhishek1]

Thank you so much for confirmation Rob. 

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Better way to handle 'MessageNotWriteableException: Message is currently read-only'

[akabhishek1]

Hi Team,

Is it possible for you to release "0.43.0" with latest fixes, as we are
having this issue in multiple applications. Please let us know your opinion.

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

how to Schedule Message with Qpid JMS Client

[akabhishek1]

Hi Team,

We have requirement to schedule message. We are using
"qpid-jms-client-0.40.0".

As this feature is available in active MQ but i am not sure this feature is
available in Qpid or not?

Could you please suggest, Is this feature available in Qpid JMS or Not? If
yes then how can i schedule message, any reference link?

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Better way to handle 'MessageNotWriteableException: Message is currently read-only'

[akabhishek1]

Hi Tim,

Thank you so much for your quick reply and fix. 
Could you please suggest time frame to release "0.43.0", So I can inform to
team accordingly ?

Could you please also suggest your opinion on below code snippet, which one
is better approach 1 or 2? Or i can use anyone?

//*** (1) Code At Time Of Error*** 
final Message origionalJmsMessage = message; // message = javax.jms.Message 
try { 
sendMessage(message); 
} catch (IllegalStateException illegalStateException) { 
// Reset Producer due to Idle TimeOut after 10m 
resetProducer();
sendMessage(origionalJmsMessage); 
}   


//*** (2) Code After Correction*** 
Message origionalJmsMessage; 
try { 
sendMessage(message); // message = javax.jms.Message 
} catch (IllegalStateException illegalStateException) { 
// Reset Producer due to Idle TimeOut after 10m 
resetProducer();
origionalJmsMessage = 
((org.apache.qpid.jms.message.JmsMessage)message).copy(); 
sendMessage(origionalJmsMessage); 
}   

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Better way to handle 'MessageNotWriteableException: Message is currently read-only'

[akabhishek1]

Hi Team,

ServiceBus forcefully disconnect producer for 10m idle timeout. We are
handling this situation with below code snippet ("//***Code At Time Of
Error***"), but we are getting strange error
"javax.jms.MessageNotWriteableException: Message is currently read-only".
This is a very rare error, and we haven't received this error from one year.

We are not not able to reproduce this issue as well. However, we came to
conclusion that copy of message to origionalJmsMessage "final Message
origionalJmsMessage = message", that could be issue. Because i am copying
"message" via reference instead of value. 

So i changed the logic to copy value by "Message origionalJmsMessage =
(org.apache.qpid.jms.message.JmsMessage)message).copy();"

I am not sure, this approach is bullet proof or not. 

Could you please suggest your opinion to preserve JMS message in a better
way and send again for handling idle timeout of the producer.

//Artifact Version
qpid-jms-client - 0.40.0
Broker - Azure ServiceBus
Java - JBK8
OS - Linux

//***Code At Time Of Error***
final Message origionalJmsMessage = message; // message = javax.jms.Message
try {
sendMessage(message);
} catch (IllegalStateException illegalStateException) {
// Reset Producer due to Idle TimeOut after 10m
resetProducer();
sendMessage(origionalJmsMessage);
}   


//**Code After Correction***
Message origionalJmsMessage;
try {
sendMessage(message); // message = javax.jms.Message
} catch (IllegalStateException illegalStateException) {
// Reset Producer due to Idle TimeOut after 10m
resetProducer();
origionalJmsMessage =
((org.apache.qpid.jms.message.JmsMessage)message).copy();
sendMessage(origionalJmsMessage);
}   

//***Exception Stack Trace***
Caused by: javax.jms.MessageNotWriteableException: Message is currently
read-only 
org.apache.qpid.jms.message.JmsMessage.checkReadOnly(JmsMessage.java:509) 
org.apache.qpid.jms.message.JmsMessage.setJMSDeliveryMode(JmsMessage.java:242) 
org.apache.qpid.jms.JmsSession.send(JmsSession.java:817) 
org.apache.qpid.jms.JmsSession.send(JmsSession.java:811) 
org.apache.qpid.jms.JmsMessageProducer.sendMessage(JmsMessageProducer.java:252) 
org.apache.qpid.jms.JmsMessageProducer.send(JmsMessageProducer.java:182) 


Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Not able to recreate consumer after InvalidDestinationException, even tried 32K+ times to recreate

[akabhishek1]

Hi Robbie, 

We need one suggestion from you for handling JMS exception in better way. 

Pre Conditions- We have 
a) One javax.jms.Connection
b) Registered exception listner. So we will get connection related exception
on "onException" block
c) 50+ javax.jms.Session
d) 32+ javax.jms.MessageProducer (every producer have separate Session)
e) 6 javax.jms.MessageConsumer for Queue_NAME="test1" (with separate
Session)
f)  6 javax.jms.MessageConsumer for Queue_NAME="test2" (with separate
Session)


Condition - We got InvalidDestinationException for consumer (queue=TEST1) on
"onException" block

As I understand from online forum, I should to follow below approach in
"onException" block.
Step 1. public onException(JMSException e)
{
connection.setExceptionListener(null);
connection.close();
connectToServer();
}
Step 2. createConsumers(); --> For "test1" queue
Step 3. createConsumers(); --> For "test2" queue

Concern - We have concern on recreating connection on every thrown exception
on "onException" block. 
  As "InvalidDestinationException" is not a connection 
exception,
connection is still valid, we can still publish message with same connection
object.
  If we reset the connection then we need to reset consumer(for 
queue
"Test2") and producer as well, which is not ideal.

Problems - As we receive "JMSException" object in onException, and
"JMSException" is the super exception of all JMS Exception.
   So we can't identify that which exception are related with 
connection
exception (connection broken from Broker). 
   So we can't filter that reset connection of this type of 
exception. 


Could you please suggest best practice to perform action on "onException"
block ?

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Not able to recreate consumer after InvalidDestinationException, even tried 32K+ times to recreate

[akabhishek1]

Hi Robbie,

Is there anyway to identify that, is Connection active or Not (i mean broken
from wire)? 

It will help me to fix this issue. Please suggest your opinion.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Not able to recreate consumer after InvalidDestinationException, even tried 32K+ times to recreate

[akabhishek1]

Hi Robbie,

Brilliant !! Thank you so much for detail analysis on this issue. 
Your analysis helps alot to replicate this issue. I am working on this to
resolve this issue.
Thanks Again !!

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Not able to recreate consumer after InvalidDestinationException, even tried 32K+ times to recreate

[akabhishek1]

Hi Team,

We are having reconnection issue in PROD with ServiceBus on one of our
subscription after InvalidDestinationException. We are using
"qpid-jms-client-0.34.0" to connect ServiceBus.

We got InvalidDestinationException at "2018-12-07 16:55:13,228" and after
that application tries to recreate consumer every 10s but it was not able to
recreate upto 4 days. 
As we verified the Subscription and Topic were available.

For resolving this issue, we restarted the application and application
resume listening on same subscription. Restarting the application is not
good idea but we didn't have any better solution to fix this critical issue.

PFA "2018-12-07_JMS_Client.log" and "2018-12-07_Proton-J.log" (that wraps
the time when error happened) in "Qpid_Issue.zip" file.
Please find below sourceCode snippet for creating connection and consumer. 

Any reasons that you are aware of within Qpid, which can cause this issue
and a possible resolution/work-around?

Our environment: Azure Service Bus with 4 MUs (possibly the MUs were change
near the time, but it can be a coincidence, as the restart has brought the
application to the desired state).

Possibilities:
- network blip
- change of MUs
- disabling the subscription/topic for a small period of time

However, in any of the case above, the connection stays available, and so
the consumer should be recreated after certain attempts.

This is a PROD issue and we are responsible to offer a resolution and
therefore any pointer from your side is more than welcome.



Code to establish connection --> 

Hashtable hashtable = new Hashtable<>();
hashtable.put("connectionfactory.SBCF", "amqps://"+ SBUS_NAME

+".servicebus.windows.net?transport.tcpKeepAlive=true&amqp.traceFrames=true&jms.validatePropertyNames=false&jms.forceSyncSend=true");
hashtable.put(Context.INITIAL_CONTEXT_FACTORY,
QPID_CONNECTION_FACTORY_CLASS);
Context context = new InitialContext(hashtable);
ConnectionFactory connectionFactory = (ConnectionFactory)
context.lookup("SBCF");
Connection connection = connectionFactory.createConnection(USERNAME,
PASSWORD);
connection.setExceptionListener(this); // Settted ExceptionListener
connection.start();

Code to create consumer @ every 10s -->
Session session = connection.createSession(false,
Session.CLIENT_ACKNOWLEDGE);
Destination destination = session.createQueue(QUEUE_NAME);
MessageConsumer consumer = session.createConsumer(destination);
consumer.setMessageListener(this);

Regards,
Abhishek Kumar
Qpid_Issue.zip
  



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

how to understand proton-j logs?

[akabhishek1]

Hi Team,

We are using qpid-jms-client to publish and subscribe the messages. As this
client internally uses proton-j for publishing and subscribing messages.

I can see proton-j logs but i don't know what does mean.  Could you please
guide me what does mean by below sample logs.  If you have any sample
document to understand log that will be very great. 
At least high level of log meaning that will be also great for initial point
of view. 


Sample Log --> 
2018-11-15 17:34:24,159 [AmqpProvider
:(1):[amqps://**.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - SENT:
Attach{name='qpid-jms:receiver:ID:b9fd1c39-f95a-4c14-b83d-5a2e318bb125:1:4:1:ipt-test',
handle=0, role=RECEIVER, sndSettleMode=UNSETTLED, rcvSettleMode=FIRST,
source=Source{address='ipt-test', durable=NONE, expiryPolicy=LINK_DETACH,
timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null,
filter=null, defaultOutcome=Modified{deliveryFailed=true,
undeliverableHere=null, messageAnnotations=null},
outcomes=[amqp:accepted:list, amqp:rejected:list, amqp:released:list,
amqp:modified:list], capabilities=[queue]}, target=Target{address='null',
durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false,
dynamicNodeProperties=null, capabilities=null}, unsettled=null,
incompleteUnsettled=false, initialDeliveryCount=null, maxMessageSize=null,
offeredCapabilities=null, desiredCapabilities=null, properties=null}
2018-11-15 17:34:24,206 [AmqpProvider
:(1):[amqps://**.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - RECV:
Attach{name='qpid-jms:receiver:ID:b9fd1c39-f95a-4c14-b83d-5a2e318bb125:1:4:1:ipt-test',
handle=0, role=SENDER, sndSettleMode=MIXED, rcvSettleMode=SECOND,
source=Source{address='ipt-test', durable=NONE, expiryPolicy=LINK_DETACH,
timeout=0, dynamic=false, dynamicNodeProperties=null, distributionMode=null,
filter=null, defaultOutcome=Modified{deliveryFailed=true,
undeliverableHere=null, messageAnnotations=null},
outcomes=[amqp:accepted:list, amqp:rejected:list, amqp:released:list,
amqp:modified:list], capabilities=[queue]}, target=Target{address='null',
durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false,
dynamicNodeProperties=null, capabilities=null}, unsettled=null,
incompleteUnsettled=false, initialDeliveryCount=0, maxMessageSize=266240,
offeredCapabilities=null, desiredCapabilities=null, properties=null}
2018-11-15 17:34:24,206 [AmqpProvider
:(1):[amqps://**.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - SENT: Flow{nextIncomingId=1,
incomingWindow=2047, nextOutgoingId=1, outgoingWindow=2147483647, handle=0,
deliveryCount=0, linkCredit=1000, available=null, drain=false, echo=false,
properties=null}
2018-11-15 17:34:50,516 [AmqpProvider
:(1):[amqps://**.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - RECV: Empty Frame
2018-11-15 17:35:16,815 [AmqpProvider
:(1):[amqps://**.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - RECV: Empty Frame
2018-11-15 17:35:43,142 [AmqpProvider
:(1):[amqps://**.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - RECV: Empty Frame
2018-11-15 17:36:09,409 [AmqpProvider
:(1):[amqps://**.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - RECV: Empty Frame
2018-11-15 17:36:35,659 [AmqpProvider
:(1):[amqps://**.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - RECV: Empty Frame
2018-11-15 17:36:50,126 [AmqpProvider
:(1):[amqps://**.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - RECV: Transfer{handle=0,
deliveryId=0, deliveryTag=~#d\x11r\x8a\xf6I\x8e\x8f"\x98\xf6O\xd27,
messageFormat=0, settled=null, more=false, rcvSettleMode=null, state=null,
resume=false, aborted=false, batchable=true}
2018-11-15 17:36:50,228 [AmqpProvider
:(1):[amqps://**.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - SENT: Disposition{role=RECEIVER,
first=0, last=0, settled=true, state=Accepted{}, batchable=false}
2018-11-15 17:36:50,562 [AmqpProvider
:(1):[amqps://**.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - RECV: Transfer{handle=0,
deliveryId=0, deliveryTag=c\xc7\xbf\x83C'\xefE\x8bo+\xea\xba\x98\xb3H,
messageFormat=0, settled=null, more=false, rcvSettleMode=null, state=null,
resume=false, aborted=false, batchable=true}
2018-11-15 17:36:50,577 [AmqpProvider
:(1):[amqps://**.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - SENT: Disposition{role=RECEIVER,
first=0, last=0, settled=true, state=Accepted{}, batchable=false}
2018-11-15 17:36:54,842 [AmqpProvider
:(1):[amqps://**.servicebus.windows.net:-1]] TRACE
org.apache.qpid.jms.provider.amqp.FRAMES - RECV: Transfer{handle=0,
deliveryId=1, deliveryTag=\x1f>\xf63P\xba\x10I\xa9\x92\x86\xf1U\xc0-\x81,
messageFormat=0, settled=null, more=false, rcvSettleMode=null, state=null,
re

ServiceBus - Send message to DeadLetterQueue

[akabhishek1]

Hi Team, 

We are using "qpid-jms-client-0.34.0" to publish message on Azure
ServiceBus. 

We have requirement to send message on "DeadLetterQueue". I am able to
receive message from "DeadLetterQueue" but not able to send message. 

Can we send message to "DeadLetterQueue" via client? Could you please
confirm this. 

If yes, could you please give some pointer, so i can implement and test.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Message going in topic with the use of "session.createQueue(name)"

[akabhishek1]

Hi Robbie And Rob,

Thank you so much for details. 

I completely understand now, we can't do much. Thanks again. 

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Message going in topic with the use of "session.createQueue(name)"

[akabhishek1]

Hi Rob, 

Thanks for your reply. I also got same comment from Robbie and Timothy. 

So, I raised this issue with Microsoft and they accepted this issue and they
fixed issues in their SDK which is  “azure-servicebus-1.2.7”. 

I have done testing with “azure-servicebus-1.2.7”  and getting expected
error now. They are also using "proton-j-0.22.0" internally. I don't know,
what they have done changes to fix this issue.

After applying their fixes, i am thinking there should be done form client
side. That's why, they fixed the issue.  I am sorry to say this, i don't
have proton-j knowledge otherwise i could share technical implementation
details with you. 

This is not a blocker for us but this will definitely add value in JMS Qpid
Client. 

Regards,
Abhishek Kumar  



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Message going in topic with the use of "session.createQueue(name)"

[akabhishek1]

Hi Tim,

I have created 

Destination dest = session.createQueue("test-resource"); 

I am expecting message should go in Queue and if "queue" is not available
then throw 
exception.

Current Behaviour - If "test-resource" named topic exists instead of a Queue
then message is successfully delivered to topic "test-resource" instead of
throwing exception.

Feature Request - if i create Queue Destination then message should go in
Queue and if same resource not available as Queue then throw exception. 

Please let me know for any concern/question.

Regards,
Abhishek Kumar




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Message going in topic with the use of "session.createQueue(name)"

[akabhishek1]

HI Robbie And Tim,

It's quite old topic, but still i want to provide update on this issue. 

I raised this issue with Microsoft and they accepted this issue and released
fixes in “azure-servicebus-1.2.7”. 

I have done testing with “azure-servicebus-1.2.7”  and getting expected
error now. They are using "proton-j-0.22.0" internally. I don't know, what
they have done changes to fix this issue but we are getting expected error.

I want to request you, if possible please take a look on this issue again.
It will be great if we can get same feature in Qpid JMS. 

Regards,
Abhishek Kumar





--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: how AMQP connection can be active upto lifetime of application

[akabhishek1]

Hi Robbie And Gordon,

Thank you so much for your valuable information.  

We can not create producer with null destination or invalid destination, it
throws error. 

So for fixing this issue, i created temporary producer with valid
Destination. After creating "temporary producer" this problem solves and
started getting different error in 15 minute. 

*Error Is *- "The connection was inactive for more than the allowed 30
milliseconds and is closed by container 'LinkTracker'.
TrackingId:#21f77a_G6, SystemTracker:gateway7, Timestamp:7/4/2018
3:47:12 PM [condition = amqp:connection:forced]"

*For Fixing this issue* - I tried to create New Session and New Producer
from existing Connection, BUT i am not able to create new session and
producer with *existing Connection object*. 

*OPTION* - So i have only option to handle this situation. Whenever i got
this type of exception, then create New Connection, New Session and New
Producer. This is not a good idea but i don't have any other option. 

I understand that there is no issue from QPid JMS Client. *Could you please
suggest is there any other option to handling this scenario?*

I am afraid to create consumer for dummy queue because there are lot of
application depends on this common component. They might be afraid for
creating dummy queue. 

Please suggest your opinion, i will further from your opinion. 

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: how AMQP connection can be active upto lifetime of application

[akabhishek1]

Hi Gordon And Robbie,

Thanks a lot for your response. I think you are right it seems upper level
exception. 
After doing research, i came on below conclusion. 

Problem - If we create only Connection(without producer and consumer) and
kept Connection idle without creating producer and consumer, then we are
receiving "idle timeOut" error. 
Workaround - If we create Connection with producer/consumer and kept idle
without doing anything, then we are not receiving any error. 
Conclusion - It seems, we definitely need to have one active
producer/consumer 

Requirement - Need to create producer on demand.

Could you please suggest best approach to handle this scenario.

Please find below code snippet and attached example
(TestQpidSendIdleTimeout.java).

//CODE

Connection connection = connectionFactory.createConnection(USERNAME,
PASSWORD);
connection.setExceptionListener(this); // Settted ExceptionListener
connection.start();

//*** WORK_AROUND ***//
//if we create producer, then we will not get inactive connection 
timeOut
error. 
//it seems we should definitely have one producer/consumer otherwise we
will get
//connection timeOut error

/*Session session = connection.createSession(false,
Session.CLIENT_ACKNOWLEDGE);
Destination destination = session.createQueue(QUEUE_NAME);
MessageProducer messageProducer = session.createProducer(destination);*/

Regards,
Abhishek Kumar

TestQpidSendIdleTimeout.java
  



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: how to set ServiceBus Message properties like MessageID, ContentType, PartitionKey ?

[akabhishek1]

Hi Robbie,

Could you please suggest your opinion on below two query. 

1. JMS Type - While sending the message to ServiceBus, we can set JMS type 
like "message.setJMSType("Abhishek")" . 
  After publishing this message, i can see JMS Type as "Label" 
of serviceBus message header. 
  I think value of JMSType should populate as
"ContentType". Is it 
correct/expected behaviour? Please find attached 
screenshot(ServiceBus_Message_Properties_Type.png). 
  Pl ServiceBus_Message_Properties_Type.png

 
ease suggest. 

2. PartitionKey - As this is part of infrastructure header properties. Is 
there any way to publish/receive PartitionKey from qpid-jms-client? 

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: how to set ServiceBus Message properties like MessageID, ContentType, PartitionKey ?

[akabhishek1]

HI Robbie,

I have two questions. Could you please suggest your opinion on below two
questions. 

1. JMS Type - While sending the message to ServiceBus, we can set JMS type
like "message.setJMSType("Abhishek")" .
  After publishing this message, i can see JMS Type as "Label"
of serviceBus message header.
  I think vaklue of JMSType should populate as 
"ContentType". Is it
correct/expected behaviour? Please find attached
screenshot(ServiceBus_Message_Properties_Type.png).
  Please suggest. 

2. PartitionKey - As this is part of infrastructure header properties. Is
there any way to publish/receive PartitionKey from qpid-jms-client?

Regards,
Abhishek Kumar
ServiceBus_Message_Properties_Type.png

  



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: how to set ServiceBus Message properties like MessageID, ContentType, PartitionKey ?

[akabhishek1]

HI Robbie, 

Thank you so much for your quick reply.Highly appreciated for your help over
weekend. 

It solves my problem. I am able to set contentType with the use of facade
and also able to send property name with dot and hyphen. 

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: how to set ServiceBus Message properties like MessageID, ContentType, PartitionKey ?

[akabhishek1]

Hi Robbie, 

Thanks a lot for reply and detail explanation. 

I went through
"http://docs.oasis-open.org/amqp/core/v1.0/amqp-core-messaging-v1.0.html#section-message-format";
and 
understood that there could be the infrastructure property as well.

Some Difference between AMQP and JMS standard
1. AMQP  support property name as String but JMS supports only letter,
digit, symbol, punctuation character etc which allows "java.lang.Character
-> isJavaIdentifierPart"
   SO JMS can not allow property name with dot, hyphen.
2. AMQP supports infrastructure property as part of header property.
Qpid-Client-JMS does not provide facility to insert infrastructure property.
   We can not change ContentType as well(part of header property). 


My understanding is that Service Bus is probably using AMQP compatible
messages, but we are using the "qpid-jms-client" library to handle these
messages.  
This seems to cause a mismatch between what AMQP / Service Bus is sending in
messages and what the JMS client can manage.  
Is using the JMS client the bst approach?  Are there other options?

I also did some research based on "qpid-amqp-1-0-client-jms-0.32". With the
use of this, i can get hold on
"org.apache.qpid.amqp_1_0.jms.impl.MessageImpl". 
I am able to set "ContentType". Please find attached example. The main
problem - "qpid-amqp-1-0-client-jms-0.32" is a old library and no release
after "Mar, 2015".

Is that possible for you to provide same functionality in "qpid-jms-client"
? Please suggest. 

We are worried about "ContentType" only. We can use ASIS message id and
default partition key. 

Please suggest your opinion depends on your input we will proceed further. 

Regards,
Abhishek Kumar

TestQpid_1_0_JMS_TemplateSend.java

  



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: how to set ServiceBus Message properties like MessageID, ContentType, PartitionKey ?

[akabhishek1]

Please find attached screenShot.


 



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Message going in topic with the use of "session.createQueue(name)"

[akabhishek1]

Hi Tim, 

Just wanted to update you. We have opened this issue with Microsoft as well,
they checked same functionality with .Net Sdk and they are getting below
exception, so i am assuming there is no issue from server/broker side. 

I also tested with Microsoft 'azure-servicebus-1.2.5', they are also not
throwing exception for wrong operation. But 'azure-servicebus-1.2.5' also
using internally 'proton-j'.

Do you think, is this issue related with 'proton-j' ? Please suggest your
opinion.

*Exception from .Net*
System.InvalidOperationException:
'Cannot open a Queue client for entity type Topic. 
TrackingId:#-7571-4b5b-8289-0ea83ede915c_B3, 
SystemTracker::topic:topicdemo~111, 
Timestamp:6/14/2018 9:19:29 AM
TrackingId:-0a5f-4981-a173-fd6fccd0e95b_G0_B3, 
SystemTracker:mrkarsbns:topic:topicdemo~111, 
Timestamp:6/14/2018 

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Message going in topic with the use of "session.createQueue(name)"

[akabhishek1]

Hi Tim,

Thanks a lot for your quick reply and clarification. Log file is correct,
please take a look again on below link and find "test-send". You can see the
details. 

http://qpid.2158936.n2.nabble.com/file/t396358/QPid_Send_Topic_Logs.txt

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Message going in topic with the use of "session.createQueue(name)"

[akabhishek1]

Hi Tim,

Thanks a lot for your quick reply and clarification. Log file is correct.
Please take a look on below link and find again "test-send". 

http://qpid.2158936.n2.nabble.com/file/t396358/QPid_Send_Topic_Logs.txt
  

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: Message going in topic with the use of "session.createQueue(name)"

[akabhishek1]

HI Tim,

Thanks a lot for your reply. Yes correct i am using "qpid-jms-client" to
connect azure service bus. 

Please find attached "QPid_Send_Topic_Logs.txt" log file and test class
"TestQpidSend.java".

*Steps To Reproduce*
1. Create topic "test-topic" in service bus. 
2. Run as java application
3. You can see published message is available in topic "test-topic"
I think this is a issue, because i have created queue destination
"Destination destination = session.createQueue(QUEUE_NAME);"

So message should publish in queue, if that queue is not available then
should through exception. If any user/developer by mistake created topic
instead of queue vice-versa, then message will go in wrong direction. 

Could you please suggest your opinion, how can i solve this issue? 

Alternatively - I tried to find validation of resource type (like Queue or
topic). But i have not found any way to validate resource,  i mean that
resource is topic or queue? Please suggest if you have any clue to validate
resource type? 

Regards,
Abhishek Kumar

QPid_Send_Topic_Logs.txt
  
TestQpidSend.java
  




--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: [VOTE] Release Apache Qpid JMS 0.33.0

[akabhishek1]

Hi Robbie, 

I tested for issue "https://issues.apache.org/jira/browse/QPIDJMS-386";. It
is working fine.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: producer exception handling for 10m idle TimeOut -> "qpid-jms-client - 0.32.0"

[akabhishek1]

Hi Robbie,

Good Morning !! Thank you so much for your both reply. With your replies,
our doubts has been cleared and concluded for reliability. We have chosen
"forceSync + send(without listener) + failover(if required)".  

This will give us 1/3 performance on comparison with Async but beneficial
from reliable point of view.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: producer exception handling for 10m idle TimeOut -> "qpid-jms-client - 0.32.0"

[akabhishek1]

Hi Robbie, 
While I am using - 
*forceSyncSend=true*
 and with a CompletionListener, I would expect the callback event to come on
the same thread that invoked the send(), however in my observation - I am
sending the message using the main thread and I am getting a response on to
the  
JmsSession [ID:12c9fd67-aed5-4930-8da2-31cb3f99729d:1:1] completion
dispatcher
 and that means any exception arising from the send() operation will not be
propagated to the main thread. 

*Is this behavior correct?*

Does that mean that the difference of behavior between Sync and Async is the
callback of onCompletion() or onException() and nothing more...(as both are
working on their own threads) and only the server side code would be able to
understand the notification/exception but will not be able to propagate it
to the client

Attached is the code -  TestQpidSendSync.java
  

/Steps to reproduce/ - 
1. Start the main 
2. Terminate the connection to the service bus while the main thread is
running 
3. it will attempt couple of times before throwing exceptions in the
CompletionListener's onException()



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

Re: producer exception handling for 10m idle TimeOut -> "qpid-jms-client - 0.32.0"

[akabhishek1]

Hi Robbie,

Thanks a lot for your reply and clarification. You are right for threading
issue on "onException()". I am still doing R&D with your suggestion, i will
come back to you shortly with results.

Today morning at 10:55AM(UTC+1), i have taken master code and installed
locally "0.33.0-SNAPSHOT" version. 

While doing research, i found one issue related with message loss and not
exposing error to application. Which i think, you need to take a look before
releasing of "0.33.0" version.

Issue - Message are getting loss while connection drop and errors are not
bubbling up at application level for failover connection.

Steps to reproduce issue(Note - sending message asynchronously without
CompletionListener with failover connection)
1. Change valid QueueName, SBUS_NAME, USERNAME and PASSWORD in below
example.
2. Run java application
3. Every 5s("due to Thread.sleep(5000)") one message will publish at
ServiceBus.
4. Disconnect Lan cable/internet after publishing 1 or 2 message. So there
is no connection between ServiceBus from your desktop/Laptop.
5. Wait for 3-4 minute, you can see that, “for loop” continue sending for
next 4 message.
   After sending 4 message, for loop will be in suspended/wait status
(thread will be in suspended/wait status). So no next processing is going
on.

6. Wait for 5 minute

7. After that connect cable/internet. So After connecting to internet, for
loop will start sending message(thread resumed also)
8. You can wait for 1-2 minute until you see message " All message sent
successfully "
9. At the end of processing, we are expecting 30 message should be available
on Queue. 
   ISSUE - But we can see only 26 messages on queue with no exception at
application level. 
   SO we are having loss of 4 messages. I waited upto 30m but still
not received loss messages at Queue.
   
NOTE: - I have also tested without connection failover. I am getting
exception on "onException()" but having loss of 4 message. I think we can’t
do anything here because we don’t have failover.

Could you please take a look on this issue and let me know for any queries.

 CODE 
package org.test;

import java.util.Hashtable;

import javax.jms.CompletionListener;
import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.Destination;
import javax.jms.ExceptionListener;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageConsumer;
import javax.jms.MessageListener;
import javax.jms.MessageProducer;
import javax.jms.Session;
import javax.jms.TextMessage;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;

import org.mule.transport.servicebus.exception.ServicebusSendException;

public class TestQpidSend implements ExceptionListener{

private static final String QUEUE_NAME = "";
private static final String SBUS_NAME = "";
private static final String USERNAME = "";
private static final String PASSWORD = "";
private static final String QPID_CONNECTION_FACTORY_CLASS =
"org.apache.qpid.jms.jndi.JmsInitialContextFactory";

SendCompletionHandler sendCompletionHandler = new 
SendCompletionHandler();

public static void main(String[] args) throws Exception {
TestQpidSend test = new TestQpidSend();
test.send();

//Loop for not terminating the application
int i = 0;
while(i > 0){
i++;
}
}

/* Enable constructor, if you are running in any logger for server logs 
*/
 
 /*public TestQpidRcvr() throws NamingException, JMSException {
System.err.println("* I am in ");
startListning();
}*/

private void send() throws NamingException, JMSException,
InterruptedException {
Hashtable hashtable = new Hashtable<>();
hashtable.put("connectionfactory.SBCF", "failover:(amqps://"+ 
SBUS_NAME
+".servicebus.windows.net?transport.tcpKeepAlive=true&amqp.traceFrames=true)?failover.reconnectDelay=2000&failover.maxReconnectAttempts=-1&failover.warnAfterReconnectAttempts=10&failover.startupMaxReconnectAttempts=3&jms.prefetchPolicy.all=1000&jms.forceAsyncSend=true");

//hashtable.put("connectionfactory.SBCF", "amqps://"+ SBUS_NAME
+".servicebus.windows.net?transport.tcpKeepAlive=true&amqp.traceFrames=true&jms.prefetchPolicy.all=1000&jms.forceAsyncSend=true");

hashtable.put(Context.INITIAL_CONTEXT_FACTORY,
QPID_CONNECTION_FACTORY_CLASS);

Context context = new InitialContext(hashtable);
ConnectionFactory connectionFactory = (ConnectionFactory)
context.lookup("SBCF");

Connection connection = 
connectionFactory.createConnection(USERNAME,
PASSWO

Re: producer exception handling for 10m idle TimeOut -> "qpid-jms-client - 0.32.0"

[akabhishek1]

HI Robbie And Team,

I need one more important suggestion from you. We are getting connection
exception in "onException(Message message, Exception exception)" block while
sending message to ServiceBus broker.

The problem is -> "onException(Message message, Exception exception)" is
overriden method of CompletionHandler. So we can't throw any checked
exception.

We have requirement to throw exception, so user can aware about payload and
exception details. 

Could you please suggest , how can we throw custom exception from
"onException(Message message, Exception exception)" block?

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: MessageListener stop listening message and producer stop sending message, -> "qpid-jms-client - 0.32.0"

[akabhishek1]

Hi Robbie,

Thanks a lot for information. I will wait for fixes. 

I am having some issue with producer. I will send new email for avoiding the
confusion.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: MessageListener stop listening message and producer stop sending message, -> "qpid-jms-client - 0.32.0"

[akabhishek1]

Hi Robbie,

Awesome News. Thank you so much for quick fix. 

I cloned jms-client code from master github and build locally. I tested
locally and it is working fine. I am getting the error as expected.

Could you please also suggest, when are you planning to release 0.33.0
version? Any lum-sum time/date would be also fine, so i can inform
internally to my team mates.

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: MessageListener stop listening message and producer stop sending message, -> "qpid-jms-client - 0.32.0"

[akabhishek1]

HI Robbie, 

Could you please suggest your opinion, is there any work around for this
issue, so i can implement same until final fix ? 

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org

Re: MessageListener stop listening message and producer stop sending message, -> "qpid-jms-client - 0.32.0"

[akabhishek1]

HI Robbie,

Thanks a lot for confirmation. Is it possible for you to fix this issue in
next release? 

Could you please also suggest timeline about availability of this feature,
so i can plan accordingly?

Regards,
Abhishek Kumar



--
Sent from: http://qpid.2158936.n2.nabble.com/Apache-Qpid-users-f2158936.html

-
To unsubscribe, e-mail: users-unsubscr...@qpid.apache.org
For additional commands, e-mail: users-h...@qpid.apache.org