Re: OM 505 Problem reconnect Stream

2020-05-18 Thread Peter Dähn

Hi Maxim,

seemed to work so far.
/
//openmeetings@openmeetings:~$ grep "Unexpected error while clean-up" 
catalina.out.6//
//ERROR 05-15 17:24:26.348 o.a.o.c.r.KurentoHandler:201 
[ntLoopGroup-2-1] - Unexpected error while clean-up//

//openmeetings@openmeetings:~$ grep "Kurento connected" catalina.out.6//
//INFO  05-15 17:24:53.653 o.a.o.c.r.KurentoHandler:160 
[pool-1-thread-1] - !!! Kurento connected//
//INFO  05-15 17:25:23.754 o.a.o.c.r.KurentoHandler:160 
[rbeatExec-e1-t0] - !!! Kurento connected/


Greetings Peter

Am 16.05.20 um 15:38 schrieb Maxim Solodovnik:

On Sat, 16 May 2020 at 14:48, Peter Dähn  wrote:


Hi Maxim,

I added this parameter to coturn... I hope this helps...

You asked for

*ERROR 05-15 17:24:26.348 o.a.o.c.r.KurentoHandler:201 [ntLoopGroup-2-1] -
Unexpected error while clean-up*


Can I see stack-trace for this exception?


This was after restart kms. As I said: No connection possible after
restart of kms.

*DEBUG 05-15 17:24:04.532 o.a.o.w.c.OmWebSocketPanel:88 [io-5082-exec-10]
- Sending WebSocket PING*
*DEBUG 05-15 17:24:12.317 o.a.o.w.c.OmWebSocketPanel:88 [io-5082-exec-12]
- Sending WebSocket PING*
*INFO  05-15 17:24:26.156 o.k.j.c.JsonRpcClientNettyWebSocket:275
[ntLoopGroup-2-1] -  channel closed*
*INFO  05-15 17:24:26.156 o.k.j.c.JsonRpcClientNettyWebSocket:275
[ntLoopGroup-5-1] -  channel closed*
*DEBUG 05-15 17:24:26.241 o.a.o.c.r.KurentoHandler:150 [ntLoopGroup-2-1] -
JsonRpcWsClient disconnected from ws://127.0.0.1:/kurento because
Channel closed.*
*DEBUG 05-15 17:24:26.241 o.a.o.c.r.KurentoHandler:150 [ntLoopGroup-5-1] -
JsonRpcWsClient disconnected from ws://127.0.0.1:/kurento because
Channel closed.*
*INFO  05-15 17:24:26.275 o.a.o.c.r.KurentoHandler:141 [ntLoopGroup-2-1] -
!!! Kurento disconnected*
*INFO  05-15 17:24:26.276 o.a.o.c.r.KurentoHandler:141 [ntLoopGroup-5-1] -
!!! Kurento disconnected*


For whatever reason the messages are doubled ...



*DEBUG 05-15 17:24:26.276 o.a.o.c.r.KurentoHandler:190 [ntLoopGroup-2-1] -
Client will destroyed ...*
*WARN  05-15 17:24:26.276 o.a.o.c.r.KurentoHandler:117 [Thread-88202] -
Media Server is not accessible*
*DEBUG 05-15 17:24:26.307 o.a.o.d.d.r.PollDao:104 [Thread-88202] -  ::
hasPoll :: 12561*
*DEBUG 05-15 17:24:26.310 o.a.o.d.d.r.PollDao:98 [Thread-88202] -  ::
getArchived :: 12561*
*WARN  05-15 17:24:26.312 o.k.j.c.JsonRpcClientNettyWebSocket:308
[ntLoopGroup-2-1] -  Trying to close a JsonRpcClientNettyWebSocket with
channel == null*
*WARN  05-15 17:24:26.312 o.k.j.c.JsonRpcClientNettyWebSocket:291
[ntLoopGroup-2-1] -  Trying to close a JsonRpcClientNettyWebSocket with
group == null*
*DEBUG 05-15 17:24:26.312 o.a.o.c.r.KurentoHandler:192 [ntLoopGroup-2-1] -
 Client is destroyed*
*WARN  05-15 17:24:26.312 o.a.o.c.r.KurentoHandler:117 [Thread-88202] -
Media Server is not accessible*
*ERROR 05-15 17:24:26.348 o.a.o.c.r.KurentoHandler:201 [ntLoopGroup-2-1] -
Unexpected error while clean-up*
*WARN  05-15 17:24:26.349 o.a.o.c.r.KurentoHandler:117 [Thread-88202] -
Media Server is not accessible*
*WARN  05-15 17:24:26.355 o.a.o.c.r.KurentoHandler:117 [Thread-88203] -
Media Server is not accessible*
*DEBUG 05-15 17:24:26.355 o.a.o.d.d.r.PollDao:104 [Thread-88203] -  ::
hasPoll :: 12561*
*WARN  05-15 17:24:26.355 o.a.o.c.r.KurentoHandler:117 [Thread-88202] -
Media Server is not accessible*


Is there "!!! Kurento connected" later in the logs?




Greetings Peter

Am 16.05.20 um 05:35 schrieb Maxim Solodovnik:

Hello Peter,

On Fri, 15 May 2020 at 23:21, Peter Dähn   wrote:


Hi Maxim,

1st: I meant is there a way to prevent those reconnection?


Yesterday there was a thread in Russian 
mailing-listhttps://markmail.org/message/camwly4nwmyot67o
Konstantin point out "stale-nonce=0" TURN parameter might be required
(otherwise default timeout of 600 seconds will happen on TURN server)
I have added this parameter to my coturn config

this might help
hard to say if this is the only reason :(((

re-connections were added due to multiple user requests
I believe auto-recover after error is something good :)))



2nd: yes restart was successful and it didn't help...

Today it happens again.. some of my "findings"



Below errors are expected during multiple re-connections
I will add some guards so it will not pollute the logs



*sudo grep ERROR /OM_HOME/logs/catalina.out*
*ERROR 05-15 17:14:42.505 o.a.c.i.AbstractFaultChainInitiatorObserver:114
[nio-5082-exec-6] - An unexpected error occurred during error handling. No
further error processing will occur.*
*ERROR 05-15 17:21:38.308 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
- Error while processing incoming message*
*ERROR 05-15 17:21:38.352 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
- Error while processing incoming message*
*ERROR 05-15 17:21:38.356 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
- Error while processing incoming message*
*ERROR 05-15 17:21:38.398 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
- Error while processing incoming messag

Re: OM 505 Problem reconnect Stream

2020-05-16 Thread Maxim Solodovnik
On Sat, 16 May 2020 at 14:48, Peter Dähn  wrote:

> Hi Maxim,
>
> I added this parameter to coturn... I hope this helps...
>
> You asked for
>
> *ERROR 05-15 17:24:26.348 o.a.o.c.r.KurentoHandler:201 [ntLoopGroup-2-1] -
> Unexpected error while clean-up*
>
>
> Can I see stack-trace for this exception?
>
>
> This was after restart kms. As I said: No connection possible after
> restart of kms.
>
> *DEBUG 05-15 17:24:04.532 o.a.o.w.c.OmWebSocketPanel:88 [io-5082-exec-10]
> - Sending WebSocket PING*
> *DEBUG 05-15 17:24:12.317 o.a.o.w.c.OmWebSocketPanel:88 [io-5082-exec-12]
> - Sending WebSocket PING*
> *INFO  05-15 17:24:26.156 o.k.j.c.JsonRpcClientNettyWebSocket:275
> [ntLoopGroup-2-1] -  channel closed*
> *INFO  05-15 17:24:26.156 o.k.j.c.JsonRpcClientNettyWebSocket:275
> [ntLoopGroup-5-1] -  channel closed*
> *DEBUG 05-15 17:24:26.241 o.a.o.c.r.KurentoHandler:150 [ntLoopGroup-2-1] -
> JsonRpcWsClient disconnected from ws://127.0.0.1:/kurento because
> Channel closed.*
> *DEBUG 05-15 17:24:26.241 o.a.o.c.r.KurentoHandler:150 [ntLoopGroup-5-1] -
> JsonRpcWsClient disconnected from ws://127.0.0.1:/kurento because
> Channel closed.*
> *INFO  05-15 17:24:26.275 o.a.o.c.r.KurentoHandler:141 [ntLoopGroup-2-1] -
> !!! Kurento disconnected*
> *INFO  05-15 17:24:26.276 o.a.o.c.r.KurentoHandler:141 [ntLoopGroup-5-1] -
> !!! Kurento disconnected*
>

For whatever reason the messages are doubled ...


>
> *DEBUG 05-15 17:24:26.276 o.a.o.c.r.KurentoHandler:190 [ntLoopGroup-2-1] -
> Client will destroyed ...*
> *WARN  05-15 17:24:26.276 o.a.o.c.r.KurentoHandler:117 [Thread-88202] -
> Media Server is not accessible*
> *DEBUG 05-15 17:24:26.307 o.a.o.d.d.r.PollDao:104 [Thread-88202] -  ::
> hasPoll :: 12561*
> *DEBUG 05-15 17:24:26.310 o.a.o.d.d.r.PollDao:98 [Thread-88202] -  ::
> getArchived :: 12561*
> *WARN  05-15 17:24:26.312 o.k.j.c.JsonRpcClientNettyWebSocket:308
> [ntLoopGroup-2-1] -  Trying to close a JsonRpcClientNettyWebSocket with
> channel == null*
> *WARN  05-15 17:24:26.312 o.k.j.c.JsonRpcClientNettyWebSocket:291
> [ntLoopGroup-2-1] -  Trying to close a JsonRpcClientNettyWebSocket with
> group == null*
> *DEBUG 05-15 17:24:26.312 o.a.o.c.r.KurentoHandler:192 [ntLoopGroup-2-1] -
>  Client is destroyed*
> *WARN  05-15 17:24:26.312 o.a.o.c.r.KurentoHandler:117 [Thread-88202] -
> Media Server is not accessible*
> *ERROR 05-15 17:24:26.348 o.a.o.c.r.KurentoHandler:201 [ntLoopGroup-2-1] -
> Unexpected error while clean-up*
> *WARN  05-15 17:24:26.349 o.a.o.c.r.KurentoHandler:117 [Thread-88202] -
> Media Server is not accessible*
> *WARN  05-15 17:24:26.355 o.a.o.c.r.KurentoHandler:117 [Thread-88203] -
> Media Server is not accessible*
> *DEBUG 05-15 17:24:26.355 o.a.o.d.d.r.PollDao:104 [Thread-88203] -  ::
> hasPoll :: 12561*
> *WARN  05-15 17:24:26.355 o.a.o.c.r.KurentoHandler:117 [Thread-88202] -
> Media Server is not accessible*
>

Is there "!!! Kurento connected" later in the logs?


>
>
> Greetings Peter
>
> Am 16.05.20 um 05:35 schrieb Maxim Solodovnik:
>
> Hello Peter,
>
> On Fri, 15 May 2020 at 23:21, Peter Dähn   
> wrote:
>
>
> Hi Maxim,
>
> 1st: I meant is there a way to prevent those reconnection?
>
>
> Yesterday there was a thread in Russian 
> mailing-listhttps://markmail.org/message/camwly4nwmyot67o
> Konstantin point out "stale-nonce=0" TURN parameter might be required
> (otherwise default timeout of 600 seconds will happen on TURN server)
> I have added this parameter to my coturn config
>
> this might help
> hard to say if this is the only reason :(((
>
> re-connections were added due to multiple user requests
> I believe auto-recover after error is something good :)))
>
>
>
> 2nd: yes restart was successful and it didn't help...
>
> Today it happens again.. some of my "findings"
>
>
>
> Below errors are expected during multiple re-connections
> I will add some guards so it will not pollute the logs
>
>
>
> *sudo grep ERROR /OM_HOME/logs/catalina.out*
> *ERROR 05-15 17:14:42.505 o.a.c.i.AbstractFaultChainInitiatorObserver:114
> [nio-5082-exec-6] - An unexpected error occurred during error handling. No
> further error processing will occur.*
> *ERROR 05-15 17:21:38.308 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
> - Error while processing incoming message*
> *ERROR 05-15 17:21:38.352 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
> - Error while processing incoming message*
> *ERROR 05-15 17:21:38.356 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
> - Error while processing incoming message*
> *ERROR 05-15 17:21:38.398 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
> - Error while processing incoming message*
> *ERROR 05-15 17:21:38.401 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
> - Error while processing incoming message*
> *ERROR 05-15 17:21:38.405 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
> - Error while processing incoming message*
> *ERROR 05-15 17:21:38.408 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
> - Error while processing incoming message*
> *E

Re: OM 505 Problem reconnect Stream

2020-05-16 Thread Peter Dähn

Hi Maxim,

I added this parameter to coturn... I hope this helps...

You asked for

*ERROR 05-15 17:24:26.348 o.a.o.c.r.KurentoHandler:201 [ntLoopGroup-2-1] -
Unexpected error while clean-up*

Can I see stack-trace for this exception?


This was after restart kms. As I said: No connection possible after 
restart of kms.

/
//DEBUG 05-15 17:24:04.532 o.a.o.w.c.OmWebSocketPanel:88 
[io-5082-exec-10] - Sending WebSocket PING//
//DEBUG 05-15 17:24:12.317 o.a.o.w.c.OmWebSocketPanel:88 
[io-5082-exec-12] - Sending WebSocket PING//
//INFO  05-15 17:24:26.156 o.k.j.c.JsonRpcClientNettyWebSocket:275 
[ntLoopGroup-2-1] - channel closed//
//INFO  05-15 17:24:26.156 o.k.j.c.JsonRpcClientNettyWebSocket:275 
[ntLoopGroup-5-1] - channel closed//
//DEBUG 05-15 17:24:26.241 o.a.o.c.r.KurentoHandler:150 
[ntLoopGroup-2-1] - JsonRpcWsClient disconnected from 
ws://127.0.0.1:/kurento because Channel closed.//
//DEBUG 05-15 17:24:26.241 o.a.o.c.r.KurentoHandler:150 
[ntLoopGroup-5-1] - JsonRpcWsClient disconnected from 
ws://127.0.0.1:/kurento because Channel closed.//
//INFO  05-15 17:24:26.275 o.a.o.c.r.KurentoHandler:141 
[ntLoopGroup-2-1] - !!! Kurento disconnected//
//INFO  05-15 17:24:26.276 o.a.o.c.r.KurentoHandler:141 
[ntLoopGroup-5-1] - !!! Kurento disconnected//
//DEBUG 05-15 17:24:26.276 o.a.o.c.r.KurentoHandler:190 
[ntLoopGroup-2-1] - Client will destroyed ...//
//WARN  05-15 17:24:26.276 o.a.o.c.r.KurentoHandler:117 [Thread-88202] - 
Media Server is not accessible//
//DEBUG 05-15 17:24:26.307 o.a.o.d.d.r.PollDao:104 [Thread-88202] -  :: 
hasPoll :: 12561//
//DEBUG 05-15 17:24:26.310 o.a.o.d.d.r.PollDao:98 [Thread-88202] -  :: 
getArchived :: 12561//
//WARN  05-15 17:24:26.312 o.k.j.c.JsonRpcClientNettyWebSocket:308 
[ntLoopGroup-2-1] - Trying to close a JsonRpcClientNettyWebSocket with 
channel == null//
//WARN  05-15 17:24:26.312 o.k.j.c.JsonRpcClientNettyWebSocket:291 
[ntLoopGroup-2-1] - Trying to close a JsonRpcClientNettyWebSocket with 
group == null//
//DEBUG 05-15 17:24:26.312 o.a.o.c.r.KurentoHandler:192 
[ntLoopGroup-2-1] -  Client is destroyed//
//WARN  05-15 17:24:26.312 o.a.o.c.r.KurentoHandler:117 [Thread-88202] - 
Media Server is not accessible//
//ERROR 05-15 17:24:26.348 o.a.o.c.r.KurentoHandler:201 
[ntLoopGroup-2-1] - Unexpected error while clean-up//
//WARN  05-15 17:24:26.349 o.a.o.c.r.KurentoHandler:117 [Thread-88202] - 
Media Server is not accessible//
//WARN  05-15 17:24:26.355 o.a.o.c.r.KurentoHandler:117 [Thread-88203] - 
Media Server is not accessible//
//DEBUG 05-15 17:24:26.355 o.a.o.d.d.r.PollDao:104 [Thread-88203] -  :: 
hasPoll :: 12561//
//WARN  05-15 17:24:26.355 o.a.o.c.r.KurentoHandler:117 [Thread-88202] - 
Media Server is not accessible/


Greetings Peter

Am 16.05.20 um 05:35 schrieb Maxim Solodovnik:

Hello Peter,

On Fri, 15 May 2020 at 23:21, Peter Dähn  wrote:


Hi Maxim,

1st: I meant is there a way to prevent those reconnection?


Yesterday there was a thread in Russian mailing-list
https://markmail.org/message/camwly4nwmyot67o
Konstantin point out "stale-nonce=0" TURN parameter might be required
(otherwise default timeout of 600 seconds will happen on TURN server)
I have added this parameter to my coturn config

this might help
hard to say if this is the only reason :(((

re-connections were added due to multiple user requests
I believe auto-recover after error is something good :)))



2nd: yes restart was successful and it didn't help...

Today it happens again.. some of my "findings"



Below errors are expected during multiple re-connections
I will add some guards so it will not pollute the logs



*sudo grep ERROR /OM_HOME/logs/catalina.out*
*ERROR 05-15 17:14:42.505 o.a.c.i.AbstractFaultChainInitiatorObserver:114
[nio-5082-exec-6] - An unexpected error occurred during error handling. No
further error processing will occur.*
*ERROR 05-15 17:21:38.308 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
- Error while processing incoming message*
*ERROR 05-15 17:21:38.352 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
- Error while processing incoming message*
*ERROR 05-15 17:21:38.356 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
- Error while processing incoming message*
*ERROR 05-15 17:21:38.398 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
- Error while processing incoming message*
*ERROR 05-15 17:21:38.401 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
- Error while processing incoming message*
*ERROR 05-15 17:21:38.405 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
- Error while processing incoming message*
*ERROR 05-15 17:21:38.408 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
- Error while processing incoming message*
*ERROR 05-15 17:21:38.448 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
- Error while processing incoming message*
*ERROR 05-15 17:21:38.451 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
- Error while processing incoming message*
*ERROR 05-15 17:21:38.454 o.a.o.w.c.OmWebSocketPanel:144 [nio-5082-exec-3]
- Error

Re: OM 505 Problem reconnect Stream

2020-05-14 Thread Maxim Solodovnik
On Thu, 14 May 2020 at 19:16, Peter Dähn  wrote:

> Hi Maxim,
>
> can I overcome 1 in some way?
>

Not sure I get this question :(
If it is about "bullet-proof way of restarting KMS"
I believe you can stop it using
1) kill -9 `pid from file`
2) kill -9 `pid from output of ps`
then restart


>
> And 2nd Restart worked (I checked out logfiles) and I had 52GB RAM free
> and CPU usage was about 3%. Over all server has 60GB RAM, 16 physical
> cores and runs om, coturn and kms. Time of logfiles there were 15 user
> online...
>

So restart was successful and it doesn't help?


>
> Greetings Peter
>
> Am 13.05.20 um 17:03 schrieb Maxim Solodovnik:
> > Hello Peter,
> >
> > On Wed, 13 May 2020 at 20:30, Peter Dähn  wrote:
> >
> >> Hi,
> >>
> >> I still have a problem to get sable connection for participants.
> >>
> >> For some of them it never works and for some it work a while... for some
> >> seems to help tu turn on own cam/mic to hear others!?!?... I am really
> >> helpless... :-(
> >>
> >> catalalina out shows this kind of messages if connection fails.
> >>
> >> *DEBUG 05-13 14:31:30.961 o.a.o.c.r.StreamProcessor:91
> [io-5082-exec-19] -
> >> Incoming message from user with ID '14421':
> >>
> {"type":"kurento","id":"broadcastStarted","uid":"d6805f4c-8fc6-4a7a-9cf5-dd2bd85d1f65","sdpOffer":"v=0\r\no=mozilla...THIS_IS_SDPARTA-71.0
> >> 7880734367584313215 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0
> >> 0\r\na=sendrecv\r\na=fingerprint:sha-256
> >>
> 59:FD:77:17:E5:21:C7:B5:09:5D:D1:AB:58:4F:26:CD:9A:04:84:24:03:A8:26:E4:C1:43:3E:5C:7B:72:13:5E\r\na=group:BUNDLE
> >> 0\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=audio 9
> >> UDP/TLS/RTP/SAVPF 109 9 0 8 101\r\nc=IN IP4
> >> 0.0.0.0\r\na=sendrecv\r\na=extmap:1
> >> urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:2/recvonly
> >> urn:ietf:params:rtp-hdrext:csrc-audio-level\r\na=extmap:3
> >> urn:ietf:params:rtp-hdrext:sdes:mid\r\na=fmtp:109
> >> maxplaybackrate=48000;stereo=1;useinbandfec=1\r\na=fmtp:101
> >>
> 0-15\r\na=ice-pwd:5447d03e80580f7c2893f37f9605b0c4\r\na=ice-ufrag:90a19ebb\r\na=mid:0\r\na=msid:{5ffed8b7-c193-42ad-823d-1783352f933d}
> >> {8a5bd209-28fa-43a4-aec7-3823895d2f5a}\r\na=rtcp-mux\r\na=rtpmap:109
> >> opus/48000/2\r\na=rtpmap:9 G722/8000/1\r\na=rtpmap:0
> >> PCMU/8000\r\na=rtpmap:8 PCMA/8000\r\na=rtpmap:101
> >> telephone-event/8000\r\na=setup:actpass\r\na=ssrc:3478667262
> >> cname:{d02f827b-578a-4abe-92d4-25274bde1262}\r\n"}*
> >> *ERROR 05-13 14:31:31.020 o.a.o.w.c.OmWebSocketPanel:144
> [io-5082-exec-19]
> >> - Error while processing incoming message*
> >> *java.lang.NullPointerException: null*
> >>
> > This Exception is most probably is caused by client re-connection
> > i.e. client have started to re-connect and did it several times
> >
> >
> >> *at
> >>
> org.apache.openmeetings.core.remote.KStream.startBroadcast(KStream.java:88)*
> >> *at
> >>
> org.apache.openmeetings.core.remote.StreamProcessor.startBroadcast(StreamProcessor.java:212)*
> >> *at
> >>
> org.apache.openmeetings.core.remote.StreamProcessor.handleBroadcastStarted(StreamProcessor.java:187)*
> >> *at
> >>
> org.apache.openmeetings.core.remote.StreamProcessor.onMessage(StreamProcessor.java:110)*
> >> *at
> >>
> org.apache.openmeetings.core.remote.KurentoHandler.onMessage(KurentoHandler.java:233)*
> >> *at
> >>
> org.apache.openmeetings.core.remote.Wicket_Proxy_KurentoHandler$$FastClassByCGLIB$$756e116d.invoke()*
> >> *at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)*
> >> *at
> >>
> org.apache.wicket.proxy.LazyInitProxyFactory$AbstractCGLibInterceptor.intercept(LazyInitProxyFactory.java:364)*
> >> *at
> >>
> org.apache.openmeetings.core.remote.Wicket_Proxy_KurentoHandler$$EnhancerByCGLIB$$98325b15.onMessage()*
> >> *at
> >>
> org.apache.openmeetings.web.common.OmWebSocketPanel$2.onMessage(OmWebSocketPanel.java:125)*
> >> *at
> >> org.apache.wicket.protocol.ws
> .api.WebSocketBehavior.onEvent(WebSocketBehavior.java:64)*
> >> *at
> >>
> org.apache.wicket.settings.FrameworkSettings.dispatchEvent(FrameworkSettings.java:147)*
> >> *at
> >>
> org.apache.wicket.ComponentEventSender.dispatchToComponent(ComponentEventSender.java:292)*
> >> *at
> >>
> org.apache.wicket.ComponentEventSender$ComponentEventVisitor.component(ComponentEventSender.java:329)*
> >> *at
> >>
> org.apache.wicket.ComponentEventSender$ComponentEventVisitor.component(ComponentEventSender.java:306)*
> >> *at
> org.apache.wicket.util.visit.Visits.visitChildren(Visits.java:144)*
> >> *at
> org.apache.wicket.util.visit.Visits.visitChildren(Visits.java:162)*
> >> *at
> org.apache.wicket.util.visit.Visits.visitChildren(Visits.java:123)*
> >> *at
> org.apache.wicket.util.visit.Visits.visitChildren(Visits.java:192)*
> >> *at
> >>
> org.apache.wicket.MarkupContainer.visitChildren(MarkupContainer.java:988)*
> >> *at
> >>
> org.apache.wicket.ComponentEventSender.breadth(ComponentEventSender.java:160)*
> >> *at
> >>
> org.a

Re: OM 505 Problem reconnect Stream

2020-05-14 Thread Peter Dähn

Hi Maxim,

can I overcome 1 in some way?

And 2nd Restart worked (I checked out logfiles) and I had 52GB RAM free 
and CPU usage was about 3%. Over all server has 60GB RAM, 16 physical 
cores and runs om, coturn and kms. Time of logfiles there were 15 user 
online...


Greetings Peter

Am 13.05.20 um 17:03 schrieb Maxim Solodovnik:

Hello Peter,

On Wed, 13 May 2020 at 20:30, Peter Dähn  wrote:


Hi,

I still have a problem to get sable connection for participants.

For some of them it never works and for some it work a while... for some
seems to help tu turn on own cam/mic to hear others!?!?... I am really
helpless... :-(

catalalina out shows this kind of messages if connection fails.

*DEBUG 05-13 14:31:30.961 o.a.o.c.r.StreamProcessor:91 [io-5082-exec-19] -
Incoming message from user with ID '14421':
{"type":"kurento","id":"broadcastStarted","uid":"d6805f4c-8fc6-4a7a-9cf5-dd2bd85d1f65","sdpOffer":"v=0\r\no=mozilla...THIS_IS_SDPARTA-71.0
7880734367584313215 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0
0\r\na=sendrecv\r\na=fingerprint:sha-256
59:FD:77:17:E5:21:C7:B5:09:5D:D1:AB:58:4F:26:CD:9A:04:84:24:03:A8:26:E4:C1:43:3E:5C:7B:72:13:5E\r\na=group:BUNDLE
0\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=audio 9
UDP/TLS/RTP/SAVPF 109 9 0 8 101\r\nc=IN IP4
0.0.0.0\r\na=sendrecv\r\na=extmap:1
urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:2/recvonly
urn:ietf:params:rtp-hdrext:csrc-audio-level\r\na=extmap:3
urn:ietf:params:rtp-hdrext:sdes:mid\r\na=fmtp:109
maxplaybackrate=48000;stereo=1;useinbandfec=1\r\na=fmtp:101
0-15\r\na=ice-pwd:5447d03e80580f7c2893f37f9605b0c4\r\na=ice-ufrag:90a19ebb\r\na=mid:0\r\na=msid:{5ffed8b7-c193-42ad-823d-1783352f933d}
{8a5bd209-28fa-43a4-aec7-3823895d2f5a}\r\na=rtcp-mux\r\na=rtpmap:109
opus/48000/2\r\na=rtpmap:9 G722/8000/1\r\na=rtpmap:0
PCMU/8000\r\na=rtpmap:8 PCMA/8000\r\na=rtpmap:101
telephone-event/8000\r\na=setup:actpass\r\na=ssrc:3478667262
cname:{d02f827b-578a-4abe-92d4-25274bde1262}\r\n"}*
*ERROR 05-13 14:31:31.020 o.a.o.w.c.OmWebSocketPanel:144 [io-5082-exec-19]
- Error while processing incoming message*
*java.lang.NullPointerException: null*


This Exception is most probably is caused by client re-connection
i.e. client have started to re-connect and did it several times



*at
org.apache.openmeetings.core.remote.KStream.startBroadcast(KStream.java:88)*
*at
org.apache.openmeetings.core.remote.StreamProcessor.startBroadcast(StreamProcessor.java:212)*
*at
org.apache.openmeetings.core.remote.StreamProcessor.handleBroadcastStarted(StreamProcessor.java:187)*
*at
org.apache.openmeetings.core.remote.StreamProcessor.onMessage(StreamProcessor.java:110)*
*at
org.apache.openmeetings.core.remote.KurentoHandler.onMessage(KurentoHandler.java:233)*
*at
org.apache.openmeetings.core.remote.Wicket_Proxy_KurentoHandler$$FastClassByCGLIB$$756e116d.invoke()*
*at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)*
*at
org.apache.wicket.proxy.LazyInitProxyFactory$AbstractCGLibInterceptor.intercept(LazyInitProxyFactory.java:364)*
*at
org.apache.openmeetings.core.remote.Wicket_Proxy_KurentoHandler$$EnhancerByCGLIB$$98325b15.onMessage()*
*at
org.apache.openmeetings.web.common.OmWebSocketPanel$2.onMessage(OmWebSocketPanel.java:125)*
*at
org.apache.wicket.protocol.ws.api.WebSocketBehavior.onEvent(WebSocketBehavior.java:64)*
*at
org.apache.wicket.settings.FrameworkSettings.dispatchEvent(FrameworkSettings.java:147)*
*at
org.apache.wicket.ComponentEventSender.dispatchToComponent(ComponentEventSender.java:292)*
*at
org.apache.wicket.ComponentEventSender$ComponentEventVisitor.component(ComponentEventSender.java:329)*
*at
org.apache.wicket.ComponentEventSender$ComponentEventVisitor.component(ComponentEventSender.java:306)*
*at org.apache.wicket.util.visit.Visits.visitChildren(Visits.java:144)*
*at org.apache.wicket.util.visit.Visits.visitChildren(Visits.java:162)*
*at org.apache.wicket.util.visit.Visits.visitChildren(Visits.java:123)*
*at org.apache.wicket.util.visit.Visits.visitChildren(Visits.java:192)*
*at
org.apache.wicket.MarkupContainer.visitChildren(MarkupContainer.java:988)*
*at
org.apache.wicket.ComponentEventSender.breadth(ComponentEventSender.java:160)*
*at
org.apache.wicket.ComponentEventSender.send(ComponentEventSender.java:68)*
*at org.apache.wicket.Component.send(Component.java:4418)*
*at
org.apache.wicket.protocol.ws.api.WebSocketMessageBroadcastHandler$1.run(WebSocketMessageBroadcastHandler.java:74)*
*at
org.apache.wicket.protocol.ws.WebSocketSettings$SameThreadExecutor.run(WebSocketSettings.java:393)*
*at
org.apache.wicket.protocol.ws.api.WebSocketMessageBroadcastHandler.respond(WebSocketMessageBroadcastHandler.java:97)*
*at
org.apache.wicket.request.cycle.RequestCycle$HandlerExecutor.respond(RequestCycle.java:917)*
*at
org.apache.wicket.request.RequestHandlerExecutor.execute(RequestHandlerExecutor.java:63)*
*at
org.apache.wicket.request.cycle.RequestCycle.ex

Re: OM 505 Problem reconnect Stream

2020-05-13 Thread Maxim Solodovnik
Hello Peter,

On Wed, 13 May 2020 at 20:30, Peter Dähn  wrote:

> Hi,
>
> I still have a problem to get sable connection for participants.
>
> For some of them it never works and for some it work a while... for some
> seems to help tu turn on own cam/mic to hear others!?!?... I am really
> helpless... :-(
>
> catalalina out shows this kind of messages if connection fails.
>
> *DEBUG 05-13 14:31:30.961 o.a.o.c.r.StreamProcessor:91 [io-5082-exec-19] -
> Incoming message from user with ID '14421':
> {"type":"kurento","id":"broadcastStarted","uid":"d6805f4c-8fc6-4a7a-9cf5-dd2bd85d1f65","sdpOffer":"v=0\r\no=mozilla...THIS_IS_SDPARTA-71.0
> 7880734367584313215 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0
> 0\r\na=sendrecv\r\na=fingerprint:sha-256
> 59:FD:77:17:E5:21:C7:B5:09:5D:D1:AB:58:4F:26:CD:9A:04:84:24:03:A8:26:E4:C1:43:3E:5C:7B:72:13:5E\r\na=group:BUNDLE
> 0\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=audio 9
> UDP/TLS/RTP/SAVPF 109 9 0 8 101\r\nc=IN IP4
> 0.0.0.0\r\na=sendrecv\r\na=extmap:1
> urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:2/recvonly
> urn:ietf:params:rtp-hdrext:csrc-audio-level\r\na=extmap:3
> urn:ietf:params:rtp-hdrext:sdes:mid\r\na=fmtp:109
> maxplaybackrate=48000;stereo=1;useinbandfec=1\r\na=fmtp:101
> 0-15\r\na=ice-pwd:5447d03e80580f7c2893f37f9605b0c4\r\na=ice-ufrag:90a19ebb\r\na=mid:0\r\na=msid:{5ffed8b7-c193-42ad-823d-1783352f933d}
> {8a5bd209-28fa-43a4-aec7-3823895d2f5a}\r\na=rtcp-mux\r\na=rtpmap:109
> opus/48000/2\r\na=rtpmap:9 G722/8000/1\r\na=rtpmap:0
> PCMU/8000\r\na=rtpmap:8 PCMA/8000\r\na=rtpmap:101
> telephone-event/8000\r\na=setup:actpass\r\na=ssrc:3478667262
> cname:{d02f827b-578a-4abe-92d4-25274bde1262}\r\n"}*
> *ERROR 05-13 14:31:31.020 o.a.o.w.c.OmWebSocketPanel:144 [io-5082-exec-19]
> - Error while processing incoming message*
> *java.lang.NullPointerException: null*
>

This Exception is most probably is caused by client re-connection
i.e. client have started to re-connect and did it several times


>
> *at
> org.apache.openmeetings.core.remote.KStream.startBroadcast(KStream.java:88)*
> *at
> org.apache.openmeetings.core.remote.StreamProcessor.startBroadcast(StreamProcessor.java:212)*
> *at
> org.apache.openmeetings.core.remote.StreamProcessor.handleBroadcastStarted(StreamProcessor.java:187)*
> *at
> org.apache.openmeetings.core.remote.StreamProcessor.onMessage(StreamProcessor.java:110)*
> *at
> org.apache.openmeetings.core.remote.KurentoHandler.onMessage(KurentoHandler.java:233)*
> *at
> org.apache.openmeetings.core.remote.Wicket_Proxy_KurentoHandler$$FastClassByCGLIB$$756e116d.invoke()*
> *at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)*
> *at
> org.apache.wicket.proxy.LazyInitProxyFactory$AbstractCGLibInterceptor.intercept(LazyInitProxyFactory.java:364)*
> *at
> org.apache.openmeetings.core.remote.Wicket_Proxy_KurentoHandler$$EnhancerByCGLIB$$98325b15.onMessage()*
> *at
> org.apache.openmeetings.web.common.OmWebSocketPanel$2.onMessage(OmWebSocketPanel.java:125)*
> *at
> org.apache.wicket.protocol.ws.api.WebSocketBehavior.onEvent(WebSocketBehavior.java:64)*
> *at
> org.apache.wicket.settings.FrameworkSettings.dispatchEvent(FrameworkSettings.java:147)*
> *at
> org.apache.wicket.ComponentEventSender.dispatchToComponent(ComponentEventSender.java:292)*
> *at
> org.apache.wicket.ComponentEventSender$ComponentEventVisitor.component(ComponentEventSender.java:329)*
> *at
> org.apache.wicket.ComponentEventSender$ComponentEventVisitor.component(ComponentEventSender.java:306)*
> *at org.apache.wicket.util.visit.Visits.visitChildren(Visits.java:144)*
> *at org.apache.wicket.util.visit.Visits.visitChildren(Visits.java:162)*
> *at org.apache.wicket.util.visit.Visits.visitChildren(Visits.java:123)*
> *at org.apache.wicket.util.visit.Visits.visitChildren(Visits.java:192)*
> *at
> org.apache.wicket.MarkupContainer.visitChildren(MarkupContainer.java:988)*
> *at
> org.apache.wicket.ComponentEventSender.breadth(ComponentEventSender.java:160)*
> *at
> org.apache.wicket.ComponentEventSender.send(ComponentEventSender.java:68)*
> *at org.apache.wicket.Component.send(Component.java:4418)*
> *at
> org.apache.wicket.protocol.ws.api.WebSocketMessageBroadcastHandler$1.run(WebSocketMessageBroadcastHandler.java:74)*
> *at
> org.apache.wicket.protocol.ws.WebSocketSettings$SameThreadExecutor.run(WebSocketSettings.java:393)*
> *at
> org.apache.wicket.protocol.ws.api.WebSocketMessageBroadcastHandler.respond(WebSocketMessageBroadcastHandler.java:97)*
> *at
> org.apache.wicket.request.cycle.RequestCycle$HandlerExecutor.respond(RequestCycle.java:917)*
> *at
> org.apache.wicket.request.RequestHandlerExecutor.execute(RequestHandlerExecutor.java:63)*
> *at
> org.apache.wicket.request.cycle.RequestCycle.execute(RequestCycle.java:283)*
> *at
> org.apache.wicket.request.cycle.RequestCycle.processRequest(RequestCycle.java:254)*
> *at
> org.apache.wicket.request.cycle.R

OM 505 Problem reconnect Stream

2020-05-13 Thread Peter Dähn

Hi,

I still have a problem to get sable connection for participants.

For some of them it never works and for some it work a while... for some 
seems to help tu turn on own cam/mic to hear others!?!?... I am really 
helpless... :-(


catalalina out shows this kind of messages if connection fails.
/
//DEBUG 05-13 14:31:30.961 o.a.o.c.r.StreamProcessor:91 
[io-5082-exec-19] - Incoming message from user with ID '14421': 
{"type":"kurento","id":"broadcastStarted","uid":"d6805f4c-8fc6-4a7a-9cf5-dd2bd85d1f65","sdpOffer":"v=0\r\no=mozilla...THIS_IS_SDPARTA-71.0 
7880734367584313215 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0 
0\r\na=sendrecv\r\na=fingerprint:sha-256 
59:FD:77:17:E5:21:C7:B5:09:5D:D1:AB:58:4F:26:CD:9A:04:84:24:03:A8:26:E4:C1:43:3E:5C:7B:72:13:5E\r\na=group:BUNDLE 
0\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=audio 9 
UDP/TLS/RTP/SAVPF 109 9 0 8 101\r\nc=IN IP4 
0.0.0.0\r\na=sendrecv\r\na=extmap:1 
urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:2/recvonly 
urn:ietf:params:rtp-hdrext:csrc-audio-level\r\na=extmap:3 
urn:ietf:params:rtp-hdrext:sdes:mid\r\na=fmtp:109 
maxplaybackrate=48000;stereo=1;useinbandfec=1\r\na=fmtp:101 
0-15\r\na=ice-pwd:5447d03e80580f7c2893f37f9605b0c4\r\na=ice-ufrag:90a19ebb\r\na=mid:0\r\na=msid:{5ffed8b7-c193-42ad-823d-1783352f933d} 
{8a5bd209-28fa-43a4-aec7-3823895d2f5a}\r\na=rtcp-mux\r\na=rtpmap:109 
opus/48000/2\r\na=rtpmap:9 G722/8000/1\r\na=rtpmap:0 
PCMU/8000\r\na=rtpmap:8 PCMA/8000\r\na=rtpmap:101 
telephone-event/8000\r\na=setup:actpass\r\na=ssrc:3478667262 
cname:{d02f827b-578a-4abe-92d4-25274bde1262}\r\n"}//
//ERROR 05-13 14:31:31.020 o.a.o.w.c.OmWebSocketPanel:144 
[io-5082-exec-19] - Error while processing incoming message//

//java.lang.NullPointerException: null//
//    at 
org.apache.openmeetings.core.remote.KStream.startBroadcast(KStream.java:88)//
//    at 
org.apache.openmeetings.core.remote.StreamProcessor.startBroadcast(StreamProcessor.java:212)//
//    at 
org.apache.openmeetings.core.remote.StreamProcessor.handleBroadcastStarted(StreamProcessor.java:187)//
//    at 
org.apache.openmeetings.core.remote.StreamProcessor.onMessage(StreamProcessor.java:110)//
//    at 
org.apache.openmeetings.core.remote.KurentoHandler.onMessage(KurentoHandler.java:233)//
//    at 
org.apache.openmeetings.core.remote.Wicket_Proxy_KurentoHandler$$FastClassByCGLIB$$756e116d.invoke()//

//    at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)//
//    at 
org.apache.wicket.proxy.LazyInitProxyFactory$AbstractCGLibInterceptor.intercept(LazyInitProxyFactory.java:364)//
//    at 
org.apache.openmeetings.core.remote.Wicket_Proxy_KurentoHandler$$EnhancerByCGLIB$$98325b15.onMessage()//
//    at 
org.apache.openmeetings.web.common.OmWebSocketPanel$2.onMessage(OmWebSocketPanel.java:125)//
//    at 
org.apache.wicket.protocol.ws.api.WebSocketBehavior.onEvent(WebSocketBehavior.java:64)//
//    at 
org.apache.wicket.settings.FrameworkSettings.dispatchEvent(FrameworkSettings.java:147)//
//    at 
org.apache.wicket.ComponentEventSender.dispatchToComponent(ComponentEventSender.java:292)//
//    at 
org.apache.wicket.ComponentEventSender$ComponentEventVisitor.component(ComponentEventSender.java:329)//
//    at 
org.apache.wicket.ComponentEventSender$ComponentEventVisitor.component(ComponentEventSender.java:306)//
//    at 
org.apache.wicket.util.visit.Visits.visitChildren(Visits.java:144)//
//    at 
org.apache.wicket.util.visit.Visits.visitChildren(Visits.java:162)//
//    at 
org.apache.wicket.util.visit.Visits.visitChildren(Visits.java:123)//
//    at 
org.apache.wicket.util.visit.Visits.visitChildren(Visits.java:192)//
//    at 
org.apache.wicket.MarkupContainer.visitChildren(MarkupContainer.java:988)//
//    at 
org.apache.wicket.ComponentEventSender.breadth(ComponentEventSender.java:160)//
//    at 
org.apache.wicket.ComponentEventSender.send(ComponentEventSender.java:68)//

//    at org.apache.wicket.Component.send(Component.java:4418)//
//    at 
org.apache.wicket.protocol.ws.api.WebSocketMessageBroadcastHandler$1.run(WebSocketMessageBroadcastHandler.java:74)//
//    at 
org.apache.wicket.protocol.ws.WebSocketSettings$SameThreadExecutor.run(WebSocketSettings.java:393)//
//    at 
org.apache.wicket.protocol.ws.api.WebSocketMessageBroadcastHandler.respond(WebSocketMessageBroadcastHandler.java:97)//
//    at 
org.apache.wicket.request.cycle.RequestCycle$HandlerExecutor.respond(RequestCycle.java:917)//
//    at 
org.apache.wicket.request.RequestHandlerExecutor.execute(RequestHandlerExecutor.java:63)//
//    at 
org.apache.wicket.request.cycle.RequestCycle.execute(RequestCycle.java:283)//
//    at 
org.apache.wicket.request.cycle.RequestCycle.processRequest(RequestCycle.java:254)//
//    at 
org.apache.wicket.request.cycle.RequestCycle.processRequestAndDetach(RequestCycle.java:222)//
//    at 
org.apache.wicket.protocol.ws.api.AbstractWebSocketProcessor.broadcastMessage(AbstractWebSocketProcessor.java:272)//
//    at 
org.apache.wicket.protocol.ws.api.AbstractWebSocketP