Hi David,
I tried similar config with 4 nodes:
> cas.ticket.registry.hazelcast.cluster.members=${HAZELCAST_CLUSTER_MEMBERS}
> cas.ticket.registry.hazelcast.cluster.asyncBackupCount=4
> cas.ticket.registry.hazelcast.cluster.backupCount=0
> cas.ticket.registry.hazelcast.cluster.port=5701
> cas.ticket.registry.hazelcast.cluster.portAutoIncrement=false
> cas.ticket.registry.hazelcast.cluster.instanceName=localhost
>
> cas.ticket.registry.hazelcast.cluster.publicAddress=${HAZELCAST_PUBLIC_ADDRESS}
> cas.ticket.registry.hazelcast.cluster.tcpipEnabled=true
> cas.ticket.registry.hazelcast.crypto.enabled=false
I see this output on each node i.e. hazelcast creates a cluster and sees
all nodes:
2020-02-06 21:20:49,235 INFO
> [com.hazelcast.internal.cluster.ClusterService] -
> <[ecdc-rant-affiliateidp-dev-1]:5701 [dev] [3.12.4]
> Members {size:4, ver:4} [
> Member [wcdc-rant-affiliateidp-dev-1]:5701 -
> a245c93b-beb0-4929-b831-e40a323cad8b
> Member [ecdc-rant-affiliateidp-dev-2]:5701 -
> bcbcd799-8cb8-4e5d-8802-5d95d4015ffd
> Member [wcdc-rant-affiliateidp-dev-2]:5701 -
> 9d3f52c9-1475-462e-844a-1b534efdca73
> Member [ecdc-rant-affiliateidp-dev-1]:5701 -
> e9f81f52-7a99-4428-a402-5a2f48cba838 this
> ]
> >
However tickets distribution doesn't work. Nodes 1, 2, 3 don't know about
session on Node 4.
I don't see any errors in the logs related to hazelcast but this one
appears time to time
> 2020-02-06 17:31:56,248 ERROR
> [org.apereo.cas.web.flow.executor.EncryptedTranscoder] - <Null input buffer>
> java.lang.IllegalArgumentException: Null input buffer
> at javax.crypto.Cipher.doFinal(Unknown Source) ~[?:?]
> at
> org.apereo.cas.util.cipher.BaseBinaryCipherExecutor.decode(BaseBinaryCipherExecutor.java:92)
>
> ~[cas-server-core-util-api-6.1.3.jar!/:6.1.3]
On Wednesday, February 5, 2020 at 9:28:43 PM UTC+2, David Curry wrote:
>
> Maksim,
>
> If you don't want to ever lose tickets, then you would want all nodes to
> back up all other nodes. So if you have 3 member nodes, you would want 2
> async backup nodes (asyncBackupCount) and also you'd probably want to
> disable the default sync backup (backupCount) node since it will block.
> Here are the settings we're running with in production (although this is
> CAS 5):
>
> cas.ticket.registry.hazelcast.cluster.members:
> cas01.newschool.edu,cas02.newschool.edu,cas03.newschool.edu,
> cas04.newschool.edu,cas05.newschool.edu
> cas.ticket.registry.hazelcast.cluster.asyncBackupCount: 4
> cas.ticket.registry.hazelcast.cluster.backupCount: 0
> cas.ticket.registry.hazelcast.cluster.port: 5701
> cas.ticket.registry.hazelcast.cluster.portAutoIncrement: false
> cas.ticket.registry.hazelcast.crypto.encryption.key:
> xxxIoXN6SBU5bF+iAVTKgw==
> cas.ticket.registry.hazelcast.crypto.signing.key:
>
> xxxmEbPGT_MXg0JWYLTe4oFaOaklocCqlY2VuHBdAHuh0V6-PdQxmgi4tTA3CZZos8TUbzg-L9nYHJpA5RqcvA
> cas.ticket.registry.hazelcast.crypto.enabled: true
>
> This works well for us behind an F5 load balancer; we do not use sticky
> sessions. We can (and do) reboot servers in the pool without anyone getting
> re-prompted to log in (just don't reboot them all at once).
>
> The crypto stuff (last three lines) is not needed for this to work, but
> you (arguably) might want it in production. You can leave it off while
> getting things to work and enable it later.
>
> One other thing -- did you remember to open 5701 in the firewall on all
> the servers?
>
> --
>
> DAVID A. CURRY, CISSP
> *DIRECTOR • INFORMATION SECURITY & PRIVACY*
> THE NEW SCHOOL • INFORMATION TECHNOLOGY
>
> 71 FIFTH AVE., 9TH FL., NEW YORK, NY 10003
> +1 646 909-4728 • david...@newschool.edu <javascript:>
>
>
> On Wed, Feb 5, 2020 at 1:40 PM Ray Bon <rb...@uvic.ca <javascript:>>
> wrote:
>
>> Maksim,
>>
>> There is this config setting
>> cas.ticket.registry.hazelcast.cluster.members=
>> Add IPs of all members to the list.
>> https://apereo.github.io/cas/6.1.x/configuration/Configuration-Properties.html#hazelcast-ticket-registry
>>
>> and the link to common settings.
>>
>> There are some hazelcast loggers in log4j2.xml
>>
>> <AsyncLogger name="com.hazelcast"
>> level="${sys:hazelcast.log.level}" includeLocation="true" />
>>
>> Ray
>>
>> On Wed, 2020-02-05 at 09:16 -0800, 'Maksim Kopeyka' via CAS Community
>> wrote:
>>
>> Ray,
>>
>> I asked about CAS functionality to distribute tickets across nodes. I
>> need specific CAS functionality based on Hazelcast and seems to me this
>> functionality doesn't work as expected so I need to check it somehow. Maybe
>> with some debug logging.
>> I have a cluster with several nodes of CAS with hazelcast ticket registry
>> and I have a load balancer. If I turn off sticky sessions CAS asks me about
>> username/password every time. This cluster works in the same way without
>> hazelcast ticket registry. So seems to me this functionality doesn't work
>> and I don't see any errors in the logs.
>>
>> On Wednesday, February 5, 2020 at 7:03:55 PM UTC+2, rbon wrote:
>>
>> Maksim,
>>
>> I do not know if there is a stand alone client for accessing hazelcast
>> data. In the docs,
>> https://docs.hazelcast.org/docs/4.0/manual/html-single/index.html, there
>> is a section on clients and one on management. It looks like you would have
>> to create an application yourself but someone has probably done that
>> already.
>>
>> Ray
>>
>> On Wed, 2020-02-05 at 08:11 -0800, 'Maksim Kopeyka' via CAS Community
>> wrote:
>>
>> Hi Ray,
>>
>> Seems to me Hazelcast doesn't distribute data across all nodes because
>> each node doesn't have information about sessions on other nodes.
>> How to check data distribution?
>>
>> On Friday, January 31, 2020 at 11:02:05 PM UTC+2, rbon wrote:
>>
>> Maksim,
>>
>> Hazelcast is distributed but not replicated. Thus, when a server goes
>> down, the tickets on that server are lost. You have to relogin only if your
>> ticket was on that server.
>> Hazelcast has some mechanism of determining which node has which ticket.
>> It may also be possible to make hazelcast replicated but I have not tried.
>>
>> Ray
>>
>> P.S. you have two node3s in your config.
>>
>> On Fri, 2020-01-31 at 11:51 -0800, 'Maksim Kopeyka' via CAS Community
>> wrote:
>>
>> Hi Andy,
>>
>> Your example is very helpful. Thank you.
>> I see how hazelcast tickets registry works on my local env. I turned off
>> active container and another container continues to work with my active
>> session without relogin.
>>
>> However on my remote env. with CAS 6.1.3 it doesn't work in this way.
>> Load balancer ask me to re-login If I turn off active container.
>> Each node with hazelcast sees other nodes. I see such messages on all
>> nodes in case node1 is turned off:
>>
>> WARN [com.hazelcast.nio.tcp.TcpIpConnectionErrorHandler] - <[node2]:5701
>> [dev] [3.12.4] Removing connection to endpoint [node1]:5701 Cause => java
>> .net.SocketException {Connection refused to address node1/xx.xx.xx.xx:
>> 5701}, Error-Count: 5>
>> WARN [com.hazelcast.internal.cluster.impl.MembershipManager] - <[node2]:
>> 5701 [dev] [3.12.4] Member [node1]:5701 -
>> b1fba639-dfff-4536-b5f4-a8681920594d
>> is suspected to be dead for reason: No connection>
>> WARN [com.hazelcast.nio.tcp.TcpIpConnectionErrorHandler] - <[node2]:5701
>> [dev] [3.12.4] Removing connection to endpoint [node1]:5701 Cause => java
>> .net.SocketException {Connection refused to address node1/xx.xx.xx.xx:
>> 5701}, Error-Count: 6>
>> WARN [com.hazelcast.nio.tcp.TcpIpConnectionErrorHandler] - <[node2]:5701
>> [dev] [3.12.4] Removing connection to endpoint [node1]:5701 Cause => java
>> .net.SocketException {Connection refused to address node1/xx.xx.xx.xx:
>> 5701}, Error-Count: 7>
>> WARN [com.hazelcast.nio.tcp.TcpIpConnectionErrorHandler] - <[node2]:5701
>> [dev] [3.12.4] Removing connection to endpoint [node1]:5701 Cause => java
>> .net.SocketException {Connection refused to address node1/xx.xx.xx.xx:
>> 57001}, Error-Count: 8>
>>
>>
>>
>> This is my hazelcast settings:
>>
>> cas.ticket.registry.hazelcast.cluster.members=node1:5701,node2:5701,node3
>> :5701,node3:5701
>> cas.ticket.registry.hazelcast.cluster.asyncBackupCount=3
>> cas.ticket.registry.hazelcast.cluster.port=5701
>> cas.ticket.registry.hazelcast.cluster.portAutoIncrement=false
>> cas.ticket.registry.hazelcast.cluster.instanceName=localhost
>> cas.ticket.registry.hazelcast.cluster.publicAddress=node1:5701
>> cas.ticket.registry.hazelcast.cluster.tcpipEnabled=true
>>
>>
>> Why hazelcast doesn't share data across the cluster?
>>
>> These messages I see on startup
>>
>> WARN [com.hazelcast.instance.AddressPicker] - <[LOCAL] [dev] [3.12.4] You
>> configured your member address as host name. Please be aware of that
>> your dns can be spoofed. Make sure that your dns configurations are
>> correct.>
>> WARN [com.hazelcast.instance.AddressPicker] - <[LOCAL] [dev] [3.12.4] You
>> configured your member address as host name. Please be aware of that
>> your dns can be spoofed. Make sure that your dns configurations are
>> correct.>
>> WARN [com.hazelcast.instance.AddressPicker] - <[LOCAL] [dev] [3.12.4] You
>> configured your member address as host name. Please be aware of that
>> your dns can be spoofed. Make sure that your dns configurations are
>> correct.>
>> WARN [com.hazelcast.instance.AddressPicker] - <[LOCAL] [dev] [3.12.4] You
>> configured your member address as host name. Please be aware of that
>> your dns can be spoofed. Make sure that your dns configurations are
>> correct.>
>> WARN [com.hazelcast.instance.AddressPicker] - <[LOCAL] [dev] [3.12.4]
>> Could not find a matching address to start with! Picking one of non-loopback
>> addresses.>
>> INFO [org.apereo.cas.util.CoreTicketUtils] - <Ticket registry
>> encryption/signing
>> is turned off. This MAY NOT be safe in a clustered production environment
>> . Consider using other choices to handle encryption, signing and
>> verification of ticket registry tickets, and verify the chosen ticket
>> registry does support this behavior.>
>>
>>
>> On Wednesday, January 22, 2020 at 3:18:34 AM UTC+2, Andy Ng wrote:
>>
>> Hi Maksim,
>>
>> Pretty sure:
>> cas.ticket.registry.hazelcast.cluster.public-address
>> and
>> cas.ticket.registry.hazelcast.cluster.publicAddress
>>
>> Both works the same, since spring property allows both camelCase and
>> kebak-case.
>>
>>
>> And I did successfully use docker CAS and use Hazelcast as ticketing
>> system, however I am using it for demo so I just included a whole bunch of
>> private IP so it works......
>>
>> here's my CAS properties:
>>
>>
>> cas.ticket.registry.hazelcast.cluster.members=172.20.0.1,172.20.0.2,172.20.0.3,172.20.0.4,172.20.0.5,172.20.0.6,172.20.0.7,172.20.0.8,172.20.0.9,172.20.0.10
>> cas.ticket.registry.hazelcast.cluster.instanceName=localhost
>> my project link as well so you can reference if want to:
>> https://github.com/NgSekLong/SelectUrCAS/blob/master/source/ticket-registry/hazelcast/cas.yml
>>
>>
>> Cheers!
>> - Andy
>>
>> --
>>
>>
>> Ray Bon
>> Programmer Analyst
>> Development Services, University Systems
>> 2507218831 | CLE 019 | rb...@uvic.ca
>>
>> I respectfully acknowledge that my place of work is located within the
>> ancestral, traditional and unceded territory of the Songhees, Esquimalt and
>> WSÁNEĆ Nations.
>>
>> --
>>
>>
>> Ray Bon
>> Programmer Analyst
>> Development Services, University Systems
>> 2507218831 | CLE 019 | rb...@uvic.ca
>>
>> I respectfully acknowledge that my place of work is located within the
>> ancestral, traditional and unceded territory of the Songhees, Esquimalt and
>> WSÁNEĆ Nations.
>>
>> --
>>
>> Ray Bon
>> Programmer Analyst
>> Development Services, University Systems
>> 2507218831 | CLE 019 | rb...@uvic.ca <javascript:>
>>
>> I respectfully acknowledge that my place of work is located within the
>> ancestral, traditional and unceded territory of the Songhees, Esquimalt and
>> WSÁNEĆ Nations.
>>
>> --
>> - Website: https://apereo.github.io/cas
>> - Gitter Chatroom: https://gitter.im/apereo/cas
>> - List Guidelines: https://goo.gl/1VRrw7
>> - Contributions: https://goo.gl/mh7qDG
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "CAS Community" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to cas-...@apereo.org <javascript:>.
>> To view this discussion on the web visit
>> https://groups.google.com/a/apereo.org/d/msgid/cas-user/cfe74ef4f3aeb0c1dfc506a7257e659418e41f0e.camel%40uvic.ca
>>
>> <https://groups.google.com/a/apereo.org/d/msgid/cas-user/cfe74ef4f3aeb0c1dfc506a7257e659418e41f0e.camel%40uvic.ca?utm_medium=email&utm_source=footer>
>> .
>>
>
--
- Website: https://apereo.github.io/cas
- Gitter Chatroom: https://gitter.im/apereo/cas
- List Guidelines: https://goo.gl/1VRrw7
- Contributions: https://goo.gl/mh7qDG
---
You received this message because you are subscribed to the Google Groups "CAS
Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to cas-user+unsubscr...@apereo.org.
To view this discussion on the web visit
https://groups.google.com/a/apereo.org/d/msgid/cas-user/8d82a0af-84cb-4e9e-96e9-7e1e688eba1d%40apereo.org.
