Hazelcast Ticket Registry - WAN Replication

Hazelcast WAN Replication allows you to keep multiple Hazelcast clusters in sync by replicating their state over WAN environments such as the Internet.

:warning: Usage Warning!

Using Hazelcast WAN Replication requires a Hazelcast Enterprise subscription. Make sure you have acquired the proper license, SDK and tooling from Hazelcast before activating this feature. Please contact Hazelcast for more information.

Hazelcast supports two different operation modes of WAN Replication:

  • Active-Passive: This mode is mostly used for failover scenarios where you want to replicate an active cluster to one or more passive clusters, for the purpose of maintaining a backup.
  • Active-Active: Every cluster is equal, each cluster replicates to all other clusters. This is normally used to connect different clients to different clusters for the sake of the shortest path between client and server.

Defining WAN replication endpoints in CAS is done using static endpoints and discovery.

The following settings and properties are available from the CAS configuration catalog:

cas.ticket.registry.hazelcast.cluster.wan-replication.enabledWhether WAN should be enabled.
false

Whether WAN should be enabled.

Type
Boolean
Default
false
Defined by
HazelcastWANReplicationProperties
cas.ticket.registry.hazelcast.cluster.wan-replication.replication-nameName of this replication group.
apereo-cas

Name of this replication group.

Type
String
Default
apereo-cas
Defined by
HazelcastWANReplicationProperties
cas.ticket.registry.hazelcast.cluster.wan-replication.targetsList of target clusters to be used for synchronization and replication.
no default

List of target clusters to be used for synchronization and replication.

Type
List<HazelcastWANReplicationTargetClusterProperties>
Default
none
Defined by
HazelcastWANReplicationProperties
cas.ticket.registry.hazelcast.cluster.wan-replication.targets[0].acknowledge-typeAccepted acknowledgement-type values are listed below.
ACK_ON_OPERATION_COMPLETE
Accepted acknowledgement-type values are listed below.
  • ACK_ON_RECEIPT: ACK after WAN operation is received by the target cluster (without waiting the result of actual operation invocation).
  • ACK_ON_OPERATION_COMPLETE: Wait till the operation is complete on target cluster.
Type
String
Default
ACK_ON_OPERATION_COMPLETE
Defined by
HazelcastWANReplicationTargetClusterProperties
cas.ticket.registry.hazelcast.cluster.wan-replication.targets[0].batch-maximum-delay-millisecondsMaximum amount of time, in milliseconds, to be waited before sending a batch of events in case batch.size is not reached.
1000

Maximum amount of time, in milliseconds, to be waited before sending a batch of events in case batch.size is not reached.

Type
int
Default
1000
Defined by
HazelcastWANReplicationTargetClusterProperties
cas.ticket.registry.hazelcast.cluster.wan-replication.targets[0].batch-sizeMaximum size of events that are sent to the target cluster in a single batch.
500

Maximum size of events that are sent to the target cluster in a single batch.

Type
int
Default
500
Defined by
HazelcastWANReplicationTargetClusterProperties
cas.ticket.registry.hazelcast.cluster.wan-replication.targets[0].cluster-nameSets the cluster name used as an endpoint group password for authentication on the target endpoint.
no default

Sets the cluster name used as an endpoint group password for authentication on the target endpoint. If there is no separate publisher ID property defined, this cluster name will also be used as a WAN publisher ID. This ID is then used for identifying the publisher.

Type
String
Default
none
Defined by
HazelcastWANReplicationTargetClusterProperties
cas.ticket.registry.hazelcast.cluster.wan-replication.targets[0].consistency-check-strategyStrategy for checking the consistency of data between replicas.
NONE

Strategy for checking the consistency of data between replicas.

Type
String
Default
NONE
Defined by
HazelcastWANReplicationTargetClusterProperties
cas.ticket.registry.hazelcast.cluster.wan-replication.targets[0].endpointsComma separated list of endpoints in this replication group.
no default

Comma separated list of endpoints in this replication group. IP addresses and ports of the cluster members for which the WAN replication is implemented. These endpoints are not necessarily the entire target cluster and WAN does not perform the discovery of other members in the target cluster. It only expects that these IP addresses (or at least some of them) are available.

Type
String
Default
none
Defined by
HazelcastWANReplicationTargetClusterProperties
cas.ticket.registry.hazelcast.cluster.wan-replication.targets[0].executor-thread-countThe number of threads that the replication executor will have.
2

The number of threads that the replication executor will have. The executor is used to send WAN events to the endpoints and ideally you want to have one thread per endpoint. If this property is omitted and you have specified the endpoints property, this will be the case. If necessary you can manually define the number of threads that the executor will use. Once the executor has been initialized there is thread affinity between the discovered endpoints and the executor threads - all events for a single endpoint will go through a single executor thread, preserving event order. It is important to determine which number of executor threads is a good value. Failure to do so can lead to performance issues - either contention on a too small number of threads or wasted threads that will not be performing any work.

Type
int
Default
2
Defined by
HazelcastWANReplicationTargetClusterProperties
cas.ticket.registry.hazelcast.cluster.wan-replication.targets[0].propertiesThe WAN publisher properties.
no default

The WAN publisher properties.

Type
Map<String, Comparable>
Default
none
Defined by
HazelcastWANReplicationTargetClusterProperties
cas.ticket.registry.hazelcast.cluster.wan-replication.targets[0].publisher-class-namePublisher class name for WAN replication.
com.hazelcast.enterprise.wan.replication.WanBatchReplication

Publisher class name for WAN replication.

Type
String
Default
com.hazelcast.enterprise.wan.replication.WanBatchReplication
Defined by
HazelcastWANReplicationTargetClusterProperties
cas.ticket.registry.hazelcast.cluster.wan-replication.targets[0].publisher-idReturns the publisher ID used for identifying the publisher.
no default

Returns the publisher ID used for identifying the publisher.

Type
String
Default
none
Defined by
HazelcastWANReplicationTargetClusterProperties
cas.ticket.registry.hazelcast.cluster.wan-replication.targets[0].queue-capacityFor huge clusters or high data mutation rates, you might need to increase the replication queue size.
10000

For huge clusters or high data mutation rates, you might need to increase the replication queue size. The default queue size for replication queues is 10,000. This means, if you have heavy put/update/remove rates, you might exceed the queue size so that the oldest, not yet replicated, updates might get lost.

Type
int
Default
10000
Defined by
HazelcastWANReplicationTargetClusterProperties
cas.ticket.registry.hazelcast.cluster.wan-replication.targets[0].queue-full-behaviorAccepted queue-full behavior values are listed below.
THROW_EXCEPTION
Accepted queue-full behavior values are listed below.
  • THROW_EXCEPTION: Instruct WAN replication implementation to throw an exception and doesn't allow further processing.
  • DISCARD_AFTER_MUTATION: Instruct WAN replication implementation to drop new events when WAN event queues are full.
  • THROW_EXCEPTION_ONLY_IF_REPLICATION_ACTIVE: Similar to THROW_EXCEPTION but only throws exception when WAN replication is active. * Discards the new events if WAN replication is stopped.
Type
String
Default
THROW_EXCEPTION
Defined by
HazelcastWANReplicationTargetClusterProperties
cas.ticket.registry.hazelcast.cluster.wan-replication.targets[0].response-timeout-millisecondsTime, in milliseconds, to be waited for the acknowledgment of a sent WAN event to target cluster.
60000

Time, in milliseconds, to be waited for the acknowledgment of a sent WAN event to target cluster.

Type
int
Default
60000
Defined by
HazelcastWANReplicationTargetClusterProperties
cas.ticket.registry.hazelcast.cluster.wan-replication.targets[0].snapshot-enabledWhen set to true, only the latest events (based on key) are selected and sent in a batch.
no default

When set to true, only the latest events (based on key) are selected and sent in a batch.

Type
boolean
Default
none
Defined by
HazelcastWANReplicationTargetClusterProperties

Required settings may be needed to activate or affect the feature; review them even when they have a default. Optional settings only need to be set to change a default or to turn on the behavior they control. Third party settings belong to libraries such as Spring Boot that CAS builds on; their own documentation may have more detail.

Notes on configuration

Configuration Metadata

The collection of configuration properties listed in this section are automatically generated from the CAS source and components that contain the actual field definitions, types, descriptions, modules, etc. This metadata may not always be 100% accurate, or could be lacking details and sufficient explanations.

Be Selective

This section is meant as a guide only. Do NOT copy/paste the entire collection of settings into your CAS configuration; rather pick only the properties that you need. Do NOT enable settings unless you are certain of their purpose and do NOT copy settings into your configuration only to keep them as reference. All these ideas lead to upgrade headaches, maintenance nightmares and premature aging.

YAGNI

Note that for nearly ALL use cases, declaring and configuring properties listed here is sufficient. You should NOT have to explicitly massage a CAS XML/Java/etc configuration file to design an authentication handler, create attribute release policies, etc. CAS at runtime will auto-configure all required changes for you. If you are unsure about the meaning of a given CAS setting, do NOT turn it on without hesitation. Review the codebase or better yet, ask questions to clarify the intended behavior.

Naming Convention

Property names can be specified in very relaxed terms. For instance cas.someProperty, cas.some-property, cas.some_property are all valid names. While all forms are accepted by CAS, there are certain components (in CAS and other frameworks used) whose activation at runtime is conditional on a property value, where this property is required to have been specified in CAS configuration using kebab case. This is both true for properties that are owned by CAS as well as those that might be presented to the system via an external library or framework such as Spring Boot, etc.

:information_source: Note

When possible, properties should be stored in lower-case kebab format, such as cas.property-name=value. The only possible exception to this rule is when naming actuator endpoints; The name of the actuator endpoints (i.e. ssoSessions) MUST remain in camelCase mode.

Settings and properties that are controlled by the CAS platform directly always begin with the prefix cas. All other settings are controlled and provided to CAS via other underlying frameworks and may have their own schemas and syntax. BE CAREFUL with the distinction. Unrecognized properties are rejected by CAS and/or frameworks upon which CAS depends. This means if you somehow misspell a property definition or fail to adhere to the dot-notation syntax and such, your setting is entirely refused by CAS and likely the feature it controls will never be activated in the way you intend.

Validation

Configuration properties are automatically validated on CAS startup to report issues with configuration binding, especially if defined CAS settings cannot be recognized or validated by the configuration schema. Additional validation processes are also handled via Configuration Metadata and property migrations applied automatically on startup by Spring Boot and family.

Indexed Settings

CAS settings able to accept multiple values are typically documented with an index, such as cas.some.setting[0]=value. The index [0] is meant to be incremented by the adopter to allow for distinct multiple configuration blocks.