Configure Service Replication

In the event that CAS service definitions are not managed globally via a centralized store, definitions need to be kept in sync throughout all CAS nodes in a cluster when more than one node is deployed. When the management strategy of such definitions is to store them on disk local to each node (such as JSON or YAML) files, the following mechanisms may be used to copy files from one host to another.

Native

A background task can be scheduled with the likes of rsync to copy files from from host to another. The job needs to of course run periodically to ensure configuration is kept in sync. This is the simplest option as CAS is completely ignorant of extra process in the background.

On Linux machines, rsync may be installed as:

1
2
# yum install rsync (On Red Hat based systems)
# apt-get install rsync (On Debian based systems)

As an example, this command will sync a directory /etc/cas/services from a local machine to a remote server:

1
rsync -avz /etc/cas/services root@192.168.0.101:/etc/cas/services

The exact opposite of the above command may be carried as such:

1
rsync -avzh root@192.168.0.100:/etc/cas/services /etc/cas/services
  • To execute the transfer operation over ssh, use the ssh --progress flags.
  • To test the command execution in mock mode, use the --dry-run flag.

Hazelcast

If you’d rather not resort to outside tooling and processes or if the native options for your deployment are not that attractive, you can take advantage of CAS’ own tooling that provides a distributed cache via Hazelcast to broadcast service definition files across the cluster and add/remove/update each node as needed. As service definitions are loaded by CAS, events are broadcasted to all CAS nodes in the cluster to pick up the changes and keep definitions in sync.

Support is enabled by including the following dependency in the overlay:

implementation "org.apereo.cas:cas-server-support-service-registry-stream-hazelcast:${project.'cas.version'}"
<dependency>
  <groupId>org.apereo.cas</groupId>
  <artifactId>cas-server-support-service-registry-stream-hazelcast</artifactId>
  <version>${cas.version}</version>
</dependency>
dependencyManagement {
  imports {
    mavenBom "org.apereo.cas:cas-server-support-bom:${project.'cas.version'}"
  }
}

dependencies {  
  implementation "org.apereo.cas:cas-server-support-service-registry-stream-hazelcast"
}

The following settings and properties are available from the CAS configuration catalog:

The configuration settings listed below are tagged as Required in the CAS configuration metadata. This flag indicates that the presence of the setting may be needed to activate or affect the behavior of the CAS feature and generally should be reviewed, possibly owned and adjusted. If the setting is assigned a default value, you do not need to strictly put the setting in your copy of the configuration, but should review it nonetheless to make sure it matches your deployment expectations.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.jclouds.credential=
  • Cloud Provider credential, can be thought of as a password for cloud services.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.jclouds.identity=
  • Cloud Provider identity, can be thought of as a user name for cloud services.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.jclouds.provider=
  • String value that is used to identify ComputeService provider. For example, "google-compute-engine" is used for Google Cloud services. See here for more info.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.azure.client-id=
  • The Azure Active Directory Service Principal client ID.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.azure.client-secret=
  • The Azure Active Directory Service Principal client secret.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.azure.cluster-id=
  • The name of the tag on the hazelcast vm resources. With every Hazelcast Virtual Machine you deploy in your resource group, you need to ensure that each VM is tagged with the value of cluster-id defined in your Hazelcast configuration. The only requirement is that every VM can access each other either by private or public IP address.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.azure.group-name=
  • The Azure resource group name of the cluster. You can find this in the Azure portal or CLI.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.azure.subscription-id=
  • The Azure subscription ID.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.azure.tenant-id=
  • The Azure Active Directory tenant ID.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.aws.access-key=
  • AWS access key.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.aws.secret-key=
  • AWS secret key.

  • cas.service-registry.stream.hazelcast.config.cluster.core.instance-name=
  • The instance name.

  • cas.service-registry.stream.hazelcast.config.cluster.network.members=
  • Sets the well known members. If members is empty, calling this method will have the same effect as calling clear(). A member can be a comma separated string, e..g '10.11.12.1,10.11.12.2' which indicates multiple members are going to be added.

  • cas.service-registry.stream.hazelcast.config.cluster.network.port=5701
  • You can specify the ports which Hazelcast will use to communicate between cluster members. The name of the parameter for this is port and its default value is 5701. By default, Hazelcast will try 100 ports to bind. Meaning that, if you set the value of port as 5701, as members are joining to the cluster, Hazelcast tries to find ports between 5701 and 5801.

    The configuration settings listed below are tagged as Optional in the CAS configuration metadata. This flag indicates that the presence of the setting is not immediately necessary in the end-user CAS configuration, because a default value is assigned or the activation of the feature is not conditionally controlled by the setting value.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.jclouds.credential-path=
  • Used for cloud providers which require an extra JSON or P12 key file. This denotes the path of that file. Only tested with Google Compute Engine. (Required if Google Compute Engine is used.)

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.jclouds.endpoint=
  • Defines the endpoint for a generic API such as OpenStack or CloudStack (optional).

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.jclouds.group=
  • Filters instance groups (optional). When used with AWS it maps to security group.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.jclouds.port=-1
  • Port which the hazelcast instance service uses on the cluster member. Default value is 5701. (optional)

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.jclouds.regions=
  • Defines region for a cloud service (optional). Can be used with comma separated values for multiple values.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.jclouds.role-name=
  • Used for IAM role support specific to AWS (optional, but if defined, no identity or credential should be defined in the configuration).

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.jclouds.tag-keys=
  • Filters cloud instances with tags (optional). Can be used with comma separated values for multiple values.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.jclouds.tag-values=
  • Filters cloud instances with tags (optional) Can be used with comma separated values for multiple values.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.jclouds.zones=
  • Defines zone for a cloud service (optional). Can be used with comma separated values for multiple values.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.aws.connection-timeout-seconds=5
  • The maximum amount of time Hazelcast will try to connect to a well known member before giving up. Setting this value too low could mean that a member is not able to connect to a cluster. Setting the value too high means that member startup could slow down because of longer timeouts (for example, when a well known member is not up). Increasing this value is recommended if you have many IPs listed and the members cannot properly build up the cluster. Its default value is 5.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.aws.host-header=
  • Host header. i.e. ec2.amazonaws.com. The URL that is the entry point for a web service.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.aws.iam-role=
  • If you do not want to use access key and secret key, you can specify iam-role. Hazelcast fetches your credentials by using your IAM role. This setting only affects deployments on Amazon EC2. If you are deploying CAS in an Amazon ECS environment, the role should not be specified. The role is fetched from the task definition that is assigned to run CAS.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.aws.port=-1
  • Hazelcast port. Typically may be set to 5701. You can set searching for other ports rather than 5701 if you've members on different ports.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.aws.region=us-east-1
  • AWS region. i.e. us-east-1. The region where your members are running.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.aws.security-group-name=
  • If a security group is configured, only instances within that security group are selected.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.aws.tag-key=
  • If a tag key/value is set, only instances with that tag key/value will be selected.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.aws.tag-value=
  • If a tag key/value is set, only instances with that tag key/value will be selected.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.docker-swarm.dns-provider.enabled=false
  • Enable provider.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.docker-swarm.dns-provider.peer-services=
  • Comma separated list of docker services and associated ports to be considered peers of this service. Note, this must include itself (the definition of serviceName and servicePort) if the service is to cluster with other instances of this service.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.docker-swarm.dns-provider.service-name=
  • Name of the docker service that this instance is running in.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.docker-swarm.dns-provider.service-port=5701
  • Internal port that hazelcast is listening on.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.docker-swarm.member-provider.docker-network-names=
  • Comma delimited list of Docker network names to discover matching services on.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.docker-swarm.member-provider.docker-service-labels=
  • Comma delimited list of relevant Docker service label=values to find tasks/containers on the networks.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.docker-swarm.member-provider.docker-service-names=
  • Comma delimited list of relevant Docker service names to find tasks/containers on the networks.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.docker-swarm.member-provider.enabled=false
  • Enable provider.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.docker-swarm.member-provider.hazelcast-peer-port=5701
  • The raw port that hazelcast is listening on. IMPORTANT: This is NOT a docker "published" port, nor is it necessarily a EXPOSEd port. It is the hazelcast port that the service is configured with, this must be the same for all matched containers in order to work, and just using the default of 5701 is the simplest way to go.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.docker-swarm.member-provider.skip-verify-ssl=false
  • If Swarm Mgr URI is SSL, to enable skip-verify for it.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.docker-swarm.member-provider.swarm-mgr-uri=
  • Swarm Manager URI (overrides DOCKER_HOST).

  • cas.service-registry.stream.hazelcast.duration=PT1M
  • Duration that indicates how long should items be kept in the hazelcast cache. Note that generally this number needs to be short as once an item is delivered to a target, it is explicitly removed from the cache/queue. This duration needs to be adjusted if the latency between the CAS nodes in the cluster is too large. Having too short a value will cause the record to expire before it reaches other members of the cluster.

    This settings supports the java.time.Duration syntax. The format of the value will be PTnHnMnS, where n is the relevant hours, minutes or seconds part of the duration. Any fractional seconds are placed after a decimal point in the seconds section. If a section has a zero value, it is omitted. The hours, minutes and seconds will all have the same sign. Example values could be in the form of PT20S, PT15M, PT10H, PT6D, P2DT3H4M. If the value is set to 0 or never, the duration will be set to zero. If the value is blank, set to -1, or infinite, the value will effectively represent an unending duration.
  • cas.service-registry.stream.hazelcast.config.cluster.core.async-backup-count=0
  • Hazelcast supports both synchronous and asynchronous backups. By default, backup operations are synchronous. In this case, backup operations block operations until backups are successfully copied to backup members (or deleted from backup members in case of remove) and acknowledgements are received. Therefore, backups are updated before a put operation is completed, provided that the cluster is stable. Asynchronous backups, on the other hand, do not block operations. They are fire and forget and do not require acknowledgements; the backup operations are performed at some point in time.

  • cas.service-registry.stream.hazelcast.config.cluster.core.backup-count=1
  • To provide data safety, Hazelcast allows you to specify the number of backup copies you want to have. That way, data on a cluster member will be copied onto other member(s). To create synchronous backups, select the number of backup copies. When this count is 1, a map entry will have its backup on one other member in the cluster. If you set it to 2, then a map entry will have its backup on two other members. You can set it to 0 if you do not want your entries to be backed up, e.g., if performance is more important than backing up. The maximum value for the backup count is 6. Sync backup operations have a blocking cost which may lead to latency issues.

  • cas.service-registry.stream.hazelcast.config.cluster.core.cp-member-count=0
  • CP Subsystem is a component of a Hazelcast cluster that builds a strongly consistent layer for a set of distributed data structures. Its data structures are CP with respect to the CAP principle, i.e., they always maintain linearizability and prefer consistency over availability during network partitions. Besides network partitions, CP Subsystem withstands server and client failures. All members of a Hazelcast cluster do not necessarily take part in CP Subsystem. The number of Hazelcast members that take part in CP Subsystem is specified here. CP Subsystem must have at least 3 CP members.

  • cas.service-registry.stream.hazelcast.config.cluster.core.eviction-policy=LRU
  • Hazelcast supports policy-based eviction for distributed maps. Currently supported policies are LRU (Least Recently Used) and LFU (Least Frequently Used) and NONE. See this for more info.

  • cas.service-registry.stream.hazelcast.config.cluster.core.logging-type=slf4j
  • Hazelcast has a flexible logging configuration and doesn't depend on any logging framework except JDK logging. It has in-built adaptors for a number of logging frameworks and also supports custom loggers by providing logging interfaces. To use built-in adaptors you should set this setting to one of predefined types below.

    • jdk: JDK logging
    • log4j: Log4j
    • slf4j: Slf4j
    • none: Disable logging

  • cas.service-registry.stream.hazelcast.config.cluster.core.map-merge-policy=PUT_IF_ABSENT
  • Define how data items in Hazelcast maps are merged together from source to destination. By default, merges map entries from source to destination if they don't exist in the destination map. Accepted values are:

    • PUT_IF_ABSENT: Merges data structure entries from source to destination if they don't exist in the destination data structure.
    • HIGHER_HITS: * Merges data structure entries from source to destination data structure if the source entry has more hits than the destination one.
    • DISCARD: Merges only entries from the destination data structure and discards all entries from the source data structure.
    • PASS_THROUGH: Merges data structure entries from source to destination directly unless the merging entry is null
    • EXPIRATION_TIME: Merges data structure entries from source to destination data structure if the source entry will expire later than the destination entry. This policy can only be used if the clocks of the nodes are in sync.
    • LATEST_UPDATE: Merges data structure entries from source to destination data structure if the source entry was updated more frequently than the destination entry. This policy can only be used if the clocks of the nodes are in sync.
    • LATEST_ACCESS: Merges data structure entries from source to destination data structure if the source entry has been accessed more recently than the destination entry. This policy can only be used if the clocks of the nodes are in sync.

  • cas.service-registry.stream.hazelcast.config.cluster.core.max-no-heartbeat-seconds=300
  • Max timeout of heartbeat in seconds for a node to assume it is dead.

  • cas.service-registry.stream.hazelcast.config.cluster.core.max-size=85
  • Sets the maximum size of the map.

  • cas.service-registry.stream.hazelcast.config.cluster.core.max-size-policy=USED_HEAP_PERCENTAGE
    • FREE_HEAP_PERCENTAGE: Policy based on minimum free JVM heap memory percentage per JVM.
    • FREE_HEAP_SIZE: Policy based on minimum free JVM heap memory in megabytes per JVM.
    • FREE_NATIVE_MEMORY_PERCENTAGE: Policy based on minimum free native memory percentage per Hazelcast instance.
    • FREE_NATIVE_MEMORY_SIZE: Policy based on minimum free native memory in megabytes per Hazelcast instance.
    • PER_NODE: Policy based on maximum number of entries stored per data structure (map, cache etc) on each Hazelcast instance.
    • PER_PARTITION: Policy based on maximum number of entries stored per data structure (map, cache etc) on each partition.
    • USED_HEAP_PERCENTAGE: Policy based on maximum used JVM heap memory percentage per data structure (map, cache etc) on each Hazelcast instance .
    • USED_HEAP_SIZE: Policy based on maximum used JVM heap memory in megabytes per data structure (map, cache etc) on each Hazelcast instance.
    • USED_NATIVE_MEMORY_PERCENTAGE: Policy based on maximum used native memory percentage per data structure (map, cache etc) on each Hazelcast instance.
    • USED_NATIVE_MEMORY_SIZE: Policy based on maximum used native memory in megabytes per data structure (map, cache etc) on each Hazelcast instance .

  • cas.service-registry.stream.hazelcast.config.cluster.core.partition-member-group-type=
  • With PartitionGroupConfig, you can control how primary and backup partitions are mapped to physical Members. Hazelcast will always place partitions on different partition groups so as to provide redundancy. Accepted value are: PER_MEMBER, HOST_AWARE, CUSTOM, ZONE_AWARE, SPI. In all cases a partition will never be created on the same group. If there are more partitions defined than there are partition groups, then only those partitions, up to the number of partition groups, will be created. For example, if you define 2 backups, then with the primary, that makes 3. If you have only two partition groups only two will be created.

    • PER_MEMBER Partition Groups</code>: This is the default partition scheme and is used if no other scheme is defined. Each Member is in a group of its own.
    • HOST_AWARE Partition Groups</code>: In this scheme, a group corresponds to a host, based on its IP address. Partitions will not be written to any other members on the same host. This scheme provides good redundancy when multiple instances are being run on the same host.
    • CUSTOM Partition Groups</code>: In this scheme, IP addresses, or IP address ranges, are allocated to groups. Partitions are not written to the same group. This is very useful for ensuring partitions are written to different racks or even availability zones.
    • ZONE_AWARE Partition Groups</code>: In this scheme, groups are allocated according to the metadata provided by Discovery SPI Partitions are not written to the same group. This is very useful for ensuring partitions are written to availability zones or different racks without providing the IP addresses to the config ahead.
    • SPI Partition Groups</code>: In this scheme, groups are allocated according to the implementation provided by Discovery SPI.

  • cas.service-registry.stream.hazelcast.config.cluster.core.timeout=5
  • Connection timeout in seconds for the TCP/IP config and members joining the cluster.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.enabled=false
  • Whether discovery should be enabled via the configured strategies below.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.multicast.enabled=false
  • Enables a multicast configuration using a group address and port. Contains the configuration for the multicast discovery mechanism. With the multicast discovery mechanism Hazelcast allows Hazelcast members to find each other using multicast. So Hazelcast members do not need to know concrete addresses of members, they just multicast to everyone listening. It depends on your environment if multicast is possible or allowed; otherwise you need to have a look at the tcp/ip cluster

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.multicast.group=
  • The multicast group address used for discovery. With the multicast auto-discovery mechanism, Hazelcast allows cluster members to find each other using multicast communication. The cluster members do not need to know the concrete addresses of the other members, as they just multicast to all the other members for listening. Whether multicast is possible or allowed depends on your environment.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.multicast.port=0
  • The multicast port used for discovery.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.multicast.time-to-live=32
  • Gets the time to live for the multicast package in seconds. This is the default time-to-live for multicast packets sent out on the socket

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.multicast.timeout=2
  • specifies the time in seconds that a member should wait for a valid multicast response from another member running in the network before declaring itself the leader member (the first member joined to the cluster) and creating its own cluster. This only applies to the startup of members where no leader has been assigned yet. If you specify a high value, such as 60 seconds, it means that until a leader is selected, each member will wait 60 seconds before moving on. Be careful when providing a high value. Also, be careful not to set the value too low, or the members might give up too early and create their own cluster.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.multicast.trusted-interfaces=
  • Multicast trusted interfaces for discovery. With the multicast auto-discovery mechanism, Hazelcast allows cluster members to find each other using multicast communication. The cluster members do not need to know the concrete addresses of the other members, as they just multicast to all the other members for listening. Whether multicast is possible or allowed depends on your environment.

  • cas.service-registry.stream.hazelcast.config.cluster.network.ipv4-enabled=true
  • IPv6 support has been switched off by default, since some platforms have issues in use of IPv6 stack. And some other platforms such as Amazon AWS have no support at all. To enable IPv6 support set this setting to false.

  • cas.service-registry.stream.hazelcast.config.cluster.network.local-address=
  • If this property is set, then this is the address where the server socket is bound to.

  • cas.service-registry.stream.hazelcast.config.cluster.network.network-interfaces=
  • You can specify which network interfaces that Hazelcast should use. Servers mostly have more than one network interface, so you may want to list the valid IPs. Range characters ('*' and '-') can be used for simplicity. For instance, 10.3.10.* refers to IPs between 10.3.10.0 and 10.3.10.255. Interface 10.3.10.4-18 refers to IPs between 10.3.10.4 and 10.3.10.18 (4 and 18 included). If network interface configuration is enabled (it is disabled by default) and if Hazelcast cannot find an matching interface, then it will print a message on the console and will not start on that node. Interfaces can be separated by a comma.

  • cas.service-registry.stream.hazelcast.config.cluster.network.outbound-ports=
  • The outbound ports for the Hazelcast configuration.

  • cas.service-registry.stream.hazelcast.config.cluster.network.port-auto-increment=true
  • You may also want to choose to use only one port. In that case, you can disable the auto-increment feature of port.

  • cas.service-registry.stream.hazelcast.config.cluster.network.public-address=
  • The default public address to be advertised to other cluster members and clients.

  • cas.service-registry.stream.hazelcast.config.cluster.network.tcpip-enabled=true
  • Enable TCP/IP config. Contains the configuration for the Tcp/Ip join mechanism. The Tcp/Ip join mechanism relies on one or more well known members. So when a new member wants to join a cluster, it will try to connect to one of the well known members. If it is able to connect, it will now about all members in the cluster and doesn't rely on these well known members anymore.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.enabled=false
  • Whether WAN should be enabled.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.replication-name=apereo-cas
  • Name of this replication group.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.targets=
  • List of target clusters to be used for synchronization and replication.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.targets[0].acknowledge-type=ACK_ON_OPERATION_COMPLETE
  • Accepted values are:

    • ACK_ON_RECEIPT: ACK after WAN operation is received by the target cluster (without waiting the result of actual operation invocation).
    • ACK_ON_OPERATION_COMPLETE: Wait till the operation is complete on target cluster.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.targets[0].batch-maximum-delay-milliseconds=1000
  • Maximum amount of time, in milliseconds, to be waited before sending a batch of events in case batch.size is not reached.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.targets[0].batch-size=500
  • Maximum size of events that are sent to the target cluster in a single batch.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.targets[0].cluster-name=
  • Sets the cluster name used as an endpoint group password for authentication on the target endpoint. If there is no separate publisher ID property defined, this cluster name will also be used as a WAN publisher ID. This ID is then used for identifying the publisher.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.targets[0].consistency-check-strategy=NONE
  • Strategy for checking the consistency of data between replicas.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.targets[0].endpoints=
  • Comma separated list of endpoints in this replication group. IP addresses and ports of the cluster members for which the WAN replication is implemented. These endpoints are not necessarily the entire target cluster and WAN does not perform the discovery of other members in the target cluster. It only expects that these IP addresses (or at least some of them) are available.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.targets[0].executor-thread-count=2
  • The number of threads that the replication executor will have. The executor is used to send WAN events to the endpoints and ideally you want to have one thread per endpoint. If this property is omitted and you have specified the endpoints property, this will be the case. If necessary you can manually define the number of threads that the executor will use. Once the executor has been initialized there is thread affinity between the discovered endpoints and the executor threads - all events for a single endpoint will go through a single executor thread, preserving event order. It is important to determine which number of executor threads is a good value. Failure to do so can lead to performance issues - either contention on a too small number of threads or wasted threads that will not be performing any work.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.targets[0].properties=
  • The WAN publisher properties.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.targets[0].publisher-class-name=com.hazelcast.enterprise.wan.replication.WanBatchReplication
  • Publisher class name for WAN replication.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.targets[0].publisher-id=
  • Returns the publisher ID used for identifying the publisher.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.targets[0].queue-capacity=10_000
  • For huge clusters or high data mutation rates, you might need to increase the replication queue size. The default queue size for replication queues is 10,000. This means, if you have heavy put/update/remove rates, you might exceed the queue size so that the oldest, not yet replicated, updates might get lost.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.targets[0].queue-full-behavior=THROW_EXCEPTION
  • Accepted values are:

    • THROW_EXCEPTION: Instruct WAN replication implementation to throw an exception and doesn't allow further processing.
    • DISCARD_AFTER_MUTATION: Instruct WAN replication implementation to drop new events when WAN event queues are full.
    • THROW_EXCEPTION_ONLY_IF_REPLICATION_ACTIVE: Similar to THROW_EXCEPTION but only throws exception when WAN replication is active. * Discards the new events if WAN replication is stopped.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.targets[0].response-timeout-milliseconds=60_000
  • Time, in milliseconds, to be waited for the acknowledgment of a sent WAN event to target cluster.

  • cas.service-registry.stream.hazelcast.config.cluster.wan-replication.targets[0].snapshot-enabled=
  • When set to true, only the latest events (based on key) are selected and sent in a batch.

  • cas.service-registry.stream.hazelcast.config.core.enable-compression=false
  • Enables compression when default java serialization is used.

  • cas.service-registry.stream.hazelcast.config.core.enable-management-center-scripting=true
  • Enables scripting from Management Center.

  • cas.service-registry.stream.hazelcast.config.core.license-key=
  • Hazelcast enterprise license key.

  • cas.service-registry.stream.hazelcast.config.cluster.discovery.kubernetes.api-retries=3
  • Defines the number of retries to Kubernetes API. Defaults to: 3.