Hi Everyone!
We're trying to get a JGroups UDP Multicast cluster running here, but cannot get the nodes to join it.
We have 3 Hybris servers.
Behavior is as follows:
1. Starting node 1
2. Starting node 2
3. Can see the list of available nodes in the cluster on each node in hac->Monitoring->Cluster
4. Starting node 3
5. Cluster between node 1 and 2 is missing, however, the nodes are working properly separately
6. Getting log messages on node 3 as follows:
INFO | jvm 1 | main | 2016/08/17 12:58:53.689 | WARN [CronJob Timer master] [DefaultBroadcastService] broadcast method jgroups::de.hybris.platform.cluster.jgroups.JGroupsBroadcastMethod@5d79b381 is not initialized yet - cannot send message [?://?|ver:4010100|15481803675868800-2124288858064-0|1|1of1 (content: 79 bytes)] INFO | jvm 1
| main | 2016/08/17 12:58:53.789 | WARN [CronJob Timer master] [DefaultBroadcastService] broadcast method jgroups::de.hybris.platform.cluster.jgroups.JGroupsBroadcastMethod@5d79b381 is not initialized yet - cannot send message [?://?|ver:4010100|15481803675868800-2124288858064-1|1|1of1 (content: 79 bytes)]INFO | jvm 1 | main | 2016/08/17 12:59:02.798 | WARN [TransferQueueBundler,hybris-jgroups,hybrisnode-0] [UDP] JGRP000032: hybrisnode-0: no physical address for a3e0a1d1-4976-1c1c-967b-3a28c957c5cc, dropping message
local.properties cluster config:
clustermode=true
cluster.id=0 #changes on nodes from 0 to 2 as we have 3 nodes in the cluster
cluster.maxid=3
cluster.ping.interval=60
cluster.ping.load.on.startup=true
cluster.broadcast.methods=jgroups cluster.broadcast.method.jgroups=de.hybris.platform.cluster.jgroups.JGroupsBroadcastMethod cluster.broadcast.method.jgroups.udp.mcast_port=44448 cluster.broadcast.method.jgroups.channel.name=hybris-jgroups cluster.broadcast.method.jgroups.configuration=jgroups-udp.xml log4j.logger.de.hybris.platform.cluster.PingBroadcastHandler=DEBUG log4j.logger.de.hybris.platform.cluster.udp.UnicastBroadcastMethod=DEBUG log4j.logger.de.hybris.platform.cache.udp=debug
jgroups-udp.xml is a standard without any amendments. Just like here:
https://wiki.hybris.com/display/release5/Configuring+a+hybris+Cluster#ConfiguringahybrisCluster-JGroupsUDP
Firewall is disabled on all Hybris servers.
Selinux is disabled on all Hybris servers.
There are no any restriction within the network.
All Hybris servers within one subnet.
In tomcat.generaloptions I've enabled the following properties:
-Djava.net.preferIPv6Addresses=false
-Djava.net.preferIPv4Stack=true
-Djava.net.preferIPv4Addresses=true
All servers have single network interface "eth0"
All servers are splitted on physical hypervisor, however, there are no any restrictions for multicast between them.
I've performed the test and it is successful.
I've started receiver on 2 servers and sender on a third server as follows:
java -Djava.net.preferIPv6Addresses=false -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Addresses=true -cp ext/core/lib/jgroups-3.4.1.Final.jar org.jgroups.tests.McastReceiverTest -port 44448 java -Djava.net.preferIPv6Addresses=false -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Addresses=true -cp ext/core/lib/jgroups-3.4.1.Final.jar org.jgroups.tests.McastSenderTest -port 44448
Another interesting thing is that servers working perfect in pairs, however, once I'm starting any of the nodes as a third cluster is failing and each node working separately.
Say, I've stopped the cluster.
Started nodes 1 and 3.
Cluster is available.
However, when I'm starting node 2, cluster fails.