openstack 组件通讯是通过ZeroMQ+ceilometer发送组件调用信息，具体是通过TCP通讯，发送数据和接收数据是用同一个端口（在配置文件指定），下面通过代码稍作解析：
IceHouse/ceilometer/ceilometer/openstack/common/rpc/impl_zmq.py
def _multi_send(method, context, topic, msg, timeout=None, envelope=False, _msg_id=None): """Wraps the sending of messages. Dispatches to the matchmaker and sends message to all relevant hosts. """ conf = CONF LOG.debug(_("%(msg)s") % {'msg': ' '.join(map(pformat, (topic, msg)))}) queues = _get_matchmaker().queues(topic) LOG.debug(_("Sending message(s) to: %s"), queues) # Don't stack if we have no matchmaker results if not queues: LOG.warn(_("No matchmaker results. Not casting.")) # While not strictly a timeout, callers know how to handle # this exception and a timeout isn't too big a lie. raise rpc_common.Timeout(_("No match from matchmaker.")) # This supports brokerless fanout (addresses > 1) for queue in queues: (_topic, ip_addr) = queue _addr = "tcp://%s:%s" % (ip_addr, conf.rpc_zmq_port) if method.__name__ == '_cast': eventlet.spawn_n(method, _addr, context, _topic, msg, timeout, envelope, _msg_id) return return method(_addr, context, _topic, msg, timeout, envelope)
debug日志:/var/log/ceilometer/compute.log
compute.log:2015-05-18 11:36:58.972 2436 DEBUG ceilometer.openstack.common.rpc.common [-] Sending message(s) to: [(u'metering.ceilometer', u'ceilometer')] _multi_send /usr/lib/python2.6/site-packages/ceilometer/openstack/common/rpc/impl_zmq.py:750 compute.log:2015-05-18 11:36:58.982 2436 DEBUG ceilometer.openstack.common.rpc.common [-] Sending message(s) to: [(u'metering.ceilometer', u'ceilometer')] _multi_send /usr/lib/python2.6/site-packages/ceilometer/openstack/common/rpc/impl_zmq.py:750
1. conf.rpc_zmq_port是通过/etc/nova/nova.conf中指定的，也即是nova服务（/usr/bin/oslo-messaging-zmq-receiver）启动的监听端口：
[root@test1 ceilometer]# netstat -lntp | grep 9502 tcp 0 0 192.168.213.202:9502 0.0.0.0:* LISTEN 17860/haproxy tcp 0 0 192.168.213.88:9502 0.0.0.0:* LISTEN 7611/python [root@test1 ceilometer]# ps -ef | grep 7611 nova 7611 1 0 May13 ? 00:00:02 /usr/bin/python /usr/bin/oslo-messaging-zmq-receiver --config-file /etc/nova/nova.conf root 26349 20271 0 11:44 pts/2 00:00:00 grep 7611
2. 通过上面代码可以看到nova与ceilometer通讯的指定的目的端口也是rpc_zmq_port，由此可以得出，在all in one环境中，需要对cinder、nova、neutron和ceilometer做端口转发，才能在不修改源码的基础上做到组件之间通讯(组件与ceilometer建立tcp连接)，用haproxy：
/etc/haproxy/haproxy.cfg
#/etc/haproxy/haproxy.cfg #--------------------------------------------------------------------- # Example configuration for a possible web application. See the # full configuration options online. # # http://haproxy.1wt.eu/download/1.4/doc/configuration.txt # #--------------------------------------------------------------------- #--------------------------------------------------------------------- # Global settings #--------------------------------------------------------------------- global # to have these messages end up in /var/log/haproxy.log you will # need to: # # 1) configure syslog to accept network log events. This is done # by adding the '-r' option to the SYSLOGD_OPTIONS in # /etc/sysconfig/syslog # # 2) configure local2 events to go to the /var/log/haproxy.log # file. A line like the following can be added to # /etc/sysconfig/syslog # # local2.* /var/log/haproxy.log # log /dev/log local2 chroot /var/lib/haproxy pidfile /var/run/haproxy.pid maxconn 32768 user haproxy group haproxy daemon # turn on stats unix socket stats socket /var/lib/haproxy/stats #--------------------------------------------------------------------- # common defaults that all the 'listen' and 'backend' sections will # use if not designated in their block #--------------------------------------------------------------------- defaults mode tcp log global #option httplog option dontlognull #option http-server-close #option forwardfor except 127.0.0.0/8 option redispatch retries 3 timeout http-request 10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive 10s timeout check 10s maxconn 32768 listen haproxy_stats bind *:8888 mode http stats uri / #stats refresh 5 # stats realm HAproxy stats # stats auth admin:admin listen zmq-cinder bind 192.168.213.202:9501 mode tcp #log 127.0.0.1 local2 balance roundrobin server test1 192.168.213.88:9500 listen zmq-nova bind 192.168.213.202:9502 mode tcp #log 127.0.0.1 local2 balance roundrobin server test1 192.168.213.88:9500 listen zmq-neutron bind 192.168.213.202:9505 mode tcp #log 127.0.0.1 local2 balance roundrobin server test1 192.168.213.88:9500 listen zmq-ceilometer bind 192.168.213.202:9500 mode tcp #log 127.0.0.1 local2 balance roundrobin server test1 192.168.213.88:9500
3. 建立TCP连接时指定目的IP是在nova配置文件的[matchmaker_ring]模块中指定：
[matchmaker_ring] ringfile = /etc/nova/matchmaker_ring.json
{ "conductor":["test1"], "scheduler":["test1"], "compute": ["test1"], "cert": ["test1"], "consoleauth":["test1"], "backendtask":["test1"], "metering": ["ceilometer"], "notifications-info": ["ceilometer"], "notifications-error": ["ceilometer"] }