OpenStack-RPC-server的构建(三)

         继续上篇文章的分析，我们知道self.rpcserver是一个MessageHandlingServer对象。然后我们将重点分析/usr/lib/python2.7/site-packages/nova/service.py的start方法中的self.rpcserver.start()这条语句。

# usr/lib/python2.7/site-packages/oslo_messaging/server.pydef start(self):        """Start handling incoming messages.        This method causes the server to begin polling the transport for        incoming messages and passing them to the dispatcher. Message        processing will continue until the stop() method is called.        The executor controls how the server integrates with the applications        I/O handling strategy - it may choose to poll for messages in a new        process, thread or co-operatively scheduled coroutine or simply by        registering a callback with an event loop. Similarly, the executor may        choose to dispatch messages in a new thread, coroutine or simply the        current thread.        """        if self._executor is not None:            return        try:            listener = self.dispatcher._listen(self.transport)        except driver_base.TransportDriverError as ex:            raise ServerListenError(self.target, ex)        self._executor = self._executor_cls(self.conf, listener,                                            self.dispatcher)        self._executor.start()

        首先listener= self.dispatcher._listen(self.transport)都做了什么，我们往下看。

    # usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py    def _listen(self, transport):        return transport._listen(self._target)    # usr/lib/python2.7/site-packages/oslo_messaging/transport.py    def _listen(self, target):        if not (target.topic and target.server):            raise exceptions.InvalidTarget('A server\'s target must have '                                           'topic and server names specified',                                           target)        return self._driver.listen(target)

        根据我们前面的分析，这里的self._driver是oslo_messaging._drivers.impl_rabbit.RabbitDriver对象，因此这里调用的是RabbitDriver对象的listen方法。由于RabbitDriver子类继承AMQPDriverBase父类，且RabbitDriver未实现listen方法，所以此时调用的是AMQPDriverBase类的listen方法。

# usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.pydef listen(self, target):        conn = self._get_connection(rpc_amqp.PURPOSE_LISTEN)        listener = AMQPListener(self, conn)        conn.declare_topic_consumer(exchange_name=self._get_exchange(target),                                    topic=target.topic,                                    callback=listener)        conn.declare_topic_consumer(exchange_name=self._get_exchange(target),                                    topic='%s.%s' % (target.topic,                                                     target.server),                                    callback=listener)        conn.declare_fanout_consumer(target.topic, listener)    return listener

        这里主要产生了两个topic和一个fanout类型的consumer。所以我们在执行rabbitmqctllist_consumers命令时，会有scheduler，scheduler.jun和scheduler_fanout_xxx(xxx表示随机序列号)三个consumer(这三个名称其实是queue名称)。在具体分析topic和fanout这两种方式之前，本篇文章我们主要分析conn = self._get_connection(rpc_amqp.PURPOSE_LISTEN)都做了些什么。

    # usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py    def _get_connection(self, purpose=rpc_amqp.PURPOSE_SEND):        return rpc_amqp.ConnectionContext(self._connection_pool,                                          purpose=purpose)

        那么self._connection_pool是什么东东呢？这里我们需要看AMQPDriverBase父类类以及子类RabbitDriver。

# usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.pyclass AMQPDriverBase(base.BaseDriver):    def __init__(self, conf, url, connection_pool,                 default_exchange=None, allowed_remote_exmods=None):        super(AMQPDriverBase, self).__init__(conf, url, default_exchange,                                             allowed_remote_exmods)        self._default_exchange = default_exchange        self._connection_pool = connection_pool        self._reply_q_lock = threading.Lock()        self._reply_q = None        self._reply_q_conn = None        self._waiter = None

# usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.pyclass RabbitDriver(amqpdriver.AMQPDriverBase):    def __init__(self, conf, url,                 default_exchange=None,                 allowed_remote_exmods=None):        opt_group = cfg.OptGroup(name='oslo_messaging_rabbit',                                 title='RabbitMQ driver options')        conf.register_group(opt_group)        conf.register_opts(rabbit_opts, group=opt_group)        conf.register_opts(rpc_amqp.amqp_opts, group=opt_group)        connection_pool = rpc_amqp.ConnectionPool(            conf, conf.oslo_messaging_rabbit.rpc_conn_pool_size,            url, Connection)        super(RabbitDriver, self).__init__(conf, url,                                           connection_pool,                                           default_exchange,                                           allowed_remote_exmods)

         首先，在RabbitDriver类中，cfg先注册一个opt_group的group选项，其名称为oslo_messaging_rabbit，然后将rabbit_opts和rpc_amqp.amqp_opts中的所有参数注册到该group选项中去。然后再创建连接池。如下。

        connection_pool是rpc_amqp.ConnectionPool对象，其中连接池的大小是conf.oslo_messaging_rabbit.rpc_conn_pool_size，该值是从conf.register_opts(rpc_amqp.amqp_opts, group=opt_group)中的rpc_amqp.amqp_opts选项中读取的。其默认大小是30。其中rpc_amqp.ConnectionPool对象将Connection类名作为参数传入去构造。因为从后面的分析，我们可知，用户第一次去取connection时，如果发现没有可用的connection时，它们会使用Connection类去构造一个connection对象。ConnectionPool的初始化如下：

# usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp.pyclass ConnectionPool(pool.Pool):    """Class that implements a Pool of Connections."""    def __init__(self, conf, rpc_conn_pool_size, url, connection_cls):        self.connection_cls = connection_cls        self.conf = conf        self.url = url        super(ConnectionPool, self).__init__(rpc_conn_pool_size)        self.reply_proxy = None

        这里self.connection_cls即为Connection类。

        从上面的分析，我们知道self._connection_pool是一个ConnectionPool对象。然后我们继续分析_get_connection方法中的rpc_amqp.ConnectionContext(self._connection_pool,purpose=purpose)，这里purpose的值为rpc_amqp.PURPOSE_LISTEN，然后利用这两个参数去构造ConnectionContext对象。如下：

# usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp.pyclass ConnectionContext(rpc_common.Connection):    """The class that is actually returned to the create_connection() caller.    This is essentially a wrapper around Connection that supports 'with'.    It can also return a new Connection, or one from a pool.    The function will also catch when an instance of this class is to be    deleted.  With that we can return Connections to the pool on exceptions    and so forth without making the caller be responsible for catching them.    If possible the function makes sure to return a connection to the pool.    """    def __init__(self, connection_pool, purpose):        """Create a new connection, or get one from the pool."""        self.connection = None        self.connection_pool = connection_pool        pooled = purpose == PURPOSE_SEND        if pooled:            self.connection = connection_pool.get()        else:            # a non-pooled connection is requested, so create a new connection            self.connection = connection_pool.create(purpose)        self.pooled = pooled        self.connection.pooled = pooled

        从上可以看出，当purpose为PURPOSE_SEND时，即生产者到达时，它将从地址池中取connection，而不是创建connection。而purpose为PURPOSE_LISTEN才是创建connection。

        这里self.connection_pool为传进来的ConnectionPool对象，因为purpose为PURPOSE_LISTEN，所以pooled为False，因此执行self.connection = connection_pool.create(purpose)。如下：

 # usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp.py # TODO(comstud): Timeout connections not used in a while    def create(self, purpose=None):        if purpose is None:            purpose = PURPOSE_SEND        LOG.debug('Pool creating new connection')        return self.connection_cls(self.conf, self.url, purpose)

        根据上面构造ConnectionPool对象的过程可知，self.connection_cls为Connection类。因此这里会构造Connection对象。而该类在/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py文件中，如下：

#/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.pyclass Connection(object):    """Connection object."""    pools = {}    def __init__(self, conf, url, purpose):        self.consumers = []        self.consumer_num = itertools.count(1)        self.conf = conf        self.driver_conf = self.conf.oslo_messaging_rabbit        self.max_retries = self.driver_conf.rabbit_max_retries        # Try forever?        if self.max_retries <= 0:            self.max_retries = None        self.interval_start = self.driver_conf.rabbit_retry_interval        self.interval_stepping = self.driver_conf.rabbit_retry_backoff        # max retry-interval = 30 seconds        self.interval_max = 30        self._login_method = self.driver_conf.rabbit_login_method        if url.virtual_host is not None:            virtual_host = url.virtual_host        else:            virtual_host = self.driver_conf.rabbit_virtual_host        self._url = ''        if self.driver_conf.fake_rabbit:            LOG.warn("Deprecated: fake_rabbit option is deprecated, set "                     "rpc_backend to kombu+memory or use the fake "                     "driver instead.")            self._url = 'memory://%s/' % virtual_host        elif url.hosts:            if url.transport.startswith('kombu+'):                LOG.warn(_LW('Selecting the kombu transport through the '                             'transport url (%s) is a experimental feature '                             'and this is not yet supported.') % url.transport)            for host in url.hosts:                transport = url.transport.replace('kombu+', '')                transport = transport.replace('rabbit', 'amqp')                self._url += '%s%s://%s:%s@%s:%s/%s' % (                    ";" if self._url else '',                    transport,                    parse.quote(host.username or ''),                    parse.quote(host.password or ''),                    self._parse_url_hostname(host.hostname) or '',                    str(host.port or 5672),                    virtual_host)        elif url.transport.startswith('kombu+'):            # NOTE(sileht): url have a + but no hosts            # (like kombu+memory:///), pass it to kombu as-is            transport = url.transport.replace('kombu+', '')            self._url = "%s://%s" % (transport, virtual_host)        else:            for adr in self.driver_conf.rabbit_hosts:                hostname, port = netutils.parse_host_port(                    adr, default_port=self.driver_conf.rabbit_port)                self._url += '%samqp://%s:%s@%s:%s/%s' % (                    ";" if self._url else '',                    parse.quote(self.driver_conf.rabbit_userid),                    parse.quote(self.driver_conf.rabbit_password),                    self._parse_url_hostname(hostname), port,                    virtual_host)        self._initial_pid = os.getpid()        self.do_consume = True        self._consume_loop_stopped = False        self.channel = None        # NOTE(sileht): if purpose is PURPOSE_LISTEN        # we don't need the lock because we don't        # have a heartbeat thread        if purpose == rpc_amqp.PURPOSE_SEND:            self._connection_lock = ConnectionLock()        else:            self._connection_lock = DummyConnectionLock()        self.connection = kombu.connection.Connection(            self._url, ssl=self._fetch_ssl_params(),            login_method=self._login_method,            failover_strategy="shuffle",            heartbeat=self.driver_conf.heartbeat_timeout_threshold)        LOG.info(_LI('Connecting to AMQP server on %(hostname)s:%(port)d'),                 self.connection.info())        # NOTE(sileht): kombu recommend to run heartbeat_check every        # seconds, but we use a lock around the kombu connection        # so, to not lock to much this lock to most of the time do nothing        # expected waiting the events drain, we start heartbeat_check and        # retreive the server heartbeat packet only two times more than        # the minimum required for the heartbeat works        # (heatbeat_timeout/heartbeat_rate/2.0, default kombu        # heartbeat_rate is 2)        self._heartbeat_wait_timeout = (            float(self.driver_conf.heartbeat_timeout_threshold) /            float(self.driver_conf.heartbeat_rate) / 2.0)        self._heartbeat_support_log_emitted = False        # NOTE(sileht): just ensure the connection is setuped at startup        self.ensure_connection()        # NOTE(sileht): if purpose is PURPOSE_LISTEN        # the consume code does the heartbeat stuff        # we don't need a thread        if purpose == rpc_amqp.PURPOSE_SEND:            self._heartbeat_start()        LOG.info(_LI('Connected to AMQP server on %(hostname)s:%(port)d'),                 self.connection.info())        # NOTE(sileht):        # value choosen according the best practice from kombu:        # http://kombu.readthedocs.org/en/latest/reference/kombu.common.html#kombu.common.eventloop        self._poll_timeout = 1        if self._url.startswith('memory://'):            # Kludge to speed up tests.            self.connection.transport.polling_interval = 0.0            self._poll_timeout = 0.05    # FIXME(markmc): use oslo sslutils when it is available as a library    _SSL_PROTOCOLS = {        "tlsv1": ssl.PROTOCOL_TLSv1,        "sslv23": ssl.PROTOCOL_SSLv23    }    _OPTIONAL_PROTOCOLS = {        'sslv2': 'PROTOCOL_SSLv2',        'sslv3': 'PROTOCOL_SSLv3',        'tlsv1_1': 'PROTOCOL_TLSv1_1',        'tlsv1_2': 'PROTOCOL_TLSv1_2',    }    for protocol in _OPTIONAL_PROTOCOLS:        try:            _SSL_PROTOCOLS[protocol] = getattr(ssl,                                               _OPTIONAL_PROTOCOLS[protocol])        except AttributeError:            pass

        1. 在构造oslo_messaging层的Connection对象时，首先解析TransportURL对象(即url)，该对象在/usr/lib/python2.7/site-packages/oslo_messaging/transport.py中的get_transport方法中创建的。具体分析get_transport方法可知，对于Nova-scheduler组件而言，url的属性只有conf和aliase有值。所以virtual_host为：’/’。然后我们执行中间红色的那段解析url的代码，由于self.driver_conf.rabbit_hosts是列表形式(可以看impl_rabbit文件最上面的设置配置项目的rabbit_hosts)，说明rabbit-server可以分布式部署到多个主机上。在我的环境中，我部署在OpenStack的Controller节点上，且rabbit_hosts=’192.168.118.1:5672’，然后经过netutils.parse_host_port方法解析出hostname和port的值，即hostname=’192.168.118.1’，port=’5672’。最后构造Connection的self._url=’ amqp://guest:guest@192.168.118.1:5672//’。此时self._url构造完成。

        2. 在这个类Connection(oslo_messaging层)中有一个self.connection，该self.connection是kombu层的Connection对象，其中将刚才构造的self._url作为参数去构造self.connection。这里ssl在conf中未设置，所以ssl=False(具体到_fetch_ssl_params方法中去查看)，然后login_method=‘AMQPLAIN’,heartbeat=0，由于在配置文件中未设置这些变量的值，所以目前都是采用默认值。所以我们可以看看kombu层的怎么去构造Connection对象的。

#/usr/lib/python2.7/site-packages/kombu/connection.pyclass Connection(object):    """A connection to the broker.    :param URL:  Broker URL, or a list of URLs, e.g.    .. code-block:: python        Connection('amqp://guest:guest@localhost:5672//')        Connection('amqp://foo;amqp://bar', failover_strategy='round-robin')        Connection('redis://', transport_options={            'visibility_timeout': 3000,        })        import ssl        Connection('amqp://', login_method='EXTERNAL', ssl={            'ca_certs': '/etc/pki/tls/certs/something.crt',            'keyfile': '/etc/something/system.key',            'certfile': '/etc/something/system.cert',            'cert_reqs': ssl.CERT_REQUIRED,        })    .. admonition:: SSL compatibility        SSL currently only works with the py-amqp & amqplib transports.        For other transports you can use stunnel.    :keyword hostname: Default host name/address if not provided in the URL.    :keyword userid: Default user name if not provided in the URL.    :keyword password: Default password if not provided in the URL.    :keyword virtual_host: Default virtual host if not provided in the URL.    :keyword port: Default port if not provided in the URL.    :keyword ssl: Use SSL to connect to the server. Default is ``False``.      May not be supported by the specified transport.    :keyword transport: Default transport if not specified in the URL.    :keyword connect_timeout: Timeout in seconds for connecting to the      server. May not be supported by the specified transport.    :keyword transport_options: A dict of additional connection arguments to      pass to alternate kombu channel implementations.  Consult the transport      documentation for available options.    :keyword heartbeat: Heartbeat interval in int/float seconds.        Note that if heartbeats are enabled then the :meth:`heartbeat_check`        method must be called at an interval twice the frequency of the        heartbeat: e.g. if the heartbeat is 10, then the heartbeats must be        checked every 5 seconds (the rate can also be controlled by        the ``rate`` argument to :meth:`heartbeat_check``).    .. note::        The connection is established lazily when needed. If you need the        connection to be established, then force it by calling        :meth:`connect`::            >>> conn.connect()        and always remember to close the connection::            >>> conn.release()    """    port = None    virtual_host = '/'    connect_timeout = 5    _closed = None    _connection = None    _default_channel = None    _transport = None    _logger = False    uri_prefix = None    #: The cache of declared entities is per connection,    #: in case the server loses data.    declared_entities = None    #: This is set to True if there is still more data to read    #: after a call to :meth:`drain_nowait`.    more_to_read = False    #: Iterator returning the next broker URL to try in the event    #: of connection failure (initialized by :attr:`failover_strategy`).    cycle = None    #: Additional transport specific options,    #: passed on to the transport instance.    transport_options = None    #: Strategy used to select new hosts when reconnecting after connection    #: failure.  One of "round-robin", "shuffle" or any custom iterator    #: constantly yielding new URLs to try.    failover_strategy = 'round-robin'    #: Heartbeat value, currently only supported by the py-amqp transport.    heartbeat = None    hostname = userid = password = ssl = login_method = None    def __init__(self, hostname='localhost', userid=None,                 password=None, virtual_host=None, port=None, insist=False,                 ssl=False, transport=None, connect_timeout=5,                 transport_options=None, login_method=None, uri_prefix=None,                 heartbeat=0, failover_strategy='round-robin',                 alternates=None, **kwargs):        alt = [] if alternates is None else alternates        # have to spell the args out, just to get nice docstrings :(        params = self._initial_params = {            'hostname': hostname, 'userid': userid,            'password': password, 'virtual_host': virtual_host,            'port': port, 'insist': insist, 'ssl': ssl,            'transport': transport, 'connect_timeout': connect_timeout,            'login_method': login_method, 'heartbeat': heartbeat        }        if hostname and not isinstance(hostname, basestring):            alt.extend(hostname)            hostname = alt[0]        if hostname and '://' in hostname:            if ';' in hostname:                alt.extend(hostname.split(';'))                hostname = alt[0]            if '+' in hostname[:hostname.index('://')]:                # e.g. sqla+mysql://root:masterkey@localhost/                params['transport'], params['hostname'] = \                    hostname.split('+', 1)                self.uri_prefix = params['transport']            else:                if transport not in URI_PASSTHROUGH:                    params.update(parse_url(hostname))        self._init_params(**params)        # fallback hosts        self.alt = alt        self.failover_strategy = failover_strategies.get(            failover_strategy or 'round-robin') or failover_strategy        if self.alt:            self.cycle = self.failover_strategy(self.alt)            next(self.cycle)  # skip first entry        # backend_cls argument will be removed shortly.        self.transport_cls = self.transport_cls or kwargs.get('backend_cls')        if transport_options is None:            transport_options = {}        self.transport_options = transport_options        if _LOG_CONNECTION:  # pragma: no cover            self._logger = True        if uri_prefix:            self.uri_prefix = uri_prefix        self.declared_entities = set()    def _init_params(self, hostname, userid, password, virtual_host, port,                     insist, ssl, transport, connect_timeout,                     login_method, heartbeat):        transport = transport or 'amqp'        if transport == 'amqp' and supports_librabbitmq():            transport = 'librabbitmq'        self.hostname = hostname        self.userid = userid        self.password = password        self.login_method = login_method        self.virtual_host = virtual_host or self.virtual_host        self.port = port or self.port        self.insist = insist        self.connect_timeout = connect_timeout        self.ssl = ssl        self.transport_cls = transport        self.heartbeat = heartbeat and float(heartbeat)

        根据在oslo_messaging层构造的self.connection对象(kombu层)传下来的参数可知，hostname=’ amqp://guest:guest@192.168.118.1:5672//’，所以将执行第一块红色标记的代码，在该代码中，重新更新params列表，因为刚开始params列表中的hostname的值并没有解析，所以需要解析hostname后才能作为最后params列表。

        然后调用self._init_params构造connection对象的相关属性。其中supports_librabbitmq()=False(因为环境支持’eventlet’，所以未采用’default’，所以返回False，细究请进入supports_librabbitmq代码中查看)。因此transport仍然等于’amqp’，所以self.transport_cls= ‘amqp’。

        此时kombu层的connection对象构造完成，让我们再次回到oslo_messaging层的connection对象的构造(/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py)，下面我们分析self.connection.info()的里面做了什么？

#/usr/lib/python2.7/site-packages/kombu/connection.pyRESOLVE_ALIASES = {'pyamqp': 'amqp',                   'librabbitmq': 'amqp'}…………… def _info(self, resolve=True):        transport_cls = self.transport_cls        if resolve:            transport_cls = RESOLVE_ALIASES.get(transport_cls, transport_cls)        D = self.transport.default_connection_params        hostname = self.hostname or D.get('hostname')        if self.uri_prefix:            hostname = '%s+%s' % (self.uri_prefix, hostname)        info = (('hostname', hostname),                ('userid', self.userid or D.get('userid')),                ('password', self.password or D.get('password')),                ('virtual_host', self.virtual_host or D.get('virtual_host')),                ('port', self.port or D.get('port')),                ('insist', self.insist),                ('ssl', self.ssl),                ('transport', transport_cls),                ('connect_timeout', self.connect_timeout),                ('transport_options', self.transport_options),                ('login_method', self.login_method or D.get('login_method')),                ('uri_prefix', self.uri_prefix),                ('heartbeat', self.heartbeat))        if self.alt:            info += (('alternates', self.alt),)        return info    def info(self):        """Get connection info."""        return OrderedDict(self._info())

        因为RESOLVE_ALIASES字典中没有’amqp’(transport_cls的值)的key，所以仍然保持原值’amqp’。

        关于这个D = self.transport.default_connection_params语句，我们分为两部分介绍，第一部分是self.transport，第二部分是self.transport.default_connection_params。其中transport和default_connection_params都是方法名，但是这里的样子怎么看都是属性名，为什么呢？这是由于python自身的语法设置的，即在方法名前设置@property的装饰器，因此调用这种方法就不用加上()调用了。

 #/usr/lib/python2.7/site-packages/kombu/connection.py @property    def transport(self):        if self._transport is None:            self._transport = self.create_transport()        return self._transport    def create_transport(self):        return self.get_transport_cls()(client=self)    create_backend = create_transport   # FIXME    def get_transport_cls(self):        """Get the currently used transport class."""        transport_cls = self.transport_cls        if not transport_cls or isinstance(transport_cls, basestring):            transport_cls = get_transport_cls(transport_cls)        return transport_cls#/usr/lib/python2.7/site-packages/kombu/transport/__init__.pyTRANSPORT_ALIASES = {    'amqp': 'kombu.transport.pyamqp:Transport',    'pyamqp': 'kombu.transport.pyamqp:Transport',    'librabbitmq': 'kombu.transport.librabbitmq:Transport',    'memory': 'kombu.transport.memory:Transport',    'redis': 'kombu.transport.redis:Transport',    'SQS': 'kombu.transport.SQS:Transport',    'sqs': 'kombu.transport.SQS:Transport',    'beanstalk': 'kombu.transport.beanstalk:Transport',    'mongodb': 'kombu.transport.mongodb:Transport',    'couchdb': 'kombu.transport.couchdb:Transport',    'zookeeper': 'kombu.transport.zookeeper:Transport',    'django': 'kombu.transport.django:Transport',    'sqlalchemy': 'kombu.transport.sqlalchemy:Transport',    'sqla': 'kombu.transport.sqlalchemy:Transport',    'ghettoq.taproot.Redis': _ghettoq('Redis', 'redis', 'redis'),    'ghettoq.taproot.Database': _ghettoq('Database', 'django', 'django'),    'ghettoq.taproot.MongoDB': _ghettoq('MongoDB', 'mongodb'),    'ghettoq.taproot.Beanstalk': _ghettoq('Beanstalk', 'beanstalk'),    'ghettoq.taproot.CouchDB': _ghettoq('CouchDB', 'couchdb'),    'filesystem': 'kombu.transport.filesystem:Transport',    'zeromq': 'kombu.transport.zmq:Transport',    'zmq': 'kombu.transport.zmq:Transport',    'amqplib': 'kombu.transport.amqplib:Transport',}_transport_cache = {}def resolve_transport(transport=None):    if isinstance(transport, basestring):        try:            transport = TRANSPORT_ALIASES[transport]        except KeyError:            if '.' not in transport and ':' not in transport:                from kombu.utils.text import fmatch_best                alt = fmatch_best(transport, TRANSPORT_ALIASES)                if alt:                    raise KeyError(                        'No such transport: %s.  Did you mean %s?' % (                            transport, alt))                raise KeyError('No such transport: %s' % transport)        else:            if callable(transport):                transport = transport()        return symbol_by_name(transport)    return transportdef get_transport_cls(transport=None):    """Get transport class by name.    The transport string is the full path to a transport class, e.g.::        "kombu.transport.pyamqp:Transport"    If the name does not include `"."` (is not fully qualified),    the alias table will be consulted.    """    if transport not in _transport_cache:        _transport_cache[transport] = resolve_transport(transport)    return _transport_cache[transport]

        在/usr/lib/python2.7/site-packages/kombu/connection.py中的get_transport_cls方法中的红色的get_transport_cls是/usr/lib/python2.7/site-packages/kombu/transport/__init__.py中的get_transport_cls方法，在/usr/lib/python2.7/site-packages/kombu/transport/__init__.py的get_transport_cls方法中，由于传递进去的参数transport =‘amqp’，且初始化时_transport_cache字典没有任何值，因此调用resolve_transport方法去得到’amqp’所对应的TRANSPORT_ALIASES，在这里’amqp’所对应的TRANSPORT_ALIASES值为：'kombu.transport.pyamqp:Transport'.然后在/usr/lib/python2.7/site-packages/kombu/connection.py的create_transport方法中创建该transport对象。所以self.transport为'kombu.transport.pyamqp:Transport'对象。

        对于self.transport.default_connection_params的解释，我们往下看。

#/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py@property    def default_connection_params(self):        return {'userid': 'guest', 'password': 'guest',                'port': self.default_port,                'hostname': 'localhost', 'login_method': 'AMQPLAIN'}

        这里只是将设置connection的默认值参数而已，这是为了防止用户在构造kombu层的connection对象时，没有设置相关的参数，所以在这里设置默认参数作为保护机制。所以在/usr/lib/python2.7/site-packages/kombu/connection.py中的_info方法构造info字典时，先判断connection对象的相关属性是否被设置，如果没被设置则采用默认值进行设置。

        我们继续回到kombu层的connection对象构造完成，下面我们分析self.ensure_connection()的里面做了什么？

 #/usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.pydef ensure_connection(self):        self.ensure(method=lambda: True)    def ensure(self, method, retry=None,               recoverable_error_callback=None, error_callback=None,               timeout_is_error=True):        """Will retry up to retry number of times.        retry = None means use the value of rabbit_max_retries        retry = -1 means to retry forever        retry = 0 means no retry        retry = N means N retries        NOTE(sileht): Must be called within the connection lock        """        current_pid = os.getpid()        if self._initial_pid != current_pid:            LOG.warn("Process forked after connection established! "                     "This can result in unpredictable behavior. "                     "See: http://docs.openstack.org/developer/"                     "oslo_messaging/transport.html")            self._initial_pid = current_pid        if retry is None:            retry = self.max_retries        if retry is None or retry < 0:            retry = None        def on_error(exc, interval):            LOG.debug(_("Received recoverable error from kombu:"),                      exc_info=True)            recoverable_error_callback and recoverable_error_callback(exc)            interval = (self.driver_conf.kombu_reconnect_delay + interval                        if self.driver_conf.kombu_reconnect_delay > 0                        else interval)            info = {'err_str': exc, 'sleep_time': interval}            info.update(self.connection.info())            if 'Socket closed' in six.text_type(exc):                LOG.error(_LE('AMQP server %(hostname)s:%(port)d closed'                              ' the connection. Check login credentials:'                              ' %(err_str)s'), info)            else:                LOG.error(_LE('AMQP server on %(hostname)s:%(port)d is '                              'unreachable: %(err_str)s. Trying again in '                              '%(sleep_time)d seconds.'), info)            # XXX(nic): when reconnecting to a RabbitMQ cluster            # with mirrored queues in use, the attempt to release the            # connection can hang "indefinitely" somewhere deep down            # in Kombu.  Blocking the thread for a bit prior to            # release seems to kludge around the problem where it is            # otherwise reproduceable.            # TODO(sileht): Check if this is useful since we            # use kombu for HA connection, the interval_step            # should sufficient, because the underlying kombu transport            # connection object freed.            if self.driver_conf.kombu_reconnect_delay > 0:                time.sleep(self.driver_conf.kombu_reconnect_delay)        def on_reconnection(new_channel):            """Callback invoked when the kombu reconnects and creates            a new channel, we use it the reconfigure our consumers.            """            self._set_current_channel(new_channel)            self.consumer_num = itertools.count(1)            for consumer in self.consumers:                consumer.reconnect(new_channel)            LOG.info(_LI('Reconnected to AMQP server on '                         '%(hostname)s:%(port)d'),                     {'hostname': self.connection.hostname,                      'port': self.connection.port})        def execute_method(channel):            self._set_current_channel(channel)            method()        recoverable_errors = (self.connection.recoverable_channel_errors +                              self.connection.recoverable_connection_errors)        try:            autoretry_method = self.connection.autoretry(                execute_method, channel=self.channel,                max_retries=retry,                errback=on_error,                interval_start=self.interval_start or 1,                interval_step=self.interval_stepping,                on_revive=on_reconnection,            )            ret, channel = autoretry_method()            self._set_current_channel(channel)            return ret        except recoverable_errors as exc:            LOG.debug(_("Received recoverable error from kombu:"),                      exc_info=True)            error_callback and error_callback(exc)            self._set_current_channel(None)            # NOTE(sileht): number of retry exceeded and the connection            # is still broken            msg = _('Unable to connect to AMQP server on '                    '%(hostname)s:%(port)d after %(retry)d '                    'tries: %(err_str)s') % {                        'hostname': self.connection.hostname,                        'port': self.connection.port,                        'err_str': exc,                        'retry': retry}            LOG.error(msg)            raise exceptions.MessageDeliveryFailure(msg)        except Exception as exc:            error_callback and error_callback(exc)            raise

        这里self.connection是kombu层的connection对象，所以当我们想知道autoretry方法做了什么呢？继续往下看。

    #/usr/lib/python2.7/site-packages/kombu/connection.py    def autoretry(self, fun, channel=None, **ensure_options):        """Decorator for functions supporting a ``channel`` keyword argument.        The resulting callable will retry calling the function if        it raises connection or channel related errors.        The return value will be a tuple of ``(retval, last_created_channel)``.        If a ``channel`` is not provided, then one will be automatically        acquired (remember to close it afterwards).        See :meth:`ensure` for the full list of supported keyword arguments.        Example usage::            channel = connection.channel()            try:                ret, channel = connection.autoretry(publish_messages, channel)            finally:                channel.close()        """        channels = [channel]        create_channel = self.channel        class Revival(object):            __name__ = fun.__name__            __module__ = fun.__module__            __doc__ = fun.__doc__            def revive(self, channel):                channels[0] = channel            def __call__(self, *args, **kwargs):                if channels[0] is None:                    self.revive(create_channel())                kwargs['channel'] = channels[0]                return fun(*args, **kwargs), channels[0]        revive = Revival()        return self.ensure(revive, revive, **ensure_options)    def ensure(self, obj, fun, errback=None, max_retries=None,               interval_start=1, interval_step=1, interval_max=1,               on_revive=None):        """Ensure operation completes, regardless of any channel/connection        errors occurring.        Will retry by establishing the connection, and reapplying        the function.        :param fun: Method to apply.        :keyword errback: Optional callback called each time the connection          can't be established. Arguments provided are the exception          raised and the interval that will be slept ``(exc, interval)``.        :keyword max_retries: Maximum number of times to retry.          If this limit is exceeded the connection error will be re-raised.        :keyword interval_start: The number of seconds we start sleeping for.        :keyword interval_step: How many seconds added to the interval          for each retry.        :keyword interval_max: Maximum number of seconds to sleep between          each retry.        **Example**        This is an example ensuring a publish operation::            >>> def errback(exc, interval):            ...     print("Couldn't publish message: %r. Retry in %ds" % (            ...             exc, interval))            >>> publish = conn.ensure(producer, producer.publish,            ...                       errback=errback, max_retries=3)            >>> publish(message, routing_key)        """        def _ensured(*args, **kwargs):            got_connection = 0            for retries in count(0):  # for infinity                try:                    return fun(*args, **kwargs)                except self.recoverable_connection_errors, exc:                    if got_connection:                        raise                    if max_retries is not None and retries > max_retries:                        raise                    self._debug('ensure connection error: %r', exc, exc_info=1)                    self._connection = None                    self._do_close_self()                    errback and errback(exc, 0)                    remaining_retries = None                    if max_retries is not None:                        remaining_retries = max(max_retries - retries, 1)                    self.ensure_connection(errback,                                           remaining_retries,                                           interval_start,                                           interval_step,                                           interval_max)                    new_channel = self.channel()                    self.revive(new_channel)                    obj.revive(new_channel)                    if on_revive:                        on_revive(new_channel)                    got_connection += 1                except self.recoverable_channel_errors, exc:                    if max_retries is not None and retries > max_retries:                        raise                    self._debug('ensure channel error: %r', exc, exc_info=1)                    errback and errback(exc, 0)        _ensured.__name__ = "%s(ensured)" % fun.__name__        _ensured.__doc__ = fun.__doc__        _ensured.__module__ = fun.__module__        return _ensured

        由于这里有点绕，我们需仔细分析，在oslo_mesaging层调用autoretry方法时，首先是得到一个_ensured的方法名，因为autoretry方法返回_ensured方法名，然后执行ret,channel = autoretry_method()，这时才将调用_ensured方法，在执行_ensured方法时，将执行returnfun(*args, **kwargs)，其中fun是在autoretry方法传递下来的Revival对象，因此会触发执行Revival类的__call__方法。同时由于channels[0]为None，所以执行self.revive(create_channel())。这里create_channel是self.channel的方法名，当create_channel()时，则调用self.channel方法。这里将创建amqp层的connection对象。这里真正建立Nova-scheduler组件到RabbitMQ-server的连接。如下。

    #/usr/lib/python2.7/site-packages/kombu/connection.py      def channel(self):        """Create and return a new channel."""        self._debug('create channel')        chan = self.transport.create_channel(self.connection)        if _LOG_CHANNEL:  # pragma: no cover            from .utils.debug import Logwrapped            return Logwrapped(chan, 'kombu.channel',                              '[Kombu channel:%(channel_id)s] ')    return chan

这里主要分为两部分：

1. 创建到RabbitMQ-server的connection。

2. 在connection的基础上创建channel。

首先，我们解释创建connection的代码流程，即self.connection代码如下。

    #/usr/lib/python2.7/site-packages/kombu/connection.py    @property    def connected(self):        """Returns true if the connection has been established."""        return (not self._closed and                self._connection is not None and                self.transport.verify_connection(self._connection))    @property    def connection(self):        """The underlying connection object.        .. warning::            This instance is transport specific, so do not            depend on the interface of this object.        """       #这里self._closed为False,表示RabbitMQ-server服务没有关闭       #且判断该组件是否连接到RabbitMQ-server服务，如果未连接上       #RabbitMQ-server服务，则需要创建连接connection.        if not self._closed:            if not self.connected:                self.declared_entities.clear()                self._default_channel = None                self._connection = self._establish_connection()                self._closed = False            return self._connection    #/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py    def establish_connection(self):        """Establish connection to the AMQP broker."""        conninfo = self.client        for name, default_value in self.default_connection_params.items():            if not getattr(conninfo, name, None):                setattr(conninfo, name, default_value)        if conninfo.hostname == 'localhost':            conninfo.hostname = '127.0.0.1'        opts = dict({            'host': conninfo.host,            'userid': conninfo.userid,            'password': conninfo.password,            'login_method': conninfo.login_method,            'virtual_host': conninfo.virtual_host,            'insist': conninfo.insist,            'ssl': conninfo.ssl,            'connect_timeout': conninfo.connect_timeout,            'heartbeat': conninfo.heartbeat,        }, **conninfo.transport_options or {})        #下面创建了amqp层的connection        conn = self.Connection(**opts)        conn.client = self.client        return conn

#/usr/lib/python2.7/site-packages/amqp/connection.py  class Connection(AbstractChannel):    """The connection class provides methods for a client to establish a    network connection to a server, and for both peers to operate the    connection thereafter.    GRAMMAR::        connection          = open-connection *use-connection close-connection        open-connection     = C:protocol-header                              S:START C:START-OK                              *challenge                              S:TUNE C:TUNE-OK                              C:OPEN S:OPEN-OK        challenge           = S:SECURE C:SECURE-OK        use-connection      = *channel        close-connection    = C:CLOSE S:CLOSE-OK                            / S:CLOSE C:CLOSE-OK    """    Channel = Channel    #: Final heartbeat interval value (in float seconds) after negotiation    heartbeat = None    #: Original heartbeat interval value proposed by client.    client_heartbeat = None    #: Original heartbeat interval proposed by server.    server_heartbeat = None    #: Time of last heartbeat sent (in monotonic time, if available).    last_heartbeat_sent = 0    #: Time of last heartbeat received (in monotonic time, if available).    last_heartbeat_received = 0    #: Number of bytes sent to socket at the last heartbeat check.    prev_sent = None    #: Number of bytes received from socket at the last heartbeat check.    prev_recv = None    def __init__(self, host='localhost', userid='guest', password='guest',                 login_method='AMQPLAIN', login_response=None,                 virtual_host='/', locale='en_US', client_properties=None,                 ssl=False, connect_timeout=None, channel_max=None,                 frame_max=None, heartbeat=0, on_blocked=None,                 on_unblocked=None, confirm_publish=False, **kwargs):        """Create a connection to the specified host, which should be        a 'host[:port]', such as 'localhost', or '1.2.3.4:5672'        (defaults to 'localhost', if a port is not specified then        5672 is used)        If login_response is not specified, one is built up for you from        userid and password if they are present.        The 'ssl' parameter may be simply True/False, or for Python >= 2.6        a dictionary of options to pass to ssl.wrap_socket() such as        requiring certain certificates.        """        channel_max = channel_max or 65535        frame_max = frame_max or 131072        if (login_response is None) \                and (userid is not None) \                and (password is not None):            login_response = AMQPWriter()            login_response.write_table({'LOGIN': userid, 'PASSWORD': password})            login_response = login_response.getvalue()[4:]  # Skip the length                                                            # at the beginning        d = dict(LIBRARY_PROPERTIES, **client_properties or {})        self._method_override = {(60, 50): self._dispatch_basic_return}        self.channels = {}        # The connection object itself is treated as channel 0        super(Connection, self).__init__(self, 0)        self.transport = None        # Properties set in the Tune method        self.channel_max = channel_max        self.frame_max = frame_max        self.client_heartbeat = heartbeat        self.confirm_publish = confirm_publish        # Callbacks        self.on_blocked = on_blocked        self.on_unblocked = on_unblocked        self._avail_channel_ids = array('H', range(self.channel_max, 0, -1))        # Properties set in the Start method        self.version_major = 0        self.version_minor = 0        self.server_properties = {}        self.mechanisms = []        self.locales = []        # Let the transport.py module setup the actual        # socket connection to the broker.        #        self.transport = self.Transport(host, connect_timeout, ssl)        self.method_reader = MethodReader(self.transport)        self.method_writer = MethodWriter(self.transport, self.frame_max)        self.wait(allowed_methods=[            (10, 10),  # start        ])        self._x_start_ok(d, login_method, login_response, locale)        self._wait_tune_ok = True        while self._wait_tune_ok:            self.wait(allowed_methods=[                (10, 20),  # secure                (10, 30),  # tune            ])     return self._x_open(virtual_host)

        这里创建amqp层的connection具体暂时先不细究，后面我会根据网上的相应的demo做具体的分析。同时创建channel的细节也不细究，后面做具体分析。

        此时连接到RabbitMQ-server的connection创建完成，然后在connection基础上创建channel。即chan = self.transport.create_channel(self.connection)

    #/usr/lib/python2.7/site-packages/kombu/transport/pyamqp.py    def create_channel(self, connection):        return connection.channel()    #/usr/lib/python2.7/site-packages/amqp/connection.py      def channel(self, channel_id=None):        """Fetch a Channel object identified by the numeric channel_id, or        create that object if it doesn't already exist."""        try:            return self.channels[channel_id]        except KeyError:            return self.Channel(self, channel_id)#/usr/lib/python2.7/site-packages/amqp/channel.py  class Channel(AbstractChannel):    """Work with channels    The channel class provides methods for a client to establish a    virtual connection - a channel - to a server and for both peers to    operate the virtual connection thereafter.    GRAMMAR::        channel             = open-channel *use-channel close-channel        open-channel        = C:OPEN S:OPEN-OK        use-channel         = C:FLOW S:FLOW-OK                            / S:FLOW C:FLOW-OK                            / functional-class        close-channel       = C:CLOSE S:CLOSE-OK                            / S:CLOSE C:CLOSE-OK    """    def __init__(self, connection, channel_id=None, auto_decode=True):        """Create a channel bound to a connection and using the specified        numeric channel_id, and open on the server.        The 'auto_decode' parameter (defaults to True), indicates        whether the library should attempt to decode the body        of Messages to a Unicode string if there's a 'content_encoding'        property for the message.  If there's no 'content_encoding'        property, or the decode raises an Exception, the message body        is left as plain bytes.        """        if channel_id:            connection._claim_channel_id(channel_id)        else:            channel_id = connection._get_free_channel_id()        AMQP_LOGGER.debug('using channel_id: %d', channel_id)        super(Channel, self).__init__(connection, channel_id)        self.is_open = False        self.active = True  # Flow control        self.returned_messages = Queue()        self.callbacks = {}        self.cancel_callbacks = {}        self.auto_decode = auto_decode        self.events = defaultdict(set)        self.no_ack_consumers = set()        # set first time basic_publish_confirm is called        # and publisher confirms are enabled for this channel.        self._confirm_selected = False        if self.connection.confirm_publish:            self.basic_publish = self.basic_publish_confirm        self._x_open()

        此时完成了组件到RabbitMQ-server的connection和channel的创建，我们继续回到kombu层的autoretry方法中的__call__函数，它执行oslo_messaging层ensure方法传递下来的fun方法(execute_method)，其返回fun方法执行的结果以及channel的信息。在执行execute_method方法时，首先需将oslo_messaging层的self.channel更新为最新创建的amqp层的channel，然后执行method方法，由于我们这里传递给oslo_messaging层的ensure方法的method方法为method=lambda: True，所以执行该方法的结果为True。

        此时oslo_messaging层的创建connection对象中的self.ensure_connection()方法的代码流程分析完成。

        总结：本文主要分析在oslo_messaging层创建connection对象，在创建connection对象时，主要分为两部分:

        1．解析OpenStack层下传的参数，如url，通过解析这些参数，然后作为创建kombu层的connection对象的参数。

        2. 执行self.ensure_connection()方法通过调用kombu层的方法来创建oslo_messaging层到amqp层的connection和channel。

        后面的章节我们将分析在创建的connection连接上创建consumer的代码流程。