nova boot代码流程分析(五)：VM启动从neutron-dhcp-agent获取IP与MAC

1.   network和subnet创建代码流程

[root@jun ~(keystone_user1)]# neutron net-create demo-net

[root@jun ~(keystone_user1)]# neutron subnet-create  demo-net 1.1.1.0/24 --name demo-subnet --gateway 1.1.1.1 --enable_dhcp true

这里，我们主要分析上面两个命令的代码流程，我们关注的重点是neutron-server如何下发的命令到neutron-dhcp-agent去创建相应的network的代码流程。

首先neutronclient发送HTTP请求给neutron-server，去执行下面的create函数。

#/neutron/api/v2/base.py:Controller    def create(self, request, body=None, **kwargs):        """Creates a new instance of the requested entity."""        parent_id = kwargs.get(self._parent_id_name)        self._notifier.info(request.context,                            self._resource + '.create.start',                            body)        body = Controller.prepare_request_body(request.context, body, True,                                               self._resource, self._attr_info,                                               allow_bulk=self._allow_bulk)        action = self._plugin_handlers[self.CREATE]        ... ... ...        def notify(create_result):            notifier_method = self._resource + '.create.end'            self._notifier.info(request.context,                                notifier_method,                                create_result)            self._send_dhcp_notification(request.context,                                         create_result,                                         notifier_method)            return create_result        kwargs = {self._parent_id_name: parent_id} if parent_id else {}        if self._collection in body and self._native_bulk:            # plugin does atomic bulk create operations            obj_creator = getattr(self._plugin, "%s_bulk" % action)            objs = obj_creator(request.context, body, **kwargs)            # Use first element of list to discriminate attributes which            # should be removed because of authZ policies            fields_to_strip = self._exclude_attributes_by_policy(                request.context, objs[0])            return notify({self._collection: [self._filter_attributes(                request.context, obj, fields_to_strip=fields_to_strip)                for obj in objs]})        else:            obj_creator = getattr(self._plugin, action)            if self._collection in body:                # Emulate atomic bulk behavior                objs = self._emulate_bulk_create(obj_creator, request,                                                 body, parent_id)                return notify({self._collection: objs})            else:                kwargs.update({self._resource: body})                obj = obj_creator(request.context, **kwargs)                self._send_nova_notification(action, {},                                             {self._resource: obj})                return notify({self._resource: self._view(request.context,                                                          obj)})

这个create函数实现的功能为：

1. 通过调用core_plugin的core resource所对应的action方法(如create_network)在数据库中保存resource信息。

2. 在保存自身的信息到数据库中后，通知agent去做相应的操作。

其实对于port，network，subnet的create操作都会走到这个create函数。这里我们主要分析创建network(包括subnet)时与neutron-dhcp-agent之间的操作。

在network(或subnet)的信息保存到数据库之后，通过notify函数(create函数内部的函数)通知neutron-dhcp-agent做相应的操作。

network和subnet发送的给neutron-dhcp-agent的方法(notifier_method)分别为network.create.end和subnet.create.end

#/neutron/api/v2/base.py:Controller    def _send_dhcp_notification(self, context, data, methodname):        if cfg.CONF.dhcp_agent_notification:            if self._collection in data:                for body in data[self._collection]:                    item = {self._resource: body}                    self._dhcp_agent_notifier.notify(context, item, methodname)            else:                self._dhcp_agent_notifier.notify(context, data, methodname)

dhcp_agent_notification为/etc/neuton/neutron.conf配置文件中的参数。

# Allow sending resource operation notification to DHCP agent

# dhcp_agent_notification = True

dhcp_agent_notification = True

 

#/neutron/api/rpc/agentnotifiers/dhcp_rpc_agent_api.py:DhcpAgentNotifyAPI    def notify(self, context, data, method_name):        # data is {'key' : 'value'} with only one key        if method_name not in self.VALID_METHOD_NAMES:            return        obj_type = data.keys()[0]        if obj_type not in self.VALID_RESOURCES:            return        obj_value = data[obj_type]        network_id = None        if obj_type == 'network' and 'id' in obj_value:            network_id = obj_value['id']        elif obj_type in ['port', 'subnet'] and 'network_id' in obj_value:            network_id = obj_value['network_id']        if not network_id:            return        method_name = method_name.replace(".", "_")        if method_name.endswith("_delete_end"):            if 'id' in obj_value:                self._notify_agents(context, method_name,                                    {obj_type + '_id': obj_value['id']},                                    network_id)        else:            self._notify_agents(context, method_name, data, network_id)#/neutron/api/rpc/agentnotifiers/dhcp_rpc_agent_api.py:DhcpAgentNotifyAPI    def _notify_agents(self, context, method, payload, network_id):        """Notify all the agents that are hosting the network."""        # fanout is required as we do not know who is "listening"        no_agents = not utils.is_extension_supported(            self.plugin, constants.DHCP_AGENT_SCHEDULER_EXT_ALIAS)        fanout_required = method == 'network_delete_end' or no_agents        # we do nothing on network creation because we want to give the        # admin the chance to associate an agent to the network manually        cast_required = method != 'network_create_end'        if fanout_required:            self._fanout_message(context, method, payload)        elif cast_required:            admin_ctx = (context if context.is_admin else context.elevated())            network = self.plugin.get_network(admin_ctx, network_id)            agents = self.plugin.get_dhcp_agents_hosting_networks(                context, [network_id])            # schedule the network first, if needed            schedule_required = (                method == 'subnet_create_end' or                method == 'port_create_end' and                not self._is_reserved_dhcp_port(payload['port']))            if schedule_required:                agents = self._schedule_network(admin_ctx, network, agents)            enabled_agents = self._get_enabled_agents(                context, network, agents, method, payload)            for agent in enabled_agents:                self._cast_message(                    context, method, payload, agent.host, agent.topic)

在_notify_agents函数可以看出，由于创建network时，fanout_required和cast_required的值为False，所以创建network时，并不会通过neutron-dhcp-agent做什么操作。而创建port或subnet时，则会通过rpc方式执行neutron-dhcp-agent的方法，从而做一些操作。所以下面我们主要分析在创建subnet时，neutron-dhcp-agent做了哪些操作。

#/neutron/api/rpc/agentnotifiers/dhcp_rpc_agent_api.py:DhcpAgentNotifyAPI    def _cast_message(self, context, method, payload, host,                      topic=topics.DHCP_AGENT):        """Cast the payload to the dhcp agent running on the host."""        cctxt = self.client.prepare(topic=topic, server=host)        cctxt.cast(context, method, payload=payload)#/neutron/agent/dhcp/agent.py:DhcpAgent    @utils.synchronized('dhcp-agent')    def subnet_update_end(self, context, payload):        """Handle the subnet.update.end notification event."""        network_id = payload['subnet']['network_id']        self.refresh_dhcp_helper(network_id)    # Use the update handler for the subnet create event.    subnet_create_end = subnet_update_end

看到/neutron/agent/dhcp/agent.py:DhcpAgent类是不是很熟悉，这正是上一篇文章中分析的在neutron-dhcp-agent服务启动时，创建的DhcpAgent对象。

#/neutron/agent/dhcp/agent.py:DhcpAgent    def refresh_dhcp_helper(self, network_id):        """Refresh or disable DHCP for a network depending on the current state        of the network.        """        old_network = self.cache.get_network_by_id(network_id)        if not old_network:            # DHCP current not running for network.            return self.enable_dhcp_helper(network_id)        network = self.safe_get_network_info(network_id)        if not network:            return        old_cidrs = set(s.cidr for s in old_network.subnets if s.enable_dhcp)        new_cidrs = set(s.cidr for s in network.subnets if s.enable_dhcp)        if new_cidrs and old_cidrs == new_cidrs:            self.call_driver('reload_allocations', network)            self.cache.put(network)        elif new_cidrs:            if self.call_driver('restart', network):                self.cache.put(network)        else:            self.disable_dhcp_helper(network.id)

refresh_dhcp_helper函数将对比self.cache中存放的network信息与新创建的network信息，通过对比结果做相应操作。

1. 如果self.cache中没有新创建的network信息，则调用enable_dhcp_helper函数，该函数最终调用driver(dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq)的enable函数。

2. 如果self.cache中有新创建的network信息，则判断old network与new network的cidr是否相等，根据结果判断调用driver中的reload_allocations函数还是restart函数，又或者是disable函数。

对于执行neutron.agent.linux.dhcp.Dnsmasq driver中的相关函数(enable, reload_allocations, restart,disable)，我们在这里本篇文章就不分析了，具体查看《neutron-dhcp-agent服务启动流程》。

PS：在《neutron-dhcp-agent服务启动流程》文章中，我们也有分析，如果创建的subnet并没有enable dhcp，则neutron-dhcp-agent不会为该network创建namespace，创建device以及创建dnsmasq进程。具体代码如下。

#/neutron/agent/dhcp/agent.py:DhcpAgent    def configure_dhcp_for_network(self, network):        if not network.admin_state_up:            return        enable_metadata = self.dhcp_driver_cls.should_enable_metadata(                self.conf, network)        dhcp_network_enabled = False        for subnet in network.subnets:            if subnet.enable_dhcp:                if self.call_driver('enable', network):                    dhcp_network_enabled = True                    self.cache.put(network)                break        if enable_metadata and dhcp_network_enabled:            for subnet in network.subnets:                if subnet.ip_version == 4 and subnet.enable_dhcp:                    self.enable_isolated_metadata_proxy(network)                    break

小结一下：

1. 创建network，subnet和port时，首先调用core_plugin提供的函数向数据库写入自身的信息。

2. 写完数据库之后，subnet和port通过rpc方式，执行neutron-dhcp-agent启动时创建的DhcpAgent对象提供的函数(这里创建network时，并不会执行neutron-dhcp-agent所提供的函数方法)。

3. neutron-dhcp-agent对创建subnet的操作可以参考《neutron-dhcp-agent服务启动流程》文章。

对于neutron-dhcp-agent对创建VM时所创建的port的操作我们在下一小节分析。

2. VM启动时从neutron-dhcp-agent获取IP与MAC的代码流程分析

2.1 更新dnsmasq配置文件信息

接到《nova boot代码流程分析(三)：nova与neutron的plugin交互》的创建port的代码流程分析，因为在那篇文章中，主要关注的如何将创建的IP和MAC保存到数据库，并未关注VM实际如何从neutron-dhcp-agent获取IP和MAC的代码流程，所以在这篇文章中，我们将关注VM实际获取IP和MAC的代码流程。

#/nova/network/neutronv2/api.py:API    def _create_port(self, port_client, instance, network_id, port_req_body,                     fixed_ip=None, security_group_ids=None,                     available_macs=None, dhcp_opts=None):        """Attempts to create a port for the instance on the given network.        :param port_client: The client to use to create the port.        :param instance: Create the port for the given instance.        :param network_id: Create the port on the given network.        :param port_req_body: Pre-populated port request. Should have the            device_id, device_owner, and any required neutron extension values.        :param fixed_ip: Optional fixed IP to use from the given network.        :param security_group_ids: Optional list of security group IDs to            apply to the port.        :param available_macs: Optional set of available MAC addresses,            from which one will be used at random.        :param dhcp_opts: Optional DHCP options.        :returns: ID of the created port.        :raises PortLimitExceeded: If neutron fails with an OverQuota error.        :raises NoMoreFixedIps: If neutron fails with            IpAddressGenerationFailure error.        """        try:            if fixed_ip:                port_req_body['port']['fixed_ips'] = [                    {'ip_address': str(fixed_ip)}]            port_req_body['port']['network_id'] = network_id            port_req_body['port']['admin_state_up'] = True            port_req_body['port']['tenant_id'] = instance.project_id            if security_group_ids:                port_req_body['port']['security_groups'] = security_group_ids            if available_macs is not None:                if not available_macs:                    raise exception.PortNotFree(                        instance=instance.uuid)                mac_address = available_macs.pop()                port_req_body['port']['mac_address'] = mac_address            if dhcp_opts is not None:                port_req_body['port']['extra_dhcp_opts'] = dhcp_opts            port_id = port_client.create_port(port_req_body)['port']['id']            LOG.debug('Successfully created port: %s', port_id,                      instance=instance)            return port_id        except neutron_client_exc.IpAddressInUseClient:            LOG.warning(_LW('Neutron error: Fixed IP %s is '                            'already in use.'), fixed_ip)            msg = _("Fixed IP %s is already in use.") % fixed_ip            raise exception.FixedIpAlreadyInUse(message=msg)        except neutron_client_exc.OverQuotaClient:            LOG.warning(_LW(                'Neutron error: Port quota exceeded in tenant: %s'),                port_req_body['port']['tenant_id'], instance=instance)            raise exception.PortLimitExceeded()        except neutron_client_exc.IpAddressGenerationFailureClient:            LOG.warning(_LW('Neutron error: No more fixed IPs in network: %s'),                        network_id, instance=instance)            raise exception.NoMoreFixedIps(net=network_id)        except neutron_client_exc.MacAddressInUseClient:            LOG.warning(_LW('Neutron error: MAC address %(mac)s is already '                            'in use on network %(network)s.') %                        {'mac': mac_address, 'network': network_id},                        instance=instance)            raise exception.PortInUse(port_id=mac_address)        except neutron_client_exc.NeutronClientException:            with excutils.save_and_reraise_exception():                LOG.exception(_LE('Neutron error creating port on network %s'),                              network_id, instance=instance)

这部分代码是从《nova boot代码流程分析(三)：nova与neutron的plugin交互》文章中提取出来的代码，该代码是nova-compute通过HTTP请求在neutron中创建port信息。然后到达我们第一小节分析的create函数。

#/neutron/api/v2/base.py:Controller    def create(self, request, body=None, **kwargs):        """Creates a new instance of the requested entity."""        parent_id = kwargs.get(self._parent_id_name)        self._notifier.info(request.context,                            self._resource + '.create.start',                            body)        body = Controller.prepare_request_body(request.context, body, True,                                               self._resource, self._attr_info,                                               allow_bulk=self._allow_bulk)        action = self._plugin_handlers[self.CREATE]        ... ... ...        def notify(create_result):            notifier_method = self._resource + '.create.end'            self._notifier.info(request.context,                                notifier_method,                                create_result)            self._send_dhcp_notification(request.context,                                         create_result,                                         notifier_method)            return create_result        kwargs = {self._parent_id_name: parent_id} if parent_id else {}        if self._collection in body and self._native_bulk:            # plugin does atomic bulk create operations            obj_creator = getattr(self._plugin, "%s_bulk" % action)            objs = obj_creator(request.context, body, **kwargs)            # Use first element of list to discriminate attributes which            # should be removed because of authZ policies            fields_to_strip = self._exclude_attributes_by_policy(                request.context, objs[0])            return notify({self._collection: [self._filter_attributes(                request.context, obj, fields_to_strip=fields_to_strip)                for obj in objs]})        else:            obj_creator = getattr(self._plugin, action)            if self._collection in body:                # Emulate atomic bulk behavior                objs = self._emulate_bulk_create(obj_creator, request,                                                 body, parent_id)                return notify({self._collection: objs})            else:                kwargs.update({self._resource: body})                obj = obj_creator(request.context, **kwargs)                self._send_nova_notification(action, {},                                             {self._resource: obj})                return notify({self._resource: self._view(request.context,                                                          obj)})

create函数首先创建IP和MAC信息保存到neutron数据库中，具体参考《nova boot代码流程分析(三)：nova与neutron的plugin交互》文章。然后调用notify函数发送port.create.end给neutron-dhcp-agent。最终neutron-dhcp-agent将调用port_create_end函数。

#/neutron/agent/dhcp/agent.py:DhcpAgent    @utils.synchronized('dhcp-agent')    def port_update_end(self, context, payload):        """Handle the port.update.end notification event."""        updated_port = dhcp.DictModel(payload['port'])        network = self.cache.get_network_by_id(updated_port.network_id)        if network:            driver_action = 'reload_allocations'            if self._is_port_on_this_agent(updated_port):                orig = self.cache.get_port_by_id(updated_port['id'])                # assume IP change if not in cache                old_ips = {i['ip_address'] for i in orig['fixed_ips'] or []}                new_ips = {i['ip_address'] for i in updated_port['fixed_ips']}                if old_ips != new_ips:                    driver_action = 'restart'            self.cache.put_port(updated_port)            self.call_driver(driver_action, network)    def _is_port_on_this_agent(self, port):        thishost = utils.get_dhcp_agent_device_id(            port['network_id'], self.conf.host)        return port['device_id'] == thishost    # Use the update handler for the port create event.    port_create_end = port_update_end

具体如何到达该代码流程，可以参考第1小节创建subnet的代码流程。这里将port_update_end函数赋给port_create_end函数，所以最终是调用port_update_end函数。port_update_end函数将比较self.cache中保存的ip信息与最新数据库中相对应的network中对应的ip信息(从neutron-server传递下来的，即payload)。正常流程下，因为我们创建VM会创建新的port信息到数据库中，而neutron-dhcp-agent的self.cache中还保存着上一次更新的network信息，所以导致数据库中port信息与self.cache中保存的port信息不一致，因此将更新self.cache中的port信息，且执行reload_allocations函数，重新更新dnsmasq的配置文件。

#/neutron/agent/linux/dhcp.py:Dnsmasq    def reload_allocations(self):        """Rebuild the dnsmasq config and signal the dnsmasq to reload."""        # If all subnets turn off dhcp, kill the process.        if not self._enable_dhcp():            self.disable()            LOG.debug('Killing dnsmasq for network since all subnets have '                      'turned off DHCP: %s', self.network.id)            return        self._release_unused_leases()        self._spawn_or_reload_process(reload_with_HUP=True)        LOG.debug('Reloading allocations for network: %s', self.network.id)        self.device_manager.update(self.network, self.interface_name)

调用_release_unused_leases函数release在leases文件(/var/lib/neutron/dhcp/43c0e274-28e3-482e-a32b-d783980fc3ed/leases)中未被使用的ip和mac。然后再重新加载dnsmasq进程所需的配置文件，由于加载所需的配置文件我们在《neutron-dhcp-agent服务启动流程》中有分析，所以我们这里主要分析release在leases文件中未被使用的ip和mac。

#/neutron/agent/dhcp/agent.py:DhcpAgent    def _release_unused_leases(self):        filename = self.get_conf_file_name('host')        old_leases = self._read_hosts_file_leases(filename)        new_leases = set()        for port in self.network.ports:            for alloc in port.fixed_ips:                new_leases.add((alloc.ip_address, port.mac_address))        for ip, mac in old_leases - new_leases:            self._release_lease(mac, ip)#/neutron/agent/dhcp/agent.py:DhcpAgent    def _release_lease(self, mac_address, ip):        """Release a DHCP lease."""        cmd = ['dhcp_release', self.interface_name, ip, mac_address]        ip_wrapper = ip_lib.IPWrapper(namespace=self.network.namespace)        ip_wrapper.netns.execute(cmd, run_as_root=True)

_release_unused_leases函数读取host文件(/var/lib/neutron/dhcp/43c0e274-28e3-482e-a32b-d783980fc3ed/host)中的被dnsmasq分配的ip和mac信息，将这些信息与数据库中network的ip和mac信息作对比，如果dnsmasq分配的ip和mac信息并未在数据库中，说明此ip和mac信息未被使用，所以调用_release_lease函数对其进行释放。

_release_lease函数是通过dhcp_release命令向dnsmasq进程监听接口ns-xxx发送DHCPRELEASE请求来释放未被使用的ip和mac信息。即删除leases文件未被使用的ip和mac信息。下面是一个测试用例。

开始leases文件信息：

[root@nova 43c0e274-28e3-482e-a32b-d783980fc3ed]# cat leases

1464597255 fa:16:3e:da:42:50 1.1.1.2 host-1-1-1-2 *

1464597255 fa:16:3e:d0:eb:87 1.1.1.9 host-1-1-1-9 *

1464597255 fa:16:3e:d1:d7:72 1.1.1.1 host-1-1-1-1 *

假设我们将release红色部分的ip和mac信息，则执行以下操作。

[root@nova ~]# ip netns exec qdhcp-43c0e274-28e3-482e-a32b-d783980fc3ed dhcp_release ns-f7620da4-39 1.1.1.9 fa:16:3e:d0:eb:87

在/var/log/messages文件中，可以看到如下日志。

May 29 04:38:09 nova dnsmasq-dhcp[3759]: DHCPRELEASE(ns-f7620da4-39) 1.1.1.9 fa:16:3e:d0:eb:87

执行dhcp_release命令后的leases文件信息：

[root@nova 43c0e274-28e3-482e-a32b-d783980fc3ed]# cat leases

1464597255 fa:16:3e:da:42:50 1.1.1.2 host-1-1-1-2 *

1464597255 fa:16:3e:d1:d7:72 1.1.1.1 host-1-1-1-1 *

可以看出，ip为1.1.1.9相关的信息被删除了。

这是_release_unused_leases函数的功能，而reload_allocations函数剩下的代码则是重新配置dnsmasq的配置文件，假设我们创建VM在数据库中分配的ip为1.1.1.10，那么在host文件中将把该ip及相对应的mac写入进去，待VM启动且发送dhcp discover请求时将该IP和mac分配于它。

2.2 分配ip和mac给VM

在创建VM开始时，首先为VM在数据库中创建port信息，创建port信息完成后，通知neutron-dhcp-agent更新dnsmasq的配置文件(包括host和addn_hosts文件)信息。如下

[root@nova 43c0e274-28e3-482e-a32b-d783980fc3ed]# cat addn_hosts

1.1.1.1 host-1-1-1-1.openstacklocal host-1-1-1-1

1.1.1.2 host-1-1-1-2.openstacklocal host-1-1-1-2

1.1.1.10        host-1-1-1-10.openstacklocal host-1-1-1-10

 [root@nova 43c0e274-28e3-482e-a32b-d783980fc3ed]# cat host

fa:16:3e:d1:d7:72,host-1-1-1-1.openstacklocal,1.1.1.1

fa:16:3e:da:42:50,host-1-1-1-2.openstacklocal,1.1.1.2

fa:16:3e:3c:a3:3e,host-1-1-1-10.openstacklocal,1.1.1.10

这是在VM启动之前，就已经保存到dnsmasq的配置文件中的信息。

等待VM启动时，发送dhcp discover请求。

VM启动的有关dhcp的log如下(这是dhcp client端)。

Starting network...

udhcpc (v1.20.1) started

Sending discover...

Sending select for 1.1.1.10...

Lease of 1.1.1.10 obtained, lease time 86400

cirros-ds 'net' up at 6.03

/var/log/messages的有关dhcp的log如下(这是dhcp server端)。

May 29 05:05:34 nova dnsmasq-dhcp[4388]: DHCPDISCOVER(ns-f7620da4-39) fa:16:3e:3c:a3:3e

May 29 05:05:34 nova dnsmasq-dhcp[4388]: DHCPOFFER(ns-f7620da4-39) 1.1.1.10 fa:16:3e:3c:a3:3e

May 29 05:05:34 nova dnsmasq-dhcp[4388]: DHCPREQUEST(ns-f7620da4-39) 1.1.1.10 fa:16:3e:3c:a3:3e

May 29 05:05:34 nova dnsmasq-dhcp[4388]: DHCPACK(ns-f7620da4-39) 1.1.1.10 fa:16:3e:3c:a3:3e host-1-1-1-10

同时neutron-dhcp-agent更新dnsmasq的leases文件信息。

[root@nova 43c0e274-28e3-482e-a32b-d783980fc3ed]# cat leases

1464599134 fa:16:3e:3c:a3:3e 1.1.1.10 host-1-1-1-10 01:fa:16:3e:3c:a3:3e

1464598886 fa:16:3e:da:42:50 1.1.1.2 host-1-1-1-2 *

1464598886 fa:16:3e:d1:d7:72 1.1.1.1 host-1-1-1-1 *

Dhcp client与dhcp server的交互过程如下图所示。

这里创建VM时，从neutron-dhcp-agent获取ip和mac的代码流程便分析完成。总结一下：

1. 创建VM时，nova-compute与neutron的plugin交互，在neutron的数据库中创建VM所需的port信息。

2. neutron数据库中的port信息创建完成后，通知neutron-dhcp-agent去执行port_create_end函数。该函数将数据库中的port中的ip和mac信息加载到dnsmasq所需的配置文件中(包括host和addn_hosts文件)。

3. 在VM启动时，广播dhcp discover请求，当dnsmasq进程的监听接口ns-xxx监听到这种请求时，dnsmasq进程将根据配置文件(host和leases文件)中的内容去判定是否有未分配的ip和mac为请求者进行提供。

4. 最终VM便真实的获取到与保存在数据库中的ip和mac信息。neutron-dhcp-agent只是将所创建VM的ip和mac信息从数据库中获取到自己的配置文件中，然后等到VM启动时，为它提供。因此neutron-dhcp-agent相当于在VM和数据库之间起了个中间桥梁的作用。