OpenStack Liberty High Availability 概述和指导

OpenStack Liberty 高可用性概述和参考-第一部分

这篇文章由Avishay Traeger 和 Shimshom Zimmerman编写。 
      OpenStack设计目的是在商用硬件上运行,但是没有自身的机制处理硬件和软件故障。OpenStack成功部署的一个重要组成部分是创建一个高可用性(HA)软件架构体系。这样的架构体系的首要组件是支持高可用OpenStack服务，它们分布在不同的硬件节点；因此，如果一个硬件节点故障时OpenStack集群还可以持续提供服务功能。这样的架构体系的第二个重要组件是有能力在一个新的计算节点上重启一个虚拟机实例当一个计算节点发生故障。OpenStack不能保证独立客户实例达到99.99%可用，但是内部OpenStack服务这个目标是可以达到的。OpenStack依赖于其他组件，如消息队列、数据库和存储；为了达到高可用OpenStack所有这些组件都需要HA 。 
      本文简要概述最常见的方法实现OpenStack高可用在OpenStack的当前发布版本-OpenStack Liberty。这里有多个OpenStack高可用（HA）架构，你可以根据你的需求选择合适的架构。本文将介绍两个架构：第一个主要基于Corosync/Pacemaker和第二个基于Consul。

OpenStack Liberty HA基础知识

      某些OpenStack服务是无状态的。无状态服务是一个不存储任何交互请求上下文在本地。任何上下文或信息相关不仅仅是单个请求在当前被处理存储全局，例如在数据库中。这些属性意味着任何请求的结果不依赖于任何其他请求的结果。这使这些服务的部署运行在“active/active”方式中（多个实例的运行在任何时候,所有的服务请求）。 
      无状态服务意味着拥有多个服务实例运行在不同的节点上，监控这些实例可用性和转发请求到可用实例。如果所选择的实例无法处理请求，它会重试相同的请求使用不同的实例。OpenStack无状态服务有：nova-api, nova-conductor, nova-scheduler, cinder-api, cinder-scheduler, glance-api, keystone-api,和 neutron-api。 
      OpenStack同样也使用有状态服务。相对于无状态服务，它存储交互请求上下文到本地；因此，在任何时候只有一个实例可能是活跃。此外，OpenStack还依赖一些服务，如数据库和消息队列为有状态服务。运行多个有状态服务实例是高可用架构的一部分。对于OpenStack服务，这些意味着可以运行在“active/passive”方式中（在任何时候只有一个实例服务于请求）。对于数据库和消息队列，我们一般选择使用内部状态共享和所有实例一致性。

授权服务

HA HTTP终端HAProxy

      OpenStack服务为HTTP 提供通过RESTful APIs。最常见的方法是为OpenStack服务满足HA模式是通过外部工具提供一个虚拟IP(VIP)管理的组合，如Pacemaker和通过HAProxy为HTTP终端提供负载均衡。为了避免单点故障（SPOF），集群在不同的节点上应该包含多个HAProxy服务实例。当一个服务使用者发送一个请求到REST API，这个请求会发送到激活的控制节点（当前持有VIP）；这个请求会被活跃的HAProxy实例处理并转发到相应的服务。 
      通常只有一个HAProxy实例是活跃的，另一个实例是处于待机模式(active/passive配置)。OpenStack服务可以配置为active/active或active/passive模式。OpenStack无状态服务除了HAProxy通常不需要其他工具可以工作在active/active或active/passive模式下。OpenStack有状态服务只能在active/passive模式下运行。OpenStack Dashboard服务（Horizon）属于粘性会话需要HAProxy因此在active/active模式下工作。

集群资源管理Pacemaker

      Pacemaker是主要推荐工具管理HAProxy实例、数据库、消息队列和OpenStack服务。Pacemaker提供其控制下服务的资源管理器。Pacemaker可以通过使用资源代理启动和停止服务。资源代理监控一个指定资源（服务）和采取相关措施如果这个资源没有运行或反馈。例如，资源代理可以转移虚拟IP从一个故障节点到一个可用节点，这是一个实现的虚拟同步协议。这个协议通常提供集群节点间可靠连接。支持基于选举的HA，Corosync/Pacemaker通常要求奇数（三个或更多）服务实例。 
      

服务发现和HA的Consul

      另一个工具可以帮助在OpenStack集群实现HA是Consul。Consul是一个开源服务发现工具，使用分布式键值对存储。Consul还没有广泛用于OpenStack部署，需要一些工作被集成在一个OpenStack集群。
      Consul在集群中运行一个守护进程在所有节点上（一些是服务模式，一些是代理模式），利用Raft一致算法和底层的Serf gossip协议。 
      在集群节点上Consul 代理可以配置监控在本地运行的服务。这些代理发布这些监控服务的健康状态到整个Consul集群。此外，Consul在每个节点上运行本地DNS服务，它允许由名称解析以及各种集群服务的负责均衡。例如，如果我们使用数据库的Galera 集群HA（见下文），有三个Galera实例运行在集群中，Consul将通过一个可配置的健康检查脚本不断监测它们的健康。任何服务想要访问数据库可以通过名称galera.service.consul访问，这将本地解决集群中Galera实例的健康问题。 
      运行运行一个集群管理服务可以很简单监测服务的健康和集群节点，可以在节点间进行迁移如果有需要。一旦Consul监测启动，这里不需要主动通知这些服务的客户端-这是由DNS服务器自动完成。 
      集群的分布式键值存储允许方便的存储配置参数。此外,它有订阅机制在注册表键/值的变化,和整个集群范围的锁。 

Learn everything you need to know about OpenStack Deployment Tools.

共享服务

MySQL/MariaDB 使用Galera

      MySQL/MariaDB的Galera集群为OpenStack数据库提供了高可用性。它支持同步复制和主/主多主机拓扑结构。Galera集群需要一个奇数实例运行（三个或更多）。OpenStack支持多节点写入Galera节点还没有准备好进行生产，所以推荐的方法是配置HAProxy只使用一个激活的Galera节点，虽然Galera内部支持主/主拓扑结构和写入到任何集群节点。在这样的配置中，只有一个节点是活动的，其余的节点是standby节点。Galera使用同步复制并确保每个集群节点是相同的。如果任何节点的状态降得远远落后与Galera缓存，整个副本分发到该节点。这将导致主切换到捐赠者角色，运行进行不同步的节点迎头赶上。

MongoDB

      OpenStack 计量服务可以使用MongoDB存储它的数据。MongoDB可以配置为HA模式通过HAProxt和Corosync/Pacemaker。

RabbitMQ 镜像队列

RabbitMQ提供active/active HA使用镜像队列。这不是一个理想的解决方案，因为即使镜像队列消息仍有可能丢失。无论如何，RabbitMQ仍然是一个事实上的标准OpenStack的消息队列。OpenStack服务使用Oslo消息库为RabbitMQ连接而不需要HAProxy的HA模式。 

Learn more about the benefits and challenges of OpenStack

HA 技巧和窍门

• Compute 节点(Nova 服务)与控制节点和共享服务之间通信是通过消息队列和数据库。Compute节点应该使用VIP和HAProxy去连接控制节点上的服务终端。
• 卷存储，Cinder管理。必须高可用。默认Cinder后端创建逻辑卷使用LVM在一个节点上的一个或多个磁盘，通过iSCSI出口。因为存储驻留在单个节点上,它不适应节点故障。Cinder支持管理卷在大多数存储设备,以及几个开源软件选项。一个流行开源选项是Ceph，内部支持HA。Ceph需要多个Ceph监控实例形成法定人数。此外，运行Ceph OSD的数量取决于所需的复制因子和集群能力。
• Swift API依赖于控制器节点上相同的HAProxy设置VIP其他RESTful API。用于生产环境需要专用的节点,至少有两个Swift代理和至少三个Swift存储。
• 可以开始尝试OpenStack在单个节点上的服务配置为高可用性,并添加更多的节点迁移到生产之前。所有的服务描述支持这个,可以添加额外的节点没有停机时间。
• 如果RESTful API端点安全通过SSL,您应该在所有节点有相同的SSL证书。

---

原文：http://www.stratoscale.com/blog/openstack/openstack-liberty-high-availability-overview-and-guidelines-part-1/

OpenStack Liberty High Availability 概述和指导-第二部分

 本文由 Avishay Trager 和 Shimshon Zimmerman编写。 
      在第一部分我们主要描述了OpenStack Liberty 高可用性的基本知识。在本文我们聚焦于HA模式和OpenStack Liberty版本的OpenStack服务。

专用硬件节点

      我们将开始与一个配置即服务以隔离方式运行在专用节点。隔离配置的两个主要好处是服务之间的资源冲突最小化，节点故障对系统的影响有限；因此他们更容易管理。缺点是过度采购和操作开销额外节点不充分的利用。在这里,我们描述了不同的节点类型和每个OpenStack最低推荐节点数:

OpenStack director/master 节点

      这节点是可选的，但是强烈建议使用一个专用的服务器来存储主配置，编排OpenStack安装和管理安装的云平台。许多OpenStack部署工具要求一个专用的服务器作为OpenStack director/master节点。

Controller节点

      一个高可用OpenStack部署应该拥有至少三个controller节点。controller节点主要负责运行服务如下：

• 所有OpenStack服务除nova-compute外负责运行用户工作负载(VMs或容器)。请根据OpenStack High Availability Guide 安装和配置OpenStack服务。
• 数据库服务，如MariaDB/MySQL和Galera(法定人数所需奇数节点)。
• 消息队列服务，如RabbitMQ。
• ntp或chrony-OpenStack服务和其他外部组件，如Ceph，需要高度时间精确，因此在所有节点之间应该时刻时间同步。 
        部署超过三个控制器节点和在它们之间传播服务是可能的,但是三个通常就足以处理数百个计算节点。

Compute节点

      Compute节点用户工作负载运行,通过nova-compute实例管理它们。至少拥有两个计算节点才允许进行迁移/疏散工作负载从一个故障节点到一个活跃节点。

Storage 节点

      Storage节点运行部署存储软件包含基于卷软件、镜像和对象存储如Ceph或Swift(而不是存储设备)。Ceph要求至少三个Ceph监控实例安装在controller节点，并至少需要2-3个专用节点安装Ceph OSD实例(至少尽可能多的节点配置副本的数量)。Swift我们推荐至少两个Swift代理实例和至少三个专用Swift存储节点。Swift代理实例可以安装在controller节点，但可能会消耗大量的CPU负载和网络资源。

其他节点

      你或许想安装测量服务和它的数据库在专用服务器上，因为它可以很好用来监控现有的集群。另一个例子是一个专用服务器(或服务器)OpenStack数据库。

融合部署

      如上所述，专用节点运行指定服务是非常简单的因为可以减少资源之间使用竞争和减少故障节点的影响，但是它导致节点无法充分利用。使用专用的节点模型为基础,我们可以考虑给节点多个角色。 
      如果总隔离的一个极端,总会聚在另一端。这里，控制服务或许运行在任意节点，每个节点运行用户工作负载和存储服务。这种类型的部署是最具成本效益的,但最难以管理。必须特别注意,这样每个服务系统中得到足够的资源以防止退化。例如Ceph在某些情况下可以使用大量的RAM和CPU，使他们无法工作负载。我们建议保留某种形式的资源，如Linux cgroups当追求这样的架构时。 

Avoid common Pitfalls with OpenStack, click here

OpenStack服务的HA

Compute(Nova)

实例高可用

      从Liberty开始，Nova允许从检测到故障节点后转移客户机。“mark host down”方法添加到了Nova API。这种方法允许外部工具像Pacemaker或Consul通知Nova主机关闭的快于一个标准的周期性任务检测它。之后，一个客户机开始转移。

Storage(Cinder)

卷迁移API

      卷迁移的API允许移动两个后端之间的卷及其数据的方式对用户是透明的和工作负载。有两个层次的存储支持这个特性：“generic”和“specialized”。随着Liberty发布，generic方法在任何存储后端都可以工作。在Liberty之间，支持受限于iSCSI和FC后端。注意generic方法只支持Nova的libvirt驱动。在这种方法中，被复制的数据通过附加卷的VM进行流动，或一个Cinder节点不附加卷。某些卷后端支持specialized流，在某些情况下,存储后端处理数据迁移(一般如果两个池之间的迁移是在同一存储后端)。

卷复制API

      卷复制的API允许使用OpenStack Cinder管理存储后端块级别复制功能。尽管在Liberty中取得了一些进步支持卷复制,设计和实现还有待最后确认。

网络(Neutron)

Neutron HA模式

OpenStack网络支持以下HA模式：

• ML2 插件和Open vSwitch(OVS)的虚拟路由冗余协议(VRRP)
• ML2 插件和Linux bridges的虚拟路由冗余协议(VRRP)
• 分布式虚拟路由(DVR)支持扁平和VLAN外部网络，VXLAN和GRE项目网络（从Juno版开始）和VLAN项目网络（从Kilo版开始）

分布式DHCP代理

      Neutron DHCP代理内在支持HA，要求达到高可用需要每个网络超过一个DHCP代理。

分布式元数据代理

      Neutron元数据代理不支持内建HA。OpenStack Liberty之前，使元数据服务高可用你不得不使用元数据代理在active/passive模式下。在Liberty中可以运行分布式元数据代理在active/active模式下。

新LBaaS实现

      新的LBaaS实现现在基于operator-grade可伸缩负载均衡Octavia并且不再是实验。

编排(Heat)

每个资源状态的持久性模式

      在OpenStack Liberty版本中，引入了一个新的可选模式持续每个资源状态状态在堆栈更新过程中。这提高了容错性和Heat引擎中可以从故障中恢复。有一个计划在即将到来的OpenStack发布中实现新自动恢复特性。

共享文件系统(Manila)

共享迁移

      共享迁移允许你迁移一个共享文件从一个主机池到另一个主机池并且可以在不同后端之间进行迁移。

容器(Magnum)

支持多主机Kubernetes

      多主机Kubernetes支持提供高可用Kubernetes使用Magnum超过1个主机节点数。

Kubernetes与Neutron 负载均衡

      Kubernetes现在与Neutron负载平衡器集成。

容器化服务(Kolla)

Kolla项目添加到OpenStack大租户在Liberty版本周期中。Kolla提供Docker容器OpenStack服务和Ansible playbooks。因此容器和部署工具是一个快速和可伸缩方法安装OpenStack。Kolla在30分钟内可以安装数百个高考用OpenStack Liberty集群。 
      不管你用什么方式部署OpenStack，一个高可用性(HA)软件架构上运行将对你的成功至关重要。确保您的部署通过简化流程的“开箱即用”的云解决方案。

---

原文：http://www.stratoscale.com/blog/openstack/openstack-liberty-high-availability-overview-and-guidelines-part-2/

原文：

Written by Stratoscale

 OpenStack Liberty High Availability Overview and Guidelines, Part 1

This post has been written by Avishay Traeger and Shimshom Zimmerman.

OpenStack is designed to run on commodity hardware, but has no native mechanisms for handling hardware and software failures. An important part of deploying OpenStack successfully is creating a highly available (HA) software architecture for it to run in. The first main component of such an architecture is support for highly available OpenStack services, which are distributed across hardware nodes; therefore, if one hardware node fails then the OpenStack cluster will continue to function. The second main component of a highly available architecture is the ability to re-launch a VM instance on a new compute node when one fails. OpenStack does not guarantee 99.99% availability for individual guest instances, but for internal OpenStack services this target is achievable. OpenStack relies on other components, such as message queues, database and, storage; all of these components need to be HA in order to achieve HA OpenStack.

This article gives a brief overview of the most common approaches for achieving OpenStack high availability in OpenStack’s current release – OpenStack Liberty. There is more than one HA architecture for OpenStack and you can choose the most appropriate one for your requirements. This article covers two architectures: the first one is based primarily on Corosync/Pacemaker and the second is based on Consul.

Openstack Liberty HA basics

Some of OpenStack’s services are stateless. A stateless service is one that does not store any inter-request context locally. Any context or information that is related to more than a single request that is currently being processed is stored globally, for example in a database. This property means that the result of any request does not depend on the result of any other request. This enables deployment of these services to run in an “active/active” manner (in which multiple instances are running at any point in time, all of which serve requests).

Stateless services entail having several instances of the services running on different nodes, monitoring the instances’ availability and forwarding requests to the available instances. If the selected instance fails to process the request, it retries the same request using a different instance. Examples of OpenStack stateless services are: nova-api, nova-conductor, nova-scheduler, cinder-api, cinder-scheduler, glance-api, keystone-api, and neutron-api.

OpenStack also uses stateful services. Contrary to stateless services, these do store inter-request context locally; therefore,only one instance may be active at any point in time. Additionally, some of the services that OpenStack depends on, such as databases and message queues are stateful as well. Running more than one instance of a stateful service is a part of highly available architecture. For OpenStack services, this entails either running in an “active/passive” manner (only one instance serves requests at any point in time). For databases and message queues, we will generally opt for making the internal state shared and consistent for all of the instances.

Enabling services

HAProxy for HA HTTP endpoints

OpenStack services provide RESTful APIs over HTTP. The most common approach for enabling HA mode for OpenStack services is a combination of virtual IP (VIP) management via an external tool, such as Pacemaker and load balancing for the HTTP endpoints via HAProxy. To avoid a single point of failure, the cluster should contain several instances of HAProxy on different nodes. When a service consumer makes a request  to the REST API, the request goes to the active controller node currently holding the VIP; the request is processed by an active HAProxy instance and forwarded to the corresponding service.

Usually only one HAProxy instance is active and the other instances are in a standby mode (active/passive configuration). OpenStack services can be configured in active/active or active/passive mode. OpenStack stateless services usually do not require any other tools except HAProxy and can work in active/active or active/passive modes. OpenStack stateful services may run only in active/passive mode. OpenStack Dashboard service (Horizon) requires sticky sessions to be enabled on HAProxy in order to work in active/active mode.

Pacemaker for cluster resource management

Pacemaker is the most recommended tool to manage instances of HAProxy, database, message queue and OpenStack services. Pacemaker provides resource management for the services under its control. Pacemaker can start and stop services using a resource agent. The resource agent monitors a specific resource (service) and takes action if the resource is not running or responding. For example, the resource agent can move the virtual IP from a failed node to an available one. Pacemaker can be used in a combination with Corosync. Corosync uses the Totem protocol, which is an implementation of Virtual Synchrony protocol. This protocol is used to provide reliable connectivity between cluster nodes. To support quorum-based HA, Corosync/Pacemaker usually requires an odd number (three or more) service instances.

Consul for Service Discovery and HA

Another tool which can help achieve HA in an OpenStack cluster is Consul. Consul is an open-source service discovery tool, with a distributed key-value store. Consul is still not widely used in OpenStack deployments, and requires some work to be integrated in an OpenStack cluster.

Consul runs a daemon on all nodes in the cluster (some in server mode, and some in agent mode), and utilizes the Raft consensus algorithm and the Serf gossip protocol in its underlying layers.

Consul agents on the cluster nodes can be configured to monitor the services running locally. The agents publish the health status of those monitored services to the entire consul cluster. In addition, Consul on each node runs a local DNS server, which allows a by-name resolution, and load-balancing of the various cluster services. For example, if we use Galera Cluster for database HA (see below), and have three Galera instances running in the cluster, Consul will constantly monitor their health with a configurable health-check script. Any service that wants to access the database can access galera.service.consul by name, which will be locally resolved to one of the healthy instances of Galera in the cluster.

This allows running a cluster manager service which can easily monitor the health of the services and the cluster nodes, and move them between the nodes in the cluster if needed. Once Consul monitoring is enabled, there is no need to actively notify the clients of those services that the services have moved – this is done automagically by the DNS server.

The distributed key-value store allows for convenient storage of cluster configuration parameters. In addition, it has mechanisms for subscribing on key/value changes in the registry, and cluster-wide locks.

Learn everything you need to know about OpenStack Deployment Tools  

Shared services

MySQL/MariaDB with Galera

Galera Cluster for MySQL/MariaDB provides HA for the OpenStack database. It supports synchronous replication and active/active multi-master topology. Galera Cluster requires an odd number of instances running (three or more). OpenStack support for multi-node writing to Galera nodes is not yet ready for production, so the recommended approach is to configure HAProxy to use only one active Galera node, although Galera internally supports active/active topology and writing to any cluster node. In such configuration, only one node is active, remaining nodes are standby masters. Galera uses synchronous replication and ensures that each cluster node is identical. If the status of any node falls too far behind the Galera cache, an entire replica is distributed to that node. This causes a master to switch to the donor role, allowing an out-of-sync node to catch up.

MongoDB

OpenStack Telemetry service can use MongoDB to store its data. MongoDB can be configured in HA mode with HAProxy and Corosync/Pacemaker.

RabbitMQ with mirrored queues

RabbitMQ provides active/active HA using mirrored queues. This is not an ideal solution, because even with mirrored queues messages may still be lost. However, RabbitMQ is still a standard de facto message queue for OpenStack. OpenStack services use the Oslo messaging library for the connectivity with RabbitMQ and do not require HAProxy for HA mode.

Learn more about the benefits and challenges of OpenStack

HA tips and tricks

• Compute nodes (Nova service) communicate with controller nodes and shared services such as message queue and database. Compute nodes should use VIP and HAProxy to connect to services endpoints on a controller node.
• Volume storage, managed by Cinder, must be highly available as well. The default Cinder backend creates logical volumes using LVM on one or more disks that are local to a node, and exporting them via iSCSI. Because the storage resides on a single node, it is not resilient to node failure. Cinder supports managing volumes on most storage appliances, as well as several open-source software options. A popular open-source option is Ceph, which has internal support for HA. Ceph requires multiple Ceph monitor instances to form a quorum. Additionally, the number of running Ceph OSD depends on desired replication factor and cluster capacity.
• Swift API relies on the same HAProxy setup with VIP on controller nodes as the other RESTful APIs. For the production environment you need dedicated nodes, at least two for Swift Proxy and at least three for Swift Storage.
• It is possible to begin experimenting with OpenStack on a single node with services configured to be highly available, and add more nodes before moving to production. All of the services described support this, and the additional nodes can be added with no downtime.
• If RESTful API endpoints are secured by SSL, you should have the same SSL certificates on all nodes.

In the second article, we will cover HA for specific OpenStack services in the Liberty release.

Authors: Avishay Traeger and Shimshom Zimmerman

OpenStack Liberty High Availability Overview and Guidelines, Part 2

This post has been written by Avishay Trager and Shimshon Zimmerman.

In part 1 of this article we covered the basics of OpenStack Liberty high availability. In this article we will focus on specific HA modes and OpenStack services in the OpenStack Liberty release.

Dedicated hardware nodes

We will begin with a configuration where services run in a segregated manner on dedicated nodes. The two main benefits of a segregated configuration are that resource contention between services is minimized, and node failures have a limited effect on the system; therefore they are easier to manage. The drawback is the overhead of purchasing and operating extra nodes that are not fully utilized. Here we describe the various node types and the minimum recommended number of nodes for each:

OpenStack director/master node

This node is optional, but it is highly recommended to use a dedicated server to store the master configuration, orchestrate OpenStack installation and manage the installed cloud. Also many OpenStack deployments tools require a dedicated server for OpenStack director/master node.

Controller nodes

A highly available OpenStack deployment should have at least three controller nodes. The controller nodes are responsible for running the following:

• All of the OpenStack services except nova-compute, which is responsible for running user workloads (i.e., VMs or containers). Please refer to the OpenStack High Availability Guide when installing and configuring OpenStack services.
• The database services, for example MariaDB/MySQL and Galera (an odd number of nodes is required for this quorum).
• The message queue servers, for example, RabbitMQ.
• ntp or chrony – the OpenStack services and other external components, such as Ceph, are very sensitive to clock drifts, so time should be synchronized across all nodes.  

Deploying more than three controller nodes and spreading the services among them is possible, but three is usually enough to handle hundreds of compute nodes.

Compute nodes

Compute nodes run the user workloads, along with the nova-compute instances that manage them. Having at least two compute nodes allows migrating/evacuating workloads from a failed node to an active one.

Storage nodes

Storage nodes run storage software for deployments that include software-based volume, image, and/or object storage such as Ceph or Swift (as opposed to storage appliances). Ceph requires at least three Ceph Monitor instances which may be installed on controller nodes, and at least 2-3 dedicated nodes for Ceph OSD instances (at least as many nodes as the number of configured replicas). For Swift we recommend  at least two Swift Proxy instances and at least three dedicated nodes for Swift Storage. The Swift Proxy instances may be installed on controller nodes, but may consume large amounts of CPU and network resources under load.

Other nodes

You may want to install Telemetry service and its database on a dedicated server because it is a good practice to use a monitoring service outside the existing cluster. Another example is a dedicated server (or servers) for OpenStack database.

 

Converged deployments

As described above, dedicating nodes to running specific services is simpler due to reduced resource contention and less impact from node failures, but it leads to under-utilized nodes. Using the dedicated node model as a base, we can consider giving nodes multiple roles.

If total segregation is on one end of the spectrum, total convergence is on the other end. Here, control services may run on any node, and every node runs user workloads and storage services. This type of deployment is the most cost-effective, but the most difficult to manage. Special care must be taken so that each service receives adequate resources to prevent degradation in the system. Ceph, for example, in some cases can use a lot of RAM and CPU, making them unavailable for workloads. We recommend some form of resource reservation, such as Linux cgroups when pursuing such an architecture.

Avoid common Pitfalls with OpenStack, click here

HA for OpenStack Services

Compute (Nova): 

Instance high availability

Starting from Liberty, Nova allows evacuating a guest after detecting node failure. The “mark host down” method has been added to Nova API. This method allows external tools like Pacemaker or Consul to notify Nova that a host is down faster than a standard periodic task detects it. After that, a guest evacuation can be started.

Storage (Cinder)

Volume migration API

The volume migration API enables moving a volume and its data between two backends in a way that it is transparent to users and workloads. There are two levels of storage support for this feature: “generic” and “specialized”. As of the Liberty release, the generic method works between any type of storage backends. Before Liberty, support was limited to iSCSI and FC backends. Note that the generic method is supported only by Nova’s libvirt driver. In this method, the data being copied flows either through a VM for attached volumes, or a Cinder node for unattached volumes. Some Cinder backends support a specialized flow, whereby in some cases the storage backend itself handles the data migration (commonly if the migration is between two pools on the same storage backend).

Volume replication API

The volume replication API enables managing a storage backend’s block-level replication feature using OpenStack Cinder. While progress has been made in Liberty to support volume replication, the design and implementation are still being finalized.

Networking (Neutron)

Neutron HA modes

OpenStack Networking supports the following HA modes:

• Virtual Router Redundancy Protocol (VRRP) with ML2 plugin and Open vSwitch (OVS)
• Virtual Router Redundancy Protocol (VRRP) with ML2 plugin and Linux bridges
• Distributed Virtual Routing (DVR) supports flat and VLAN external networks, VXLAN and GRE project networks (starting from Juno release) and VLAN project networks (starting from Kilo release).

Distributed DHCP agents

Neutron DHCP agent has built-in support for HA. More than one DHCP agent per network  is required to achieve high availability.

Distributed metadata agents

Neutron metadata agent does not have built-in support for HA. Before OpenStack Liberty, to make metadata service highly available you had to use metadata agents in active/passive mode. In Liberty it is possible to run distributed metadata agents in active/active mode.

New LBaaS implementation

The new LBaaS implementation is now based on the operator-grade scalable load balancer Octavia and is no longer experimental.

Orchestration (Heat)

Per resource state persistence mode

In OpenStack Liberty, a new optional mode has been introduced to persist per resource state during stack updates. This improves fault tolerance and enables recovery from a failure of the Heat engine. There is a plan to implement new auto healing features in the upcoming OpenStack releases.

Learn how Stratoscale Symphony takes advantage of the OpenStack ecosystem with full support for OpenStack APIs.

Shared file system (Manila)

Share migration

Share migration allows you to migrate a share from one host pool to another and to perform a migration between different backends as well.

Containers (Magnum)

Multi-master Kubernetes bay support

Multi-master Kubernetes bay support offers highly available Kubernetes using Magnum with a number of master nodes greater than 1.

Kubernetes with Neutron load balancers

Kubernetes is now integrated with Neutron load balancers.

Containerized services (Kolla)

Kolla project has been added to the OpenStack Big Tent during the Liberty cycle. Kolla provides Docker containers with OpenStack services and Ansible playbooks. Such containers and deployment tools are a fast and scalable way to install OpenStack. Kolla can install a hundred-node highly available OpenStack Liberty cluster in 30 minutes.

Regardless of your approach to OpenStack deployment, a highly available (HA) software architecture for it to run on will be critical to your success. Ensure your deployment by simplifying the process with an “out of the box” cloud solution.