不看死不瞑目的文档 ：Solaris container - Container Example

Combining the Resource Management and Zones features available in Solaris 10 OS, a system administrator can create Solaris Containers tailored for a specific use. Building on the examples in the previous chapters, assume the administrator wants to run the following workloads on a single SMP system:

l        The production sales database

l        The production marketing database

l        A development environment for the marketing database with multiple developers working on application development

l        A development environment for the sales database

l        System management

The following issues prevent successful consolidation onto a single system:

l        The databases are managed by two different DBA organizations that each have their own (conflicting) standards

l        The database systems use different naming services

l        The development systems use the same usernames, file system paths and Oracle SIDs as the production environment

l        The database instances should be guarateed a certain minimum and maximum amount of CPU capacity at all times

l        The production systems should get preferential treatment over the development systems

l        The sales database is the most important workload

l        Developers should not be able to monopolize the CPU resources on the development systems

l        The marketing department is willing to pay for a maximum of two CPUs

The problem is that the sales and marketing databases cannot co-exist on a single system because of different database administration standards and the use of different naming services. This can be overcome by using a separate zone for each workload. The issue of the development environments sharing naming with production can also be overcome with zones. Each zone has its own namespace for users, file systems, network ports and naming services.

The guarantee for minimum and maximum CPU capacity can be ensured by using Dynamic Resource Pools and the Fair Share Scheduler. Resource Pools provide hard upper and lower bounds at the granularity of a CPU. By creating a pool for the sales production database, a pool for the marketing database, and a pool for all other workloads, the production databases can be given guaranteed CPU capacity.

The demand for preferential treatment of the production systems can be implemented using the Fair Share Scheduler by assigning the production zones more zone.cpu-shares than development zones. When contention for CPU resources arises, the production zones are given more CPU resources than the other zones.

To prevent a developer from monopolizing the CPU resources in a development zone, each developer is assigned to a project with a certain amount of project.cpu-shares. The Fair Share Scheduler is used inside the development zones to distribute CPU resources according to the shares.

This leads to the following design:

l        The resouce pool sales with a processor set large of at least two CPUs and no upper bound on CPUs

l        Bound to this pool is a single zone sales, thus allowing exclusive access to the CPUs in the large processor set for the sales production database only

l        Inside the zone there is a single project ora_sales, used to limit the amount of shared memory

l        The pool uses the default Time Sharing (TS) scheduler since there is no need to arbitrate between zones in the pool or between projects in the zone

l        A resource pool called marketing with a processor set small of at least one and at most two CPUs

l        Bound to this pool are two zones, mkt for the marketing production database and mkt_dev for the development database This pool uses the Fair Share Scheduler (FSS) to arbitrate between the two zones bound to the pool and between projects in those zones

l        Inside each zone is a single project ora_mkt to limit shared memory for the Oracle instance

l        Each developer is assigned a unique project in the development zone with a suitable amount of CPU shares

l        The default resource pool pool_default with a processor set with at least one CPU

l        Bound to this pool are the global zone and the sales_dev zone for the sales database developers

l        This pool uses the FSS scheduler to arbitrate between the two zones bound to the pool, and between projects in those zones

l        Each developer is assigned a unique project in the development zone with a suitable amount of CPU shares Container Construction

Creating the Pools

The pool configuration built in Chapter 6 almost matches the design, and could be used as a basis to create the required pools. However, the pools are created from scratch in order to show all relevant steps in a single location.

1. Enable the Resource Pools facility and save the default configuration to the /etc/pooladm.conf file. (The default configuration constists of a processor set pset_default with all CPUs and a single pool pool_default.)

global # pooladm -e

global # pooladm -s

2. Create the sales resource pool with TS as the scheduler and the large processor set with at least two CPUs.

global # poolcfg -c ‘create pset large (uint pset.min=2; uint pset.max=65536)’

global # poolcfg -c ‘create pool sales (string pool.scheduler=”TS”)’

global # poolcfg -c ‘associate pool sales (pset large)’

3. Create the marketing resource pool with FSS as the scheduler and the small processor set with between one and two CPUs.

global # poolcfg -c ‘create pset small (uint pset.min=1; uint pset.max=2)’

global # poolcfg -c ‘create pool marketing (string pool.scheduler=”FSS”)’

global # poolcfg -c ‘associate pool marketing (pset small)’

4. Set the scheduler for the default pool to the Fair Share Scheduler and instantiate the pool configuration just created:

global # poolcfg -c 'modify pool pool_default (string pool.scheduler="FSS")'

global # pooladm -c

global # poolcfg -dc info

system blondie

string system.comment

int system.version 1

boolean system.bind-default true

int system.poold.pid 29072

pool marketing

int pool.sys_id 5

boolean pool.active true

boolean pool.default false

string pool.scheduler FSS

int pool.importance 1

string pool.comment

pset small

pool sales

int pool.sys_id 6

boolean pool.active true

boolean pool.default false

string pool.scheduler TS

int pool.importance 1

string pool.comment

pset large

pool pool_default

int pool.sys_id 0

boolean pool.active true

boolean pool.default true

string pool.scheduler FSS

int pool.importance 1

string pool.comment

pset pset_default

pset large

int pset.sys_id 1

boolean pset.default false

uint pset.min 2

uint pset.max 65536

string pset.units population

uint pset.load 0

uint pset.size 2

string pset.comment

cpu

int cpu.sys_id 3

string cpu.comment

string cpu.status on-line

cpu

int cpu.sys_id 2

string cpu.comment

string cpu.status on-line

pset small

int pset.sys_id 2

boolean pset.default false

uint pset.min 1

uint pset.max 2

string pset.units population

uint pset.load 0

uint pset.size 2

string pset.comment

cpu

int cpu.sys_id 1

string cpu.comment

string cpu.status on-line

cpu

int cpu.sys_id 0

string cpu.comment

string cpu.status on-line

pset pset_default

int pset.sys_id -1

boolean pset.default true

uint pset.min 1

uint pset.max 65536

string pset.units population

uint pset.load 17

uint pset.size 2

string pset.comment

cpu

int cpu.sys_id 11

string cpu.comment

string cpu.status on-line

cpu

int cpu.sys_id 10

string cpu.comment

string cpu.status on-line

Binding Zones to Pools

Currently all zones are bound to the default pool because the pool property of the created zones has not been set, resulting in the zones being bound to the pool with the

pool.default attribute set to true. Setting the zone's pool property to the name of a resource pool binds that zone and all of its processes to that pool when the zone is booted. Note that since the sales zone is bound to a resource pool with the normal TS scheduler, the zone.cpu-shares resource control is no longer applicable and is therefore removed from the zone configuration.

 

global # zonecfg -z sales set pool=sales

global # zonecfg -z sales remove rctl name=zone.cpu-shares

global # zonecfg -z mkt set pool=marketing

 

To bind a running zone to a pool without rebooting the zone, the poolbind(1M) command can be used. This dynamically rebinds the zone and its processes to a pool until the next zone boot. To have this change persist accross zone reboots, the zone's property should be set as shown above.

 

global # poolbind -q `pgrep -z sales -x init`

28399 pool_default

global # poolbind -p sales -i zoneid sales

global # poolbind -q `pgrep -z sales -x init`

28399 sales

global # poolbind -p marketing -i zoneid mkt

global # poolbind -q `pgrep -z mkt -x init`

28545 marketing

 

The poolbind -q `pgrep -z sales -x init` command is used to ascertain to which zone the current pool is bound by querying the binding of the init(1M) process of that zone. As can been seen, the sales zone was bound to the pool pool_default and is now bound to the sales pool.

Creating Development Zones

The development environments for both databases get their own zones, enabing them to use the same user names, projects and file system paths as the production environments. The development zone for the sales database, sales_dev, is bound to the default pool and shares the pool with all processes of the global zone. To prevent the sales_dev zone from monopolizing CPU resources, its zone.cpu-shares is set to the same value as that of the global zone. This gives both zones equal access to CPU resources. When the Fair Share Scheduler is active in a resource pool, it only looks at processes in that pool. The amount of shares for the sales_dev zone is only relevant in relation to those of the global zone.

 

global # zonecfg -z sales_dev

sales_dev: No such zone configured

Use 'create' to begin configuring a new zone.

zonecfg:sales_dev> create

zonecfg:sales_dev> set zonepath=/export/zones/sales_dev

zonecfg:sales_dev> set autoboot=true

zonecfg:sales_dev> set pool=pool_default

zonecfg:sales_dev> add rctl

zonecfg:sales_dev:rctl> set name=zone.cpu-shares

zonecfg:sales_dev:rctl> add value (priv=privileged,limit=1,action=none)

zonecfg:sales_dev:rctl> end

[...]

global # chmod 700 /export/zones/sales_dev

global # zoneadm -z sales_dev install

[...]

global # zoneadm -z sales_dev boot

 

The development environment of the marketing database uses the same pool as the zone for the marketing production database. The Fair Share Scheduler is used to give preferential access to the production zone. By setting the zone.cpu-shares of the mkt zone to 50, and the zone.cpu-shares of the mkt_dev zone to 10, the production database is granted five times as much CPU resources as the development database.

 

global # zonecfg -z mkt 'select rctl name=zone.cpu-shares; set

value=(priv=privileged,limit=50,action=none);end'

global # zonecfg -z mkt_dev

mkt_dev: No such zone configured

Use 'create' to begin configuring a new zone.

zonecfg:mkt_dev> create

zonecfg:mkt_dev> set zonepath=/export/zones/mkt_dev

zonecfg:mkt_dev> set autoboot=true

zonecfg:mkt_dev> set pool=marketing

zonecfg:mkt_dev> add rctl

zonecfg:mkt_dev:rctl> set name=zone.cpu-shares

zonecfg:mkt_dev:rctl> add value (priv=privileged,limit=10,action=none)

zonecfg:mkt_dev:rctl> end

[...]

global # chmod 700 /export/zones/mkt_dev

global # zoneadm -z mkt_dev install

[...]

global # zoneadm -z mkt_dev boot

 

The pool bindings for the zones can be verified using the poolbind -q pid command on every zone's init(1M) process.

 

global # poolbind -q `pgrep -z sales_dev -x init`

6718 pool_default

global # poolbind -q `pgrep -z sales -x init`

28399 sales

global # poolbind -q `pgrep -z mkt -x init`

28545 marketing

global # poolbind -q `pgrep -z mkt_dev -x init`

6579 marketing

global # poolbind -q `pgrep -z global -x init`

1 pool_default

Creating Development Users and Projects

Once all zones are created, it is time to create users and projects inside the development zones and set the appropriate resource controls to implement the design. The Fair Share Scheduler is used to prevent the developers from consuming the CPU resources. In both zones three users and three projects are created.

 

global # zlogin mkt_dev

mkt_dev # mkdir -p /export/home

mkt_dev # groupadd dba

mkt_dev # useradd -g dba -m -d /export/home/oracle -s /bin/bash oracle

mkt_dev # projadd -U oracle ora_mkt

mkt_dev # projmod -sK "project.max-shm-memory=(privileged,2G,deny)" ora_mkt

mkt_dev # projmod -sK "project.cpu-shares=(privileged,100,none)" ora_mkt

mkt_dev # useradd -m -d /export/home/dev1 -s /bin/bash dev1

mkt_dev # useradd -m -d /export/home/dev2 -s /bin/bash dev2

mkt_dev # projadd user.dev1

mkt_dev # projadd user.dev2

mkt_dev # projmod -sK "project.cpu-shares=(privileged,10,none)" user.dev1

mkt_dev # projmod -sK "project.cpu-shares=(privileged,10,none)" user.dev2

[Oracle installation omitted for brevity...]

 

global # zlogin sales_dev

sales_dev # mkdir -p /export/home

sales_dev # groupadd dba

sales_dev # useradd -g dba -m -d /export/home/oracle -s /bin/bash oracle

sales_dev # projadd -U oracle ora_sales

sales_dev # projmod -sK "project.max-shm-memory=(privileged,2G,deny)" ora_sales

sales_dev # projmod -sK "project.cpu-shares=(privileged,100,none)" ora_sales

sales_dev # useradd -m -d /export/home/dev1 -s /bin/bash dev1

sales_dev # useradd -m -d /export/home/dev2 -s /bin/bash dev2

sales_dev # projadd user.dev1

sales_dev # projadd user.dev2

sales_dev # projmod -sK "project.cpu-shares=(privileged,10,none)" user.dev1

sales_dev # projmod -sK "project.cpu-shares=(privileged,10,none)" user.dev2

[Oracle installation omitted for brevity...]

Verifying the Configuration

The configuration just built can be verified using the following steps:

1.      Start the prstat -Z command in the global zone to observe the CPU utilization of the zones.

2.      Start the poolstat -r pset 5 command in the global zone to observe utlization in the resource pools.

3.      Create load using the nspin -n 4 command in the mkt zone as the user oracle in the ora_mkt project. Note the CPU consumption of the mkt zone peaks around 33% since the marketing resource pool to which the zone is bound consists of two CPUs. The other CPUs are idle.

4.      Add the same load in the mkt_dev zone. The combined CPU usage of the mkt and mkt_dev zones is approximately 33% since they share the same resource pool. The mkt zone receives approximately 27% and the mkt_dev zone about 6% because the mkt zone has five times more zone.cpu-shares than the mkt_dev zone.

5.      Add the same load in the sales zone. The sales zone receives 33% since it is bound to the sales pool, which also has two CPUs. The CPU consumption of the mkt and mkt_dev zones is not impacted by the CPU usage of the sales zone.

6.      Add load in the sales_dev zone. This zone is bound to the default pool. As a result, it is able to use all of the remaining CPU capacity since it is the only zone in that pool using CPU resources.

7.      Add the same load in the global zone. The global zone is also bound to the default pool, and has the same amount of zone.cpu-shares as the sales_dev zone. The CPU usage of both zones is therefore equal, and approximately 16 percent. The resulting prstat –Z command output looks as follows:

 

global # prstat -Z

PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP

1987 100 1144K 672K cpu3 20 0 0:04:17 8.4% nspin/1

[...]

2031 root 1144K 808K run 7 0 0:00:49 4.1% nspin/1

ZONEID NPROC SIZE RSS MEMORY TIME CPU ZONE

2 15 42M 30M 0.4% 0:17:17 33% sales

1 33 104M 69M 0.8% 0:23:00 27% mkt

0 65 388M 179M 2.2% 0:03:36 17% global

3 33 105M 71M 0.8% 0:06:27 16% sales_dev

4 33 103M 69M 0.8% 0:03:53 5.7% mkt_dev

Total: 179 processes, 586 lwps, load averages: 19.81, 14.84, 8.04

8.      Add load in the sales_dev zone in the user.dev1 and user.dev2 projects. The total CPU usage of the sales_dev zone remains the same. However, in the zone the CPU should now be divided across the three projects according to the project.cpu-shares in the zone. Notice that a zone bound to a resource pool is only aware of the CPUs in the associated processor set. As a result, the sales_dev zone only knows about two CPUs, and the usage shown in the output of the prstat command is therefore based on two CPUs. That is why inside the zone the three projects seem to use 50%. (The other 50% is used by the global zone that is also bound to the same pool.) The user.dev1 and user.dev2 projects receive 10/120ths each of that 50% since they each have 10 project.cpu-shares and ora_sales has 100 project.cpu-shares.

 

sales_dev # prstat -J

PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP

2016 oracle 1144K 672K run 35 0 0:24:48 10% nspin/1

2088 dev1 1136K 704K run 1 0 0:00:03 1.4% nspin/1

[...]

2113 dev2 1144K 848K run 27 0 0:00:02 1.2% nspin/1

PROJID NPROC SIZE RSS MEMORY TIME CPU PROJECT

100 5 7456K 4864K 0.1% 1:39:06 40% ora_sales

101 5 7464K 5072K 0.1% 0:00:12 5.3% user.dev1

102 5 7464K 5072K 0.1% 0:00:08 5.0% user.dev2

1 5 11M 9336K 0.1% 0:00:00 0.2% user.root

0 26 93M 63M 0.8% 0:00:14 0.0% system

3 1 2904K 2064K 0.0% 0:00:00 0.0% default

Total: 47 processes, 132 lwps, load averages: 14.43, 10.28, 8.84

 

This example illustrates some of the ways that Solaris Containers technologies can be used to facilitate consolidation. It should be noted that not all features must be used at the same time. Depending on the circumstances, some Solaris Container technologies such as Resource Management, Resource Pools and Zones, can be mixed and matched to meet the specific needs for a consolidation project. In some cases, just using the Fair Share Scheduler may be sufficient to meet requirements, while in other cases Solaris Zones can be the key technology to a succesful consolidation.