Cloudsuite WebServing deployement

来源：互联网发布：mac如何下载魔兽世界编辑：程序博客网时间：2024/05/16 12:49

The official guide of webserving doesn't indicate all the necessary configurations. I refined this guide based on my own experience. Hope you can enjoy this!

My experimental setup are three kvm VMs. The setup of backend and frontend machines could be eased by just copying the VM image.

My modifications are marked as red below:

Prerequisite Software Packages

MySQL v5.5
Nginx v1.0
PHP v5.3
APC v3.1
Faban v1.0
Olio v0.2
Tomcat v6.0

Download the Web Serving Benchmark.

You need JDK, GCC, and Apache Ant installed on all machines. The JAVA_HOME enviroment variable must point to the JDK installation path. We use OpenJDK 1.6.

I just installed openjdk and build-essentials

Benchmark Components

The benchmark consists of three main components: A web server, a database backend and a client to emulate real world accesses to the web server, each running on a separate machine (Client and backend can run on the same machine). The following instructions provide the necessary details to install the benchmark. We provide the instructions to install the three main components and prepare the applications for benchmarking. Please note that CloudStone provides the option to add an additional layer of memcached machines. However, in CloudSuite, we focus on benchmarking the web server behavior using this benchmark and we do not setup a memcached server.

Setting up the Client Machine

The following instructions will help the user set up the Faban driver which is used to benchmark the Olio application. We start by the client machine because we will need Faban installation on the backend and frontend. Faban must be installed on all nodes on the same absolute path. It has a single master and several agents. The Faban master must run on the client machine. You just need to set up the Faban master and then copy it to the backend and frontend machines, where it will be used as an agent.

Setting up the Faban driver

Use the Faban kit in our web benchmark package or download it from link.
Untar faban-kit-022311.tar.gz in a directory on the client node. A subdirectory named faban will be created which we will refer to it as $FABAN_HOME.
tar xzvf faban-kit-022311.tar.gz
export FABAN_HOME=/path/to/faban
Note: More details on Faban installation can be found at link.
Download a copy of Olio's PHP implementation from link by selecting apache-olio-php-src-0.2.tar.gz or use the copy included in our web benchmark package.
Untar apache-olio-php-src-0.2.tar.gz in a directory. This creates an apache-olio-php-src-0.2 subdirectory, which we will refer to it as $OLIO_HOME.
tar xzvf apache-olio-php-src-0.2.tar.gz
export OLIO_HOME=/path/to/apache-olio-php-src-0.2
Note: Detailed installation and configuration instructions for Olio can be found at link.
Download MySQL Connector/J (JDBC Driver for MySQL) and unpack it (or use the one provided in our package). Copy the mysql-connector-java-5.0.8-bin.jar file into $OLIO_HOME/workload/php/trunk/lib.
tar -xzvf mysql-connector-java-5.0.8.tar.gz
cp mysql-connector-java-5.0.8/mysql-connector-java-5.0.8-bin.jar $OLIO_HOME/workload/php/trunk/lib
cd $FABAN_HOME/
cp samples/services/ApacheHttpdService/build/ApacheHttpdService.jar services
cp samples/services/MysqlService/build/MySQLService.jar services
cp samples/services/MemcachedService/build/MemcachedService.jar services
cd $OLIO_HOME/workload/php/trunk
cp build.properties.template build.properties
Edit build.properties and set faban.home to $FABAN_HOME and faban.url to http://IP.address.of.client_machine:9980, where the Faban master will run.
ant deploy.jar
cp $OLIO_HOME/workload/php/trunk/build/OlioDriver.jar $FABAN_HOME/benchmarks
Start the Faban master on the client machine:
$FABAN_HOME/master/bin/startup.sh
Point your browser to: http://IP.address.of.client_machine:9980. You should see the OlioWorkload welcome note.
Note: This step is necessary because it will unpack the content of OlioDriver.jar which will be required later.
Copy the faban directory ($FABAN_HOME) to the frontend, backend, and client machines. Faban directories must be in the same path on every machine.
The tricky points are: The faban that provided in web-release seems doesn't work well on my Ubuntu 12.04 server setup. If I point my browser to client:9980 it prompts me the welcome note of the Olioworkload though. The error will soon occur after you click schedule run. I just downloaded the latest faban and solved this issue.
If you execute a run and encountered "can not contact your_frontend", this may because faban calls a script "interface" in $FABAN_HOME/bin/Linux. In this script the client pings front end and back end using "ping -R", the "-R" is usually banned in most of the network environments. Just rename this script or modify it.

Setting up the Backend Machine

Setting up MySQL

The following instructions assume a Linux-based installation. For more detailed instructions, please refer to link.

Note: You need to have a copy of MySQL Community Server. You can find it in our web benchmark package or download it from link.
You need to select the Linux platform, and then, download the Linux-generic, x86_64-bit, compressed tar archive.

Here are the installation commands:
(You need root privileges to execute some of the commands below)

Install the libaio1 (e.g., sudo apt-get install libaio1)
groupadd mysql
(addgroup and adduser commands might differ in different Linux versions)
useradd -r -g mysql mysql
tar xzvf mysql-5.5.20-linux2.6-x86_64.tar.gz
chown -R mysql mysql-5.5.20-linux2.6-x86_64
chgrp -R mysql mysql-5.5.20-linux2.6-x86_64
cd mysql-5.5.20-linux2.6-x86_64
cp support-files/my-medium.cnf /etc/my.cnf
scripts/mysql_install_db --user=mysql
To start mysql, in the installation directory, run:
bin/mysqld_safe --defaults-file=/etc/my.cnf --user=mysql &
For convenience, I just apt-get install mysql-server. If you want to compile it from source, remember do not put the source in /root/ directory.

Setting up and populating the database

Run MySQL from the mysql installation directory:
- bin/mysql -uroot
Create the Olio user:
- mysql> create user 'olio'@'%' identified by 'olio';
- mysql> grant all privileges on *.* to 'olio'@'localhost' identified by 'olio' with grant option;Note: This command grants privilege to an olio user running on the local host. In CloudStone, the olio users running on all frontend machines need the same privileges:
- mysql> grant all privileges on *.* to 'olio'@'ip.address.of.frontend' identified by 'olio' with grant option;
Crate Olio database and tables:
- mysql> create database olio;
- mysql> use olio;
- mysql> \. $FABAN_HOME/benchmarks/OlioDriver/bin/schema.sql
- mysql> exit
Populate the database:
- cd $FABAN_HOME/benchmarks/OlioDriver/bin
- chmod +x dbloader.sh
- ./dbloader.sh <dbserver> <load_scale>
  dbserver is the address of the database server (e.g., localhost) and load_scale is the scaling factor. This number shows the maximum number of concurrent users that can be simulated. The database will be loaded with 100*load_scale users. For information on the scaling factor to populate the database, refer to link.

Setting up Tomcat

Download Tomcat or use the copy in our package. In the current CloudSuite distribution, we are using Tomcat v6.0.35.
Untar the package. We refer to the root folder of Tomcat as CATALINA_HOME.
tar xzvf apache-tomcat-6.0.35.tar.gz
export CATALINA_HOME=/path/to/apache-tomcat-6.0.35
cd $CATALINA_HOME/bin
tar zxvf commons-daemon-native.tar.gz
cd commons-daemon-1.0.7-native-src/unix/
./configure
make
cp jsvc ../..

Setting up the Geocoder Emulator

The Geocoder Emulator is a servlet deployed on Tomcat and required by CloudStone.

Copy $OLIO_HOME/geocoder from the client machine to the backend machine in a directory which we will refer to it as $GEOCODER_HOME.
export GEOCODER_HOME=/path/to/a/directory
mkdir $GEOCODER_HOME
scp -r IP.address.of.client_machine:$OLIO_HOME/geocoder $GEOCODER_HOME
cd $GEOCODER_HOME/geocoder
cp build.properties.template build.properties
Edit build.properties and set the servlet.lib.path to point to the $CATALINA_HOME/lib directory.
servlet.lib.path=/usr/local/apache-tomcat-6.0.35/lib
ant all
cp dist/geocoder.war $CATALINA_HOME/webapps
Start Tomcat:
$CATALINA_HOME/bin/startup.sh

Setting up the Frontend machine

Setting up the web application

Create a directory which will be used as the root of the web application. You can use the root directory of the web server (e.g., /var/www) to install the web application. We will call this directory $APP_DIR.
export APP_DIR=/path/to/webapp/root
mkdir -p $APP_DIR
Download a copy of Olio's PHP implementation from link by selecting apache-olio-php-src-0.2.tar.gz or use the copy included in our web benchmark package.
Untar apache-olio-php-src-0.2.tar.gz in a directory. This creates an apache-olio-php-src-0.2 subdirectory, which we will refer to it as $OLIO_HOME.
tar xzvf apache-olio-php-src-0.2.tar.gz
export OLIO_HOME=/path/to/apache-olio-php-src-0.2
cp -r $OLIO_HOME/webapp/php/trunk/* $APP_DIR
We created a patch for the CloudStone which includes a few bug fixes and resolves compatibility issues with PHP 5.3. The patch is available in our package with the name cloudstone.patch. Copy this patch to $APP_DIR and apply it to the Olio installation:
cp cloudstone.patch $APP_DIR
cd $APP_DIR
patch -p1 < cloudstone.patch
You need to use $APP_DIR/etc/php.ini as the setting for your PHP installation (see below) or copy the settings appropriately to the php.ini for your installation (e.g., /usr/local/lib). We will use $APP_DIR/etc/php.ini through the rest of the document.
Edit $APP_DIR/etc/config.php:
- Set the database host name:
  $olioconfig['dbTarget'] = 'mysql:host=IP.address.of.dbserver;dbname=olio';
- Comment out the 'MemCached' component and uncomment the 'NoCache' line:
  //$olioconfig['cacheSystem'] = 'MemCached';
  $olioconfig['cacheSystem'] = 'NoCache';
- Edit the geocoderURL parameter and replace GEOCODER_HOST:8080 with the actual host address and port number, where Tomcat is running (i.e., the backend machine).
  $olioconfig['geocoderURL'] = 'http://IP.address.of.backend:8080/geocoder/geocode';

Installing Nginx

The following dependencies are required, make sure to install them before you continue:
libpcre3 libpcre3-dev libpcrecpp0 libssl-dev zlib1g-dev (Depending on the distribution the names might be pcre, pcre-devel, ...)
You need the Nginx web server which is included in our web benchmark package. Alternatively, you can download it from here.
tar zxvf nginx-1.0.11.tar.gz
cd nginx-1.0.11
./configure
Note: You can change the installation directory by specifying --sbin-path=/path in the configure command.
make
sudo make install
To test the installation:
- Start the server: sudo /usr/local/nginx/sbin/nginx
- Point your browser to: http://IP.address.of.webserver. You should see a welcome note.
- Stop the server: sudo /usr/local/nginx/sbin/nginx -s stop
Edit the Nginx configuration file (e.g., /usr/local/nginx/conf/nginx.conf) and make the following changes:
- Under location /, set the root parameter to $APP_DIR/public_html and append index.php to the index parameter:
  location / {
  root $APP_DIR/public_html;
  index index.html index.htm index.php;
  }
- Set the correct port number to access PHP-FPM by adding the following lines under the server section:
  location ~ \.php$ {
  root $APP_DIR/public_html ; # (e.g., /var/www/public_html );
  fastcgi_pass 127.0.0.1:9000;
  fastcgi_index index.php;
  fastcgi_param SCRIPT_FILENAME $APP_DIR/public_html/$fastcgi_script_name; # (e.g., /var/www/public_html/$fastcgi_script_name)
  include fastcgi_params;}
- Turn the access log off by adding the following line under the server section:
  access_log off;
- To saturate the web server you might need to increase the maximum number of connections. As a rough estimate, 700-1000 connections per core should be able to saturate the web server.
  The number of concurrent connections to Nginx is controlled by two parameters in the nginx configuration file: worker_processes and worker_connections. Theworker_processes parameter dictates the number of child processes, which will be forked, and worker_connections dictates the maximum number of connections per child process. The total number of concurrent connections is the multiplication of these two parameters.
Start the server: sudo /usr/local/nginx/sbin/nginx
I just apt-get the nginx and copy the config file from install package.

Installing PHP

The installation is made to support FPM and the libraries required by Olio. You can use the PHP version which is available in our release or download it from here.

Install the following dependencies:
libxm12-dev curl libcurl3 libcurl3-dev libjpeg-dev libpng-dev
(e.g., sudo apt-get install libxm12-dev curl libcurl3 libcurl13-dev libjpeg-dev libpng-dev)
To allow the usage of the MySQL functionality, you should point to the path of an uncompressed MySQL distribution (i.e., the one used in the backend). You must untar it and use the path in the following command. No installation of MySQL is needed.
tar zxvf php-5.3.9.tar.gz
cd php-5.3.9./configure --enable-fpm --with-curl --with-pdo-mysql=/path/to/mysql --with-gd --with-jpeg-dir --with-png-dir --with-config-file-path=/path/to/php.ini
Warning: --with-config-file-path argument should only include the path ($APP_DIR/etc/), not the file name (php.ini) in the end.
make
sudo make install
Set an environment variable named PHPRC pointing to the directory, which contains php.ini. You should put a command in your shell startup script to automatically do that:
export PHPRC=/path/to/php.ini-directory (e.g., export PHPRC=$APP_DIR/etc/).
Make the tmp/http_sessions directory which is specified by php.ini to store PHP sessions:
mkdir -p /tmp/http_sessions //chmod 777 it. Sometimes you are prompted "
```
WARNING:UIDriverAgent[0].2.doLogin: Found login prompt at
```
", check whether this directory is still there.
touch /var/log/messages
Edit the php.ini file (e.g., $APP_DIR/etc/php.ini) and make the following changes:
- Add the following line which declares the extensions directory:
  extension_dir=/path/to/php/lib/php/extensions/no-debug-non-zts-20090626/
- Add a timezone:
  date.timezone = "Europe/Zurich"
- Disable error reporting for PHP notices (or all errors). These lines should already exist in the php.ini and you need to modify them:
  error_reporting = E_ALL & ~E_NOTICE ; or error_reporting = E_NONE
  display_errors = Off
To run Nginx with PHP support, the PHP-FPM module must be started separately:
- sudo cp /usr/local/etc/php-fpm.conf.default /usr/local/etc/php-fpm.conf
- sudo addgroup nobody # (or you can modify the php-fpm.com file to set the user and group to the ones you are already using)
- sudo /usr/local/sbin/php-fpm
The number of processes created by the PHP-FPM can be controlled by setting Process Manager parameters in the php-fpm.config file. These parameters are prefixed with pm.. See the php-fpm.conf file for a description of these parameters. Modification to these parameters requires a restart of PHP-FPM to take effect.

Installing APC

You can use the APC (Alternative PHP Cache) copy in our web benchmark package or download it from link, then run the following commands for installation:

Install the following dependency: autoconf (e.g., sudo apt-get install autoconf)
tar xzvf APC-3.1.9.tgz
cd APC-3.1.9
phpize
./configure --enable-apc --enable-apc-mmap --with-php-config=/path/to/php-config # (e.g., /usr/local/bin/php-config)
make
sudo make install
Restart PHP-FPM. To make sure APC is enabled, you can check the "php-fpm -m" output. There should be "apc" listed as a module in the output.
If you can not find APC, refer this:
http://stackoverflow.com/questions/6050076/php-apc-error-loading-apc-so

Setting up the Filestore

Filestore is a directory on the local file system, which is used to store the data required for the web application. The required space depends on the load_scale and might reach tens of gigabytes.

We created a patch (cloudsuite.patch) to deal with Filestore large space requirement. When this patch is applied, Olio accesses a limited set of files on the disk. Use of this patch is optional.
cd $APP_DIR
patch -p1 < cloudsuite.patch
Create a directory referred to as $FILESTORE and give read/write permissions to everyone:
export FILESTORE=/path/where/files/will/be/stored
mkdir -p $FILESTORE
chmod a+rwx $FILESTORE
chmod +x $FABAN_HOME/benchmarks/OlioDriver/bin/fileloader.sh
Load the Filestore:
$FABAN_HOME/benchmarks/OlioDriver/bin/fileloader.sh <load_scale> $FILESTORE
load_scale is the same value used when loading the database.
Note: If you used the cloudsuite.patch then this value must be set to 102 regardless of the database load_scale.
Edit $APP_DIR/etc/config.php and set the value of the localfsRoot parameter to the path which is pointed by $FILESTORE.
$olioconfig['localfsRoot'] = '$FILESTORE'; // (e.g., $olioconfig['localfsRoot'] = '/filestore';)
Restart PHP-FPM.
DO NOT put $FILESTORE in /root/ directory, or:
Image at http://10.227.56.248:80/fileService.php?cache=false&file=e1858t.jpg size of 0 bytes is too small. Image may not exist

Running the benchmark

Make sure that the Faban master, the MySQL server, Nginx, and PHP-FPM are up and running.
Before running you test, install necessary tools on your client, front and back machines. ethtool, sysstat, gawk,
Point your browser to http://IP.address.of.client_machine:9980.
Click on the 'Schedule Run' link.
Enter $JAVA_HOME of the client machine in the 'Java Home' field under the Java tab.
Select the 'Driver' tab and set the configuration parameters as follows:
- Enter the driver (client) machine address(es) in the 'Hosts' field.
- Enter the number of 'Concurrent Users'. It defines the number of concurrent users that send request to the web server. The configuration for the web and database servers must allow that many concurrent users.
- Enter vmstat 10 in the 'Tools' field.
- Set the proper values for 'Ramp Up', 'Steady State' and 'Ramp Down' in seconds.
Select the 'Web Server' tab.
- Enter the web server address in the 'Host' field. Do change the port 8080 into 80, or it prompts you :
- UIDriverAgent[0].9.doHomePage: Connection refused
  Note: Error not counted in result.
  Either transaction start or end time is not within steady state.
- For the Web Server 'Type' field, select empty.
- Clear the 'Web Server command path' field.
- Enter the name of the monitoring tools in the 'Tools' field (separate them by semicolon): vmstat 10; mpstat 10; nicstat 10; iostat -x 10
- Enter the log (/usr/local/nginx/logs) and config (/usr/local/nginx/conf/nginx.conf) paths, and the path containing the php.ini file ($APP_DIR/etc/php.ini) in the respective fields.
Select the 'Data Servers' tab.
Under 'Database Servers':
- Enter the database server address in the 'Host' field.
- In 'JDBC connection URL' replace dbHost with the database server address and set the correct user name (olio) and password (olio) to connect to the database.
- Select No in the 'Reload Database' drop-down list.
- In the 'Load for Concurrent Users' field, enter the number of users you populated the database for (i.e., load_scale).
Under 'Data Storage Server':
- Enter the frontend server address in the 'Host' field.
- Enter the $FILESTORE path in the 'Media Directory' field.
Under 'Memcached Servers':
- Clear the 'Host:Port Pairs' field.
Click OK to start the run.
You can monitor the run and see the results by clicking on the 'View Results' link.

0 0