Using smem on Android

smem is a handy tool for analyzing system memory.

Contents

• 1Quick steps
• 2Introduction
• 3Overview of Steps to Use smem
• 4Detailed Instructions
  • 4.1Installing the smem tool
  • 4.2Building 'smemcap' (or use pre-built binary)
    • 4.2.1Pre-built binary (for G1)
  • 4.3Installing 'android-smemcap' onto your Android target
  • 4.4Collect memory information on the target
  • 4.5Retrieving the collected data from the system
  • 4.6Review the data on the host
  • 4.7Viewing information and charts
  • 4.8Sample Charts
  • 4.9Providing additional system information
• 5Using the graphical viewer
• 6Interpreting the results

Quick steps

For the impatient, here are some quick steps:

* download smem  * hg clone http://selenic.com/repo/smem  * cd smem* (install python-matplotlib and python-matplotlib-tk, if needed)* cross-compile smemcap for Android, using something like 'agcc'  * agcc smemcap.c -o android-smemcap* install android-smemcap onto your target  * adb shell mkdir /data/smem  * adb push android-smemcap /data/smem* collect memory information on target  * adb shell "/data/smem/android-smemcap >/data/smem/memdata1.tar"* retreive the collected data  * adb pull /data/smem/memdata1.tar .* review the data on host  * smem -S memdata1.tar

This document describes how to use 'smem' with Android.

Introduction

smem is a memory tool for analyzing the amount of memory usageon a Linux system. The 'ps' tool provided with Android only shows theVSIZE and RSS memory columns. Even if you install busybox on anandroid system, that tool's 'ps' will also only show you only the VSIZEvalue. VSIZE is the total size of the virtual image, and RSS is theresident set size - the amount of physical pages accounted by the kernelto a particular process. These numbers are not very useful to indicatethe actual cost in memory of a particular application or daemon.

On the smem web site it says:

smem is a tool that can give numerous reports on memory usage on Linux systems.Unlike existing tools, smem can report proportional set size (PSS), which is amore meaningful representation of the amount of memory used by libraries andapplications in a virtual memory system.

Because large portions of physical memory are typically shared among multipleapplications, the standard measure of memory usage known as resident set size (RSS)will significantly overestimate memory usage. PSS instead measures eachapplication's "fair share" of each shared area to give a realistic measure.of the activity of a system during boot.

smem consists of a data capture tool, for use on an android target,and an analysis tool for use on a development host machine.

Overview of Steps to Use smem

To use smem on an Android System, you need to performthe following steps:

* install the smem tool on your host machine* build smemcap on your host machine (or use a pre-built binary)* install smemcap on your Android target system* use smemcap to capture the memory information from /proc on the target* retrieve the collected data from the system* use the smem tool with the collected data, on the host* view memory information and charts* [optionally] provide additional system information

These steps will be described in detail in the followingsections. This paper was written based on Android 1.5software and SDK.

Detailed Instructions

Installing the smem tool

smem can be downloaded from http://selenic.com/repo/smem

Download using the mercurial tool (hg)

    $ hg clone http://selenic.com/repo/smemdestination directory: smemrequesting all changesadding changesetsadding manifestsadding file changesadded 37 changesets with 40 changes to 5 files5 files updated, 0 files merged, 0 files removed, 0 files unresolved

This will create a directory called 'smem' in the current directory.

Usually, you will want to place the 'smem' program on your path. This can be done by copying smem to /usr/local/bin, like so (as root):

   $ cp smem/smem /usr/local/bin

smem has a few requirements:

• a reasonably modern kernel (> 2.6.27 or so)
• a reasonably recent version of Python (2.4 or so)

The smem tool can be used in text mode with no further installation.

However, smem also provides the ability to produce nice charts and graphs.For this, it needs the python matplotlib plotting library.

On Fedora, you can install this with the command:

  $ yum install python-matplotlib python-matplotlib-tk

Other Linux distributions may also have packages for matplotlib.

Here are links to information if you need to directly download and/or install matplotlib

• http://matplotlib.sourceforge.net/index.html - home page
• http://sourceforge.net/projects/matplotlib - download area
• http://matplotlib.sourceforge.net/users/installing.html - install instructions

Building 'smemcap' (or use pre-built binary)

To collect the memory information from the /proc filesystem on the targetyou need to run the 'smemcap' program. smem also includes a shell scriptcalled 'capture' which can collect the data on a standard Linux desktop.However, the Android environment does not have the necessary shell environmentor command line tools for this program to work. So you will need to usethe compiled C program.

Compile the smemcap program as a native Android program against thebionic C library. Details about this are outside the scope of thisdocument, but you can use the instructions here:Compiling Native Apps for Android

If you have installed 'agcc', then the following steps should work:

  $ cd smem  $ agcc smemcap.c -o android-smemcap

Pre-built binary (for G1)

Alternatively, you can use a pre-built capture program. This one worked onmy Android G1 phone and on an OMAP EVM board.

Note: Due to constraints on file uploads on this wiki, I had to upload it as a tar file. Download it and untar with: 'tar -xvf Android-smemcap.tar'

Installing 'android-smemcap' onto your Android target

To install the android-smemcap program on your target, you just need to copy theprogram, and make sure it is executable. It is recommended to make a separate directoryfor smem capture data, on the /data portion of the target file system. (However,any writable directory can be used, such as one on an sdcard.)

Make the smem directory, copy the file:

  $ adb shell mkdir /data/smem  $ adb push android-smemcap /data/smem

You should check that the program is executable:

  $ adb shell chmod 777 /data/smem/android-smemcap  $ adb shell ls -l /data/smem  -rwxrwxrwx root     root         7391 2009-06-24 14:47 android-smemcap

Collect memory information on the target

Now, to measure the memory usage on the target, use the android-smemcap program to collect the data from the /proc filesystem.

  $ adb shell  $ cd /data/smem  $ ./android-smemcap >memdata1.tar  $ exit

You can also do this directly from the host, using the adb shell:

  $ adb shell "/data/smem/android-smemcap >/data/smem/memdata1.tar"

Note the use of quotes to have the shell redirection on the targetstay on the target. Due to the binary nature of the data outputby android-smemcap, you can not redirect it directly throughthe 'adb shell' command to the host filesystem.

Retrieving the collected data from the system

Now, collect the data from the target, and retrieve it to the host:

   $ adb pull /data/smem/memdata1.tar .

This will pull the just-created file 'memdata1.tar' from the targetand place it in the current directory.

Review the data on the host

Now, you can use the smem tool on the host, to view the target process data:

   $ smem -S memdata1.tar  PID User     Command                         Swap      USS      PSS      RSS   32 0        /system/bin/debuggerd              0       64       70      252   30 1000     /system/bin/servicemanager         0       80       86      264   36 1002     /system/bin/dbus-daemon --s        0       92      106      368   39 0        /system/bin/sh /runme.sh           0       72      106      316   37 0        /system/bin/installd               0      100      108      332 2462 0        /system/bin/sh -                   0       84      118      332   31 0        /system/bin/vold                   0      136      144      392   41 0        /sbin/adbd                         0      144      144      156    1 0        /init                              0      152      154      172 3780 0        ./android-smemcap                  0      188      195      384   40 1008     /system/bin/akmd                   0      268      303      688   33 0        /system/bin/rild                   0      444      504      984   35 1013     /system/bin/mediaserver            0      876     1458     2948  181 0        com.android.voicedialer            0      948     1550    10668  263 0        com.instamapper.gpstracker         0     1072     1633    10992  252 0        com.rerware.android.MyBacku        0     1080     1665    11348  289 0        com.android.music                  0     1204     1892    11436  271 0        com.biggu.shopsavvy                0     1236     2050    12344  241 0        com.android.alarmclock             0     1488     2087    12264  215 0        com.android.email                  0     1524     2164    12116  302 0        com.google.android.gm              0     1508     2170    11928  280 0        com.acme.android.powermanag        0     1884     2533    12688  135 0        com.android.mms                    0     2036     2711    12708  164 0        android.process.media              0     2208     2840    13076  412 0        com.biggu.shopsavvy:remote         0     2036     2904    13668  228 0        com.android.calendar               0     2296     3005    13596   34 0        zygote /bin/app_process -Xz        0     1088     3218    16688  202 0        com.google.android.apps.map        0     2472     3334    14324  106 0        com.android.phone                  0     4928     5891    16664  155 0        com.google.process.gapps           0     4628     5933    17408  108 0        android.process.acore              0     7288     9023    29560   66 0        system_server                      0    10188    12636    35056

Viewing information and charts

There are a number of interesting options that smem provides, for filtering,analyzing and visualizing the process data.

Here are some interesting options:

show pie chart 

'smem -S memdata1.tar --pie=command'

This produces a pie chart labeled by the command names (with pss as the data key)

Also: 'smem -S memdata1.tar -P com.* --pie=command'

Only showing Android applications (apps starting with "com.")

show a bar chart 

'smem -S memdata1.tar --bar=command'

See below for a sample

show all memory mappings on the system 

'smem -S memdata1.tar -m'

show the mappings for a specific program 

'smem -S memdata1.tar -m -P email'

show the system libraries 

'smem -S memdata1.tar -m -M /system/lib/*'

show memory maps for application packages 

'smem -S memdata1.tar -m -M .*.apk'

find the libraries with the most users 

'smem -S memdata1.tar -m -M lib -s pids'

That is, show mappings for libraries, with 'lib' in the name, sorted by number of processes (pids) using them.

sample output:

Map                                       PIDs   AVGPSS      PSS/system/lib/hw/copybit.msm7k.so              1        8        8/system/lib/hw/lights.msm7k.so               1        8        8/system/lib/hw/sensors.trout.so              1       12       12/system/lib/libagl.so                        1       48       48/system/lib/libandroid_servers.so            1       16       16/system/lib/libhtc_acoustic.so               1        4        4/system/lib/libhtc_ril.so                    1      168      168/system/lib/libril.so                        1       28       28/system/lib/libsoundpool.so                  1        4        4/system/lib/libsurfaceflinger.so             1      108      108/system/lib/libsystem_server.so              1        8        8/system/lib/liba2dp.so                       2        6       12/system/lib/libaudio.so                      2       14       28/system/lib/libaudioflinger.so               2       62      124/system/lib/libcamera.so                     2       14       28/system/lib/libcameraservice.so              2       20       40/system/lib/libmediaplayerservice.so         2       52      104/system/lib/libmedia_jni.so                 19        1       35/system/lib/libwebcore.so                   19       10      191/system/lib/libEGL.so                       20        0       15/system/lib/libGLESv1_CM.so                 20        0       12/system/lib/libOmxCore.so                   20        0        4/system/lib/libaes.so                       20        0        4/system/lib/libandroid_runtime.so           20       13      267/system/lib/libbluedroid.so                 20        0        8/system/lib/libcorecg.so                    20        2       52/system/lib/libcrypto.so                    20       21      424/system/lib/libdrm1.so                      20        0        7/system/lib/libdvm.so                       20       18      375/system/lib/libemoji.so                     20        0        4/system/lib/libhardware.so                  20        0        7/system/lib/libicudata.so                   20       13      260/system/lib/libicui18n.so                   20       15      304/system/lib/libicuuc.so                     20       18      363/system/lib/libmedia.so                     20        5      102/system/lib/libnativehelper.so              20        6      135/system/lib/libopencoreauthor.so            20       13      265/system/lib/libopencorecommon.so            20        9      185/system/lib/libopencoreplayer.so            20       29      593/system/lib/libpixelflinger.so              20        3       71/system/lib/libsgl.so                       20       29      583/system/lib/libskiagl.so                    20        0        4/system/lib/libsonivox.so                   20        0        7/system/lib/libsqlite.so                    20       16      325/system/lib/libssl.so                       20        4       97/system/lib/libui.so                        20       12      245/system/lib/libvorbisidec.so                20        4       94/system/lib/libdbus.so                      21        0       16/system/lib/libexpat.so                     21        3       67/system/lib/libgps.so                       21        2       52/system/lib/libhardware_legacy.so           21        1       40/system/lib/libnetutils.so                  21        0       16/system/lib/librpc.so                       21        0       16/system/lib/libwpa_client.so                21        0       12/system/lib/libutils.so                     22       18      404/system/lib/libz.so                         22        3       77/system/lib/libcutils.so                    24        5      131/system/lib/liblog.so                       25        0       24/system/lib/libstdc++.so                    29        1       40/system/lib/libc.so                         30       14      444/system/lib/libm.so                         30        2       88

show processes using libc, sorted by PSS 

'smem -S memdata1.tar -M libc'

sample output:

  PID User     Command                         Swap      USS      PSS      RSS  135 0        com.android.mms                    0        8       16      264  164 0        android.process.media              0        8       16      264  181 0        com.android.voicedialer            0        8       16      260  202 0        com.google.android.apps.map        0        8       16      264  215 0        com.android.email                  0        8       16      260  228 0        com.android.calendar               0        8       16      264  241 0        com.android.alarmclock             0        8       16      264  252 0        com.rerware.android.MyBacku        0        8       16      260  263 0        com.instamapper.gpstracker         0        8       16      260  271 0        com.biggu.shopsavvy                0        8       16      260  280 0        com.acme.android.powermanag        0        8       16      260  289 0        com.android.music                  0        8       16      260  302 0        com.google.android.gm              0        8       16      264  412 0        com.biggu.shopsavvy:remote         0        8       16      264   30 1000     /system/bin/servicemanager         0       16       19      124   32 0        /system/bin/debuggerd              0       16       19      132   39 0        /system/bin/sh /runme.sh           0       16       19      132 2462 0        /system/bin/sh -                   0       16       19      136   36 1002     /system/bin/dbus-daemon --s        0       16       20      148 3780 0        ./android-smemcap                  0       16       20      144   31 0        /system/bin/vold                   0       20       25      192   37 0        /system/bin/installd               0       20       25      168   40 1008     /system/bin/akmd                   0       20       25      176  106 0        com.android.phone                  0       12       25      276   33 0        /system/bin/rild                   0       20       30      204  108 0        android.process.acore              0       16       36      312   34 0        zygote /bin/app_process -Xz        0        8       83      472   35 1013     /system/bin/mediaserver            0       44       95      344  155 0        com.google.process.gapps           0       28      187      612   66 0        system_server                      0       32      248      772

Sample Charts

Here are some examples of graphical charts produced by smem

product a pie chart 

'smem -S memdata1.tar --pie=command'

Pie chart is labeled with the command names (and pss is used as the data key)

---

Produce a bar chart 

'smem -S memdata1.tar --bar=command'

Providing additional system information

You can provide additional information to smem (such as the kernel image size), using the -R and -K options:

Here is an example showing the 'whole system' memory information, while specifing a kernel vmlinux image:

   $ smem -S ftrace-kernel.tar -w -t -K /home/tbird/work/android/kernel-msm/build/g1/vmlinuxArea                           Used      Cache   Noncachefirmware/hardware                 0          0          0kernel image                   4515          0       4515kernel dynamic memory         17500       4784      12716userspace memory              78544      28736      49808free memory                    1988       1988          0----------------------------------------------------------                             102547      35508      67039

In this example, my kernel was a test kernel that was significantly larger than a standard productionkernel.

Using the graphical viewer

The graphical viewer provides options for zooming and panning portions of the graph, for adjusting graph display parameters, switching between views, and saving a view to an image file.

Here is a snapshot of the smem (matplotlib) chart viewer:

Here are some tips:

Sometimes it is difficult to read the chart labels. You can view the labels more easilyif you pan and zoom the chart.

To zoom in on an area, click on the 'zoom to rectangle' button. Drag a rectangle in the viewing area.For a bar chart, the rectangle must reside within the graph area of the chart (above the x axis and to theright of the y axis). Once you unclick your mouse button, a zoomed image is then shown.You can pan the zoomed image by clicking the 'pan axes' button, and dragging the image withthe left mouse button. (This "pan axes" button puts the chart in 'pan/zoom' mode. Clickthe button again to revert to regular view mode.)

You can also zoom using the right mouse button in the 'pan/zoom' mode.

You can adjust the location of the chart on the page (which is also sometimes necessary to see the full labels), by clicking on the "configure subplots" button. Experiment with changing different parameters, such as the bottomor left parameters.

Save a view to a file using the "save the figure" button. Select the image format to save in, from the drop-downlist at the bottom of the dialog. Available formats are: bmp, eps, jpg, png, ps, svg.

Interpreting the results

See the smem web site for an explanation of USS and PSS, and how these measurements should be interpretedto analyze a running system.