C++ STL容器的Benchmark[转贴]

来源:互联网 发布:单片机和嵌入式的关系 编辑:程序博客网 时间:2024/06/13 18:50

 

 

 

Standard Container Benchmark


 

 

 

作者

留言

PolyRandom




我的机器上的运行结果在最后面,不知道大家的机器上运行结果是什么。

Code:

/* Standard Container Benchmark

Version 0.9, May 23, 2003

The primary purpose of this
benchmark is to show how different standard
containers are in terms of performance. We have observed that programmers
often use sets, multisets, deques in the situations where sorted vectors
are the optimal solutions. Similarly, programmers often choose lists simply
because they do a few insertions and deletions without knowing that vectors
are more efficient at that for small containers.

Frequently, of course, performance does not matter,
but it is important that programmers are aware of the consequences of their
choices. We are not saying that only vectors should be used, there are
cases when one really needs a more complicated data structure, but one needs to
understand the price for additional functionality.

The secondary purpose of the
benchmark is to encourage compiler and library vendors to
keep improving performance. For example, it is not acceptable that some compilers give
you a sizable penalty for using vector iterators instead of pointer. It is also quite
clear that  performance of other standard containers could be improved.

The
benchmark is doing the same task 7 times using different data structures:
array, vector (using a pointer as iterator), vector (using the defailt cector iterator),
list, deque, set and multiset. The task is to remove duplicates from a sequence of doubles.
This is a simple test, but it illustrates the performance of containers.

It is clear that the
benchmark needs to be eventually extended
to slists and even to hash-sets and hash-maps, but we decided that at present it
should work only with the standard containers. As the standard grows, so can
the
benchmark. The additional benefit of not including hash based containers is
that now all the test have identical asymptotic complexity and, even more
importantly, do almost the same number of comparisons. The difference is only
in data structure overhead and cache behavior.

The number of times that a given test is run inversly proportional to NlogN where N is the
size of the sequence.  This allows one to see how containers of different size behave.
The instructive values used for the
benchmark are: 10, 100, 1000, 10000, 100000, 1000000.

The time taken for a run of the
benchmark depends on the machine used, the compiler used,
the compiler and optimizer settings used, as well as the standard library. Please note that
the time taken could be several minutes - and much more if you use a slow debug mode.

The output is a table where columns are separated by tabs and rows by newlines. This is
barely ok for a human reader, but ideal for feeding into a program, such as a spreadsheet
for display or analysis.

If you want to add your own test of a container, add the name of your container to
the "header string, write a test function like the existing ones, e.g. vector_test,
and add a call of "run" for your test in "run_tests".


Alex Stepanov
stepanov@adobe.com

Bjarne Stroustrup
bs@cs.tamu.edu

*/

#include <stddef.h>   // some older implementations lack <cstddef>
#include <time.h>
#include <math.h>
#include <stdlib.h>

#include <vector>
#include <algorithm>
#include <list>
#include <deque>
#include <set>

#include <iostream>
#include <iomanip>

typedef double element_t;

using namespace std;

vector<double> result_times; // results are puched into this vector

const char header[] =
"/tarray"
"/tvector with pointers"
"/tvector with iterators"
"/tdeque"
"/tlist"
"/tset"
"/tmultiset";

void do_head()
{
   cout << "size" << header << '/n';
}

int do_tail()
{
   // in case you want to stop for confirmation in a windows console application
   //char c;
   //cin >> c;
   return 0;
}

void do_row(int size)
{
   cout << size;
   cout << fixed << setprecision(2);
   for (size_t i = 0; i < result_times.size(); ++i)
      cout << '/t' << result_times[i];
   cout << '/n';
}


clock_t start_time;

inline void start_timer() { start_time = clock(); }

inline double timer()
{
   clock_t end_time = clock();
   return (end_time - start_time)/double(CLOCKS_PER_SEC);
}

typedef void(*Test)(element_t*, element_t*, int);
void run(Test f, element_t* first, element_t* last, int number_of_times)
{
   start_timer();
   while (number_of_times-- > 0) f(first,last,number_of_times);
   result_times.push_back(timer());
}

void array_test(element_t* first, element_t* last, int number_of_times)
{
   element_t* array = new element_t[last - first];
   copy(first, last, array);
   sort(array, array + (last - first));
   unique(array, array + (last - first));
   delete [] array;    
}

void vector_pointer_test(element_t* first, element_t* last, int number_of_times)
{
   vector<element_t> container(first, last);
   // &*container.begin() gets us a pointer to the first element
   sort(&*container.begin(), &*container.end());
   unique(&*container.begin(), &*container.end());
}

void vector_iterator_test(element_t* first, element_t* last, int number_of_times)
{
   vector<element_t> container(first, last);
   sort(container.begin(), container.end());
   unique(container.begin(), container.end());
}

void deque_test(element_t* first, element_t* last, int number_of_times)

   //       deque<element_t> container(first, last); CANNOT BE USED BECAUSE OF MVC++ 6
   deque<element_t> container(size_t(last - first), 0.0);
   copy(first, last, container.begin());
   sort(container.begin(), container.end());
   unique(container.begin(), container.end());
}

void list_test(element_t* first, element_t* last, int number_of_times)
{
   list<element_t> container(first, last);
   container.sort();
   container.unique();
}

void set_test(element_t* first, element_t* last, int number_of_times)
{
   set<element_t> container(first, last);
}

void multiset_test(element_t* first, element_t* last, int number_of_times)
{
   multiset<element_t> container(first, last);
   typedef multiset<element_t>::iterator iterator;
   {
      iterator first = container.begin();
      iterator last = container.end();

      while (first != last) {
         iterator next = first;
         if (++next == last) break;
         if (*first == *next)
            container.erase(next);
         else
            ++first;
      }
   }
}

void initialize(element_t* first, element_t* last)
{
   element_t value = 0.0;
   while (first != last) {
      *first++ = value;
      value += 1.;
   }
}

double logtwo(double x)
{
   return log(x)/log((double) 2.0);
}

const int largest_size = 1000000;

int number_of_tests(int size) {
   double n = size;
   double largest_n = largest_size;
   return int(floor((largest_n * logtwo(largest_n)) / (n * logtwo(n))));
}

void run_tests(int size)
{
   const int n = number_of_tests(size);
   const size_t length = 2*size;
   result_times.clear();

   // make a random test set of the chosen size:
   vector<element_t> buf(length);
   element_t* buffer = &buf[0];
   element_t* buffer_end = &buf[length];
   initialize(buffer, buffer + size);      // elements
   initialize(buffer + size, buffer_end);   // duplicate elements
   random_shuffle(buffer, buffer_end);

   // test the containers:
   run(array_test, buffer, buffer_end, n);
   run(vector_pointer_test, buffer, buffer_end, n);
   run(vector_iterator_test, buffer, buffer_end, n);
   run(deque_test, buffer, buffer_end, n);
   run(list_test, buffer, buffer_end, n);
   run(set_test, buffer, buffer_end, n);
   run(multiset_test, buffer, buffer_end, n);
   do_row(size);
}

int main()
{
   do_head();
   const int sizes [] = {10, 100, 1000, 10000, 100000, 1000000};
   const int n = sizeof(sizes)/sizeof(int);
   for (int i = 0; i < n; ++i) run_tests(sizes[i]);
   return do_tail();
}


Code:

P4 2.8E HT 上的结果,VC 7.1 Release + Win XP
size    array   vector with pointers    vector with iterators   deque   list    set     multiset
10      1.55    1.53    1.70    4.89    11.76   2.97    5.75
100     0.94    0.95    1.05    2.98    3.94    1.91    3.23
1000    0.98    0.98    1.08    2.48    2.94    1.66    2.94
10000   0.92    0.92    1.00    2.41    2.92    1.78    2.89
100000  0.92    0.92    1.02    2.28    3.81    3.13    4.70
1000000 0.89    0.91    0.94    2.16    4.28    4.31    5.73


Code:

P4 2.8E HT 上的结果,Dev C++ 4.9.8.0 + Win XP
size    array   vector with pointers    vector with iterators   deque   list    set     multiset
10      1.59    1.78    2.97    5.03    13.30   3.75    5.58
100     0.95    0.98    1.89    2.91    6.82    3.22    4.09
1000    1.13    1.13    1.88    2.61    5.77    2.94    3.56
10000   1.08    1.08    1.77    2.41    5.57    2.86    3.60
100000  0.95    0.95    1.61    2.16    6.75    3.94    4.97
1000000 1.00    1.02    1.74    2.36    5.50    4.14    4.88

 

 

_________________
是非成败转头空

最后进行编辑的是 PolyRandom on 周二 9 28, 2004 9:04 下午, 总计第 1 次编辑

yumagi




注册时间: 2003-06-09
帖子: 168
来自: 上海

我的Mingw也是有类似的实现问题,vector::iteratorpointer慢将近一倍左右,看样子这个实现有点问题。手头上没有VC7.1,但是看你这个结果,好像比Mingw自带的快好多吗,M$的实现是越来越好了。

 

 

_________________

我庄严地宣誓
我将用真诚的良心承担如下的许诺和保证:
我将勇敢地去捍卫真正的科学
将其开拓,为之添彩
既不为厚禄所驱,亦不为虚名所赶
只求上帝真理的神辉普照大地、发扬光大

   

PolyRandom




注册时间: 2002-09-03
帖子: 2396
来自: 上海

我的dev C++如果加上优化(-O3),那么结果将是:

Code:

size    array   vector with pointers    vector with iterators   deque   list    set     multiset
10      0.77    0.80    0.98    2.02    4.00    1.33    2.17
100     0.48    0.48    0.59    1.20    1.92    1.22    1.86
1000    0.81    0.73    0.88    1.31    1.86    1.30    1.63
10000   0.77    0.73    0.83    1.11    1.86    1.30    1.92
100000  0.75    0.80    0.84    1.11    3.84    2.64    3.72
1000000 0.70    0.72    0.81    1.08    2.45    2.55    3.22

 

 

_________________
是非成败转头空

feishq00




注册时间: 2004-07-07
帖子: 3

HPC 3000/400/512M
aCC: HP ANSI C++ B3910B A.03.13

Code:


size    array   vector with pointers    vector with iterators   deque   list    set     multiset
10      7.7     7.5     7.5     18      2.9e+02 33      42
100     4.5     4.4     4.4     9       56      17      24
1000    4       3.9     3.9     7.3     37      13      18
10000   3.7     3.6     3.6     6.4     34      11      14
100000  3.8     3.7     3.8     6.2     33      11      15
1000000 4       3.9     3.9     6.2     32      14      18


aCC -O
优化

Code:


size    array   vector with pointers    vector with iterators   deque   list    set     multiset
10      3.4     3.5     3.5     11      2.1e+02 14      18
100     1.6     1.6     1.7     5.1     23      5.6     8
1000    1.4     1.4     1.4     4       13      4.3     6
10000   1.3     1.3     1.3     3.6     12      3.8     5
100000  1.4     1.4     1.4     3.6     13      5       7.2
1000000 1.6     1.6     1.6     3.6     13      8.4     12

 

 

 

Elminster




注册时间: 2002-09-09
帖子: 1175


AMD ATHLONG XP 1600+, 512M, W2K PRO

vc 7.1

Code:

size    array   vector with pointers    vector with iterators   deque   list
set     multiset
10      1.84    1.86    2.14    6.55    11.19   3.11    5.89
100     1.13    1.13    1.30    4.17    3.58    1.95    3.36
1000    1.00    1.02    1.13    3.52    2.53    1.55    2.88
10000   0.94    0.95    1.06    3.61    3.75    2.38    5.39
100000  1.13    1.13    1.22    3.69    6.09    5.39    8.05
1000000 1.19    1.17    1.27    3.70    7.17    7.19    10.14



icc 8.0

Code:

size    array   vector with pointers    vector with iterators   deque   list
set     multiset
10      2.09    2.14    2.25    6.28    11.69   3.25    6.20
100     1.31    1.33    1.39    3.83    3.89    2.13    3.63
1000    1.19    1.19    1.31    3.27    2.86    1.67    3.11
10000   1.13    1.13    1.25    3.41    4.05    2.50    5.56
100000  1.30    1.28    1.39    3.53    6.55    5.47    8.23
1000000 1.33    1.33    1.45    3.61    7.72    7.36    10.38



显然,面对非 INTEL CPUICC 性能上有所损失。

 

 

_________________

Program be my incantation
 Computer be my golem
  Internet be my world of magic

newsuppy




P III 500/320M/Windows Server 2003/vc7.1
size array vector with pointers vector with iterators deque list set multiset
10 6.11 6.25 7.29 22.11 43.57 12.15 22.66
100 3.27 3.27 3.85 13.08 13.51 7.17 12.71
1000 2.99 2.96 3.50 11.24 10.13 5.95 12.06
10000 2.96 2.96 3.37 11.34 11.91 8.08 14.87
100000 3.13 3.12 3.46 11.69 13.87 10.40 17.28
1000000 3.29 3.29 3.58 11.39 14.68 12.36 18.97

C 1.7GHz/256M/Windows Server 2003/vc7.1
size array vector with pointers vector with iterators deque list set multiset
10 2.16 2.20 2.93 7.84 22.30 5.11 10.35
100 1.17 1.24 1.71 4.64 7.20 2.88 5.36
1000 1.18 1.21 1.67 3.77 5.26 2.41 5.45
10000 1.27 1.33 1.64 4.04 9.25 7.57 13.29
100000 1.24 1.26 1.51 3.68 11.28 11.42 16.71
1000000 1.35 1.23 1.43 3.52 12.86 14.72 20.33

 

 

 

newsuppy






Code:

P III 500/320M/Windows Server 2003
size    array   vector with pointers    vector with iterators   deque   list set  multiset
10      6.11    6.25                    7.29    22.11   43.57   12.15   22.66
100     3.27    3.27                    3.85    13.08   13.51   7.17    12.71
1000    2.99    2.96                    3.50    11.24   10.13   5.95    12.06
10000   2.96    2.96                    3.37    11.34   11.91   8.08    14.87
100000  3.13    3.12                    3.46    11.69   13.87   10.40   17.28
1000000 3.29    3.29                    3.58    11.39   14.68   12.36   18.97

C 1.7GHz/256M/Windows
size    array   vector with pointers    vector with iterators   deque   list set  multiset
10      2.16    2.20                    2.93    7.84    22.30   5.11    10.35
100     1.17    1.24                    1.71    4.64    7.20    2.88    5.36
1000    1.18    1.21                    1.67    3.77    5.26    2.41    5.45
10000   1.27    1.33                    1.64    4.04    9.25    7.57    13.29
100000  1.24    1.26                    1.51    3.68    11.28   11.42   16.71
1000000 1.35    1.23                    1.43    3.52    12.86   14.72   20.33

 

 

 

newsuppy





Celeron这种cache小的CPU在大型非数组容器面前的性能居然比PIII 500好不了多少

 

 

 

newsuppy




Code:

CPU                   size(1M)/size(10)[set]  [multiset]
P4 2.8G(512KB L2)               1.451           0.997
Athlon (256KB L2)                 2.311           1.722
PIII(512KB L2,
半速)              1.017           0.837
C1.7G(128KB L2)                  2.881           1.964
HPC 3000(512KB) (
优化)      0.6             0.667

这份数据是拿1M的时间比10的时间,容器选择了树型容器set/multiset
前四个的编译器都是vc7.1的数据。
从结果可以很明显的看出cache小的CPU效率下降明显(cache命中率下降)。
PIII 500
虽然L2 cache只是半速,但是在1M大的set/multisetsort中明显
超越了C1.7G。而对于SunUltraSPARC II是这里唯一一个64bitCPU,可以
0.6的比值,相当厉害啊。

 

原创粉丝点击