Memory management: default new & delete vs. allocator & deallocator using a pool

/***********************************************************************************************
All rights reserved by Allen Young. Welcome to communicate and discuss. If you want to quote this article, please declare its origin. Thank you!
***********************************************************************************************/

C++ provides two operators for dynamic memory management: new and delete. They are general operators, which means they can handle requests of different amount of memory. It seems to be perfect, however, the execution performance and the space efficiency are not good enough sometimes. Their flexibilities leave some room for improvement in performance and efficiency, and this is the main reason that you want to write your own version of new and delete.

    First of all, you should know that the execution process of new is divided into two steps: first allocate enough raw space, if succeeds, then construct the object and place it in the space; if fails, return immediately (actually is more complex than just return, but it is not related to the topic of this article). So it is with delete: first destruct the object, then deallocate the space. This article focuses on the allocation and deallocation of memory space (the construction and destruction is covered in another article).

    Operator new always give you more memory than you request. You may doubt about it, but it really works in that way. Let us see the form of using delete: it has only one argument which is the pointer of memory to give back to the heap. Ok, but how large is this memory? It doesn't know! So, when new is performed, it adds some extra space, which is used to store the size of the requested memory, to the head of the memory block given to user. And then delete uses that information besides the pointer passed in, to do the cleanup. This is transparent to the user, but it can lead to very low space efficiency when there are many small objects in your program because a large part of the allocated memory is used for bookkeeping.

    How to avoid this waste? The answer is using a memory pool. It can improve the space efficiency, as well as the execution performance, by using a linked list to manage the small memory blocks. At the first time that newing an object, allocate a large memory block and make it a linked list, in which the nodes are of the same size as that object. Then take one node out for use when newing, and put it back to the list when deleting. Many materials have talked about this. The following example and its implementation is from Item 10 in <Effective C++>, by Scott Meyers (Slight changes are made).

//Airplane.h

#ifndef _AIRPLANE_H
#define _AIRPLANE_H

#include <cstddef>

// this class represents the real information of an airplane
class AirplaneReq ...{
  // many data and method members
};

// this class holds the address of a real airplane
class Airplane ...{
 public:
  static void * operator new(size_t);
  static void operator delete(void *, size_t);
 public:
  union ...{
    AirplaneReq * req;
    Airplane *    next;
  };
  static const int BLOCK_SIZE;
  static Airplane * head_of_free_list;
};

#endif

    A little trick is used here: a union holds both the pointer to the real representations of a plane, and the pointer to next Airplane block. There is no conflict, because the pointer to next block is used only when that block is in the linked list. When allocated successfully, the pointer to the real representations is used only.

//Airplane.cpp

#include "Airplane.h"

const int Airplane::BLOCK_SIZE = 128;
Airplane * Airplane::head_of_free_list = 0;

void * Airplane::operator new(size_t size) ...{
  if(size != sizeof(Airplane))
    return ::operator new(size);

  Airplane * p = head_of_free_list;

  if(p) ...{
    head_of_free_list = p->next;
  }
  else ...{
    // create the link list
    Airplane * new_block = static_cast<Airplane * > (::operator new(BLOCK_SIZE * sizeof(Airplane)));
    for(int i = 0; i < BLOCK_SIZE - 1; ++i)
      new_block[i].next = &new_block[i+1];
    new_block[BLOCK_SIZE - 1].next = 0;
    p = new_block;
    head_of_free_list = &new_block[1];
  }
  
  return p;
}

void Airplane::operator delete(void * dead_object, size_t size) ...{
  if(dead_object == 0)
    return;

  if(size != sizeof(Airplane)) ...{
    ::operator delete(dead_object);
    return;
  }

  Airplane * p = static_cast<Airplane * > (dead_object);
  p->next = head_of_free_list;
  head_of_free_list = p;
}

    Now we need some test code..

//item10.cpp

#include "Airplane.h"
#include <iostream>
using namespace std;

int main() ...{
  Airplane * ap0 = new Airplane();
  cout << "ap0: " << ap0 << endl;
  Airplane * ap1 = new Airplane();
  cout << "ap1: " << ap1 << endl;
  cout << "head:" << Airplane::head_of_free_list << endl;
  Airplane * ap2 = new Airplane();
  cout << "ap2: " << ap2 << endl;
  cout << "head:" << Airplane::head_of_free_list << endl;
  delete ap2;
  Airplane * ap3 = new Airplane();
  cout << "ap3: " << ap3 << endl;
  cout << "head:" << Airplane::head_of_free_list << endl;
  delete ap1;
  Airplane * ap4 = new Airplane();
  cout << "ap4: " << ap4 << endl;
  cout << "head:" << Airplane::head_of_free_list << endl;
  return 0;
}

    Let us see its running results.
//printed in the terminal
ap0: 0x804a008
ap1: 0x804a00c
head:0x804a010
ap2: 0x804a010
head:0x804a014
ap3: 0x804a010
head:0x804a014
ap4: 0x804a00c
head:0x804a014

    Ok, now it can work. But as Scott mentioned at the end of that item, it can be better by using a pool independent of specific classes. The following is my implementation (the underlying mechanism and code come from the implementation of sgi stl allocator and <The annotated stl sources> by Hou Jie, but code is rewrite to fit our situation here).

//MemoryPool.h

#ifndef _MEMORY_POOL_H
#define _MEMORY_POOL_H

#include <cstddef>

class MemoryPool ...{
 public:
  MemoryPool(size_t);
  ~MemoryPool();
  void * Alloc(size_t);
  void Free(void *, size_t);
 public:
  union obj ...{
    union obj * next;
    char data[1];
  };
  size_t obj_size, block_size;
  obj * head_of_free_list;
};

#endif

//MemoryPool.cpp

#include "MemoryPool.h"
#include <iostream>
using namespace std;

MemoryPool::MemoryPool(size_t size)
  : obj_size(size), block_size(16)
...{}

MemoryPool::~MemoryPool() ...{
  ::operator delete((char *)head_of_free_list);
}

void * MemoryPool::Alloc(size_t size) ...{
  char * chunk;
  obj * result = head_of_free_list;
  obj * current, * next;
  if(result == 0) ...{
    chunk = static_cast<char *>(::operator new (size * block_size));
    result = (obj *)chunk;
    current = next = (obj *)(chunk + obj_size);
    for(int i = 1; i < block_size - 1; ++i) ...{
      next = (obj *)((char *)next + obj_size);
      current->next = next;
      current = next;
    }
    current->next = 0;
    head_of_free_list = (obj *)(chunk + obj_size);
  }
  else ...{
    head_of_free_list = result->next;
  }
  return result;
}

void MemoryPool::Free(void * dead_object, size_t size) ...{
  if(dead_object == 0)
    return;
  
  obj * p = (obj *)dead_object;
  p->next = head_of_free_list;
  head_of_free_list = p;
}

    The parameter of constructor is the object size. The block size is the size of the pool, i.e., the number of objects the pool can hold. You can make it the second parameter of constructor.

//AirplaneWithPool.h

#ifndef _AIRPLANE_WITH_POOL_H
#define _AIRPLANE_WITH_POOL_H

#include "MemoryPool.h"
#include <cstddef>

class AirplaneReq ...{

};

class AirplaneWithPool ...{
 public:
  static void * operator new(size_t);
  static void operator delete(void *, size_t);
 public:
  AirplaneReq * req;
  static MemoryPool mem_pool;
};

#endif

//AirplaneWithPool.cpp

#include "AirplaneWithPool.h"
#include "MemoryPool.h"

MemoryPool AirplaneWithPool::mem_pool(sizeof(AirplaneWithPool));

void * AirplaneWithPool::operator new(size_t size) ...{
  return mem_pool.Alloc(size);
}

void AirplaneWithPool::operator delete(void * dead_object, size_t size) ...{
  mem_pool.Free(dead_object, size);
}

    Some test code is needed.

//item10WithPool.cpp

#include "AirplaneWithPool.h"
#include <iostream>
using namespace std;

int main() ...{
  AirplaneWithPool * apwp0 = new AirplaneWithPool();
  cout << "apwp0: " << apwp0 << endl;
  cout << "head:  " << apwp0->mem_pool.head_of_free_list << endl;
  AirplaneWithPool * apwp1 = new AirplaneWithPool();
  cout << "apwp1: " << apwp1 << endl;
  cout << "head:  " << apwp1->mem_pool.head_of_free_list << endl;
  AirplaneWithPool * apwp2 = new AirplaneWithPool();
  cout << "apwp2: " << apwp2 << endl;
  cout << "head:  " << apwp2->mem_pool.head_of_free_list << endl;
  delete apwp1;
  AirplaneWithPool * apwp3 = new AirplaneWithPool();
  cout << "apwp3: " << apwp3 << endl;
  cout << "head:  " << apwp3->mem_pool.head_of_free_list << endl;
  delete apwp3;
  AirplaneWithPool * apwp4 = new AirplaneWithPool();
  cout << "apwp4: " << apwp4 << endl;
  cout << "head:  " << apwp4->mem_pool.head_of_free_list << endl;
  return 0;
}

    Let us see its running results.
//printed in the terminal
apwp0: 0x804a008
head:  0x804a00c
apwp1: 0x804a00c
head:  0x804a010
apwp2: 0x804a010
head:  0x804a014
apwp3: 0x804a00c
head:  0x804a014
apwp4: 0x804a00c
head:  0x804a014

    To make a comparison, let us see what will happen if the global operator new is used other than a memory pool. For the sake of simpleness, I just changed one statement in AirplaneWithPool::operator new() and AirplaneWithPool::operator delete().

//changed code
//AirplaneWithPool.cpp

#include "AirplaneWithPool.h"
#include "MemoryPool.h"

MemoryPool AirplaneWithPool::mem_pool(sizeof(AirplaneWithPool));

void * AirplaneWithPool::operator new(size_t size) ...{
  //return mem_pool.Alloc(size);
  return ::operator new(size);
}

void AirplaneWithPool::operator delete(void * dead_object, size_t size) ...{
  //mem_pool.Free(dead_object, size);
  ::operator delete(dead_object);
}

    Let us see the running results again.
//printed in the terminal
apwp0: 0x804a008
head:  0
apwp1: 0x804a018
head:  0
apwp2: 0x804a028
head:  0
apwp3: 0x804a018
head:  0
apwp4: 0x804a018
head:  0

    It is obvious that the global new wastes a lot more memory space for bookkeeping. Thus, next time when you have some small objects to create, write your own version of new and delete, and use memory pool as the underlying mechanism for memory management.

References:
Scott Meyers, <Effective C++>
Hou Jie, <The annotated stl sources>