环形缓冲

来源：互联网发布：中国教育网络期刊编辑：程序博客网时间：2024/04/28 17:33

http://en.wikipedia.org/wiki/Circular_buffer#Circular_buffer_mechanics

Amending our Terms of Use:
Please comment on a proposed amendment regarding undisclosed paid editing.

[ Help with translations! ]

Circular buffer

From Wikipedia, the free encyclopedia

This article's tone or style may not reflect the encyclopedic tone used on Wikipedia. See Wikipedia'sguide to writing better articles for suggestions. (April 2011)

This article does not cite any references or sources. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. (April 2011)

This article or section appears to contradict itself about the end pointer: it is confused with the next pointer, resulting in a contradiction. Please see the talk page for more information. (April 2013)

A ring showing, conceptually, a circular buffer. This visually shows that the buffer has no real end and it can loop around the buffer. However, since memory is never physically created as a ring, a linear representation is generally used as is done below.

A circular buffer, cyclic buffer or ring buffer is a data structure that uses a single, fixed-size buffer as if it were connected end-to-end. This structure lends itself easily to buffering data streams.

[hide]

1 Uses
2 How it works
3 Circular buffer mechanics
- 3.1 Start / End Pointers
4 Difficulties
- 4.1 Full / Empty Buffer Distinction
  - 4.1.1 Always Keep One Slot Open
  - 4.1.2 Use a Fill Count
  - 4.1.3 Mirroring
  - 4.1.4 Read / Write Counts
  - 4.1.5 Absolute indices
  - 4.1.6 Record last operation
- 4.2 Multiple Read Pointers
- 4.3 Chunked Buffer
5 Optimization
- 5.1 Optimized POSIX implementation
6 Variants
7 External links

Uses[edit]

The useful property of a circular buffer is that it does not need to have its elements shuffled around when one is consumed. (If a non-circular buffer were used then it would be necessary to shift all elements when one is consumed.) In other words, the circular buffer is well-suited as a FIFO buffer while a standard, non-circular buffer is well suited as aLIFO buffer.

Circular buffering makes a good implementation strategy for a queue that has fixed maximum size. Should a maximum size be adopted for a queue, then a circular buffer is a completely ideal implementation; all queue operations are constant time. However, expanding a circular buffer requires shifting memory, which is comparatively costly. For arbitrarily expanding queues, a linked list approach may be preferred instead.

In some situations, overwriting circular buffer can be used, e.g. in multimedia. If the buffer is used as the bounded buffer in the producer-consumer problem then it is probably desired for the producer (e.g., an audio generator) to overwrite old data if the consumer (e.g., the sound card) is unable to momentarily keep up. Also, the LZ77 family of lossless data compression algorithms operates on the assumption that strings seen more recently in a data stream are more likely to occur soon in the stream. Implementations store the most recent data in a circular buffer.

How it works[edit]

A circular buffer first starts empty and of some predefined length. For example, this is a 7-element buffer:

Assume that a 1 is written into the middle of the buffer (exact starting location does not matter in a circular buffer):

Then assume that two more elements are added — 2 & 3 — which get appended after the 1:

If two elements are then removed from the buffer, the oldest values inside the buffer are removed. The two elements removed, in this case, are 1 & 2 leaving the buffer with just a 3:

If the buffer has 7 elements then it is completely full:

A consequence of the circular buffer is that when it is full and a subsequent write is performed, then it starts overwriting the oldest data. In this case, two more elements — A & B — are added and they overwrite the 3 & 4:

Alternatively, the routines that manage the buffer could prevent overwriting the data and return an error or raise an exception. Whether or not data is overwritten is up to the semantics of the buffer routines or the application using the circular buffer.

Finally, if two elements are now removed then what would be returned is not 3 & 4 but 5 & 6 because A & B overwrote the 3 & the 4 yielding the buffer with:

Circular buffer mechanics[edit]

What is not shown in the example above is the mechanics of how the circular buffer is managed.

Start / End Pointers[edit]

Generally, a circular buffer requires four pointers:

one to the actual buffer in memory
one to the buffer end in memory (or alternately: the size of the buffer)
one to point to the start of valid data (or alternately: amount of data written to the buffer)
one to point to the end of valid data (or alternately: amount of data read from the buffer)

Alternatively, a fixed-length buffer with two integers to keep track of indices can be used in languages that do not have pointers.

Taking a couple of examples from above. (While there are numerous ways to label the pointers and exact semantics can vary, this is one way to do it.)

This image shows a partially full buffer:

This image shows a full buffer with two elements having been overwritten:

What to note about the second one is that after each element is overwritten then the start pointer is incremented as well.

Difficulties[edit]

Full / Empty Buffer Distinction[edit]

A small disadvantage of relying on pointers or relative indices of the start and end of data is, that in the case the buffer is entirely full, both pointers point to the same element:^{[contradiction]}

This is exactly the same situation as when the buffer is empty:

To solve this confusion there are a number of solutions:

Always keep one slot open.
Use a fill count to distinguish the two cases.
Use an extra mirroring bit to distinguish the two cases.
Use read and write counts to get the fill count from.
Use absolute indices.
Record last operation.

Always Keep One Slot Open[edit]

This design always keeps one slot unallocated. A full buffer has at most $({\text{size}}-1)$ slots. If both pointers refer to the same slot, the buffer is empty. If the end (write) pointer refers to the slot preceding the one referred to by the start (read) pointer, the buffer is full.

The advantage is:

The solution is simple and robust.

The disadvantages are:

One slot is lost, so it is a bad compromise when the buffer size is small or the slot is big or is implemented in hardware.
The full test requires a modulo operation

Example implementation, 'C' language

/* Circular buffer example, keeps one slot open */ #include <stdio.h>#include <stdlib.h> /* Opaque buffer element type.  This would be defined by the application. */typedef struct { int value; } ElemType; /* Circular buffer object */typedef struct {    int         size;   /* maximum number of elements           */    int         start;  /* index of oldest element              */    int         end;    /* index at which to write new element  */    ElemType   *elems;  /* vector of elements                   */} CircularBuffer; void cbInit(CircularBuffer *cb, int size) {    cb->size  = size + 1; /* include empty elem */    cb->start = 0;    cb->end   = 0;    cb->elems = (ElemType *)calloc(cb->size, sizeof(ElemType));} void cbFree(CircularBuffer *cb) {    free(cb->elems); /* OK if null */} int cbIsFull(CircularBuffer *cb) {    return (cb->end + 1) % cb->size == cb->start;} int cbIsEmpty(CircularBuffer *cb) {    return cb->end == cb->start;} /* Write an element, overwriting oldest element if buffer is full. App can   choose to avoid the overwrite by checking cbIsFull(). */void cbWrite(CircularBuffer *cb, ElemType *elem) {    cb->elems[cb->end] = *elem;    cb->end = (cb->end + 1) % cb->size;    if (cb->end == cb->start)        cb->start = (cb->start + 1) % cb->size; /* full, overwrite */} /* Read oldest element. App must ensure !cbIsEmpty() first. */void cbRead(CircularBuffer *cb, ElemType *elem) {    *elem = cb->elems[cb->start];    cb->start = (cb->start + 1) % cb->size;} int main(int argc, char **argv) {    CircularBuffer cb;    ElemType elem = {0};     int testBufferSize = 10; /* arbitrary size */    cbInit(&cb, testBufferSize);     /* Fill buffer with test elements 3 times */    for (elem.value = 0; elem.value < 3 * testBufferSize; ++ elem.value)        cbWrite(&cb, &elem);     /* Remove and print all elements */    while (!cbIsEmpty(&cb)) {        cbRead(&cb, &elem);        printf("%d\n", elem.value);    }     cbFree(&cb);    return 0;}

Use a Fill Count[edit]

This approach replaces the end pointer with a counter that tracks the number of readable items in the buffer. This unambiguously indicates when the buffer is empty or full and allows use of all buffer slots.

The performance impact should be negligible, since this approach adds the costs of maintaining the counter and computing the tail slot on writes but eliminates the need to maintain the end pointer and simplifies the fullness test.

The advantage is:

The test for full/empty is simple

The disadvantages are:

You need modulo for read and write
Read and write operation must share the counter field, so it requires synchronization in multi-threaded situation.

Note: When using semaphores in a Producer-consumer model, the semaphores act as a fill count.

Differences from previous example

/* This approach replaces the CircularBuffer 'end' field with the   'count' field and changes these functions: */ void cbInit(CircularBuffer *cb, int size) {    cb->size  = size;    cb->start = 0;    cb->count = 0;    cb->elems = (ElemType *)calloc(cb->size, sizeof(ElemType));} int cbIsFull(CircularBuffer *cb) {    return cb->count == cb->size; } int cbIsEmpty(CircularBuffer *cb) {    return cb->count == 0; } void cbWrite(CircularBuffer *cb, ElemType *elem) {    int end = (cb->start + cb->count) % cb->size;    cb->elems[end] = *elem;    if (cb->count == cb->size)        cb->start = (cb->start + 1) % cb->size; /* full, overwrite */    else        ++ cb->count;} void cbRead(CircularBuffer *cb, ElemType *elem) {    *elem = cb->elems[cb->start];    cb->start = (cb->start + 1) % cb->size;    -- cb->count;}

Mirroring[edit]

Another solution is to remember the number of times each read and write pointers have wrapped and compare this to distinguish empty and full situations. In fact only the parity of the number of wraps is necessary, so you only need to keep an extra bit. You can see this as if the buffer adds a virtual mirror and the pointers point either to the normal or to the mirrored buffer.

It is easy to see above that when the pointers (including the extra msb bit) are equal, the buffer is empty, while if the pointers differ only by the extra msb bit, the buffer is full.

The advantages are:

The test for full/empty is simple
No modulo operation is needed
The source and sink of data can implement independent policies for dealing with a full buffer and overrun while adhering to the rule that only the source of data modifies the write count and only the sink of data modifies the read count. This can result in elegant and robust circular buffer implementations even in multi-threaded environments.

The disadvantage is:

You need one more bit for read and write pointer

Differences from Always Keep One Slot Open example

/* This approach adds one bit to end and start pointers */ /* Circular buffer object */typedef struct {    int         size;   /* maximum number of elements           */    int         start;  /* index of oldest element              */    int         end;    /* index at which to write new element  */    int         s_msb;    int         e_msb;    ElemType   *elems;  /* vector of elements                   */} CircularBuffer; void cbInit(CircularBuffer *cb, int size) {    cb->size  = size;    cb->start = 0;    cb->end   = 0;    cb->s_msb = 0;    cb->e_msb = 0;    cb->elems = (ElemType *)calloc(cb->size, sizeof(ElemType));} int cbIsFull(CircularBuffer *cb) {    return cb->end == cb->start && cb->e_msb != cb->s_msb; } int cbIsEmpty(CircularBuffer *cb) {    return cb->end == cb->start && cb->e_msb == cb->s_msb; } void cbIncr(CircularBuffer *cb, int *p, int *msb) {    *p = *p + 1;    if (*p == cb->size) {        *msb ^= 1;        *p = 0;    }} void cbWrite(CircularBuffer *cb, ElemType *elem) {    cb->elems[cb->end] = *elem;    if (cbIsFull(cb)) /* full, overwrite moves start pointer */        cbIncr(cb, &cb->start, &cb->s_msb);    cbIncr(cb, &cb->end, &cb->e_msb);} void cbRead(CircularBuffer *cb, ElemType *elem) {    *elem = cb->elems[cb->start];    cbIncr(cb, &cb->start, &cb->s_msb);}

If the size is a power of two, the implementation is simpler and the separate msb variables are no longer necessary, removing the disadvantage:

Differences from Always Keep One Slot Open example

/* This approach adds one bit to end and start pointers */ /* Circular buffer object */typedef struct {    int         size;   /* maximum number of elements           */    int         start;  /* index of oldest element              */    int         end;    /* index at which to write new element  */    ElemType   *elems;  /* vector of elements                   */} CircularBuffer; void cbInit(CircularBuffer *cb, int size) {    cb->size  = size;    cb->start = 0;    cb->end   = 0;    cb->elems = (ElemType *)calloc(cb->size, sizeof(ElemType));} void cbPrint(CircularBuffer *cb) {    printf("size=0x%x, start=%d, end=%d\n", cb->size, cb->start, cb->end);} int cbIsFull(CircularBuffer *cb) {    return cb->end == (cb->start ^ cb->size); /* This inverts the most significant bit of start before comparison */ } int cbIsEmpty(CircularBuffer *cb) {    return cb->end == cb->start; } int cbIncr(CircularBuffer *cb, int p) {    return (p + 1)&(2*cb->size-1); /* start and end pointers incrementation is done modulo 2*size */} void cbWrite(CircularBuffer *cb, ElemType *elem) {    cb->elems[cb->end&(cb->size-1)] = *elem;    if (cbIsFull(cb)) /* full, overwrite moves start pointer */        cb->start = cbIncr(cb, cb->start);    cb->end = cbIncr(cb, cb->end);} void cbRead(CircularBuffer *cb, ElemType *elem) {    *elem = cb->elems[cb->start&(cb->size-1)];    cb->start = cbIncr(cb, cb->start);}

Read / Write Counts[edit]

Another solution is to keep counts of the number of items written to and read from the circular buffer. Both counts are stored in signed integer variables with numerical limits larger than the number of items that can be stored and are allowed to wrap freely.

The unsigned difference (write_count - read_count) always yields the number of items placed in the buffer and not yet retrieved. This can indicate that the buffer is empty, partially full, completely full (without waste of a storage location) or in a state of overrun.

The advantage is:

The source and sink of data can implement independent policies for dealing with a full buffer and overrun while adhering to the rule that only the source of data modifies the write count and only the sink of data modifies the read count. This can result in elegant and robust circular buffer implementations even in multi-threaded environments.

The disadvantage is:

You need two additional variables.

Absolute indices[edit]

It is possible to optimize the previous solution by using indices instead of pointers: indices can store read/write counts instead of the offset from start of the buffer, the separate variables in the above solution are removed and relative indices are obtained on the fly by division modulo the buffer's length.

The advantage is:

No extra variables are needed.

The disadvantages are:

Every access needs an additional modulo operation.
If counter wrap is possible, complex logic can be needed if the buffer's length is not a divisor of the counter's capacity.

On binary computers, both of these disadvantages disappear if the buffer's length is a power of two—at the cost of a constraint on possible buffers lengths.

Record last operation[edit]

Another solution is to keep a flag indicating whether the most recent operation was a read or a write. If the two pointers are equal, then the flag will show whether the buffer is full or empty: if the most recent operation was a write, the buffer must be full, and conversely if it was a read, it must be empty.

The advantages are:

Only a single bit needs to be stored (which may be particularly useful if the algorithm is implemented in hardware)
The test for full/empty is simple

The disadvantage is:

You need an extra variable
Read and write operation must share the flag, so it probably require synchronization in multi-threaded situation.

Multiple Read Pointers[edit]

A little bit more complex are multiple read pointers on the same circular buffer. This is useful if you have n threads, which are reading from the same buffer, but one thread writing to the buffer.

Chunked Buffer[edit]

Much more complex are different chunks of data in the same circular buffer. The writer is not only writing elements to the buffer, it also assigns these elements to chunks^{[citation needed]}.

The reader should not only be able to read from the buffer, it should also get informed about the chunk borders.

Example: The writer is reading data from small files, writing them into the same circular buffer. The reader is reading the data, but needs to know when and which file is starting at a given position.

Optimization[edit]

A circular-buffer implementation may be optimized by mapping the underlying buffer to two contiguous regions of virtual memory. (Naturally, the underlying buffer‘s length must then equal some multiple of the system’s page size.) Reading from and writing to the circular buffer may then be carried out with greater efficiency by means of direct memory access; those accesses which fall beyond the end of the first virtual-memory region will automatically wrap around to the beginning of the underlying buffer. When the read offset is advanced into the second virtual-memory region, both offsets—read and write—are decremented by the length of the underlying buffer.^[1]

Optimized POSIX implementation[edit]

#include <sys/mman.h>#include <stdlib.h>#include <unistd.h> #define report_exceptional_condition() abort () struct ring_buffer{  void *address;   unsigned long count_bytes;  unsigned long write_offset_bytes;  unsigned long read_offset_bytes;}; //Warning order should be at least 12 for Linuxvoidring_buffer_create (struct ring_buffer *buffer, unsigned long order){  char path[] = "/dev/shm/ring-buffer-XXXXXX";  int file_descriptor;  void *address;  int status;   file_descriptor = mkstemp (path);  if (file_descriptor < 0)    report_exceptional_condition ();   status = unlink (path);  if (status)    report_exceptional_condition ();   buffer->count_bytes = 1UL << order;  buffer->write_offset_bytes = 0;  buffer->read_offset_bytes = 0;   status = ftruncate (file_descriptor, buffer->count_bytes);  if (status)    report_exceptional_condition ();   buffer->address = mmap (NULL, buffer->count_bytes << 1, PROT_NONE,                          MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);   if (buffer->address == MAP_FAILED)    report_exceptional_condition ();   address =    mmap (buffer->address, buffer->count_bytes, PROT_READ | PROT_WRITE,          MAP_FIXED | MAP_SHARED, file_descriptor, 0);   if (address != buffer->address)    report_exceptional_condition ();   address = mmap (buffer->address + buffer->count_bytes,                  buffer->count_bytes, PROT_READ | PROT_WRITE,                  MAP_FIXED | MAP_SHARED, file_descriptor, 0);   if (address != buffer->address + buffer->count_bytes)    report_exceptional_condition ();   status = close (file_descriptor);  if (status)    report_exceptional_condition ();} voidring_buffer_free (struct ring_buffer *buffer){  int status;   status = munmap (buffer->address, buffer->count_bytes << 1);  if (status)    report_exceptional_condition ();} void *ring_buffer_write_address (struct ring_buffer *buffer){  /*** void pointer arithmetic is a constraint violation. ***/  return buffer->address + buffer->write_offset_bytes;} voidring_buffer_write_advance (struct ring_buffer *buffer,                           unsigned long count_bytes){  buffer->write_offset_bytes += count_bytes;} void *ring_buffer_read_address (struct ring_buffer *buffer){  return buffer->address + buffer->read_offset_bytes;} voidring_buffer_read_advance (struct ring_buffer *buffer,                          unsigned long count_bytes){  buffer->read_offset_bytes += count_bytes;   if (buffer->read_offset_bytes >= buffer->count_bytes)    {      buffer->read_offset_bytes -= buffer->count_bytes;      buffer->write_offset_bytes -= buffer->count_bytes;    }} unsigned longring_buffer_count_bytes (struct ring_buffer *buffer){  return buffer->write_offset_bytes - buffer->read_offset_bytes;} unsigned longring_buffer_count_free_bytes (struct ring_buffer *buffer){  return buffer->count_bytes - ring_buffer_count_bytes (buffer);} voidring_buffer_clear (struct ring_buffer *buffer){  buffer->write_offset_bytes = 0;  buffer->read_offset_bytes = 0;} /*Note, that initial anonymous mmap() can be avoided - after initial mmap() for descriptor fd,you can try mmap() with hinted address as (buffer->address + buffer->count_bytes) and if it fails -another one with hinted address as (buffer->address - buffer->count_bytes).Make sure MAP_FIXED is not used in such case, as under certain situations it could end with segfault.The advantage of such approach is, that it avoids requirement to map twice the amount you need initially(especially useful e.g. if you want to use hugetlbfs and the allowed amount is limited)and in context of gcc/glibc - you can avoid certain feature macros(MAP_ANONYMOUS usually requires one of: _BSD_SOURCE, _SVID_SOURCE or _GNU_SOURCE).*/

Variants[edit]

Perhaps the most common version of the circular buffer uses 8-bit bytes as elements.

Some implementations of the circular buffer use fixed-length elements that are bigger than 8-bit bytes -- 16-bit integers for audio buffers, 53-byte ATM cells for telecom buffers, etc. Each item is contiguous and has the correct data alignment, so software reading and writing these values can be faster than software that handles non-contiguous and non-aligned values.

Ping-pong buffering can be considered a very specialized circular buffer with exactly two large fixed-length elements.

The Bip Buffer is very similar to a circular buffer, except it always returns contiguous blocks (which can be variable length).^[1]

External links[edit]

^ Jump up to:^a ^b Simon Cooke. "The Bip Buffer - The Circular Buffer with a Twist". 2003.

CircularBuffer at the Portland Pattern Repository
Boost: Templated Circular Buffer Container
http://www.dspguide.com/ch28/2.htm

[hide]

Data structures

Types

Collection
Container

Abstract

Associative array
Double-ended priority queue
Double-ended queue
List
Multimap
Priority queue
Queue
Set
- multiset
Disjoint Sets
Stack

Arrays

Bit array
Circular buffer
Dynamic array
Hash table
Hashed array tree
Sparse array

Linked

Association list
Linked list
Skip list
Unrolled linked list
XOR linked list

Trees

B-tree
Binary search tree
- AA
- AVL
- red-black
- self-balancing
- splay
Heap
- binary
- binomial
- Fibonacci
R-tree
- R*
- R+
- Hilbert
Trie
- Hash tree

Graphs

Binary decision diagram
Directed acyclic word graph

List of data structures

Categories:

Computer memory
Arrays

0 0

环形缓冲

Circular buffer

Contents

Uses[edit]

How it works[edit]

Circular buffer mechanics[edit]

Start / End Pointers[edit]

Difficulties[edit]

Full / Empty Buffer Distinction[edit]

Always Keep One Slot Open[edit]

Use a Fill Count[edit]

Mirroring[edit]

Read / Write Counts[edit]

Absolute indices[edit]

Record last operation[edit]

Multiple Read Pointers[edit]

Chunked Buffer[edit]

Optimization[edit]

Optimized POSIX implementation[edit]

Variants[edit]

External links[edit]

Navigation menu

Interaction

Tools

Print/export

Languages