There is no best sorting algoritm： 没有最好关键是找到合适的。

最近想到一个问题， 最好的sorting algorithm 是什么？？

sorting 分为comparision sort 和 noncomparision sort 两大类。

要回答这个问题不是一件容易的事情。  或者说， 这个问题给出的信息太少， 没有确定的答案。

我们知道， 要评价一个算法， 指标通常是时间复杂度和空间复杂度。 但是这两个指标是抽象后出来的结果， 使用big O notation等 量化。 只能适合于理论分析， 给出我们关于那个算法是最优的一个rough idea。 但是真正实际生活中， 算法的performance远非这两个指标所能概括的。

算法的性能表现不仅和算法本身有关， 而且和输入的数据的分布情况有很大关系。

评论一个算法是否优秀， 有如下几个指标：

（1）average case, best case, and worst case efficiency.  Some algorithms, such as quick sort, perform exceptionally well for some inputs, but horribly for others. Other algorithms, such as merge sort, are unaffected by the order of input data. Even a modified version of bubble sort can finish in O(n) for the most favorable inputs. 

（2）A second factor is the "constant term". As big-O notation abstracts away many of the details of a process, it is quite useful for looking at the big picture. But one thing that gets dropped out is the constant in front of the expression: for instance, O(c*n) is just O(n). In the real world, the constant, c, will vary across different algorithms. A well-implemented quicksort should have a much smaller constant multiplier than heap sort. 

（3）A second criterion for judging algorithms is their space requirement -- do they require scratch space or can the array be sorted in place (without additional memory beyond a few variables)? Some algorithms never require extra space, whereas some are most easily understood when implemented with extra space (heap sort, for instance, can be done in place, but conceptually it is much easier to think of a separate heap). Space requirements may even depend on the data structure used (merge sort on arrays versus merge sort on linked lists, for instance). 

（4）A third criterion is stability -- does the sort preserve the order of keys with equal values? Most simple sorts do just this, but some sorts, such as heap sort, do not. 

The following chart compares sorting algorithms on the various criteria outlined above; the algorithms with higher constant terms appear first, though this is clearly an implementation-dependent concept and should only be taken as a rough guide when picking between sorts of the same big-O efficiency. 

 Time SortAverageBestWorstSpaceStabilityRemarksBubble sortO(n^2)O(n^2)O(n^2)ConstantStableAlways use a modified bubble sortModified Bubble sortO(n^2)O(n)O(n^2)ConstantStableStops after reaching a sorted arraySelection SortO(n^2)O(n^2)O(n^2)ConstantStableEven a perfectly sorted input requires scanning the entire arrayInsertion SortO(n^2)O(n)O(n^2)ConstantStableIn the best case (already sorted), every insert requires constant timeHeap SortO(n*log(n))O(n*log(n))O(n*log(n))ConstantInstableBy using input array as storage for the heap, it is possible to achieve constant spaceMerge SortO(n*log(n))O(n*log(n))O(n*log(n))DependsStableOn arrays, merge sort requires O(n) space; on linked lists, merge sort requires constant spaceQuicksortO(n*log(n))O(n*log(n))O(n^2)ConstantStableRandomly picking a pivot value (or shuffling the array prior to sorting) can help avoid worst case scenarios such as a perfectly sorted array.

stackexhange 上曾经出现类似的问题。 就是（附带答案）

Why is quicksort better than other sorting algorithms in practice?

Short Answer

The cache efficiency argument has already been explained in detail. In addition, there is an intrinsic argument, why Quicksort is fast. If implemented like with two “crossing pointers”, e.g. here, the inner loops have a very small body. As this is the code executed most often, this pays off.

Long Answer

First of all,

The Average Case does not exist!

As best and worst case often are extremes rarely occurring in practice, average case analysis is done. But any average case analysis assume some distribution of inputs! For sorting, the typical choice is the random permutation model (tacitly assumed on Wikipedia).

Why O-Notation?

Discarding constants in analysis of algorithms is done for one main reason: If I am interested in exactrunning times, I need (relative) costs of all involved basic operations (even still ignoring caching issues, pipelining in modern processors ...). Mathematical analysis can count how often each instruction is executed, but running times of single instructions depend on processor details, e.g. whether a 32-bit integer multiplication takes as much time as addition.

There are two ways out:

1. Fix some machine model.

   This is done in Don Knuth's book series “The Art of Computer Programming” for an artificial “typical” computer invented by the author. In volume 3 you find exact average case results for many sorting algorithms, e.g.

   • Quicksort: 11.667(n+1)ln(n)−1.74n−18.74
   • Mergesort: 12.5nln(n)
   • Heapsort: 16nln(n)+0.01n
   • Insertionsort: 2.25n2+7.75n−3ln(n) 
     [source]

   These results indicate that Quicksort is fastest. But, it is only proved on Knuth's artificial machine, it does not necessarily imply anything for say your x86 PC. Note also that the algorithms relate differently for small inputs:
   
   [source]

2. Analyse abstract basic operations.

   For comparison based sorting, this typically is swaps and key comparisons. In Robert Sedgewick's books, e.g. “Algorithms”, this approach is pursued. You find there

   • Quicksort: 2nln(n) comparisons and 13nln(n) swaps on average
   • Mergesort: 1.44nln(n) comparisons, but up to 8.66nln(n) array accesses (mergesort is not swap based, so we cannot count that).
   • Insertionsort: 14n2 comparisons and 14n2 swaps on average.

   As you see, this does not readily allow comparisons of algorithms as the exact runtime analysis, but results are independent from machine details.

There are multiple points that can be made regarding this question.

Quicksort is usually fast

Although Quicksort has worst-case O(n2) behaviour, it is usually fast: assuming random pivot selection, there's a very large chance we pick some number that separates the input into two similarly sized subsets, which is exactly what we want to have.

In particular, even if we pick a pivot that creates a 10%-90% split every 10 splits (which is a meh split), and a 1 element - n−1 element split otherwise (which is the worst split you can get), our running time is still O(nlogn) (note that this would blow up the constants to a point that Merge sort is probably faster though).

Quicksort is usually faster than most sorts

Quicksort is usually faster than sorts that are slower than O(nlogn) (say, Insertion sort with its O(n2) running time), simply because for large n their running times explode.

A good reason why Quicksort is so fast in practice compared to most other O(nlogn) algorithms such as Heapsort, is because it is relatively cache-efficient. Its running time is actually O(nBlog(nB)), where B is the block size. Heapsort, on the other hand, doesn't have any such speedup: it's not at all accessing memory cache-efficiently.

The reason for this cache efficiency is that it linearly scans the input and linearly partitions the input. This means we can make the most of every cache load we do as we read every number we load into the cache before swapping that cache for another. In particular, the algorithm is cache-oblivious, which gives good cache performance for every cache level, which is another win.

Cache efficiency could be further improved to O(nBlogMB(nB)), where M is the size of our main memory, if we use k-way Quicksort. Note that Mergesort also has the same cache-efficiency as Quicksort, and its k-way version in fact has better performance (through lower constant factors) if memory is a severe constrain. This gives rise to the next point: we'll need to compare Quicksort to Mergesort on other factors.

Quicksort is usually faster than Mergesort

This comparison is completely about constant factors (if we consider the typical case). In particular, the choice is between a suboptimal choice of the pivot for Quicksort versus the copy of the entire input for Mergesort (or the complexity of the algorithm needed to avoid this copying). It turns out that the former is more efficient: there's no theory behind this, it just happens to be faster.

Note that Quicksort will make more recursive calls, but allocating stack space is cheap (almost free in fact, as long as you don't blow the stack) and you re-use it. Allocating a giant block on the heap (or your hard drive, if n is really large) is quite a bit more expensive, but both are O(logn) overheads that pale in comparison to the O(n) work mentioned above.

Lastly, note that Quicksort is slightly sensitive to input that happens to be in the right order, in which case it can skip some swaps. Mergesort doesn't have any such optimizations, which also makes Quicksort a bit faster compared to Mergesort.

Use the sort that suits your needs

In conclusion: no sorting algorithm is always optimal. Choose whichever one suits your needs. If you need an algorithm that is the quickest for most cases, and you don't mind it might end up being a bit slow in rare cases, and you don't need a stable sort, use Quicksort. Otherwise, use the algorithm that suits your needs better.

shareimprove this answer

edited Mar 7 '12 at 0:57

answered Mar 6 '12 at 23:58

Alex ten Brink
5,1131849

 

 

Your last remark is especially valuable. A colleague of mine currently analyses Quicksort implementations under different input distributions. Some of them break down for many duplicates, for instance. –  Raphael♦Mar 7 '12 at 0:35

3 

@Raphael, take a look at McIllroy's "A Killer Adversary for Quicksort", Software - Practice and Experience 29(4), 341-344 (1999). It describes a devious technique to make Quicksort take O(n2) time always. Bentley and McIllroy's "Engineering a Sort Function", Software - Practice and Experience 23(11), 1249-1265 (1993) might also be relevant. –  vonbrand Feb 1 '13 at 12:37

 

"[T]here's no theory behind this, it just happens to be faster." That statement is highly unsatisfactory from a scientific point of view. Imagine Newton saying, "Butterflies fly up, apples fall down: there's no theory behind this, apples just happen to fall." –  David Richerby Jul 24 at 9:01

 

@Alex ten Brink, what do you mean with “In particular, the algorithm is cache-oblivious”? –  Hibou57 Jul 24 at 20:19

1 

@David Richerby, “That statement is highly unsatisfactory from a scientific point of view”: he may be just witnessing a fact without pretending we should be happy with it. Some algorithm families suffers from a lack of full formalisation; hashing functions are an example case. –  Hibou57 Jul 24 at 20:31