Selecting MV Strategy for Temporal ERC

In spatial erc method, how to find better MV for P or B frame is rather important.

Zero MV
Just copy！

Neighboring MV and Collocated MV
The neighboring MV and collocated MV from corrupted MV.

Median MV
mv = Median(mv1,mv2,mv3)
where mv1 = (sum of right part MV)/3
mv2 = (sum of left down two MV)/2
mv3 = Median(left three MV)
These eight candidate MV is neighboring of corrupted MB
If neighboring MV is not available or I type, setting it to zero.
This method doesn’t need calculate distortion.

BMA MV
Using neighboring mv to get restored MB, calculate distortion and select the best one among the eight MVs set. Needing lots of calculation and lacking of satisfied effect.
MV comes from：
the same block in previous picture;the median of neighboring block;the available neighboring block;the average mv;zero vector.
This method is suitable for FMO.

EBMA MV
When vector and difference are lost, try with available neighboring blocking’s difference and candidate MV from BMA. Above two techniques are invented by Wai, Amy, Bede in Princeton.

Multi Ref Frame MV
All above methods are consider that the cur frame can get neighboring MB, but unfortunately, whole frame lost.
This technique is mainly for pixel-level.
Set ref frame number = L, generate MV history then get average MV for cur frame. Then use Markov random field to get half-pel resolution.
Above step making one pixel into four. Next getting four-size scalable picture, interpolation for missing pixel, filtering, down sampling.

BNM(best neighboring MV)
Just use average available neighboring corrupted MB and get the compensated MV location and extend it to HxL, and search best mv in this domain by MSE. But this method needs neighboring corrupted MB information, not good for video stream application.

Previous Frame MV
The corrupted MB j’s previous frame’s mv Pmv(j)
Set Th(high motion vector)
If not Pmv(j), get from neighbor.
If multiple mv exists, get minimal one.

Forward Motion Vector Extrapolation
This method can conceal the lost whole frame and it is block level.
When conceal corrupted block, get forward block and previous frame to see which block is in corrupted block.
And calculate every block’s weight, and get better MV for corrupted block.

OBEC(overlapped block error concealment)
Or MVE(motion vector estimation). Use previous frame’s mv to estimate. Then copy or interpolation, in the end, using post-processing.

P-OBEC(pixel OBEC)
Extrapolate MV then using window for pixel.

MV-OBEC(MV OBEC)
Window extrapolate MV and horizontal correlations of motion vectors are higher than the vertical corrections in typical video sequences.

VBSME(variable block size motion estimation)
Used in FMO mode application, such as, video meeting.
Using multiple sub-MV and adaptively get best MV by minimal times of computation.
With high complexity.

DS(Diamond Searching)
After BMA, using the encoder MV searching strategy to get the better MV(1/4 pixel). Including LDSP and SDSP pattern.

TR(temporal replacement)
Repeat the lost frame.

MVE
Block level, just using previous frame’s MV information to reconstruct a new vector field.

PMVE
Pixel level, just using previous frame’s MV information to reconstruct a new vector field. But this method has high complex computation and high accuracy.

HMVE
Combine MVE and PMVE to a new technique.
This method produces a new state——the pixel that in the overlapped block but without overlapped itself.
This method first divide covered pixel into three parts， then first covered pixel Using the selecting strategy to find the best MV and averaging the sum. As the pixel not covered but in the covered block will be concealed by covered max MV and covered averaging MV. As for not covered block, it will use the previous frame’s MV.

BC
A new method to get backward motion vector not from the front, suiting for non-real time erc video restoration.

EBC(enhanced backward concealment)
This method improves the quality of erc based on bc and mev. Making the unrecovered pixel into two parts.

SAM(Size-Adaptive Matching)
Set a threshold for SSIM, if smaller, using division until satisfied or end.

DP AND CROSS SMOOTHNESS
Using a new criteria with cross smoothness and dp to calculate the distance. This method can get a good result for smoothness but with high computation.
This technique uses global selecting MV and global optimal.

Greedy Algorithm
This method is local optimal and needs low computation.

OAKF(optimized adaptive Kalman Filtering)
Changing Kalman filter gain to lower computation complexity with cross smoothness.

MFI（motion Field interpolation）
After MVE, there is also missing block without MV. Using simple interpolation may be good.

Multi-Source MV
MV comes from reference frame, upcoming frame and reset frame.
Then using boundary matching.

Motion Stability Estimation
Using forward and backward information to conceal corrupted frame.
Stable MV: the MV list just has only one MV after merging.
Unstable MV: the other MV.
After conceal stable MV, then conceal corrupted frame with selective MV from unstable MV list, neighboring MV, zero MV.

BWMVE(Bi-Directional Weighted MVE)
Pn(x,y)=w1Pn,n−1(x,y)+w1Pn,n+1(x,y)+w3PDMC(x,y)w1+w2+w3

w3=max(0,W−max(w1,w2)) W = 12

AVLI(Adaptive Vector Length Interpolation)
According to SAD, choosing method between VLI(Vector Length Interpolation) and BPA(Bii-directional Pixel Average).
BPA uses average pixel value.
VLI uses MV projection.

Multi-Frame Based Block and Adaptive Weight Model
Using previous two frame as reference and for frame Fn−1, there exists a window and projection all the MV in the window project into frame Fn−1, then using SAD to choose a good MV.
As for adaptive weight model, wk=1|fn−2(x+2mvkx,y+2mvky)−fn−1(x+2mvkx,y+2mvky)|

So the estimeting MV is f^n(x,y)=∑kwkfkn(x,y)