Joint Video Experts Team (jvet) of itu-t sg 6 wp and iso/iec jtc 1/sc 29/wg 11

Aspects of MVD coding

Test#	Description	Document#
AFFINE	BMS AFFINE as bench mark
4.1.3.b	4.1.3.a + MVD prediction	JVET-K0337
4.1.5.b	4.1.5.a + MVD coding	JVET-K0185

Random access results

	VTM_tool_test					BMS_tool_test
Test#	Y	U	V	EncT	DecT	Y	U	V	EncT	DecT
AFFINE	-2.99%	-2.19%	-2.21%	137%	112%	-1.92%	-1.34%	-1.36%	108%	102%
4.1.3.a	-3.22%	-2.40%	-2.43%	134%	110%	-2.12%	-1.54%	-1.56%	112%	107%
4.1.3.b	-3.42%	-2.55%	-2.61%	136%	112%	-2.24%	-1.61%	-1.67%	111%	108%
4.1.5.a	-3.24%	-2.41%	-2.47%	138%	110%	-2.10%	-1.54%	-1.53%	110%	101%
4.1.5.b	-3.23%	-2.41%	-2.43%	139%	110%	-2.10%	-1.52%	-1.53%	110%	101%

Low delay B results

	VTM_tool_test					BMS_tool_test
Test#	Y	U	V	EncT	DecT	Y	U	V	EncT	DecT
AFFINE	-2.06%	-1.33%	-1.52%	168%	108%	-1.98%	-1.52%	-1.70%	125%	104%
4.1.3.a	-2.08%	-1.46%	-1.62%	162%	92%	-2.08%	-1.62%	-1.92%	127%	108%
4.1.3.b	-2.23%	-1.50%	-1.60%	164%	99%	-2.21%	-1.80%	-1.84%	127%	109%
4.1.5.a	-2.13%	-1.26%	-1.59%	169%	107%	-2.07%	-1.69%	-1.57%	126%	103%
4.1.5.b	-2.11%	-1.40%	-1.51%	170%	107%	-2.07%	-1.72%	-1.86%	126%	104%

Test 4.1.3.b tries to exploit the similarity of CPMVs of a coding block, refining the MV predictor of a CPMV by the MVD of the other one.

Test 4.1.5.b tries to save the bits for coding two MVDs by explicitly signalling whether the pair of MVDs in a prediction direction is skipped or not.

Decision: Adopt Test 4.1.3b to BMS affine. Scheme as shown in equations below.






Aspects of Alternative motion models

Test#	Description	Document#
AFFINE	BMS AFFINE as bench mark
4.1.3.c	4.1.3.b + CU level 4-para/6-para switching	JVET-K0337
4.1.3.d	4.1.3.c + slice level 4-para/6-para switching	JVET-K0337
4.1.5.c	4.1.5.b + 4 para/6-para switching	JVET-K0185
4.1.7.b	Four and six parameter model	JVET-K0094
4.1.7.c	Four and six parameter model with the conditional affine parameter signalling	JVET-K0094
4.1.8.a	Alternative 3-para (scaling) motion model	JVET-K0124
4.1.8.b	Alternative 3-para (rotation) motion model	JVET-K0124
4.1.8.c	4.1.8.a + 4.1.8.b	JVET-K0124

Random access results

	VTM_tool_test					BMS_tool_test
Test#	Y	U	V	EncT	DecT	Y	U	V	EncT	DecT
AFFINE	-2.99%	-2.19%	-2.21%	137%	112%	-1.92%	-1.34%	-1.36%	108%	102%
4.1.3.b	-3.42%	-2.55%	-2.61%	136%	112%	-2.24%	-1.61%	-1.67%	111%	108%
4.1.3.c	-3.90%	-2.94%	-2.92%	160%	113%	-2.72%	-1.96%	-1.97%	118%	107%
4.1.3.d	-3.88%	-2.95%	-2.92%	146%	113%	-2.69%	-1.98%	-2.00%	112%	106%
4.1.5.b	-3.23%	-2.41%	-2.43%	139%	110%	-2.10%	-1.52%	-1.53%	110%	101%
4.1.5.c	-3.83%	-2.86%	-2.89%	144%	111%	-2.65%	-1.91%	-1.94%	111%	102%
4.1.7.b	-3.53%	-2.56%	-2.62%	200%	116%	-2.40%	-1.69%	-1.70%	128%	102%
4.1.7.c	-3.48%	-2.54%	-2.57%	159%	116%	-2.31%	-1.67%	-1.65%	114%	102%
4.1.8.a	-3.10%	-2.27%	-2.29%	181%	114%	-1.99%	-1.40%	-1.42%	118%	101%
4.1.8.b	-3.07%	-2.26%	-2.28%	181%	112%	-1.97%	-1.41%	-1.44%	118%	101%
4.1.8.c	-3.19%	-2.32%	-2.31%	221%	114%	-2.06%	-1.44%	-1.50%	127%	102%

Low delay B results

	VTM_tool_test					BMS_tool_test
Test#	Y	U	V	EncT	DecT	Y	U	V	EncT	DecT
AFFINE	-2.06%	-1.33%	-1.52%	168%	108%	-1.98%	-1.52%	-1.70%	125%	104%
4.1.3.b	-2.23%	-1.50%	-1.60%	164%	99%	-2.21%	-1.80%	-1.84%	127%	109%
4.1.3.c	-2.35%	-1.53%	-1.82%	210%	103%	-2.37%	-1.91%	-1.89%	145%	107%
4.1.3.d	-2.37%	-1.53%	-1.53%	199%	105%	-2.36%	-1.92%	-1.74%	139%	109%
4.1.5.b	-2.11%	-1.40%	-1.51%	170%	107%	-2.07%	-1.72%	-1.86%	126%	104%
4.1.5.c	-2.36%	-1.54%	-1.47%	174%	108%	-2.31%	-1.69%	-2.13%	128%	104%
4.1.7.b	-2.33%	-1.49%	-1.60%	297%	116%	-2.35%	-1.85%	-1.90%	170%	105%
4.1.7.c	-2.28%	-1.43%	-1.42%	218%	117%	-2.27%	-1.61%	-1.75%	141%	106%
4.1.8.a	-2.12%	-1.31%	-1.39%	252%	113%	-2.10%	-1.56%	-1.68%	152%	104%
4.1.8.b	-2.12%	-1.28%	-1.34%	252%	113%	-2.08%	-1.72%	-1.64%	152%	104%
4.1.8.c	-2.15%	-1.35%	-1.55%	324%	114%	-2.14%	-1.71%	-1.77%	176%	104%

Most of tests here add an additional 6-param affine model. 6-param affine model requires coding 3 CPMVs, compared with coding 2CPMVs for 4-param affine model in BMS affine. Adaptive selection at CU level or slice level are proposed.

Test 4.1.8 propose two types of 3-param model models, scaling model and rotation model. The 3-param model is represented by 1.5 motion vector, top-left motion vector and x or y component of the top-right motion vector.

Generally, additional motion model introduces operations handling the additional motion parameters, e.g. motion estimation, motion vector prediction, motion vector difference coding, motion parameter storage, etc.

Switchable 4/6 parameter model provides approx. 0.5-0.6% gain. The switchable 3/3/4 approach does not provide comparable gain (up to 0.2%).

Proponents are requested to provide an analysis about the number of operations, memory usage, etc. for the list construction and the inheritance, also in comparison with BMS affine.

Analysis was shown Sat. 14^th 9-10.

The solution of 4.1.3c has least complexity (still significantly less complex than the current BMS with 4 parameters, no scal. No mul. No div.). 4.1.4 is based on it with more candidates, by factor 1.4 more complex. 4.1.5 is still less complex than BMS, but higher complex than 4.1.3/4.1.4. Though it has less shift and additions, it requires scaling/mul/div.

After all, 4.1.3. is asserted to be the simplest solution. For 6 parameters it is 1.5x complex as for the case of 4 parameters, and still significantly less than current BMS with 4 parameters

Decision (BMS): Adopt JVET-K0337 (4.1.3c, 4/6 parameter model, no slice level switch).

Decision (BMS/enc): Adopt JVET-K0185 fast encoder from 4.1.5c in combination with 4.1.3c

Further study of slice level switch (as per 4.1.3d, 4.1.4.x) in CE

The harmonization with affine merge should further be studied

Line buffer reduction should further be studied

The signalling and coding mechanisms of 4.1.7c should also further be studied in combination with the new BMS.

The possibility of further switching between 3/3/4/6 models (as from the results of 3.18c might give additional gain) should also further be studied. However, such a modification should only be made when it is not penalized by increased encoder run time, and the methods of MV prediction and merge should be harmonized.