5Explorations – 3D Video
The goal of the current 3D video exploration work is to generate interpolated views from available videos of multiview camera configurations. The target application is mostly seen for upcoming generations of various (auto-)stereoscopic displays, either requiring multiple views internally or providing means for baseline adjustment. In the new format, only a low number (1-3) of video sequences would be transmitted, and rendering of additional views would be enabled by associated depth information.
The activity currently concentrates on preparing test sets and anchor coding for an upcoming Call for Proposals (CfP). By the Xi’an meeting, new more challenging test sequences (in terms of content activity and camera movement) had been provided which were initially investigated in the period before and at the Kyoto meeting, and this work was continued in the period before the 92nd meeting in Dresden. Even though the 3DV group was affected by travel problems, a significant amount of participants were present for the work, so it was possible to transfer all test sequences, conduct a review of all input documents and perform subjective viewing of the results. The following developments are reported:
-
For the interpolation of original (uncoded) sequences, a total of 10 sequences has arrived at “acceptable” quality for the 2-view (smaller baseline) scenario, whereas 7 were found acceptable for the 3-view (larger baseline) scenario. In addition, 2 more sequences were judged to be “almost acceptable with slight artifacts” for the 2-view, and 3 more sequences for the 3-view scenario. The “acceptable” category includes difficult sequences with moving background, fog, etc.
-
It can therefore be concluded that in principle the number and quality of sequences is sufficient as test material for a Call for Proposals, and it will even be possible to perform some more selection according to appropriateness. Longer test sequences (10s) would be desirable, but if it is not possible to obtain such material, the available material will be sufficient for the purpose of the Call.
-
In addition, EE4 was continued, by which it was already possible to obtain first bit rate settings for the sets of new sequences. For the 2-view scenario, the suitable bit rate ranges for 7 sequences are practically known (though particular rate points may need to be explored further), for the 3-view scenario this is the case for 6 sequences. The remaining sequences are still under investigation, and it may also happen that the rate points need to be changed when better depth maps become available for some of the sequences.
As a conclusion, reasonable progress was made and it appears possible to adhere to the existing tentative timeline (as planned in Xi’an and Kyoto) – issuing a final CfP by October 2010, with responses expected for review in March 2011. Such a timeline would also be suitable to observe a possible relationship between the HEVC development and 3DV. As an initial step, a first (internal) Draft CfP was produced in Dresden, and an update of the status document was made which is intended to accompany the CfP, giving more technical description about the background of 3DV technology and the generation of depth maps, interpolation algorithms and anchors.
Still, the definition of proper testing methodologies requires much more consideration. Some initial thoughts summarizing the discussions of Dresden are as follows:
-
Careful definition of rules will be necessary to make submissions clearly comparable, e.g.
-
Are the depth maps as used for the anchors mandatory as input?
-
Is the view synthesis algorithm as used for the anchors mandatory?
-
Can additional alternatives view synthesis algorithm be used?
-
Probably stereo output will be tested, but it may be worthwhile to make some of the quality assessments with regards to mono compatibility
-
The test methodology in general requires much more discussion
With regard to the first bullet above (and its three sub-bullets), it will be important to find a definition of reasonable restrictions which exclude advantages due to tuning of encoder settings, pre-/post-processing etc., while avoiding imposing unreasonable restrictions to potential new technology.
An AHG was appointed (N11279, as recorded in N11242) to further the work until the 93rd meeting.
Documents reviewed in AHG (see AHG report m17366)
m17459
|
3DV/FTV EE4 results on "Lovebird1" and "Kendo" sequences
|
Gun Bang
Neung-joo Hwang
Donggyu Sim
Hyunho Jo
Gi-Mun Um
Won-Sik Cheong
Namho Hur
|
m17460
|
3DV/FTV EE result of coding experiment on "kendo" sequence
|
Gun Bang
Neung-joo Hwang
Donggyu Sim
Hyunho Jo
Gi-Mun Um
Won-Sik Cheong
Namho Hur
Jinwoong Kim
|
m17474
|
3DV EE4 Results on Balloons sequence
|
Jaejoon Lee
Seok Lee
HoCheon Wey
Du Sik Park
|
m17475
|
3DV EE4 Report on Book_arrival Sequence
|
Deliang Fu
Lu Yu
|
m17476
|
3DV EE4 Report on Poznan_Street Sequence
|
Deliang Fu
Lu Yu
|
m17488
|
Results of EE1 on ?Cafe? Sequence
|
Sang-Beom Lee
Cheon Lee
Yo-Sung Ho
|
m17489
|
Results of EE4 on ?Cafe? Sequence
|
Min-Koo Kang
Cheon Lee
Yo-Sung Ho
|
m17490
|
Modified Depth Data of ?Newspaper? Sequence
|
Cheon Lee
Yo-Sung Ho
|
m17498
|
Input on Call for Proposals on 3D Video Coding
|
Clinton Priddle
Thomas Rusert
Per Fröjdh
|
m17499
|
3D Video Coding Results of Exploration Experiments (Book Arrival)
|
Gerhard Tech
Karsten Müller
|
m17500
|
3DV/FTV EE1 report on Kendo and Balloons sequences
|
Masayuki Tanimoto
Toshiaki Fujii
Mehrdad Panahpour Tehrani
Menno Wildeboer
|
m17501
|
3DV/FTV EE4 report on Kendo and Balloons sequences
|
Masayuki Tanimoto
Toshiaki Fujii
Mehrdad Panahpour Tehrani
Menno Wildeboer
|
m17503
|
3DV/FTV EE4 report on Champagne Tower sequence
|
Masayuki Tanimoto
Toshiaki Fujii
Mehrdad Panahpour Tehrani
Menno Wildeboer
|
m17521
|
3DV EE4 Results on BeerGarden Sequence
|
Patrick Lopez
Dong Tian
|
m17526
|
3DV EE4 Result on Mobile
|
fons bruls
|
m17581
|
3DV EE1 Report with Recursive Depth Estimation
|
Takanori Senoh
Kenji Yamamoto
Ryutaro Oi
Tomoyuki Mishina
Taiichiro Kurita
|
m17585
|
EE1 results on BookArrival test sequence
|
Jaewon Sung
Byeong-Moon Jeon
|
m17601
|
Results on 3DVC EE4 for Beergarden MVD2
|
Pablo Carballeira
Julián Cabrera
Fernando Jaureguizar
Gianluca Cernigliaro
Juan Casal
Narciso García
|
m17602
|
Results on 3DVC EE4 for Newspaper MVD3
|
Pablo Carballeira
Julián Cabrera
Fernando Jaureguizar
Gianluca Cernigliaro
Juan Casal
Narciso García
|
m17612
|
Estimation of temporally consistent depth maps using noise removal from video
|
Olgierd Stankiewicz
Krzysztof Wegner
Marek Doma?ski
|
m17613
|
Results of 3DV/FTV Exploration Experiment 1 (EE1) for Poznan sequences
|
Olgierd Stankiewicz
Krzysztof Wegner
|
m17614
|
Results of 3DV/FTV Exploration Experiment 4 (EE4) for Poznan sequences
|
Krzysztof Klimaszewski
Krzysztof Wegner
|
m17615
|
Results of 3DV/FTV Exploration Experiment 4 (EE4) for Newspaper sequences
|
Krzysztof Klimaszewski
Krzysztof Wegner
|
Output documents:
No.
|
Title
|
TBP
|
Available
|
11272
|
Draft Call for Proposals on 3D Video Coding Technology
|
N
|
10/04/23
|
11273
|
Draft Report on Experimental Framework in 3D Video Coding
|
N
|
10/04/23
|
11274
|
Description of Exploration Experiments in 3D Video Coding
|
N
|
10/04/30
|
-
- JCT report
Source: Gary J. Sullivan, Jens-Rainer Ohm
Dostları ilə paylaş: |
