International organisation for standardisation
Annex A-V: Audio Task Groups
Yüklə 1 Mb.
|
- Bu səhifədəki naviqasiya:
- Annex A-VI: Input/Output Documentation
- SNHC Contributions Related Review
- AHG Meetings and Reports
- Meeting Work and Results
- Conformance Work for Version 1
- Still Image Texture Coding in Version 1
- CGD for SNHC Functionalities in Version 2
- BIFS for SNHC Functionalities
- Body Animation in Version 2
- 3D Model Coding in Version 2
- Ad Hoc Groups for Dublin
Annex A-V: Audio Task Groups
Annex A-VI: Input/Output DocumentationContributed documentsThe following documents were contributed to the Audio Subgroup and were considered during this meeting:
Output DocumentsThe following output documents were produced in whole or part by the Audio Subgroup. Those shown in Italics were approved for public release.
Annex 8 SNHC meeting report Source: Peter Doenges (Evans & Sutherland), Chair SNHC Meeting ObjectivesThe main SNHC objectives for the Dublin meeting were review of the completeness and quality of Version 1 FCD work for Study of FCD, and initial development of conformance contributions to Part 4 based on the improving stability of profile/level work. The main objectives for Version 2 were to move technologies forward in preparation for CD, to continue work on FBA calibration connected with Version 2 profiling, and to advance CGD work in the development of content metrics for the bitstream. Qualified technologies were to be advanced to Visual WD (notably 3D model connectivity coding and body animation) with related VM promotions if justified. Development of experiments for CGD was targeted (along with identifying 3D platforms, data sets, etc. needed to conduct the experiments) to verify a model for estimation of terminal loading during 3D rendering from content metrics in the bitstream. Detail Dublin objectives are listed below: Version 1
Version 2 1. Assessment of outstanding work items a. 3D Model Coding i. Decision - retention of 3D regular gridded mesh vs. efficiency? ii. M1: 3D mesh connectivity - efficiency, generality, footprint iii. M2: Geometry coding including progressive shape iv. M3: Progressive connectivity coding (and links to M1, M2) v. M4: Properties coding (normals, color, texture coordinates) b. Body Animation i. BA2: BAP Compression ii. BA3: Hand BAP Interpretation iii. BA6: BAP Quantization Step Sizes iv. BA7: BDP Interpretation c. Face Animation i. FA1: Face Calibration (feature points, mesh texture & shape) ii. FA2: FAP Coding with Facial Action Basis Functions vs. FIT? iii. FA3: Error Resilience of FAP Bitstreams 2. VM 9.0 contribution with recommended changes from AHGs 3. Critical review of pre-Dublin CEs compared to interim AHG results 4. VM update per CEs if significant change for technology promotion 5. Verification of 1st bitstream exchanges, selection of WD technologies 6. If technology changes in VM, critical plan to finish WD on CD schedule 7. Quality/effectiveness of documentation/software for 2nd implementers 8. Profiling: FA calibration studies vs. Calibration, Predictable profiles 9. ISG CGD contributions and SNHC experiment design/contributors a. Adaptive decoding for terminal resources vs. media complexity b. Basic capability for scalable content with no backchannel c. Server adaptation of content with backchannel terminal metrics i. Key applications and requirements ii. Impact on, supported by content developers iii. Assistance to Systems for real solutions iv. Level of server/terminal negotiation v. Effect on profiling Most of these objectives were achieved. However for Version 2, profiling discussions were tabled in deference to Version 1 work, and no time was spent on backchannel work associated with CGD and server adaptation of content. For Version 1, significant improvements to the FCD that involve lagging editorial work with NB backing, and final conformance work with supporting test data, must still be achieved by Atlantic City. Key individuals are committed to closure on these points. For Version 2, the schedule imperatives associated with fulfilling required process steps to qualify technology fully for CD now requires another interim AHG meeting for 3D Model Coding before Atlantic City. Promotion to WD of remaining elements of 3D Model Coding is expected, and is needed to produce an integrated tool set for static and progressive compression of connectivity, geometry, and properties. SNHC Contributions & Related ReviewThe following SNHC-related contributions were reviewed during the meeting: FCD for Version 1
2D Mesh for Version 1
Face & Body Animation for Version 2
3D Model Coding for Version 2
SNHC VM for Version 2
Visual Still Texture Coding
Visual WD for Version 2
Computational Graceful Degradation & Quality of Service
Complexity Analysis
Output Document EditorsEditors or coordinators responsible for SNHC elements of output documents were:
AHG Meetings and ReportsThe following AHG meetings were held on Sunday before the WG11 meeting and reports discussed:
See those reports for details. An important development emerged from the meetings on 3D Model Coding. While there has been tremendous work by various proponents and partners as 2nd implementers to bring the work to the maturity of bitstream exchanges, these were not consistently achieved across all the M1-M4 experiments. Thus VM and WD promotions were not as aggressive as originally hoped. The Korean NB requested a response to their concern on this, and another AHG meeting is planned in Korea. Meeting Work and ResultsStudy of FCD for Version 1Little of the NB comments on FCD applied to SNHC-related functionalities except for Still Texture Coding. The responsible working group in Video continues to handle the DoC and Study inputs for Still Texture Coding with cross-review by SNHC. Iraj Sodagar presented the intended actions for this work along with the AHG report to the SNHC group. Reported inconsistencies in bitstream decoding for different implementations of Still Texture and other problems reported at and after Tokyo are receiving needed attention. The group expected pertinent items to be corrected in the Study of FCD and subsequent NB follow-up for Atlantic City. After Tokyo, a number of edits in the FCD were discovered missing after the publishing date in areas of FA and 2D Mesh as previously driven by NB comments including references back into CD and Study of CD. The reasons are important to understand (editor overloads, lack of incremental review before publishing, lack of adequate proponent participation at the right times). The results are that FCD can not transition to DIS with reliability without a thorough assessment of the FA and 2D Mesh areas of the FCD. This is obviously not the state that the work should be in nor is it the intent of the process. The process demands a very high level of quality in the text at this point. People involved have lost a bit of confidence in the process, but have pledged to close editing gaps by rebuilding what should have been in FCD from the known NB trail. NBs should take a careful look at this situation in NB meetings before Atlantic City, collaborate with other interested NBs, and make NB contributions to Atlantic City supported by the necessary unanimity of proponent organizations. The 2D Mesh area was covered by an individual contribution (M3644) in Dublin that provided corrections to the FCD in the necessary details reflecting previous NB requests. The FA area required working time in the Dublin meeting to audit the FCD for missing parts and give priority to the process of recovering the trail by which to incorporate needed changes driven by NB comments going back into Tokyo. This situation was complicated by the necessity to edit the edits of the CD. Individuals in the FA area have committed to "rebuild" a valid NB-driven version of FCD in the affected areas before Atlantic City to gain NB approval and contributions to the meeting. They have also volunteered to join the main Visual and Systems editors wherever necessary to ensure progressive verification and cross-checking of DIS editing after Atlantic City. Conformance Work for Version 1An outline for the Conformance strategy and content of Part 4 was initially developed in the Dublin meeting, along with a review of the existing state of Part 4. Then several discussions were held with proponents (specifically FA and 2D Mesh) to characterize the methods of conformance, exercising necessary modalities in the decoder, and augmenting current bitstreams to achieve adequate verification of proper function and performance as driven by profile/level results. An initial version of the Conformance additions to Part 4 with some embedding of current Part 4 language to help with convergence of consistent results was issued as N2299. Key individuals for FA and 2DM will expand this document after Dublin, and discuss its refinement as well as the development of data sets using appropriate reflectors. No AHG was formed, but the mandate is very clear, and proponents must supply an integrated contribution with NB support by Atlantic City. Still Image Texture Coding in Version 1In addition to the FCD work, several delegates discussed the need to verify wavelet texture coding in its application to 3D. Specifically, wavelet texture coding should be subjected to an end-to-end (non-normative) verification test (not a CE) relative to the resolution scalability and quality of still texture when used in 3D rendering. The test would include conversion of a downloaded wavelet image pyramid from MPEG-4 Part 5 decoding into MIP map texture for varied viewing of textured 3D models on a PC graphics accelerator or workstation. OpenGL hardware with bilinear and trilinear filtering of MIP map texture could be used with dynamic viewing to verify that wavelet texture supports the intended MPEG-4 functionality. E&S volunteered rendering on an OpenGL PC 3D graphics accelerator and Lucent Bell Labs volunteered rendering an SGI OpenGL workstation to observe wavelet-based texture in 3D. Textured 3D models would be run through the complete MPEG-4 still image texture pipeline. Texture maps would be extracted from 3D models, the textures processed through the wavelet encoder, the complete wavelet bitstream decoded while accumulating the supplied image resolution layers (11 peak), results translated to MIP map texture, and the reintegrated texture models rendered in 3D. Demos would be staged under viewing conditions of varied model orientation, distance, and perspective distortion to show the effectiveness of resolution scalability. If possible, videos would be prepared for Atlantic City. This effort will be pursued following the Dublin meeting. Sarnoff had anticipated such a demo with some work now applied in this direction. Rockwell and AT&T Research may be able to help. IM1 now includes the tools, and may be a platform for testing. Sarnoff confirmed that a schedule of tasks and deadlines before Atlantic City and spreading of this collaborative work over partners should be detailed. CGD for SNHC Functionalities in Version 2The ISG group has done a productive job of laying the basis for using CGD to support SNHC/Systems BIFS functionalities that depend on the display of models at useful rates and fidelities while driven by MPEG-4 decoders producing 2D, 3D, or mixed media streams. MPEG-4 compliant decoders should achieve adequate performance in decoding rates due to the profile/level specifications and associated conformance definitions. However, there is presently no normative profiling of model or rendering complexity for specific 2D/3D content whose terminal loading beyond the decoder can be difficult to predict from the content. Overload of terminal rendering is possible with compliant decoding. The goal of current CGD work is to provide bitstream parameterization of content complexity and structure that could be used by the terminal to select scalable versions of the content related to the 2D/3D geometry and rendering capacities of the terminal beyond the decoder. The correct model for estimating geometry and rendering complexity will ultimately be hardware/software-specific and can depend on 3D content organization, graphics architecture, viewpoint location/look angle and associated culling, scene occlusion, the context of state in the graphics pipeline, bus and memory limitations, etc. ISG has done previous work including some benchmarking and analysis of specific systems to characterize basic content metrics that are likely to produce varied effects on video composting and 3D rendering accelerators. M3567 for Dublin from ISG members provided a more substantial proposal with functional analysis of model transformations and polygon/texel rendering into the pixels of an output image, along with some approximations that predict rendering load for component pieces of a 2D/3D media stream. This estimating is simplified compared to hardware-specific models, but a useful start. During Dublin several joint working sessions and informal discussions were held between ISG and SNHC about how to design experiments validating the utility of estimating metrics in M3567. These discussions also dissected what kinds of 3D models and variations in viewing conditions should be used to help validate specific performance estimators. Some time was also spent educating and clarifying specific points about the context-dependent aspects of rendering that are not adequately accounted for now by the ISG CGD model for SNHC. Specific vendors undoubtedly have and might supply more details on the total performance prediction model for estimating rendering load. However, the agreed scope of the current work is to identify hardware architectural abstractions that provide more promise for achieving a normative methodology for hardware, software, and content developers to adapt MPEG-4 content in relation to predicted loading, and to provide scalability controls for content in the bitstream. The output document N2317 provides the results of this work. Without going into details here, several organizations (EPFL, IBM, France Telecomm, E&S, etc.) have agreed to provide models with and without texture for testing. Some organizations have agreed to supply platform assistance (e.g. E&S has shipped a high-performance PC 3D accelerator to IMEC), and may provide more performance estimation insights if this helps broaden the applicability of ISG results to a larger set of hardware and software renderers. On-going discussions should be on the ISG reflector. Some members have agreed to post other academic and industrial references on benchmarking and performance prediction to the reflector. BIFS for SNHC FunctionalitiesTwo joint meetings were held with Systems BIFS. One session discussed remaining issues on Version 1 system integrity. A separate session examined new functionalities in Version 2 to support SNHC decoding and scene composition. The Version 1 discussion dealt with specific node and synchronization issues. Version 2 discussions compared Advanced BIFS with SNHC directions including these points: Scripts & more sophisticated animation/interaction Customizing BAP streams by some extension mechanism - Adding feature control points, clothing, jewelry Dealing with progressive & incremental 3D models Harmonizing aural environment modeling - Material properties & room geometry - Consistency with 3D Model Coding
Face AnimationSome encouraging work in model-based face tracking and 3D pose estimation was reviewed. This tends to strengthen the case for the developing availability of technology on the encoder side to instrument live faces for interactive applications of face animation at very low bandwidth. Discussions continued on face calibration for Version 2 and the FCD problems for Version 1 cited earlier. The FAP default values in arithmetic coder limits needed to be fixed for Version 1. Facial Action Basis Functions, as a means to new functionality with simultaneous high coding efficiency and low decoding lag (compared with the current frame-based arithmetic coding or DCT coding of FAPs), was dropped for lack of adequate results on-time. This decision removes from consideration any new technologies that might have challenged backward compatibility with Version 1 or the principle of one function, one tool. The area of face calibration still needs more work to enable closure on useful profiling results with Requirements in Version 2 for Calibration and Predictable FA. Due to the many Version 1 priorities at the Dublin meeting, no effort was joined with Requirements to move this further along. A meeting was held with Requirements to assess the packaging of Simple FA in Version 1 profiling relative to Main Profile; results should be reviewed in the output of Requirements from Dublin. 2D MeshThe Study of FCD contribution (M3644) was made available to the Visual editors as a reflection of prior NB support. A joint session with Requirements was held to clarify level points previously recommended. A new CE (M6) was added to the 3D Model Coding work associated with a slight change in header bitstream flag semantics of 2D Mesh that would access 3D model connectivity coding now in the Visual WD for Version 2. This means that the same Version 2 tool set would provide a form of generalized 2D topological coding in addition to the 2D regular and Delaunay mesh coding now covered by the header for 2D mesh coding in Version 1. Thus no new technology is introduced, and applications that need general 2D mesh topological coding for MPEG-4 Version 2 could achieve this by invoking the 3D mesh connectivity tool now headed for CD. This would also provide stronger support for efficient coding of 2D model primitives now supported in the BIFS 2D nodes and profiling work. Due to the meeting steps remaining in Version 2 work, we must achieve bitstream testing and exchange by the interim 3D Model Coding AHG meeting before Atlantic City. Then the M6 initiative must gain the approval of the Video and SNHC groups showing that adequate discipline has been followed to incorporate M6 results in the CD when the currently envisioned detail (however minor) is not in the VM. This coupling of 2D mesh and 3D mesh connectivity coding was rehearsed and agreed a year ago in Stockholm. The necessary work before Dublin was not completed on the agreed VM/WD/CD schedule. Body Animation in Version 2Much of the work was concentrated on verifying that all conditions have been met for promoting body animation to WD including bitstream exchange and on improving the quality of the text for this purpose. INT generously donated the hand animation software reviewed in the last few meetings, and reported on successful BAP extraction for hand animation. Some valuable results were reported on body animation core experiments. Analysis and demonstrations of animated models illuminated a better understanding of bitrate vs. quantization tradeoffs. Dynamic animations helped illustrate where BAP quantization levels become unacceptable for useful body motion granularity, signing intelligibility, stable dependent motions of body parts, etc. Specific hand signing was demonstrated for alphabet examples, and VRML Consortium Humanoid Animation models were integrated with MPEG-4 body animation to observe the composite behavior of this unification of models and BAP decoding. The Dublin meeting work actively incorporated informal agreements made with the VRMLC H-Anim working group in the joint meeting on June 88, 1998, to harmonize the work of the two groups. Body animation faced one dilemma about normative specification referencing. The nomenclature used in MPEG-4 and H-Anim specifications should be the same. However the H-Anim work will likely result in an informative annex to the VRML specification without the force of normative standardization in an ISO framework. Thus MPEG-4 body animation will not make normative reference to H-Anim, but will use the same nomenclature to enhance the prospects for ready development of applications when H-Anim and MPEG-4 are combined. If there is any copyright problem, an indexing system will have to be adopted. 3D Model Coding in Version 2The baseline work on 3D Model Coding for the interim AHG meeting at IBM Research on May 18-19, 1998, and for the Dublin meeting has included: M1: 3D mesh connectivity - lossless compression _ Topological Surgery with enhanced arithmetic coding _ Dual Graph topology with adaptive context-based arithmetic coding _ Loop topology coding with islands and bridges M2: Geometry coding including progressive shape _ Predictive Residual Vector Quantization (with Lattice VQ) _ Scalar Quantization (VRML-CBF geometry compression) _ Successive Quantization (with embedded arithmetic coding) M3: Progressive connectivity coding _ Progressive forest split compression _ Progressive vertex split/edge collapse M4: Properties coding (normals, color, texture coordinates) _ Compressed representation of properties attached to a model,
_ 2D/3D Mesh Connectivity: Topological Surgery/IBM Prior run in VRML community, much work & maturity Excellent competing technologies from USC, Samsung _ Body Animation: Fusion of FA adaptation & H-Anim “Loose” specification linkage with shared nomenclature Version 2 SNHC Verification Model _ 3DMC Connectivity: promoted out of VM ??WD _ Body Animation: promoted out of VM ??WD _ 3DMC Geometry: Successive Quantization ??VM (from CE) _ 3DMC Progressive: Forest Split already in VM _ 3DMC Properties: VQ ??SQ? evaluating alternatives This resulted in the continuance of M2-M4 for bitstream exchanges and in some cases for the achievement of other required process steps. Partners were enlisted to enhance the verification of the various competing technologies to strengthen the viability of the primary candidates for CD promotion. The AHG meeting in Korea in late August will be pivotal in providing sufficient evidence that the remainder of the 3D Model Coding tool set can and should be advanced to CD. The Korean NB expressed due concern about the adequacy of resources to finish the process in a manner that justifies standardization. After much open discussion of the status of the work and the further volunteering of participation, the following response to the Korean NB was given: "WG11 thanks the Korean National Body for its comments on the progress of SNHC and in particular the 3D Model Coding tools for Version 2 MPEG-4. In response, two additional 3D Model Coding Ad Hoc Group meetings have been scheduled before the WG11 meeting in Atlantic City to ensure adequate attention to this important priority. Moreover, necessary steps have been taken to enlist additional resources to confirm the superiority of the technologies already selected for advancement to Working Draft in Atlantic City. A request for additional test models will be issued, and the 3D Model Coding Verification Model software will be made available for public evaluation, at this meeting." The group also recommended to WG11 a call for data sets to expand the 3D models used in testing and to publish the VM/WD software to the industry to encourage experimentation and feedback on the robustness of the tools. A special header for the software (expected to be issued after the Korean AHG meeting) was drafted toward the end of the WG11 meeting and approved. Owners of the M1-M4 software should review this header language for acceptability (N2397) before issuing the software. Starting with the AHG meeting at IBM Research, Samsung has proposed a variant of strip-oriented 3D mesh coding, in addition to the Dual Graph (DG) and Topological Surgery (TS) technologies entered earlier, that performs well for looping or ring mesh topologies. Examination of this coding method revealed that it is compatible with incremental transmission of the 3D model for a given level of detail. This opened consideration of how M1 WD technology (TS) and others offered so far could be modified to support error resilience. Error resilience would be supported by any coding method that preserves high lossless coding efficiency for the base mesh, but also allows ready partitioning of the coding into free-standing chunks of the model while yielding only a small loss in coding efficiency. M5, Partitioning of Data for 3D Model Coding, was devised and supported by partners to rapidly investigate this functionality by building on the technologies already under consideration. A meeting with Requirements was held to establish that this functionality is useful, and the MPEG-4 Requirements were updated. 2D/3D Mesh Unification, M6, was described earlier to access 3D connectivity coding to serve as a general topological coding method for 2D meshes as well without new technology. MPEG-4 OverviewLate in the Dublin meeting, an update of SNHC functional language and partitioning of tools into Version 1 and 2 matching the current status of work was supplied to Requirements for the MPEG-4 Overview. The submission was too late to affect output documents of Dublin, but should be refined as soon as Requirements has time to incorporate and critique the changes. Output Documents
Core ExperimentsThe following Core Experiments were formulated for Atlantic City: 3D Model Coding N2303 _ M2 Mesh Geometry/Vertex Coding Predictive Residual Vector Quantization (with Lattice VQ) Scalar Quantization (IBM VRML-CBF geometry compression) Successive Quantization (with embedded arithmetic coding) _ M3 Progressive/Scalable 3D Mesh Coding Progressive forest split compression Progressive vertex split/edge collapse _ M4 Attribute Coding and Tool Integration Compressed representation of properties attached to a model, with all bindings defined in VRML (per vertex, per face, per corner) _ M5 Partitioning of Data for 3D Model Coding Error resilience potential vs. efficiency loss _ M6 2D/3D Mesh Unification Face Body Animation N2302 _ FA1 Face Model Mesh Calibration _ BA2 BAP Coding _ BA6 BAP Quantization Step Size _ BAT BAT Interpolation
Other Output DocumentsThe following output documents were generated:
Ad Hoc Groups for DublinThe following groups were established to coordinate core experiments and documents: 3D Model Coding N2306 _ Big workload, carefully designed priorities to move VM technologies to WD _ Key work with much expanded partnering on 2nd implementations & verification - THANKS! _ Heading toward very robust 3DMC tools _ Crucial meetings for success 30-31 August 1998, Seoul Korea 11 October 1998, Atlantic City Face Body Animation N2307 _ 11 October 1998, Atlantic City SNHC VM Editing N2308 Yüklə 1 Mb. Dostları ilə paylaş:
|