TIA-1017

Description / Abstract: This document specifies the procedures to test implementations of VMR-WB-compatible variable-rate speech codecs, either by meeting the bit-exact implementation, or meeting recommended minimum performance requirements. The VMR-WB codec corresponds to Service Option 62 for Rate-set II compliant modes and Service Option 63 for the Rate-set I compliant mode, as described in TIA-1016- A [1]. The VMR-WB codec is used to digitally encode the speech signal for transmission using the encoding rates of 13300, 6200, 2700, or 1000 bps for Rate-set II operation (Modes 0, 1, and 2, or Mode 3), and the encoding rates of 8550, 4000, or 800 bps for Rate-set I operation (Mode 4). Note that the encoding rate of 2000 bps is not utilized in VMR-WB in Rate-Set I operation. Like some other speech coding standards, this standard provides a bit-exact method of testing the codec for minimum performance. In this procedure, a given set of test vectors are input to the test codec, and the output vectors from the test codec must be bit-exact with the output vectors generated from the Software Distribution which is part of this standard. If they are bit-exact, and the test codec complies with Section 2.1.2, the codec passes the minimum performance requirement, and no further testing is required. The bit-exact mode of testing, however, is only applicable to codecs whose design conforms in all respects to the algorithmic description of the VMR-WB in TIA-1016-A [1]. The bit17 exact VMR-WB validation includes specific testing of the OEM VMR-WB to the frame erasure processing characteristics of the standard VMR-WB. Should the candidate VMR-WB differ in any of these components, the codec shall be tested using the objective and subjective tests prescribed by this standard. That is, VMR-WB compliance of a "test codec" can be achieved by either:

• Complying with Section 2.1.2, and demonstrating bit-exactness according to the procedure described in Section 3.6, specifically Sections 3.6.5.1 to 3.6.5.4, or

• Following the objective and subjective testing procedures set forth in Sections 2.1 and 2.2 of this standard, using processes additionally defined in Section 3.6, specifically Sections 3.6.5.5 to 3.6.5.8.

With the exception of Section 3.6, the remaining text applies only to implementations that do not satisfy the requirement for bit-exactness.

Testing the codec is based on two classes of procedures: objective tests and subjective tests. In the event that the test codec fails any of the objective or subjective tests, the test codec fails the compliance test. Objective tests are based upon actual measurements from the speech codec function. Subjective tests are based on listening tests to judge overall speech quality. The minimum subjective requirement for the test codec is based upon the ability of the test codec to demonstrate performance no worse than that of the VMR-WB master reference codec within a fixed allowable statistical error.

The purpose of the testing is not only to ensure adequate performance between one manufacturer's encoder and decoder but also that this level of performance is maintained with operation between any pairing of manufacturers' encoders and decoders. This interoperability is a serious issue. Any variation in implementing the exact standard must be avoided if it cannot be ensured that minimum performance levels are met when interoperating with all other manufacturers' equipment meeting the standard. This standard provides a means for measuring performance levels while trying to ensure proper interoperation with other manufacturers' equipment.

The issue of interoperation can only be definitively answered by testing all combinations of encoder/decoder pairings. With the number of equipment manufacturers expected to supply equipment, this becomes a prohibitive task; therefore, the objective and subjective tests rely upon the use of a "master codec". The master codec is defined as the appropriate implementation of TIA-1016-A [1] written in the C-programming language. The master codec software, described in Sections 3.4 and 3.5, are used as part of the interoperability testing. The master codec is available as a Software Distribution associated with this document.

By convention in this document, the Courier font is used to indicate C-language and other software constructs, such as file and variable names.