Hypothetical reference decoder

AHypothetical reference decoder

(This annex forms an integral part of this Recommendation | International Standard)

This annex specifies the hypothetical reference decoder (HRD) and its use to check bitstream and decoder conformance.

Two types of bitstreams are subject to HRD conformance checking for this Recommendation | International Standard. The first such type of bitstream, called Type I bitstream, is a NAL unit stream containing only the VCL NAL units and filler data NAL units for all access units in the bitstream. The second type of bitstream, called a Type II bitstream, contains, in addition to the VCL NAL units and filler data NAL units for all access units in the bitstream, at least one of the following.

additional non-VCL NAL units other than filler data NAL units

all start code prefixes and zero_byte syntax elements that form a byte stream from the NAL unit stream (as specified in Annex B)

Figure C‑1 shows the types of bitstream conformance points checked by the HRD.

Figure C‑ Structure of byte streams and NAL unit streams for HRD conformance checks

The syntax elements of non-VCL NAL units (or their default values for some of the syntax elements), required for the HRD, are specified in the semantic subclauses of clause 7 and Annexes D and E.

Two types of HRD parameter sets are used. The HRD parameter sets are signalled through video usability information as specified in subclauses E.10 and E.11, which is part of the sequence parameters set syntax structure.

In order to check conformance of a bitstream using the HRD, all sequence parameter sets and picture parameters sets referred to in the VCL NAL units, and corresponding buffering period and picture timing SEI messages shall be conveyed to the HRD, in a timely manner, either in the bitstream (by non-VCL NAL units), or by other means not specified in this Recommendation | International Standard.

In Annexes C, D and E, the specification for 'presence' of non-VCL NAL units is also satisfied when those NAL units (or just some of them) are conveyed to decoders (or to the HRD) by other means not specified by this Recommendation | International Standard. For the purpose of counting bits, only the appropriate bits that are actually present in the bitstream are counted.

NOTE - As an example, synchronization of a non-VCL NAL unit, conveyed by means other than presence in the bitstream, with the NAL units that are present in the bitstream, can be achieved by indicating two points in the bitstream, between which the non-VCL NAL unit would have been present in the bitstream, had the encoder decided to convey it in the bitstream.

When the content of a non-VCL NAL unit is conveyed for the application by some means other than presence within the bitstream, the representation of the content of the non-VCL NAL unit is not required to use the same syntax specified in this annex.

NOTE - When HRD information is contained within the bitstream, it is possible to verify the conformance of a bitstream to the requirements of this subclause based solely on information contained in the bitstream. When the HRD information is not present in the bitstream, as is the case for all 'stand-alone' Type I bitstreams, conformance can only be verified when the HRD data is supplied by some other means not specified in this Recommendation | International Standard.

The HRD contains a coded picture buffer (CPB), an instantaneous decoding process, a decoded picture buffer (DPB), and output cropping as shown in Figure C‑2.

Figure C‑ HRD buffer model

The CPB size (number of bits) is specified by CpbSize[ SchedSelIdx ] in Annex E. DPB size (number of frame buffers) is specified by max_dec_frame_buffering in Annex E.

The HRD operates as follows. Data associated with access units that flow into the CPB according to a specified arrival schedule are delivered by the HSS. The data associated with each access unit are removed and decoded instantaneously by the instantaneous decoding process at CPB removal times. Each decoded picture is placed in the DPB at its CPB removal time unless it is output at its CPB removal time and is a non-reference picture. When a picture is placed in the DPB it is removed from the DPB at the later of the DPB output time or the time that it is marked as 'unused for reference'.

The operation of the CPB is specified in subclause C.4. The instantaneous decoder operation is specified in clauses 8 and  . The operation of the DPB is specified in subclause C.5. The output cropping is specified in subclause C.5.2.

HSS and HRD information concerning the number of enumerated delivery schedules and their associated bit rates and buffer sizes is specified in subclauses E.10.1, E.10.2, E.11.1 and E.11.2. The HRD is initialised as specified by the buffering period SEI message as specified in subclauses D.8.1 and D.9.1. The removal timing of access units from the CPB and output timing from the DPB are specified in the picture timing SEI message as specified in subclauses D.8.2 and D.9.2. All timing information relating to a specific access unit shall arrive prior to the CPB removal time of the access unit.

The HRD is used to check conformance of bitstreams and decoders as specified in subclauses C.6 and C.7, respectively.

NOTE - While conformance is guaranteed under the assumption that all frame-rates and clocks used to generate the bitstream match exactly the values signalled in the bitstream, in a real system each of these may vary from the signalled or specified value.

All the arithmetic in this annex is done with real values, so that no rounding errors can propagate. For example, the number of bits in a CPB just prior to or after removal of an access unit is not necessarily an integer.

The variable t_c is derived as follows and is called a clock tick.

t_c = num_units_in_tick time_scale  (C- )

The following is specified for expressing the constraints in this Annex.

Let access unit n be the n-th access unit in decoding order with the first access unit being access unit 0.

Let picture n be the primary coded picture or the decoded primary picture of access unit n.

A.1     Operation of coded picture buffer (CPB)

The specifications in this subclause apply independently to each set of CPB parameters that is present and to both Type I and Type II conformance.

A.1.1          Timing of bitstream arrival

The HRD may be initialised at any one of the buffering period SEI messages. Prior to initialisation, the CPB is empty.

NOTE - After initialisation, the HRD is not initialised again by subsequent buffering period SEI messages.

The access unit that is associated with the buffering period SEI message that initializes the CPB is referred to as access unit 0. All other access units are referred to as access unit n with n being incremented by 1 for the next access unit in decoding order.

The time at which the first bit of access unit n begins to enter the CPB is referred to as the initial arrival time t_ai( n ).

The initial arrival time of access units is derived as follows.

If the access unit is access unit 0, t_ai( 0 ) = 0,

Otherwise (the access unit is access unit n with n > 0), the following applies.

If cbr_flag[ SchedSelIdx ] is equal to 1, the initial arrival time for access unit n, is equal to the final arrival time (which is derived below) of access unit n ‑  , i.e.

t_ai( n ) t_af( n  1 ) (C-

Otherwise, if cbr_flag[ SchedSelIdx ] is equal to 0 and access unit n is not the first access unit of a subsequent buffering period, the initial arrival time for access unit n is derived by

t_ai( n ) Max( t_af( n  1 ), t_ai,earliest( n ) ) (C- )

where t_ai,earliest( n ) is given as follows

t_ai,earliest( n ) = t_r,n( n )
( initial_cpb_removal_delay[ SchedSelIdx ] + initial_cpb_removal_delay_offset[ SchedSelIdx ] ) (C- )

with t_r,n( n ) being the nominal removal time of access unit n from the CPB as specified in subclause C.4.2 and initial_cpb_removal_delay[ SchedSelIdx ] and initial_cpb_removal_delay_offset[ SchedSelIdx ] being specified in the previous buffering period SEI message.

Otherwise (cbr_flag[ SchedSelIdx ] is equal to 0 and the subsequent access unit n is the first access unit of a subsequent buffering period), the initial arrival time for the access unit n is derived by

t_ai( n ) = t_r,n( n ) ( initial_cpb_removal_delay[ SchedSelIdx ] ) (C- )

with initial_cpb_removal_delay[ SchedSelIdx ] being specified in the buffering period SEI message associated with access unit n.

The final arrival time for access unit n is derived by

t_af( n ) = t_ai( n ) + b( n ) BitRate[ SchedSelIdx ]  (C- )

where b( n ) is the size in bits of access unit n, counting the bits of the Type I bitstream for Type I conformance or the bits of the Type II bitstream for Type II conformance.

The values of SchedSelIdx, BitRate[ SchedSelIdx ], and CpbSize[ SchedSelIdx ] are constrained as follows.

If access unit n and access unit n - 1 are part of different coded video sequences and the content of the active sequence parameter sets of the two coded video sequences differ, the HSS may select a value SchedSelIdx1 of SchedSelIdx from among the values of SchedSelIdx provided for the coded video sequence containing access unit n that results in a BitRate[ SchedSelIdx1 ] or CpbSize[ SchedSelIdx1 ] for the second of the two coded video sequences (which contains access unit n 1) that differs from the value of BitRate[ SchedSelIdx0 ] or CpbSize[ SchedSelIdx0 ] for the value SchedSelIdx0 of SchedSelIdx  that was in use for the coded video sequence containing access unit n - 1.

Otherwise, the HSS continues to operate with the previous values of SchedSelIdx, BitRate[ SchedSelIdx ] and CpbSize[ SchedSelIdx ].

When the HSS selects values of BitRate[ SchedSelIdx ] or CpbSize[ SchedSelIdx ] that differ from those of the previous access unit, the following applies.

the variable BitRate[ SchedSelIdx ] comes into effect at time t_ai( n )

the variable CpbSize[ SchedSelIdx ] comes into effect as follows.

If the new value of CpbSize[ SchedSelIdx ] exceeds the old CPB size, it comes into effect at time t_ai( n ),

Otherwise, the new value of CpbSize[ SchedSelIdx ] comes into effect at the time t_r( n ).

A.1.2          Timing of coded picture removal

For access unit 0, the nominal removal time of the access unit from the CPB is specified by

t_r,n ) = initial_cpb_removal_delay[ SchedSelIdx ] (C- )

For the first access unit of a buffering period that does not initialise the HRD, the nominal removal time of the access unit from the CPB is specified by

t_r,n( n ) = t_r,n ( n_b ) + t_c cpb_removal_delay( n ) (C- )

where t_r,n( n_b ) is the nominal removal time of the first picture of the previous buffering period and cpb_removal_delay( n ) is specified in the picture timing SEI message associated with access unit n.

When an access unit n is the first access unit of a buffering period, n_b is set equal to n at the removal time of access unit n.

The nominal removal time t_r,n(n) of an access unit n that is not the first access unit of a buffering period is given by

t_r,n( n ) = t_r,n( n_b ) + t_c cpb_removal_delay( n ) (C- )

The removal time of access unit n is specified as follows.

If low_delay_hrd_flag is equal to 0 or t_r,n( n ) >= t_af( n ), the removal time of access unit n is specified by

t_r( n ) = t_r,n( n ) (C- )

Otherwise (low_delay_hrd_flag is equal to 1 and t_r,n( n ) < t_af( n )), the removal time of access unit n is specified by

t_r( n ) = t_r,n( n ) + t_c Ceil( ( t_af( n ) - t_r,n( n ) ) t_c (C- )

NOTE The latter case indicates that the size access unit n, b(n), is so large that it prevents removal at the nominal removal time.

A.2     Operation of the decoded picture buffer (DPB)

The decoded picture buffer contains frame buffers. Each of the frame buffers may contain a decoded frame, a decoded complementary field pair or a single (non-paired) decoded field that are marked as 'used for reference' (reference pictures) or are held for future output (reordered or delayed pictures). Prior to initialisation, the DPB is empty (the DPB fullness is set to zero). The following steps of the subclauses of this subclause all happen instantaneously at t_r( n ) and in the sequence listed.

A.2.1          Decoding of gaps in frame_num and storage of 'non-existing' frames

If applicable, gaps in frame_num are detected by the decoding process and the generated frames are marked and inserted into the DPB as specified below.

Gaps in frame_num are detected by the decoding process and the generated frames are marked as specified in subclause 8.2.5.2.

After the marking of each generated frame, each picture m marked by the sliding window process as unused for reference is removed from the DPB when it is also marked as 'non-existing' or its DPB output time is less than or equal to the CPB removal time of the current picture n; i.e., t_o,dpb( m ) <= t_r( n ). When a frame or the last field in a frame buffer is removed from the DPB, the DPB fullness is decremented by one. The non-existing generated frame is inserted into the DPB and the DPB fullness is incremented by one.

A.2.2          Picture decoding and output

Picture n is decoded and its DPB output time t_o,dpb( n ) is derived by

t_o,dpb( n ) = t_r( n ) + t_c dpb_output_delay( n ) (C- )

The output of the current picture is specified as follows.

If t_o,dpb(n) = t_r(n), the current picture is output.

NOTE - When the current picture is a reference picture it will be stored in the DPB

Otherwise ( t_o,dpb(n) > t_r(n) ), the current picture is output later and will be stored in the DPB (as specified in subclause C.5.4) and is output at time t_o,dpb(n) unless indicated not to be output by the decoding or inference of no_output_of_prior_pics_flag equal to 1 at a time that precedes t_o,dpb(n).

The output picture shall be cropped, using the cropping rectangle specified in the sequence parameter set for the sequence.

When picture n is a picture that is output and is not the last picture of the bitstream that is output, the value of Dt_o,dpb( n ) is defined as:

Dt_o,dpb( n ) = t_o,dpb( n_n ) - t_o,dpb( n ) (C- )

where n_n indicates the picture that follows after picture n in output order.

The decoded picture is temporarily stored (not in the DPB).

A.2.3          Removal of pictures from the DPB before possible insertion of the current picture

The removal of pictures from the DPB before possible insertion of the current picture proceeds as follows.

If the decoded picture is an IDR picture the following applies.

All reference pictures in the DPB are marked as 'unused for reference' as specified in subclauses and .

When the IDR picture is not the first IDR picture decoded and the value of PicWidthInMbs or FrameHeightInMbs or max_dec_frame_buffering derived from the active sequence parameter set is different from the value of PicWidthInMbs or FrameHeightInMbs or max_dec_frame_buffering derived from the sequence parameter set that was active for the preceding sequence, respectively, no_output_of_prior_pics_flag is inferred to be equal to 1 by the HRD, regardless of the actual value of no_output_of_prior_pics_flag.

NOTE - Decoder implementations should try to handle frame or DPB size changes more gracefully than the HRD in regard to changes in PicWidthInMbs or FrameHeightInMbs.

When no_output_of_prior_pics_flag is equal to 1 or is inferred to be equal to 1, all frame buffers in the DPB are emptied without output of the pictures they contain, and DPB fullness is set to 0.

Otherwise (the decoded picture is not an IDR picture), the following applies.

If the slice header of the current picture includes memory_management_control_operation equal to 5, all reference pictures in the DPB are marked as 'unused for reference'.

Otherwise (the slice header of the current picture does not include memory_management_control_operation equal to 5), the decoded reference picture marking process is invoked.

All pictures m in the DPB, for which all of the following conditions are true, are removed from the DPB.

picture m is marked as unused for reference or picture m is a non-reference picture. When a picture is a reference frame, it is considered to be marked as 'unused for reference' only when both of its fields have been marked as 'unused for reference'.

picture m is marked as 'non-existing' or its DPB output time is less than or equal to the CPB removal time of the current picture n; i.e., t_o,dpb( m ) <= t_r( n )

When a frame or the last field in a frame buffer is removed from the DPB, the DPB fullness is decremented by one.

A.2.4          Current decoded picture marking and storage

A.2.4.1     Marking and storage of a reference decoded picture into the DPB

When the current picture is a reference picture it is stored in the DPB as follows.

If the current decoded picture is a second field (in decoding order) of a complementary reference field pair, and the first field of the pair is still in the DPB, the current decoded picture is stored in the same frame buffer as the first field of the pair.

Otherwise, the current decoded picture is stored in an empty frame buffer, and the DPB fullness is incremented by one.

A.2.4.2     Storage of a non-reference picture into the DPB

When the current picture is a non-reference picture and current picture n has t_o,dpb(n) > t_r(n), it is stored in the DPB as follows.

If the current decoded picture is a second field (in decoding order) of a complementary non-reference field pair, and the first field of the pair is still in the DPB, the current decoded picture is stored in the same frame buffer as the first field of the pair.

Otherwise, the current decoded picture is stored in an empty frame buffer, and the DPB fullness is incremented by one.

A.3     Bitstream conformance

A bitstream of coded data conforming to this Recommendation | International Standard fulfils the following requirements.

The bitstream is constructed according to the syntax, semantics, and constraints specified in this Recommendation | International Standard outside of this Annex.

The bitstream is tested by the HRD as specified below:

For Type I bitstreams, the number of tests carried out is equal to cpb_cnt_minus1 + 1 where cpb_cnt_minus1 is the syntax element of hrd_parameters( ) following the vcl_hrd_parameters_present_flag or cpb_cnt_minus1 for Type I conformance is determined by the application by other means not specified in this Recommendation | International Standard. One test is carried out for each bit rate and CPB size combination specified by hrd_parameters( ) following the vcl_hrd_parameters_present_flag.

For Type II bitstreams there are two sets of tests. The number of tests of the first set is equal to cpb_cnt_minus1 + 1 where cpb_cnt_minus1 is the syntax element of hrd_parameters( ) following the vcl_hrd_parameters_present_flag or cpb_cnt_minus1 for Type II conformance is determined by the application by other means not specified in this Recommendation | International Standard.. One test is carried out for each bit rate and CPB size combination. For these tests, only VCL and filler data NAL units are counted for the input bit rate and CPB storage.

The number of tests of the second set, for Type II bitstreams, is equal to cpb_cnt_minus1 + 1 where cpb_cnt_minus1 is the syntax element of hrd_parameters( ) following the nal_hrd_parameters_present_flag or cpb_cnt_minus1 for Type II conformance is determined by the application by other means not specified in this Recommendation | International Standard. One test is carried out for each bit rate and CPB size combination specified by hrd_parameters( ) following the nal_hrd_parameters_present_flag. For these tests, all NAL units (of a Type II NAL unit stream) or all bytes (of a byte stream) are counted for the input bit rate and CPB storage.

For conformant bitstreams, all of the following conditions shall be fulfilled for each of the tests.

Initial arrival time consistency: For each access unit n, with n>0, associated with a buffering period SEI message, with Dt_g,90( n ) specified by

Dt_g,90( n ) * ( t_r,n( n ) - t_af( n - 1 ) ) (C- )

The value of initial_cpb_removal_delay[ SchedSelIdx ] shall be constrained as follows.

If cbr_flag[ SchedSelIdx ] is equal to 0,

initial_cpb_removal_delay[ SchedSelIdx ] <= Dt_g,90( n ) (C- )

Otherwise (cbr_flag[ SchedSelIdx ] is equal to 1),

Floor( Dt_g,90( n ) ) <= initial_cpb_removal_delay[ SchedSelIdx ] < Dt_g,90( n ) (C- )

NOTE When cbr_flag[ SchedSelIdx ] is equal to 1 and the precision of the clocks used (the 90 kHz clock used for initial_cpb_removal_delay[ SchedSelIdx ], and the 1 t_c Hz clock used for cpb_removal_delay) differs, the constraint above may cause a small difference of CPB buffer fullness in the operation of the HRD after initialisation at different buffering period SEI messages. Encoders must take this into account, as the HRD may be initialised at any one of the buffering period SEI messages.

CPB underflow and overflow prevention: The CPB shall never overflow or underflow.

NOTE - In terms of the arrival and removal schedules, this means that, with the exception of some access units in low-delay mode that are described below, all bits from an access unit must be in the CPB at the access unit's nominal removal time t_r,n(n). In other words, its final arrival time must be no later than its nominal removal time: t_af(n) <= t_r,n(n). Further, the nominal removal time t_r,n(n) must be no later than the time-equivalent of the buffer size CpbSize[ SchedSelIdx ] BitRate[ SchedSelIdx ]. This prevents both underflow and overflow.

CPB overflow prevention for big picture removal time: When the final arrival time t_af(n) of access unit n to the CPB exceeds its nominal removal time t_r,n(n), its size must be such that it can be removed from the buffer without overflow at t_r(n) as specified above.

NOTE The final arrival time t_af(n) of access unit n to the CPB can only exceed its nominal removal time t_r,n(n) when low_delay_hrd_flag is equal to 1.

Maximum removal rate from the CPB: The nominal removal times of pictures from the CPB (starting from the second picture in decoding order), shall satisfy the constraints on t_r,n(n) and t_r(n) expressed in subclauses A.3.1 and A.3.2 for the profile and level specified in the bitstream.

DPB overflow prevention: Immediately after any decoded picture is added to the DPB, the fullness of the DPB shall be less than or equal to the DPB size as constrained by Annexes A, D, and E for the profile and level specified in the bitstream.

DPB underflow prevention: All reference pictures shall be present in the DPB when needed for prediction. Each picture shall be present in the DPB at its DPB output time unless it is not stored in the DPB at all, or is removed from the DPB before its output time by one of the processes specified in subclause C.5.

Maximum output rate from the DPB: The value of Dt_o,dpb(n) as given by Equation C-13, which is the difference between the output time of a picture and that of the picture immediately following it in output order, shall satisfy the constraint expressed in subclause A.3.1 for the profile and level specified in the bitstream.

A.4     Decoder conformance

A decoder conforming to this Recommendation | International Standard fulfils the following requirements.

A decoder claiming conformance to a specific profile and level shall be able decode successfully all conforming bitstreams specified for decoder conformance in subclause C.6, provided that all sequence parameter sets and picture parameters sets referred to in the VCL NAL units, and appropriate buffering period and picture timing SEI messages are conveyed to the decoder, in a timely manner, either in the bitstream (by non-VCL NAL units), or by external means not specified by this Recommendation | International Standard.

There are two types of conformance that can be claimed by a decoder: output timing conformance and output order conformance.

To check conformance of a decoder, test bitstreams conforming to the claimed profile and level, as specified by subclause C.6 are delivered by a hypothetical stream scheduler (HSS) both to the HRD and to the decoder under test (DUT). All pictures output by the HRD shall also be output by the DUT and, for each picture output by the HRD, the values of all samples that are output by the DUT for the corresponding picture shall be equal to the values of the samples output by the HRD.

For output timing decoder conformance, the HSS operates as described above, with delivery schedules selected only from the subset of values of SchedSelIdx for which the bit rate and CPB size are restricted as specified in Annex A, for the specified profile and level, or with 'interpolated' delivery schedules for which the bit rate and CPB size are restricted as specified in Annex A derived from the bit rate and CPB sizes expressed for the provided values of SchedSelIdx as specified below. The same delivery schedule is used for both the HRD and DUT.

When the HRD parameters and the buffering period SEI messages are present with cpb_cnt_minus1 greater than 0, the decoder shall be capable of decoding the bitstream as delivered from the HSS operating using an 'interpolated' delivery schedule specified as having peak bit rate r, CPB size c( r ), and initial CPB removal delay ( f( r ) r ) as follows

a = ( r - BitRate[ SchedSelIdx - 1 ] ) ( BitRate[ SchedSelIdx ] BitRate[ SchedSelIdx - 1 ] ), (C- )

c( r ) = a * CpbSize[ SchedSelIdx ] a) * CpbSize[ SchedSelIdx-1 ] (C- )

f( r ) = a * initial_cpb_removal_delay[ SchedSelIdx ] * BitRate[ SchedSelIdx ] + ( 1 a initial_cpb_removal_delay[ SchedSelIdx - 1 ] * BitRate[ SchedSelIdx - 1 ] (C- )

for any SchedSelIdx > 0 and r such that BitRate[ SchedSelIdx - 1 ] <= r <= BitRate[ SchedSelIdx ] such that r and c( r ) are within the limits as specified in Annex A for the maximum bit rate and buffer size for the specified profile and level.

NOTE - initial_cpb_removal_delay[ SchedSelIdx ] can be different from one buffering period to another and have to be re-calculated.

For output timing decoder conformance, an HRD as described above is used and the timing (relative to the delivery time of the first bit) of picture output is the same for both HRD and the DUT up to a fixed delay.

For output order decoder conformance, the HSS delivers the bitstream to the DUT 'by demand' from the DUT, meaning that the HSS delivers bits (in decoding order) only when the DUT requires more bits to proceed with its processing. An HRD as described below is used, and the HSS delivers the bitstream to the HRD by one of the schedules specified in the bitstream or by an 'interpolated' schedule such that the bit rate and CPB size are restricted as specified in Annex A. The order of pictures output shall be the same for both HRD and the DUT.

NOTE - This means that for this test, the coded picture buffer of the DUT could be as small as the size of the largest access unit.

For the HRD, the CPB size is equal to CpbSize[ SchedSelIdx ] for the selected schedule and the DPB size is equal to MaxDpbSize. Removal time from the CPB for the HRD is equal to final bit arrival time and decoding is immediate. The operation of the DPB of this HRD is described below.

A.4.1          Operation of the output order DPB

The decoded picture buffer contains frame buffers. Each of the frame buffers may contain a decoded frame, a decoded complementary field pair or a single (non-paired) decoded field that is marked as 'used for reference' or is held for future output (reordered pictures). At HRD initialization, the DPB fullness, measured in frames, is set to 0. The following steps all happen instantaneously when an access unit is removed from the CPB, and in the order listed.

A.4.2          Decoding of gaps in frame_num and storage of 'non-existing' pictures

If applicable, gaps in frame_num are detected by the decoding process and the generated frames are marked and inserted into the DPB as specified below.

Gaps in frame_num are detected by the decoding process and the generated frames are marked as specified in subclause 8.2.5.2.

When there are not enough empty frame buffers (i.e., DPB size minus DPB fullness is less than the number of 'non-existing' frames to be stored), the necessary number of frame buffers is emptied by the 'bumping' process specified below.

All generated frames marked as non-existing and used for short-term reference are inserted into the DPB. The DPB fullness is incremented according to the number of additional frames stored in the DPB as a result of the insertion of the 'non-existing' frames.

A.4.3          Picture decoding

Primary coded picture n is decoded and is temporarily stored (not in the DPB).

A.4.4          Removal of pictures from the DPB before possible insertion of the current picture

The removal of pictures from the DPB before possible insertion of the current picture proceeds as follows .

If the decoded picture is an IDR picture the following applies.

All reference pictures in the DPB are marked as 'unused for reference' as specified in subclause .

When the IDR picture is not the first IDR picture decoded and the value of PicWidthInMbs or FrameHeightInMbs or max_dec_frame_buffering derived from the active sequence parameter set is different from the value of PicWidthInMbs or FrameHeightInMbs or max_dec_frame_buffering derived from the sequence parameter set that was active for the preceding sequence, respectively, no_output_of_prior_pics_flag is inferred to be equal to 1 by the HRD, regardless of the actual value of no_output_of_prior_pics_flag.

NOTE - Decoder implementations should try to handle frame or DPB size changes more gracefully than the HRD in regard to changes in PicWidthInMbs or FrameHeightInMbs.

When no_output_of_prior_pics_flag is equal to 1 or is inferred to be equal to 1, all frame buffers in the DPB are emptied without output of the pictures they contain, and DPB fullness is set to 0.

Otherwise (the decoded picture is not an IDR picture), the following applies.

If the slice header of the current picture includes memory_management_control_operation equal to 5, all reference pictures in the DPB are marked as 'unused for reference' as specified in subclause .

Otherwise (the slice header of the current picture does not include memory_management_control_operation equal to 5), the decoded reference picture marking process is invoked as specified in subclause 8.2.5. Frame buffers containing frames marked as 'non-existing' and 'unused for reference' are emptied without output of the 'non-existing' frames they contain, and the DPB fullness is decremented by the number of frame buffers emptied.

When the current picture is an IDR picture and no_output_of_prior_pics_flag is not equal to 1 and is not inferred to be equal to 1, or the current picture has memory_management_control_operation equal to 5, all non-empty frame buffers in the DPB are emptied by repeatedly invoking the bumping process specified below, and the DPB fullness is set to 0.

A.4.5          Current decoded picture marking and storage

A.4.5.1     Storage and marking of a reference decoded picture into the DPB

When the current picture is a reference picture, it is stored in the DPB as follows.

If the current decoded picture is the second field (in decoding order) of a complementary reference field pair, and the first field of the pair is still in the DPB, the current picture is stored in the same frame buffer as the first field of the pair.

Otherwise, the following operations are performed:

When there is no empty frame buffer (i.e., DPB fullness is equal to DPB size), one is emptied by the 'bumping' process specified below.

The current decoded picture is stored in an empty frame buffer and the DPB fullness is incremented by one.

A.4.5.2     Storage and marking of a non-reference decoded picture into the DPB

When the current picture is a non-reference picture, it is stored in the DPB as follows.

If the current decoded picture is the second field (in decoding order) of a complementary non-reference field pair and the first field of the pair is still in the DPB, the current picture is stored in the same frame buffer as the first field of the pair.

Otherwise, the following operations are performed:

When there is no empty frame buffer (i.e., DPB fullness is equal to DPB size), the following applies

If the current picture does not have the lowest value of PicOrderCnt( ) among all pictures in the DPB, a frame buffer is emptied by the 'bumping' process described below.

Otherwise (the current picture has the lowest value of PicOrderCnt( ) among all pictures in the DPB), the current picture is cropped, using the cropping rectangle specified in the sequence parameter set for the sequence and the cropped picture is output

When the current decoded picture has not been output, it is stored in an empty frame buffer and the DPB fullness is incremented by one.

A.4.5.3     'Bumping' process

The 'bumping' process operates when an empty frame buffer is needed for a decoded (non IDR) picture, as in the following steps:

When a frame buffer of the DPB contains a complementary non-reference field pair with both fields marked as 'needed for output' and both fields have the same PicOrderCnt( ), the first of the two fields in decoding order is considered first for bumping. When a frame buffer of the DPB contains a complementary reference field pair with both fields marked as 'needed for output' and both fields have the same PicOrderCnt( ), the two fields are considered together for bumping as specified below.

The picture marked as 'needed for output' that has the lowest value of PicOrderCnt( ) of all pictures in the DPB marked as 'needed for output', is cropped, using the cropping rectangle specified in the sequence parameter set for the sequence, the cropped picture is output, and the picture is marked as 'not needed for output'. When this picture is a field which is part of a complementary reference field pair, and the other field of the pair is still in the DPB and marked as 'needed for output', and the values of PicOrderCnt( ) of both fields are the same, the other field is cropped, using the cropping rectangle specified in the sequence parameter set for the sequence, and the cropped field is output together with the previous one, and the field is marked as 'not needed for output'.

The frame buffer that included the field, complementary reference field pair, or frame output in step 1 is checked, and the following applies.

If one of the following conditions is satisfied, the frame buffer is emptied, DPB fullness is decremented and the bumping operation is terminated.

The frame buffer includes a non-reference non-paired field

The frame buffer includes a non-reference frame with both fields marked as 'not needed for output'

The frame buffer includes a complementary non-reference field pair with both fields marked as 'not needed for output'.

The frame buffer includes a non-paired reference field marked as 'unused for reference' and 'not needed for output'.

The frame buffer includes a reference frame with both fields marked as 'unused for reference' and 'not needed for output'.

The frame buffer includes a complementary reference field pair with both fields marked as 'unused for reference' and 'not needed for output'.

Otherwise, steps 1 and 2 are repeated until termination.

BAnnex D
Supplemental enhancement information

(This annex forms an integral part of this Recommendation | International Standard)

This annex specifies syntax and semantics for SEI message payloads.

SEI messages assist in processes related to decoding, display or other purposes. However, SEI messages are not required for constructing the luma or chroma samples by the decoding process. Conforming decoders are not required to process this information for output order conformance to this Recommendation | International Standard (see Annex C for the specification of conformance). Some SEI message information is required to check bitstream conformance and for output timing decoder conformance.

In Annex D, specification for presence of SEI messages are also satisfied when those messages (or some subset of them) are conveyed to decoders (or to the HRD) by other means not specified by this Recommendation | International Standard. When present in the bitstream, SEI messages shall obey the syntax and semantics specified in subclauses and and this annex. When the content of an SEI message is conveyed for the application by some means other than presence within the bitstream, the representation of the content of the SEI message is not required to use the same syntax specified in this annex. For the purpose of counting bits, only the appropriate bits that are actually present in the bitstream are counted.

B.5     SEI payload syntax

B.5.1          Buffering period SEI message syntax

B.5.2          Picture timing SEI message syntax

B.5.3          Pan-scan rectangle SEI message syntax

B.5.4          Filler payload SEI message syntax

B.5.5          User data registered by ITU-T Recommendation T.35 SEI message syntax

B.5.6          User data unregistered SEI message syntax

B.5.7          Recovery point SEI message syntax

B.5.8          Decoded reference picture marking repetition SEI message syntax

B.5.9          Spare picture SEI message syntax

B.5.10      Scene information SEI message syntax

B.5.11      Sub-sequence information SEI message syntax

B.5.12      Sub-sequence layer characteristics SEI message syntax

B.5.13      Sub-sequence characteristics SEI message syntax

B.5.14      Full-frame freeze SEI message syntax

B.5.15      Full-frame freeze release SEI message syntax

B.5.16      Full-frame snapshot SEI message syntax

B.5.17      Progressive refinement segment start SEI message syntax

B.5.18      Progressive refinement segment end SEI message syntax

B.5.19      Motion-constrained slice group set SEI message syntax

B.5.20      Reserved SEI message syntax

B.6     SEI payload semantics

B.6.1          Buffering period SEI message semantics

When NalHrdBpPresentFlag or VclHrdBpPresentFlag are equal to 1, a buffering period SEI message can be associated with any access unit in the bitstream, and a buffering period SEI message shall be associated with each IDR access unit and with each access unit associated with a recovery point SEI message.

NOTE For some applications, the frequent presence of a buffering period SEI message may be desirable.

A buffering period is specified as the set of access units between two instances of the buffering period SEI message in decoding order.

seq_parameter_set_id specifies the sequence parameter set that contains the sequence HRD attributes. The value of seq_parameter_set_id shall be equal to the value of seq_parameter_set_id in the picture parameter set referenced by the primary coded picture associated with the buffering period SEI message. The value of seq_parameter_set_id shall be in the range of 0 to 31, inclusive.

initial_cpb_removal_delay[ SchedSelIdx ] specifies the delay for the SchedSelIdx-th CPB between the time of arrival in the CPB of the first bit of the coded data associated with the access unit associated with the buffering period SEI message and the time of removal from the CPB of the coded data associated with the same access unit, for the first buffering period after HRD initialisation. The syntax element has a length in bits given by initial_cpb_removal_delay_length_minus1 + 1. It is in units of a 90 kHz clock. initial_cpb_removal_delay[ SchedSelIdx ] shall not be equal to 0 and shall not exceed 90000 * ( CpbSize[ SchedSelIdx ] BitRate[ SchedSelIdx ] ), the time-equivalent of the CPB size in 90 kHz clock units.

initial_cpb_removal_delay_offset[ SchedSelIdx ] is used for the SchedSelIdx-th CPB in combination with the cpb_removal_delay to specify the initial delivery time of coded access units to the CPB. initial_cpb_removal_delay_offset[ SchedSelIdx ] is in units of a 90 kHz clock. The initial_cpb_removal_delay_offset[ SchedSelIdx ] syntax element is a fixed length code whose length in bits is given by initial_cpb_removal_delay_length_minus1 + 1. This syntax element is not used by decoders and is needed only for the delivery scheduler (HSS) specified in Annex C.

Over the entire coded video sequence, the sum of initial_cpb_removal_delay[ SchedSelIdx ] and initial_cpb_removal_delay_offset[ SchedSelIdx ] shall be constant for each value of SchedSelIdx.

B.6.2          Picture timing SEI message semantics

When CpbDpbDelaysPresentFlag is equal to 1, a picture timing SEI Message shall be associated with every access unit in the bitstream.

cpb_removal_delay specifies how many clock ticks (see subclause E.11.1) to wait after removal from the CPB of the access unit associated with the most recent buffering period SEI message before removing from the buffer the access unit data associated with the picture timing SEI message. This value is also used to calculate an earliest possible time of arrival of access unit data into the CPB for the HSS, as specified in Annex C. The syntax element is a fixed length code whose length in bits is given by cpb_removal_delay_length_minus1 + 1. The cpb_removal_delay is the remainder of a 2^{(cpb_removal_delay_length_minus1 + 1)} counter.

The value of cpb_removal_delay for the first picture in the bitstream shall be equal to 0.

dpb_output_delay is used to compute the DPB output time of the picture. It specifies how many clock ticks to wait after removal of an access unit from the CPB before the decoded picture can be output from the DPB (see subclause C.5).

NOTE - A picture is not removed from the DPB at its output time when it is still marked as 'used for short-term reference' or 'used for long-term reference'.

NOTE - Only one dpb_output_delay is specified for a decoded picture.

The size of the syntax element dpb_output_delay is given in bits by dpb_output_delay_length_minus1 + 1.

The output time derived from the dpb_output_delay of any picture that is output from an output timing conforming decoder as specified in subclause C.5 shall precede the output time derived from the dpb_output_delay of all pictures in any subsequent coded video sequence in decoding order.

The output time derived from the dpb_output_delay of the second field, in decoding order, of a complementary non-reference field pair shall exceed the output time derived from the dpb_output_delay of the first field of the same complementary non-reference field pair.

The picture output order established by the values of this syntax element shall be the same order as established by the values of PicOrderCnt( ) as specified by subclauses C.7.1 to C.7.5, except that when the two fields of a complementary reference field pair have the same value of PicOrderCnt( ), the two fields have different output times.

For pictures that are not output by the 'bumping' process of subclause C.7.5 because they precede, in decoding order, an IDR picture with no_output_of_prior_pics_flag equal to 1 or inferred to be equal to 1, the output times derived from dpb_output_delay shall be increasing with increasing value of PicOrderCnt( ) relative to all pictures within the same coded video sequence subsequent to any picture having a memory_management_control_operation equal to 5.

pic_struct indicates whether a picture should be displayed as a frame or one or more fields, according to Table D-1. Frame doubling (pic_struct equal to 7) indicates that the frame should be displayed two times consecutively, and frame tripling (pic_struct equal to 8) indicates that the frame should be displayed three times consecutively.

NOTE - Frame doubling can facilitate the display, for example, of 25p video on a 50p display and 29.97p video on a 59.94p display. Using frame doubling and frame tripling in combination on every other frame can facilitate the display of 23.98p video on a 59.94p display.

Table D‑ Interpretation of pic_struct

NumClockTS is determined by pic_struct as specified in Table D‑1. There are up to NumClockTS sets of clock timestamp information for a picture, as specified by clock_timestamp_flag[ i ] for each set. The sets of clock timestamp information apply to the field(s) or the frame(s) associated with the picture by pic_struct.

The contents of the clock timestamp syntax elements indicate a time of origin, capture, or alternative ideal display. This indicated time is computed as

clockTimestamp = ( ( hH * 60 + mM ) * 60 + sS ) * time_scale +
nFrames * ( num_units_in_tick * ( 1 + nuit_field_based_flag ) ) + tOffset, (D- )

in units of clock ticks of a clock with clock frequency equal to time_scale Hz, relative to some unspecified point in time for which clockTimestamp is equal to 0. Output order and DPB output timing are not affected by the value of clockTimestamp. When two or more frames with pic_struct equal to 0 are consecutive in output order and have equal values of clockTimestamp, the indication is that the frames represent the same content and that the last such frame in output order is the preferred representation.

NOTE clockTimestamp time indications may aid display on devices with refresh rates other than those well-matched to DPB output times.

clock_timestamp_flag[ i ] equal to 1 indicates that a number of clock timestamp syntax elements are present and follow immediately. clock_timestamp_flag[ i ] equal to 0 indicates that the associated clock timestamp syntax elements are not present. When NumClockTS is greater than 1 and clock_timestamp_flag[ i ] is equal to 1 for more than one value of i, the value of clockTimestamp shall be non-decreasing with increasing value of i.

ct_type indicates the scan type (interlaced or progressive) of the source material as follows:

Two fields of a coded frame may have different values of ct_type.

When clockTimestamp is equal for two fields of opposite parity that are consecutive in output order, both with ct_type equal to 0 (progressive) or ct_type equal to 2 (unknown), the two fields are indicated to have come from the same original progressive frame. Two consecutive fields in output order shall have different values of clockTimestamp when the value of ct_type for either field is 1 (interlaced).

Table D‑ Mapping of ct_type to source picture scan

nuit_field_based_flag: Used in calculating clockTimestamp, as specified in Equation D-1.

counting_type: Specifies the method of dropping values of the n_frames as specified in Table D‑3.

Table D‑ Definition of counting_type values

full_timestamp_flag equal to 1 specifies that the n_frames syntax element is followed by seconds_value, minutes_value, and hours_value. full_timestamp_flag equal to 0 specifies that the n_frames syntax element is followed by seconds_flag.

discontinuity_flag equal to 0 indicates that the difference between the current value of clockTimestamp and the value of clockTimestamp computed from the previous clock timestamp in output order can be interpreted the time difference between the times of origin or capture of the associated frames or fields. discontinuity_flag equal to 1 indicates that the difference between the current value of clockTimestamp and the value of clockTimestamp computed from the previous clock timestamp in output order should not be interpreted the time difference between the times of origin or capture of the associated frames or fields. When discontinuity_flag is equal to 0, the value of clockTimestamp shall be greater than or equal to all values of clockTimestamp present for the preceding picture in DPB output order.

cnt_dropped_flag specifies the skipping of one or more values of n_frames using the counting method specified by counting_type.

n_frames specifies the value of nFrames used to compute clockTimestamp. n_frames shall be less than

MaxFPS = Ceil( time_scale num_units_in_tick ) (D- )

NOTE n_frames is a frame-based counter. For field-specific timing indications, time_offset should be used to indicate a distinct clockTimestamp for each field.

When counting_type is equal to 2 and cnt_dropped_flag is equal to 1, n_frames shall be equal to 1 and the value of n_frames for the previous picture in output order shall not be equal to 0 unless discontinuity_flag is equal to 1.

NOTE When counting_type is equal to 2, the need for increasingly large magnitudes of tOffset in Equation D-1 when using fixed non-integer frame rates (e.g., 12.5 frames per second with time_scale equal to 25 and num_units_in_tick equal to 2 and nuit_field_based_flag equal to 0) can be avoided by occasionally skipping over the value n_frames equal to 0 when counting (e.g., counting n_frames from 0 to 12, then incrementing seconds_value and counting n_frames from 1 to 12, then incrementing seconds_value and counting n_frames from 0 to 12, etc.).

When counting_type is equal to 3 and cnt_dropped_flag is equal to 1, n_frames shall be equal to 0 and the value of n_frames for the previous picture in output order shall not be equal to MaxFPS 1 unless discontinuity_flag is equal to 1.

NOTE When counting_type is equal to 3, the need for increasingly large magnitudes of tOffset in Equation D-1 when using fixed non-integer frame rates (e.g., 12.5 frames per second with time_scale equal to 25 and num_units_in_tick equal to 2 and nuit_field_based_flag equal to 0) can be avoided by occasionally skipping over the value n_frames equal to MaxFPS when counting (e.g., counting n_frames from 0 to 12, then incrementing seconds_value and counting n_frames from 0 to 11, then incrementing seconds_value and counting n_frames from 0 to 12, etc.).

When counting_type is equal to 4 and cnt_dropped_flag is equal to 1, n_frames shall be equal to 2 and the specified value of sS shall be zero and the specified value of mM shall not be an integer multiple of ten and n_frames for the previous picture in output order shall not be equal to 0 or 1 unless discontinuity_flag is equal to 1.

NOTE When counting_type is equal to 4, the need for increasingly large magnitudes of tOffset in Equation D-1 when using fixed non-integer frame rates (e.g., 30000 1001 frames per second with time_scale equal to 60000 and num_units_in_tick equal to 1 001 and nuit_field_based_flag equal to 1) can be reduced by occasionally skipping over the value n_frames equal to MaxFPS when counting (e.g., counting n_frames from 0 to 29, then incrementing seconds_value and counting n_frames from 0 to 29, etc., until the seconds_value is zero and minutes_value is not an integer multiple of ten, then counting n_frames from 2 to 29, then incrementing seconds_value and counting n_frames from 0 to 29, etc.). This counting method is well known in industry and is often referred to as 'NTSC drop-frame' counting.

When counting_type is equal to 5 or 6 and cnt_dropped_flag is equal to 1, n_frames shall not be equal to 1 plus the value of n_frames for the previous picture in output order modulo MaxFPS unless discontinuity_flag is equal to 1.

NOTE When counting_type is equal to 5 or 6, the need for increasingly large magnitudes of tOffset in Equation D-1 when using fixed non-integer frame rates can be avoided by occasionally skipping over some values of n_frames when counting. The specific values of n_frames that are skipped are not specified when counting_type is equal to 5 or 6.

seconds_flag equal to 1 specifies that seconds_value and minutes_flag are present when full_timestamp_flag is equal to 0. seconds_flag equal to 0 specifies that seconds_value and minutes_flag are not present.

seconds_value specifies the value of sS used to compute clockTimestamp. The value of seconds_value shall be in the range of 0 to 59, inclusive. When seconds_value is not present, the previous seconds_value in decoding order shall be used as sS to compute clockTimestamp.

minutes_flag equal to 1 specifies that minutes_value and hours_flag are present when full_timestamp_flag is equal to 0 and seconds_flag is equal to 1. minutes_flag equal to 0 specifies that minutes_value and hours_flag are not present.

minutes_value specifies the value of mM used to compute clockTimestamp. The value of minutes_value shall be in the range of 0 to 59, inclusive. When minutes_value is not present, the previous minutes_value in decoding order shall be used as mM to compute clockTimestamp.

hours_flag equal to 1 specifies that hours_value is present when full_timestamp_flag is equal to 0 and seconds_flag is equal to 1 and minutes_flag is equal to 1.

hours_value specifies the value of hH used to compute clockTimestamp. The value of hours_value shall be in the range of 0 to 23, inclusive. When hours_value is not present, the previous hours_value in decoding order shall be used as hH to compute clockTimestamp.

time_offset specifies the value of tOffset used to compute clockTimestamp. The number of bits used to represent time_offset shall be equal to time_offset_length. When time_offset is not present, the value 0 shall be used as tOffset to compute clockTimestamp.

B.6.3          Pan-scan rectangle SEI message semantics

The pan-scan rectangle SEI message syntax elements specify the coordinates of a rectangle relative to the cropping rectangle of the sequence parameter set. Each coordinate of this rectangle is specified in units of one-sixteenth sample spacing relative to the luma sampling grid.

pan_scan_rect_id contains an identifying number that may be used to identify the purpose of the pan-scan rectangle (for example, to identify the rectangle as the area to be shown on a particular display device or as the area that contains a particular actor in the scene). The value of pan_scan_rect_id shall be in the range of 0 to 2³²  1, inclusive.

Values of pan_scan_rect_id from 0 to 255 and from 512 to 2³¹-1 may be used as determined by the application. Values of pan_scan_rect_id from 256 to 511 and from 2³¹ to 2³²-1 are reserved for future use by ITU-T | ISO/IEC. Decoders encountering a value of pan_scan_rect_id in the range of 256 to 511 or in the range of 2³¹ to 2³² - 1 shall ignore (remove from the bitstream and discard) it.

pan_scan_rect_cancel_flag equal to 1 indicates that the SEI message cancels the persistence of a previous pan-scan rectangle SEI message. pan_scan_rect_cancel_flag equal to 0 indicates that the SEI message does not cancel the persistence of a previous pan-scan rectangle SEI message and that pan-scan rectangle information follows.

pan_scan_cnt_minus1 specifies the number of pan-scan rectangles that are present in the SEI message. pan_scan_cnt_minus1 shall be in the range of 0 to 2, inclusive. pan_scan_cnt_minus1 equal to 0 indicates that a single pan-scan rectangle is present that applies to all fields of the decoded picture. pan_scan_cnt_minus1 shall be equal to 0 when the current picture is a field. pan_scan_cnt_minus1 equal to 1 indicates that two pan-scan rectangles are present, the first of which applies to the first field of the picture in output order and the second of which applies to the second field of the picture in output order. pan_scan_cnt_minus1 equal to 2 indicates that three pan-scan rectangles are present, the first of which applies to the first field of the picture in output order, the second of which applies to the second field of the picture in output order, and the third of which applies to a repetition of the first field as a third field in output order.

pan_scan_rect_left_offset[ i ], pan_scan_rect_right_offset[ i ], pan_scan_rect_top_offset[ i ], and pan_scan_rect_bottom_offset[ i ], specify, as signed integer quantities in units of one-sixteenth sample spacing relative to the luma sampling grid, the location of the pan-scan rectangle. The values of each of these four syntax elements shall be in the range of -2³¹ to 2³¹ - 1, inclusive.

The pan-scan rectangle is specified, in units of one-sixteenth sample spacing relative to a luma frame sampling grid, as the area of the rectangle with coordinates as follows:

If frame_mbs_only_flag is equal to 1, the pan-scan rectangle has luma frame horizontal coordinates from 32 * frame_crop_left_offset + pan_scan_rect_left_offset[ i ] to 32 * ( 8 * PicWidthInMbs frame_crop_right_offset ) + pan_scan_rect_right_offset[ i ] 1 and with vertical coordinates from 32 * frame_crop_top_offset + pan_scan_rect_top_offset[ i ] to 32 * ( 8 * PicHeightInMbs  frame_crop_bottom_offset ) + pan_scan_rect_bottom_offset[ i ] 1, inclusive. In this case, the value of 32 * frame_crop_left_offset + pan_scan_rect_left_offset[ i ] shall be less than or equal to 32 * ( 8 * PicWidthInMbs frame_crop_right_offset ) + pan_scan_rect_right_offset[ i ] 1; and the value of 32 * frame_crop_top_offset + pan_scan_rect_top_offset[ i ] shall be less than or equal to 32 *  ( 8 * PicHeightInMbs frame_crop_bottom_offset ) + pan_scan_rect_bottom_offset[ i ] 1.

Otherwise (frame_mbs_only_flag is equal to 0), the pan-scan rectangle has luma frame horizontal coordinates from 32 * frame_crop_left_offset + pan_scan_rect_left_offset[ i ] to 32 * ( 8 * PicWidthInMbs frame_crop_right_offset ) + pan_scan_rect_right_offset[ i ] 1 and with vertical coordinates from 64 * frame_crop_top_offset + pan_scan_rect_top_offset[ i ] to 64 * ( 4 * PicHeightInMbs  frame_crop_bottom_offset ) + pan_scan_rect_bottom_offset[ i ] 1, inclusive. In this case, the value of 32 * frame_crop_left_offset + pan_scan_rect_left_offset[ i ] shall be less than or equal to 32 * ( 8 * PicWidthInMbs frame_crop_right_offset ) + pan_scan_rect_right_offset[ i ] 1; and the value of 64 * frame_crop_top_offset + pan_scan_rect_top_offset[ i ] shall be less than or equal to 64 * ( 4 * PicHeightInMbs frame_crop_bottom_offset ) + pan_scan_rect_bottom_offset[ i ] 1.

When the pan-scan rectangular area includes samples outside of the cropping rectangle, the region outside of the cropping rectangle may be filled with synthesized content (such as black video content or neutral grey video content) for display.

pan_scan_rect_repetition_period indicates whether another pan-scan rectangle SEI message with the same value of pan_scan_rect_id shall be present in the bitstream and specifies the picture order count interval within which it will be present. The value of pan_scan_rect_repetition_period shall be in the range of 0 to 16 384, inclusive. When pan_scan_cnt_minus1 is greater than 0, pan_scan_rect_repetition_period shall not be greater than 1.

pan_scan_rect_repetition_period equal to 0 specifies that the pan-scan rectangle information applies to the current decoded picture only.

pan_scan_rect_repetition_period equal to 1 specifies that the pan-scan rectangle information persists in output order until any of the following conditions are true.

A new coded video sequence begins

A picture in an access unit containing a pan-scan rectangle SEI message with the same value of pan_scan_rect_id is output having PicOrderCnt( ) greater than PicOrderCnt( CurrPic ).

pan_scan_rect_repetition_period equal to 0 or equal to 1 indicates that another pan-scan rectangle SEI message with the same value of pan_scan_rect_id may or may not be present.

pan_scan_rect_repetition_period greater than 1 specifies that the pan-scan rectangle information persists until any of the following conditions are true.

A new coded video sequence begins

A picture in an access unit containing a pan-scan rectangle SEI message with the same value of pan_scan_rect_id is output having PicOrderCnt( ) greater than PicOrderCnt( CurrPic ) + pan_scan_rect_repetition_period.

pan_scan_rect_repetition_period greater than 1 indicates that another pan-scan rectangle SEI message with the same value of pan_scan_rect_id shall be present for a picture in an access unit that is output having PicOrderCnt( ) less than or equal to PicOrderCnt( CurrPic ) + pan_scan_rect_repetition_period; unless a new coded video sequence begins without output of such a picture.

B.6.4          Filler payload SEI message semantics

This message contains a series of payloadSize bytes of value 0xFF, which can be discarded.

ff_byte shall be a byte having the value 0xFF.

B.6.5          User data registered by ITU-T Recommendation T.35 SEI message semantics

This message contains user data registered as specified by ITU-T Recommendation T.35, the contents of which are not specified by this Recommendation | International Standard.

itu_t_t35_country_code shall be a byte having a value specified as a country code by ITU-T Recommendation T.35 Annex A.

itu_t_t35_country_code_extension_byte shall be a byte having a value specified as a country code by ITU-T Recommendation T.35 Annex B.

itu_t_t35_payload_byte shall be a byte containing data registered as specified by ITU-T Recommendation T.35.

The ITU-T T.35 terminal provider code and terminal provider oriented code shall be contained in the first one or more bytes of the itu_t_t35_payload_byte, in the format specified by the Administration that issued the terminal provider code. Any remaining itu_t_t35_payload_byte data shall be data having syntax and semantics as specified by the entity identified by the ITU-T T.35 country code and terminal provider code.

B.6.6          User data unregistered SEI message semantics

This message contains unregistered user data identified by a UUID, the contents of which are not specified by this Recommendation | International Standard.

uuid_iso_iec_11578 shall have a value specified as a UUID according to the procedures of ISO/IEC 11578:1996 Annex A.

user_data_payload_byte shall be a byte containing data having syntax and semantics as specified by the UUID generator.

B.6.7          Recovery point SEI message semantics

The recovery point SEI message assists a decoder in determining when the decoding process will produce acceptable pictures for display after the decoder initiates random access or after the encoder indicates a broken link in the sequence. When the decoding process is started with the access unit in decoding order associated with the recovery point SEI message, all decoded pictures at or subsequent to the recovery point in output order specified in this SEI message are indicated to be correct or approximately correct in content. Decoded pictures produced by random access at or before the picture associated with the recovery point SEI message need not be correct in content until the indicated recovery point, and the operation of the decoding process starting at the picture associated with the recovery point SEI message may contain references to pictures not available in the decoded picture buffer.

In addition, by use of the broken_link_flag, the recovery point SEI message can indicate to the decoder the location of some pictures in the bitstream that can result in serious visual artefacts when displayed, even when the decoding process was begun at the location of a previous IDR access unit in decoding order.

NOTE The broken_link_flag can be used by encoders to indicate the location of a point after which the decoding process for the decoding of some pictures may cause references to pictures that, though available for use in the decoding process, are not the pictures that were used for reference when the bitstream was originally encoded (e.g., due to a splicing operation performed during the generation of the bitstream).

The recovery point is specified as a count in units of access units subsequent to the current access unit at the position of the SEI message.

NOTE When HRD information is present in the bitstream, a buffering period SEI message should be associated with the access unit associated with the recovery point SEI message in order to establish initialisation of the HRD buffer model after a random access.

recovery_frame_cnt specifies the recovery point of output pictures in output order. All decoded pictures in output order are indicated to be correct or approximately correct in content starting at the output order position of the reference picture having the frame_num equal to the frame_num of the VCL NAL units for the current access unit incremented by recovery_frame_cnt in modulo MaxFrameNum arithmetic. recovery_frame_cnt shall be in the range of 0 to MaxFrameNum  1, inclusive.

exact_match_flag indicates whether decoded pictures at and subsequent to the specified recovery point in output order derived by starting the decoding process at the access unit associated with the recovery point SEI message shall be an exact match to the pictures that would be produced by starting the decoding process at the location of a previous IDR access unit in the NAL unit stream. The value 0 indicates that the match need not be exact and the value 1 indicates that the match shall be exact.

When decoding starts from the location of the recovery point SEI message, all references to not available reference pictures shall be inferred as references to pictures containing only macroblocks coded using Intra macroblock prediction modes and having sample values given by Y samples equal to 128, Cb samples equal to 128, and Cr samples equal to 128 (mid-level grey) for purposes of determining the conformance of the value of exact_match_flag.

NOTE When performing random access, decoders should infer all references to not available reference pictures as references to pictures containing only intra macroblocks and having sample values given by Y equal to 128, Cb equal to 128, and Cr equal to 128 (mid-level grey), regardless of the value of exact_match_flag.

When exact_match_flag is equal to 0, the quality of the approximation at the recovery point is chosen by the encoding process and is not specified by this Recommendation | International Standard.

broken_link_flag indicates the presence or absence of a broken link in the NAL unit stream at the location of the recovery point SEI message and is assigned further semantics as follows.

If broken_link_flag is equal to 1, pictures produced by starting the decoding process at the location of a previous IDR access unit may contain undesirable visual artefacts to the extent that decoded pictures at and subsequent to the access unit associated with the recovery point SEI message in decoding order should not be displayed until the specified recovery point in output order.

Otherwise (broken_link_flag is equal to 0), no indication is given regarding any potential presence of visual artefacts.

Regardless of the value of the broken_link_flag, pictures subsequent to the specified recovery point in output order are specified to be correct or approximately correct in content.

NOTE When a sub-sequence information SEI message is present in conjunction with a recovery point SEI message in which broken_link_flag is equal to 1 and when sub_seq_layer_num is equal to 0, sub_seq_id should be different from the latest sub_seq_id for sub_seq_layer_num equal to 0 that was decoded prior to the location of the recovery point SEI message. When broken_link_flag is equal to 0, the sub_seq_id in sub-sequence layer 0 should remain unchanged.

changing_slice_group_idc equal to 0 indicates that decoded pictures are correct or approximately correct in content at and subsequent to the recovery point in output order when all macroblocks of the primary coded pictures are decoded within the changing slice group period, i.e., the period between the access unit associated with the recovery point SEI message (inclusive) and the specified recovery point (exclusive) in decoding order. changing_slice_group_idc shall be equal to 0 when num_slice_groups_minus1 is equal to 0 in any primary coded picture within the changing slice group period.

When changing_slice_group_idc is equal to 1 or 2, num_slice_groups_minus1 shall be equal to 1 and the macroblock-to-slice-group map type 3, 4, or 5 shall be applied in each primary coded picture in the changing slice group period.

changing_slice_group_idc equal to 1 indicates that within the changing slice group period no sample values outside the decoded macroblocks covered by slice group 0 are used for inter prediction of any macroblock within slice group 0. In addition, changing_slice_group_idc equal to 1 indicates that when all macroblocks in slice group 0 within the changing slice group period are decoded, decoded pictures will be correct or approximately correct in content at and subsequent to the specified recovery point in output order regardless of whether any macroblock in slice group 1 within the changing slice group period is decoded.

changing_slice_group_idc equal to 2 indicates that within the changing slice group period no sample values outside the decoded macroblocks covered by slice group 1 are used for inter prediction of any macroblock within slice group 1. In addition, changing_slice_group_idc equal to 2 indicates that when all macroblocks in slice group 1 within the changing slice group period are decoded, decoded pictures will be correct or approximately correct in content at and subsequent to the specified recovery point in output order regardless of whether any macroblock in slice group 0 within the changing slice group period is decoded.

changing_slice_group_idc shall be in the range of 0 to 2, inclusive.

B.6.8          Decoded reference picture marking repetition SEI message semantics

The decoded reference picture marking repetition SEI message is used to repeat the decoded reference picture marking syntax structure that was located in the slice header of an earlier picture in the sequence in decoding order.

original_idr_flag shall be equal to 1 when the decoded reference picture marking syntax structure occurred originally in an IDR picture. original_idr_flag shall be equal to 0 when the repeated decoded reference picture marking syntax structure did not occur in an IDR picture originally.

original_frame_num shall be equal to the frame_num of the picture where the repeated decoded reference picture marking syntax structure originally occurred. The picture indicated by original_frame_num is the previous coded picture having the specified value of frame_num. The value of original_frame_num used to refer to a picture having a memory_management_control_operation equal to 5 shall be 0.

original_field_pic_flag shall be equal to the field_pic_flag of the picture where the repeated decoded reference picture marking syntax structure originally occurred.

original_bottom_field_flag shall be equal to the bottom_field_flag of the picture where the repeated decoded reference picture marking syntax structure originally occurred.

dec_ref_pic_marking( ) shall contain a copy of the decoded reference picture marking syntax structure of the picture whose frame_num was original_frame_num. The nal_unit_type used for specification of the repeated dec_ref_pic_marking( ) syntax structure shall be the nal_unit_type of the slice header(s) of the picture whose frame_num was original_frame_num (i.e., nal_unit_type as used in subclause shall be considered equal to 5 when original_idr_flag is equal to 1 and shall not be considered equal to 5 when original_idr_flag is equal to 0).

B.6.9          Spare picture SEI message semantics

This SEI message indicates that certain slice group map units, called spare slice group map units, in one or more decoded reference pictures resemble the co-located slice group map units in a specified decoded picture called the target picture. A spare slice group map unit may be used to replace a co-located, incorrectly decoded slice group map unit, in the target picture. A decoded picture containing spare slice group map units is called a spare picture.

For all spare pictures identified in a spare picture SEI message, the value of frame_mbs_only_flag shall be equal to the value of frame_mbs_only_flag of the target picture in the same SEI message. The spare pictures in the SEI message are constrained as follows.

If the target picture is a decoded field, all spare pictures identified in the same SEI message shall be decoded fields.

Otherwise (the target picture is a decoded frame), all spare pictures identified in the same SEI message shall be decoded frames.

For all spare pictures identified in a spare picture SEI message, the values of pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1 shall be equal to the values of pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1, respectively, of the target picture in the same SEI message. The picture associated (as specified in subclause 7.4.1.2.3) with this message shall appear after the target picture, in decoding order.

target_frame_num indicates the frame_num of the target picture.

spare_field_flag equal to 0 indicates that the target picture and the spare pictures are decoded frames. spare_field_flag equal to 1 indicates that the target picture and the spare pictures are decoded fields.

target_bottom_field_flag equal to 0 indicates that the target picture is a top field. target_bottom_field_flag equal to 1 indicates that the target picture is a bottom field.

A target picture is a decoded reference picture whose corresponding primary coded picture precedes the current picture, in decoding order, and in which the values of frame_num, field_pic_flag (when present) and bottom_field_flag (when present) are equal to target_frame_num, spare_field_flag and target_bottom_field_flag, respectively.

num_spare_pics_minus1 indicates the number of spare pictures for the specified target picture. The number of spare pictures is equal to num_spare_pics_minus1 + 1. The value of num_spare_pics_minus1 shall be in the range of 0 to 15, inclusive.

delta_spare_frame_num[ i ] is used to identify the spare picture that contains the i-th set of spare slice group map units, hereafter called the i-th spare picture, as specified below. The value of delta_spare_frame_num[ i ] shall be in the range of 0 to MaxFrameNum ‑ 1 ‑ !spare_field_flag, inclusive.

The frame_num of the i-th spare picture, spareFrameNum[ i ], is derived as follows for all values of i from 0 to num_spare_pics_minus1, inclusive:

candidateSpareFrameNum = target_frame_num - !spare_field_flag
for ( i = 0; i <= num_spare_pics_minus1; i++ )

spare_bottom_field_flag[ i ] equal to 0 indicates that the i-th spare picture is a top field. spare_bottom_field_flag[ i ] equal to 1 indicates that the i-th spare picture is a bottom field.

The 0-th spare picture is a decoded reference picture whose corresponding primary coded picture precedes the target picture, in decoding order, and in which the values of frame_num, field_pic_flag (when present) and bottom_field_flag (when present) are equal to spareFrameNum[ 0 ], spare_field_flag and spare_bottom_field_flag[ 0 ], respectively. The i-th spare picture is a decoded reference picture whose corresponding primary coded picture precedes the ( i ‑ 1 )‑th spare picture, in decoding order, and in which the values of frame_num, field_pic_flag (when present) and bottom_field_flag (when present) are equal to spareFrameNum[ i ], spare_field_flag and spare_bottom_field_flag[ i ], respectively.

spare_area_idc[ i ] indicates the method used to identify the spare slice group map units in the i-th spare picture. spare_area_idc[ i ] shall be in the range of 0 to 2, inclusive. spare_area_idc[ i ] equal to 0 indicates that all slice group map units in the i-th spare picture are spare units. spare_area_idc[ i ] equal to 1 indicates that the value of the syntax element spare_unit_flag[ i ][ j ] is used to identify the spare slice group map units. spare_area_idc[ i ] equal to 2 indicates that the zero_run_length[ i ][ j ] syntax element is used to derive the values of spareUnitFlagInBoxOutOrder[ i ][ j ], as described below.

spare_unit_flag[ i ][ j ] equal to 0 indicates that the j-th slice group map unit in raster scan order in the i-th spare picture is a spare unit. spare_unit_flag[ i ][ j ] equal to 1 indicates that the j-th slice group map unit in raster scan order in the i-th spare picture is not a spare unit.

zero_run_length i ][ j ] is used to derive the values of spareUnitFlagInBoxOutOrder[ i ][ j ] when spare_area_idc[ i ] is equal to 2. In this case, the spare slice group map units identified in spareUnitFlagInBoxOutOrder[ i ][ j ] appear in counter-clockwise box-out order, as specified in subclause 8.2.2.4, for each spare picture. spareUnitFlagInBoxOutOrder[ i ][ j ] equal to 0 indicates that the j-th slice group map unit in counter-clockwise box-out order in the i-th spare picture is a spare unit. spareUnitFlagInBoxOutOrder[ i ][ j ] equal to 1 indicates that the j-th slice group map unit in counter-clockwise box-out order in the i-th spare picture is not a spare unit.

When spare_area_idc[ 0 ] is equal to 2, spareUnitFlagInBoxOutOrder[ 0 ][ j ] is derived as follows:

for( j = 0, loop = 0; j < PicSizeInMapUnits; loop++ )

When spare_area_idc[ i ] is equal to 2 and the value of i is greater than 0, spareUnitFlagInBoxOutOrder[ i ][ j ] is derived as follows:

for( j = 0, loop = 0; j < PicSizeInMapUnits; loop++ )

B.6.10      Scene information SEI message semantics

A scene and a scene transition are herein defined as a set of consecutive pictures in output order.

NOTE - Decoded pictures within one scene generally have similar content. The scene information SEI message is used to label pictures with scene identifiers and to indicate scene changes. The message specifies how the source pictures for the labelled pictures were created. The decoder may use the information to select an appropriate algorithm to conceal transmission errors. For example, a specific algorithm may be used to conceal transmission errors that occurred in pictures belonging to a gradual scene transition. Furthermore, the scene information SEI message may be used in a manner determined by the application, such as for indexing the scenes of a coded sequence.

A scene information SEI message labels all pictures, in decoding order, from the primary coded picture to which the SEI message is associated (inclusive), as specified in subclause , to the primary coded picture to which the next scene information SEI message (if present) in decoding order is associated (exclusive) or (otherwise) to the last access unit in the bitstream (inclusive). These pictures are herein referred to as the target pictures.

scene_info_present_flag equal to 0 indicates that the scene or scene transition to which the target pictures belong is unspecified. scene_info_present_flag equal to 1 indicates that the target pictures belong to the same scene or scene transition.

scene_id identifies the scene to which the target pictures belong. When the value of scene_transition_type of the target pictures is less than 4, and the previous picture in output order is marked with a value of scene_transition_type less than 4, and the value of scene_id is the same as the value of scene_id of the previous picture in output order, this indicates that the source scene for the target pictures and the source scene for the previous picture (in output order) are considered by the encoder to have been the same scene. When the value of scene_transition_type of the target pictures is greater than 3, and the previous picture in output order is marked with a value of scene_transition_type less than 4, and the value of scene_id is the same as the value of scene_id of the previous picture in output order, this indicates that one of the source scenes for the target pictures and the source scene for the previous picture (in output order) are considered by the encoder to have been the same scene. When the value of scene_id is not equal to the value of scene_id of the previous picture in output order, this indicates that the target pictures and the previous picture (in output order) are considered by the encoder to have been from different source scenes.

The value of scene_id shall be in the range of 0 to 2³²-1, inclusive. Values of scene_id in the range of 0 to 255, inclusive, and in the range of 512 to 2³¹  1, inclusive, may be used as determined by the application. Values of scene_id in the range of 256 to 511, inclusive, and in the range of 2³¹ to 2³²  1, inclusive, are reserved for future use by ITU-T | ISO/IEC. Decoders encountering a value of scene_id in the range of 256 to 511, inclusive, or in the range of 2³¹ to 2³² ‑ 1, inclusive, shall ignore (remove from the bitstream and discard) it.

scene_transition_type specifies in which type of a scene transition (if any) the target pictures are involved. The valid values of scene_transition_type are specified in Table D‑4.

Table D‑ scene_transition_type values.

When scene_transition_type is greater than 3, the target pictures include contents both from the scene labelled by its scene_id and the next scene, in output order, which is labelled by second_scene_id (see below). The term the current scene is used to indicate the scene labelled by scene_id. The term the next scene is used to indicate the scene labelled by second_scene_id. It is not required for any following picture, in output order, to be labelled with scene_id equal to second_scene_id of the current SEI message.

Scene transition types are specified as follows.

No transition specifies that the target pictures are not involved in a gradual scene transition.

NOTE - When two consecutive pictures in output order have scene_transition_type equal to 0 and different values of scene_id, a scene cut occurred between the two pictures.

Fade to black indicates that the target pictures are part of a sequence of pictures, in output order, involved in a fade to black scene transition, i.e., the luma samples of the scene gradually approach zero and the chroma samples of the scene gradually approach 128.

NOTE When two pictures are labelled to belong to the same scene transition and their scene_transition_type is 'Fade to black', the later one, in output order, is darker than the previous one.

Fade from black indicates that the target pictures are part of a sequence of pictures, in output order, involved in a fade from black scene transition, i.e., the luma samples of the scene gradually diverge from zero and the chroma samples of the scene may gradually diverge from 128.

NOTE When two pictures are labelled to belong to the same scene transition and their scene_transition_type is 'Fade from black', the later one in output order is lighter than the previous one.

Dissolve indicates that the sample values of each target picture (before encoding) were generated by calculating a sum of co-located weighted sample values of a picture from the current scene and a picture from the next scene. The weight of the current scene gradually decreases from full level to zero level, whereas the weight of the next scene gradually increases from zero level to full level. When two pictures are labelled to belong to the same scene transition and their scene_transition_type is 'Dissolve', the weight of the current scene for the later one, in output order, is less than the weight of the current scene for the previous one, and the weight of the next scene for the later one, in output order, is greater than the weight of the next scene for the previous one.

'Wipe' indicates that some of the sample values of each target picture (before encoding) were generated by copying co-located sample values of a picture in the current scene and the remaining sample values of each target picture (before encoding) were generated by copying co-located sample values of a picture in the next scene. When two pictures are labelled to belong to the same scene transition and their scene_transition_type is 'Wipe', the number of samples copied from the next scene to the later picture in output order is greater than the number of samples copied from the next scene to the previous picture.

second_scene_id identifies the next scene in the gradual scene transition in which the target pictures are involved. The value of second_scene_id shall not be equal to the value of scene_id. The value of second_scene_id shall not be equal to the value of scene_id in the previous picture in output order. When the next picture in output order is marked with a value of scene_transition_type less than 4, and the value of second_scene_id is the same as the value of scene_id of the next picture in output order, this indicates that the encoder considers one of the source scenes for the target pictures and the source scene for the next picture (in output order) to have been the same scene. When the value of second_scene_id is not equal to the value of scene_id or second_scene_id (if present) of the next picture in output order, this indicates that the encoder considers the target pictures and the next picture (in output order) to have been from different source scenes.

When the value of scene_id of a picture is equal to the value of scene_id of the following picture in output order and the value of scene_transition_type in both of these pictures is less than 4, this indicates that the encoder considers the two pictures to have been from the same source scene. When the values of scene_id, scene_transition_type and second_scene_id (if present) of a picture are equal to the values of scene_id, scene_transition_type and second_scene_id (respectively) of the following picture in output order and the value of scene_transition_type is greater than 0, this indicates that the encoder considers the two pictures to have been from the same source gradual scene transition.

The value of second_scene_id shall be in the range of 0 to 2³²-1, inclusive. Values of second_scene_id in the range of 0 to 255, inclusive, and in the range of 512 to 2³¹-1, inclusive, may be used as determined by the application. Values of second_scene_id in the range of 256 to 511, inclusive, and in the range of 2³¹ to 2³²-1, inclusive, are reserved for future use by ITU-T | ISO/IEC. Decoders encountering a value of second_scene_id in the range of 256 to 511, inclusive, or in the range of 2³¹ to 2³²-1, inclusive, shall ignore (remove from the bitstream and discard) it.

B.6.11      Sub-sequence information SEI message semantics

The sub-sequence information SEI message is used to indicate the position of a picture in data dependency hierarchy that consists of sub-sequence layers and sub-sequences.

A sub-sequence layer contains a subset of the coded pictures in a sequence. Sub-sequence layers are numbered with non-negative integers. A layer having a larger layer number is a higher layer than a layer having a smaller layer number. The layers are ordered hierarchically based on their dependency on each other so that any picture in a layer shall not be predicted from any picture on any higher layer.

NOTE In other words, any picture in layer 0 must not be predicted from any picture in layer 1 or above, pictures in layer 1 may be predicted from layer 0, pictures in layer 2 may be predicted from layers 0 and 1, etc.

NOTE: The subjective quality is expected to increase along with the number of decoded layers.

A sub-sequence is a set of coded pictures within a sub-sequence layer. A picture shall reside in one sub-sequence layer and in one sub-sequence only. Any picture in a sub-sequence shall not be predicted from any picture in another sub-sequence in the same or in a higher sub-sequence layer. A sub-sequence in layer 0 can be decoded independently of any picture that does not belong to the sub-sequence.

The sub-sequence information SEI message concerns the current access unit. The primary coded picture in the access unit is herein referred to as the current picture.

The sub-sequence information SEI message shall not be present unless gaps_in_frame_num_value_allowed_flag in the sequence parameter set referenced by the picture associated with the sub-sequence SEI message is equal to 1.

sub_seq_layer_num specifies the sub-sequence layer number of the current picture. When sub_seq_layer_num is greater than 0, memory management control operations shall not be used in any slice header of the current picture. When the current picture resides in a sub-sequence whose first picture in decoding order is an IDR picture, the value of sub_seq_layer_num shall be equal to 0. For a non-paired reference field, the value of sub_seq_layer_num shall be equal to 0. sub_seq_layer_num shall be in the range of 0 to 255, inclusive.

sub_seq_id identifies the sub-sequence within a layer. When the current picture resides in a sub-sequence whose first picture in decoding order is an IDR picture, the value of sub_seq_id shall be the same as the value of idr_pic_id of the IDR picture. sub_seq_id shall be in the range of 0 to 65535, inclusive.

first_ref_pic_flag equal to 1 specifies that the current picture is the first reference picture of the sub-sequence in decoding order. When the current picture is not the first picture of the sub-sequence in decoding order, the first_ref_pic_flag shall be equal to 0.

leading_non_ref_pic_flag equal to 1 specifies that the current picture is a non-reference picture preceding any reference picture in decoding order within the sub-sequence or that the sub-sequence contains no reference pictures. When the current picture is a reference picture or the current picture is a non-reference picture succeeding at least one reference picture in decoding order within the sub-sequence, the leading_non_ref_pic_flag shall be equal to 0.

last_pic_flag equal to 1 indicates that the current picture is the last picture of the sub-sequence (in decoding order), including all reference and non-reference pictures of the sub-sequence. When the current picture is not the last picture of the sub-sequence (in decoding order), last_pic_flag shall be equal to 0.

The current picture is assigned to a sub-sequence as follows.

If one or more of the following conditions is true, the current picture is the first picture of a sub-sequence in decoding order.

no earlier picture in decoding order is labelled with the same values of sub_seq_id and sub_seq_layer_num as the current picture

the value of leading_non_ref_pic_flag is equal to 1 and the value of leading_non_ref_pic_flag is equal to 0 in the previous picture in decoding order having the same values of sub_seq_id and sub_seq_layer_num as the current picture

the value of first_ref_pic_flag is equal to 1 and the value of leading_non_ref_pic_flag is equal to 0 in the previous picture in decoding order having the same values of sub_seq_id and sub_seq_layer_num as the current picture

the value of last_pic_flag is equal to 1 in the previous picture in decoding order having the same values of sub_seq_id and sub_seq_layer_num as the current picture

Otherwise, the current picture belongs to the same sub-sequence as the previous picture in decoding order having the same values of sub_seq_id and sub_seq_layer_num as the current picture.

sub_seq_frame_num_flag equal to 0 specifies that sub_seq_frame_num is not present. sub_seq_frame_num_flag equal to 1 specifies that sub_seq_frame_num is present.

sub_seq_frame_num shall be equal to 0 for the first reference picture of the sub-sequence and for any non-reference picture preceding the first reference picture of the sub-sequence in decoding order. sub_seq_frame_num is further constrained as follows.

If the current picture is not the second field of a complementary field pair, sub_seq_frame_num shall be incremented by 1, in modulo MaxFrameNum operation, relative to the previous reference picture, in decoding order, that belongs to the sub-sequence.

Otherwise (the current picture is the second field of a complementary field pair), the value of sub_seq_frame_num shall be the same as the value of sub_seq_frame_num for the first field of the complementary field pair.

sub_seq_frame_num shall be in the range of 0 to MaxFrameNum 1, inclusive.

When the current picture is an IDR picture, it shall start a new sub-sequence in sub-sequence layer 0. Thus, the sub_seq_layer_num shall be 0, the sub_seq_id shall be different from the previous sub-sequence in sub-sequence layer 0, first_ref_pic_flag shall be 1, and leading_non_ref_pic_flag shall be equal to 0.

When the sub-sequence information SEI message is present for both coded fields of a complementary field pair, the values of sub_seq_layer_num, sub_seq_id, leading_non_ref_pic_flag and sub_seq_frame_num, when present, shall be identical for both of these pictures.

When the sub-sequence information SEI message is present only for one coded field of a complementary field pair, the values of sub_seq_layer_num, sub_seq_id, leading_non_ref_pic_flag and sub_seq_frame_num, when present, are also applicable to the other coded field of the complementary field pair.

B.6.12      Sub-sequence layer characteristics SEI message semantics

The sub-sequence layer characteristics SEI message specifies the characteristics of sub-sequence layers.

num_sub_seq_layers_minus1 plus 1 specifies the number of sub-sequence layers in the sequence. num_sub_seq_layers_minus1 shall be in the range of 0 to 255, inclusive.

A pair of average_bit_rate and average_frame_rate characterizes each sub-sequence layer. The first pair of average_bit_rate and average_frame_rate specifies the characteristics of sub-sequence layer 0. When present, the second pair specifies the characteristics of sub-sequence layers 0 and 1 jointly. Each pair in decoding order specifies the characteristics for a range of sub-sequence layers from layer number 0 to the layer number specified by the layer loop counter. The values are in effect from the point they are decoded until an update of the values is decoded.

accurate_statistics_flag equal to 1 indicates that the values of average_bit_rate and average_frame_rate are rounded from statistically correct values. accurate_statistics_flag equal to 0 indicates that the average_bit_rate and the average_frame_rate are estimates and may deviate somewhat from the correct values.

When accurate_statistics_flag is equal to 0, the quality of the approximation used in the computation of the values of average_bit_rate and the average_frame_rate is chosen by the encoding process and is not specified by this Recommendation | International Standard.

average_bit_rate indicates the average bit rate in units of 1000 bits per second. All NAL units in the range of sub-sequence layers specified above are taken into account in the calculation. The average bit rate is derived according to the access unit removal time specified in Annex C of the Recommendation | International Standard. In the following, bTotal is the number of bits in all NAL units succeeding a sub-sequence layer characteristics SEI message (including the bits of the NAL units of the current access unit) and preceding the next access unit (in decoding order) including a sub-sequence layer characteristics SEI message (if present) or the end of the stream (otherwise). t₁ is the removal time (in seconds) of the current access unit, and t₂ is the removal time (in seconds) of the latest access unit (in decoding order) before the next sub-sequence layer characteristics SEI message (if present) or the end of the stream (otherwise).

When accurate_statistics_flag is equal to 1, the following conditions shall be fulfilled as follows.

If t₁ is not equal to t₂, the following condition shall be true

average_bit_rate = = Round( bTotal ( ( t₂ t₁ ) * 1000 ) ) ) (D- )

Otherwise (t₁ is equal to t₂), the following condition shall be true

average_bit_rate = = 0 (D- )

average_frame_rate indicates the average frame rate in units of frames/(256 seconds). All NAL units in the range of sub-sequence layers specified above are taken into account in the calculation. In the following, fTotal is the number of frames, complementary field pairs and non-paired fields between the current picture (inclusive) and the next sub-sequence layer characteristics SEI message (if present) or the end of the stream (otherwise). t₁ is the removal time (in seconds) of the current access unit, and t₂ is the removal time (in seconds) of the latest access unit (in decoding order) before the next sub-sequence layer characteristics SEI message (if present) or the end of the stream (otherwise).

When accurate_statistics_flag is equal to 1, the following conditions shall be fulfilled as follows.

If t₁ is not equal to t₂, the following condition shall be true

average_frame_rate = = Round( fTotal * 256 ( t₂ t₁ ) )  (D- )

Otherwise (t₁ is equal to t₂), the following condition shall be true

average_frame_rate = = 0 (D- )

B.6.13      Sub-sequence characteristics SEI message semantics

The sub-sequence characteristics SEI message indicates the characteristics of a sub-sequence. It also indicates inter prediction dependencies between sub-sequences. This message shall be contained in the first access unit in decoding order of the sub-sequence to which the sub-sequence characteristics SEI message applies. This sub-sequence is herein called the target sub-sequence.

sub_seq_layer_num identifies the sub-sequence layer number of the target sub-sequence. sub_seq_layer_num shall be in the range of 0 to 255, inclusive.

sub_seq_id identifies the target sub-sequence. sub_seq_id shall be in the range of 0 to 65535, inclusive.

duration_flag equal to 0 indicates that the duration of the target sub-sequence is not specified.

sub_seq_duration specifies the duration of the target sub-sequence in clock ticks of a 90-kHz clock.

average_rate_flag equal to 0 indicates that the average bit rate and the average frame rate of the target sub-sequence are unspecified.

accurate_statistics_flag indicates how reliable the values of average_bit_rate and average_frame_rate are. accurate_statistics_flag equal to 1, indicates that the average_bit_rate and the average_frame_rate are rounded from statistically correct values. accurate_statistics_flag equal to 0 indicates that the average_bit_rate and the average_frame_rate are estimates and may deviate from the statistically correct values.

average_bit_rate indicates the average bit rate in (1000 bits)/second of the target sub-sequence. All NAL units of the target sub-sequence are taken into account in the calculation. The average bit rate is derived according to the access unit removal time specified in subclause C.4.2. In the following, nB is the number of bits in all NAL units in the sub-sequence. t₁ is the removal time (in seconds) of the first access unit of the sub-sequence (in decoding order), and t₂ is the removal time (in seconds) of the last access unit of the sub-sequence (in decoding order).

When accurate_statistics_flag is equal to 1, the following conditions shall be fulfilled as follows.

If t₁ is not equal to t₂, the following condition shall be true

average_bit_rate = = Round( nB ( ( t₂ t₁ ) * 1000 ) ) (D- )

Otherwise (t₁ is equal to t₂), the following condition shall be true

average_bit_rate = = 0 (D- )

average_frame_rate indicates the average frame rate in units of frames/(256 seconds) of the target sub-sequence. All NAL units of the target sub-sequence are taken into account in the calculation. The average frame rate is derived according to the access unit removal time specified in subclause C.4.2. In the following, fC is the number of frames, complementary field pairs and non-paired fields in the sub-sequence. t₁ is the removal time (in seconds) of the first access unit of the sub-sequence (in decoding order), and t₂ is the removal time (in seconds) of the last access unit of the sub-sequence (in decoding order).

When accurate_statistics_flag is equal to 1, the following conditions shall be fulfilled as follows.

If t₁ is not equal to t₂, the following condition shall be true

average_frame_rate = = Round( fC * 256 ( t₂ t₁ ) ) (D- )

Otherwise (t₁ is equal to t₂), the following condition shall be true

average_frame_rate = = 0 (D- )

num_referenced_subseqs specifies the number of sub-sequences that contain pictures that are used as reference pictures for inter prediction in the pictures of the target sub-sequence. num_referenced_subseqs shall be in the range of 0 to 255, inclusive.

ref_sub_seq_layer_num, ref_sub_seq_id, and ref_sub_seq_direction identify the sub-sequence that contains pictures that are used as reference pictures for inter prediction in the pictures of the target sub-sequence. Depending on ref_sub_seq_direction, the following applies.

If ref_sub_seq_direction is equal to 0, a set of candidate sub-sequences consists of the sub-sequences whose sub_seq_id is equal to ref_sub_seq_id, which reside in the sub-sequence layer having sub_seq_layer_num equal to ref_sub_seq_layer_num, and whose first picture in decoding order precedes the first picture of the target sub-sequence in decoding order

Otherwise (ref_sub_seq_direction is equal to 1), a set of candidate sub-sequences consists of the sub-sequences whose sub_seq_id is equal to ref_sub_seq_id, which reside in the sub-sequence layer having sub_seq_layer_num equal to ref_sub_seq_layer_num, and whose first picture in decoding order succeeds the first picture of the target sub-sequence in decoding order

The sub-sequence used as a reference for the target sub-sequence is the sub-sequence among the set of candidate sub-sequences whose first picture is the closest to the first picture of the target sub-sequence in decoding order.

B.6.14      Full-frame freeze SEI message semantics

The full-frame freeze SEI message indicates that the contents of the entire prior displayed video frame in output order should be kept unchanged, without updating the display using the contents of the current decoded picture.

full_frame_freeze_repetition_period indicates whether another full-frame freeze SEI message shall be present in the bitstream and specifies the picture order count interval within which another full-frame freeze SEI message or a full-frame freeze release SEI message will be present. The value of full_frame_freeze_repetition_period shall be in the range of 0 to 16 384, inclusive.

full_frame_freeze_repetition_period equal to 0 specifies that the full-frame freeze SEI message applies to the current decoded picture only.

full_frame_freeze_repetition_period equal to 1 specifies that the full-frame freeze SEI message persists in output order until any of the following conditions are true.

A new coded video sequence begins

A picture in an access unit containing full-frame freeze release SEI message is output having PicOrderCnt( ) greater than PicOrderCnt( CurrPic ).

full_frame_freeze_repetition_period greater than 1 specifies that the full-frame freeze SEI message persists until any one of the following conditions are true.

A new coded video sequence begins

A picture in an access unit containing a full-frame freeze release SEI message or a full-frame freeze release SEI message is output having PicOrderCnt( ) greater than PicOrderCnt( CurrPic ) + full_frame_freeze_repetition_period.

full_frame_freeze_repetition_period greater than 1 indicates that another full-frame freeze SEI message or a full-frame freeze release SEI message shall be present for a picture in an access unit that is output having PicOrderCnt( ) less than or equal to PicOrderCnt( CurrPic ) + full_frame_freeze_repetition_period; unless a new coded video sequence begins without output of such a picture.

B.6.15      Full-frame freeze release SEI message semantics

The full-frame freeze release SEI message indicates that the update of the displayed video frame should resume, starting with the contents of the current decoded picture and continuing for subsequent pictures in output order. The full-frame freeze release SEI message cancels the effect of any full-frame freeze SEI message sent with pictures that precede the current picture in output order.

B.6.16      Full-frame snapshot SEI message semantics

The full-frame snapshot SEI message indicates that the current frame is labelled for use as determined by the application as a still-image snapshot of the video content.

snapshot_id specifies a snapshot identification number. snapshot_id shall be in the range of 0 to 2³² ‑ 1, inclusive.

Values of snapshot_id in the range of 0 to 255, inclusive, and in the range of 512 to 2³¹-1, inclusive, may be used as determined by the application. Values of snapshot_id in the range of 256 to 511, inclusive, and in the range of 2³¹ to 2³²-1, inclusive, are reserved for future use by ITU-T | ISO/IEC. Decoders encountering a value of snapshot_id in the range of 256 to 511, inclusive, or in the range of 2³¹ to 2³²-1, inclusive, shall ignore (remove from the bitstream and discard) it.

B.6.17      Progressive refinement segment start SEI message semantics

The progressive refinement segment start SEI message specifies the beginning of a set of consecutive coded pictures that is labelled as the current picture followed by a sequence of one or more pictures of refinement of the quality of the current picture, rather than as a representation of a continually moving scene.

The tagged set of consecutive coded pictures shall continue until one of following conditions is true. When a condition below becomes true, the next slice to be decoded does not belong to the tagged set of consecutive coded pictures.

The next slice to be decoded belongs to an IDR picture.

num_refinement_steps_minus1 is greater than 0 and the frame_num of the next slice to be decoded is (currFrameNum + num_refinement_steps_minus1 + 1) % MaxFrameNum, where currFrameNum is the value of frame_num of the picture in the access unit containing the SEI message.

num_refinement_steps_minus1 is 0 and a progressive refinement segment end SEI message with the same progressive_refinement_id as the one in this SEI message is decoded.

The decoding order of picture within the tagged set of consecutive pictures should be the same as their output order. progressive_refinement_id specifies an identification number for the progressive refinement operation. progressive_refinement_id shall be in the range of 0 to 2³² ‑ 1, inclusive.

Values of progressive_refinement_id in the range of 0 to 255, inclusive, and in the range of 512 to 2³¹ ‑ 1, inclusive, may be used as determined by the application. Values of progressive_refinement_id in the range of 256 to 511, inclusive, and in the range of 2³¹ to 2³²-1, inclusive, are reserved for future use by ITU-T | ISO/IEC. Decoders encountering a value of progressive_refinement_id in the range of 256 to 511, inclusive, or in the range of 2³¹ to 2³² ‑ 1, inclusive, shall ignore (remove from the bitstream and discard) it.

num_refinement_steps_minus1 specifies the number of reference frames in the tagged set of consecutive coded pictures as follows.

If num_refinement_steps_minus1 is equal to 0, the number of reference frames in the tagged set of consecutive coded pictures is unknown.

Otherwise, the number of reference frames in the tagged set of consecutive coded pictures is equal to num_refinement_steps_minus1 + 1.

num_refinement_steps_minus1 shall be in the range of 0 to MaxFrameNum ‑ 1, inclusive.

B.6.18      Progressive refinement segment end SEI message semantics

The progressive refinement segment end SEI message specifies the end of a set of consecutive coded pictures that has been labelled by use of a progressive refinement segment start SEI message as an initial picture followed by a sequence of one or more pictures of the refinement of the quality of the initial picture, and ending with the current picture.

progressive_refinement_id specifies an identification number for the progressive refinement operation. progressive_refinement_id shall be in the range of 0 to 2³² ‑ 1, inclusive.

The progressive refinement segment end SEI message specifies the end of any progressive refinement segment previously started using a progressive refinement segment start SEI message with the same value of progressive_refinement_id.

Values of progressive_refinement_id in the range of 0 to 255, inclusive, and in the range of 512 to 2³¹ ‑ 1, inclusive, may be used as determined by the application. Values of progressive_refinement_id in the range of 256 to 511, inclusive, and in the range of 2³¹ to 2³² ‑ 1, inclusive, are reserved for future use by ITU-T | ISO/IEC. Decoders encountering a value of progressive_refinement_id in the range of 256 to 511, inclusive, or in the range of 2³¹ to 2³² ‑ 1, inclusive, shall ignore (remove from the bitstream and discard) it.

B.6.19      Motion-constrained slice group set SEI message semantics

This SEI message indicates that inter prediction over slice group boundaries is constrained as specified below. When present, the message shall only appear where it is associated, as specified in subclause , with an IDR access unit.

The target picture set for this SEI message contains all consecutive primary coded pictures in decoding order starting with the associated primary coded IDR picture (inclusive) and ending with the following primary coded IDR picture (exclusive) or with the very last primary coded picture in the bitstream (inclusive) in decoding order when there is no following primary coded IDR picture. The slice group set is a collection of one or more slice groups, identified by the slice_group_id[ i ] syntax element.

This SEI message indicates that, for each picture in the target picture set, the inter prediction process is constrained as follows: No sample value outside the slice group set, and no sample value at a fractional sample position that is derived using one or more sample values outside the slice group set is used to inter predict any sample within the slice group set.

num_slice_groups_in_set_minus1 + 1 specifies the number of slice groups in the slice group set. The allowed range of num_slice_groups_in_set_minus1 is 0 to num_slice_groups_minus1, inclusive. The allowed range of num_slice_groups_minus1 is specified in Annex A.

slice_group_id[ i ] identifies the slice group(s) contained within the slice group set. The allowed range is from 0 to num_slice_groups_in_set_minus1, inclusive. The size of the slice_group_id[ i ] syntax element is Ceil( Log2( num_slice_groups_minus1 + 1 ) ) bits.

exact_sample_value_match_flag equal to 0 indicates that, within the target picture set, when the macroblocks that do not belong to the slice group set are not decoded, the value of each sample in the slice group set need not be exactly the same as the value of the same sample when all the macroblocks are decoded. exact_sample_value_match_flag equal to 1 indicates that, within the target picture set, when the macroblocks that do not belong to the slice group set are not decoded, the value of each sample in the slice group set shall be exactly the same as the value of the same sample when all the macroblocks in the target picture set are decoded.

Note - When disable_deblocking_filter_idc is equal to 2 in all slices in the target picture set, exact_sample_value_match_flag should be 1.

pan_scan_rect_flag equal to 0 specifies that pan_scan_rect_id is not present. pan_scan_rect_flag equal to 1 specifies that pan_scan_rect_id is present.

pan_scan_rect_id indicates that the specified slice group set covers at least the pan-scan rectangle identified by pan_scan_rect_id within the target picture set.

Note - Multiple motion_constrained_slice_group_set SEI messages may be associated with the same IDR picture. Consequently, more than one slice group set may be active within a target picture set.

Note - The size, shape, and location of the slice groups in the slice group set may change within the target picture set.

B.6.20      Reserved SEI message semantics

This message consists of data reserved for future backward-compatible use by ITU-T | ISO/IEC. Encoders conforming to this Recommendation | International Standard shall not send reserved SEI messages until and unless the use of such messages has been specified by ITU-T | ISO/IEC. Decoders conforming to this Recommendation | International Standard that encounter reserved SEI messages shall discard their content without effect on the decoding process, except as specified in future Recommendations | International Standards specified by ITU-T | ISO/IEC. reserved_sei_message_payload_byte is a byte reserved for future use by ITU-T | ISO/IEC.

EAnnex E
Video usability information

(This annex forms an integral part of this Recommendation | International Standard)

This Annex specifies syntax and semantics of the VUI parameters of the sequence parameter sets.

VUI parameters are not required for constructing the luma or chroma samples by the decoding process. Conforming decoders are not required to process this information for output order conformance to this Recommendation | International Standard (see Annex C for the specification of conformance). Some VUI parameters are required to check bitstream conformance and for output timing decoder conformance.

In Annex E, specification for presence of VUI parameters is also satisfied when those parameters (or some subset of them) are conveyed to decoders (or to the HRD) by other means not specified by this Recommendation | International Standard. When present in the bitstream, VUI parameters shall follow the syntax and semantics specified in subclauses and and this annex. When the content of VUI parameters is conveyed for the application by some means other than presence within the bitstream, the representation of the content of the VUI parameters is not required to use the same syntax specified in this annex. For the purpose of counting bits, only the appropriate bits that are actually present in the bitstream are counted.

E.7     VUI syntax

E.7.1          VUI parameters syntax

E.7.2          HRD parameters syntax

E.8     VUI semantics

E.8.1          VUI parameters semantics

aspect_ratio_info_present_flag equal to 1 specifies that aspect_ratio_idc is present. aspect_ratio_info_present_flag equal to 0 specifies that aspect_ratio_idc is not present.

aspect_ratio_idc specifies the value of the sample aspect ratio of the luma samples. Table E‑1 shows the meaning of the code. When aspect_ratio_idc indicates Extended_SAR, the sample aspect ratio is represented by sar_width and sar_height. When the aspect_ratio_idc syntax element is not present, aspect_ratio_idc value shall be inferred to be equal to 0.

Table E‑ Meaning of sample aspect ratio indicator

sar_width indicates the horizontal size of the sample aspect ratio (in arbitrary units).

sar_height indicates the vertical size of the sample aspect ratio (in the same arbitrary units as sar_width).

sar_width and sar_height shall be relatively prime or equal to 0. When aspect_ratio_idc is equal to 0 or sar_width is equal to 0 or sar_height is equal to 0, the sample aspect ratio shall be considered unspecified by this Recommendation | International Standard.

overscan_info_present_flag equal to 1 specifies that the overscan_appropriate_flag is present. When overscan_info_present_flag is equal to 0 or is not present, the preferred display method for the video signal is unspecified.

overscan_appropriate_flag equal to 1 indicates that the cropped decoded pictures output are suitable for display using overscan. overscan_appropriate_flag equal to 0 indicates that the cropped decoded pictures output contain visually important information in the entire region out to the edges of the cropping rectangle of the picture, such that the cropped decoded pictures output should not be displayed using overscan. Instead, they should be displayed using either an exact match between the display area and the cropping rectangle, or using underscan.

NOTE For example, overscan_appropriate_flag equal to 1 might be used for entertainment television programming, or for a live view of people in a videoconference, and overscan_appropriate_flag equal to 0 might be used for computer screen capture or security camera content.

video_signal_type_present_flag equal to 1 specifies that video_format, video_full_range_flag and colour_description_present_flag are present. video_signal_type_present_flag equal to 0, specify that video_format, video_full_range_flag and colour_description_present_flag are not present.

video_format indicates the representation of the pictures as specified in Table E‑2, before being coded in accordance with this Recommendation | International Standard. When the video_format syntax element is not present, video_format value shall be inferred to be equal to 5.

Table E‑ Meaning of video_format

video_full_range_flag indicates the black level and range of the luma and chroma signals as derived from E_Y, E_PB, and E_PR analogue component signals, as follows.

If video_full_range_flag is equal to 0,

Y = Round( 219 * E_Y + 16 ) (E- )

Cb = Round( 224 * E_PB + 128 ) (E- )

Cr = Round( 224 * E_PR + 128 ) (E- )

Otherwise (video_full_range_flag is equal to 1),

Y = Round( 255 * E_Y ) (E- )

Cb = Round( 255 * E_PB + 128 ) (E- )

Cr = Round( 255 * E_PR + 128 ) (E- )

When the video_full_range_flag syntax element is not present, video_full_range_flag value shall be inferred to be equal to 0.

colour_description_present_flag equal to 1 specifies that colour_primaries, transfer_characteristics and matrix_coefficients are present. colour_description_present_flag equal to 0 specifies that colour_primaries, transfer_characteristics and matrix_coefficients are not present.

colour_primaries indicates the chromaticity coordinates of the source primaries as specified in Table E‑3 in terms of the CIE 1931 definition of x and y as specified by ISO/CIE 10527.

Table E‑ Colour primaries

When the colour_primaries syntax element is not present, the value of colour_primaries shall be inferred to be equal to 2 (the chromaticity is unspecified or is determined by the application).

transfer_characteristics indicates the opto-electronic transfer characteristic of the source picture as specified in Table E‑4 as a function of a linear optical intensity input L_c with an analogue range of 0 to 1.

Table E‑ Transfer characteristics

When the transfer_characteristics syntax element is not present, the value of transfer_characteristics shall be inferred to be equal to 2 (the transfer characteristics are unspecified or are determined by the application).

matrix_coefficients describes the matrix coefficients used in deriving luma and chroma signals from the green, blue, and red primaries, as specified in Table E‑5.

Using the following definitions:

E_R, E_G, and E_B are analogue with values in the range of 0 to 1.

White is specified as having E_R equal to 1, E_G equal to 1, and E_B equal to 1.

Then:

E_Y = K_R * E_R + ( 1 K_R K_B ) * E_G + K_B * E_B (E- )

E_PB = 0.5 * ( E_B E_Y ) ( 1 K_B )  (E-

E_PR = 0.5 * ( E_R E_Y ) ( 1 K_R )  (E-

NOTE Then E_Y is analogue with values in the range of 0 to 1, E_PB and E_PR are analogue with values in the range of -0.5 to 0.5, and white is equivalently given by E_Y = 1, E_PB = 0, E_PR = 0.

Table E‑ Matrix coefficients

When the matrix_coefficients syntax element is not present, the value of matrix_coefficients shall be inferred to be equal to 2.

chroma_loc_info_present_flag equal to 1 specifies that chroma_sample_loc_type_top_field and chroma_sample_loc_type_bottom_field are present. chroma_loc_info_present_flag equal to 0 specifies that chroma_sample_loc_type_top_field and chroma_sample_loc_type_bottom_field are not present.

chroma_sample_loc_type_top_field and chroma_sample_loc_type_bottom_field specify the location of chroma samples for the top field and the bottom field as shown in Figure E‑1. The value of chroma_sample_loc_type_top_field and chroma_sample_loc_type_bottom_field shall be in the range of 0 to 5, inclusive. When the chroma_sample_loc_type_top_field and chroma_sample_loc_type_bottom_field are not present, the values of chroma_sample_loc_type_top_field and chroma_sample_loc_type_bottom_field shall be inferred to be equal to 0.

NOTE When coding progressive source material, chroma_sample_loc_type_top_field and chroma_sample_loc_type_bottom_field should have the same value.

Figure E‑ Location of chroma samples for top and bottom fields as a function of chroma_sample_loc_type_top_field and chroma_sample_loc_type_bottom_field

timing_info_present_flag equal to 1 specifies that num_units_in_tick, time_scale and fixed_frame_rate_flag are present in the bitstream. timing_info_present_flag equal to 0 specifies that num_units_in_tick, time_scale and fixed_frame_rate_flag are not present in the bitstream.

num_units_in_tick is the number of time units of a clock operating at the frequency time_scale Hz that corresponds to one increment (called a clock tick) of a clock tick counter. num_units_in_tick shall be greater than 0. A clock tick is the minimum interval of time that can be represented in the coded data. For example, when the clock frequency of a video signal is 30000 1001 Hz, time_scale may be 30 000 and num_units_in_tick may be 1001. See Equation C-1.

time_scale is the number of time units that pass in one second. For example, a time coordinate system that measures time using a 27 MHz clock has a time_scale of 27 000 000. time_scale shall be greater than 0.

fixed_frame_rate_flag equal to 1 indicates that the temporal distance between the HRD output times of any two consecutive pictures in output order is constrained as follows. fixed_frame_rate_flag equal to 0 indicates that no such constraints apply to the temporal distance between the HRD output times of any two consecutive pictures in output order.

When fixed_frame_rate_flag is equal to 1, for all n where n indicates the n-th picture in output order and picture n is not the last picture in the bitstream in output order, the value of Dt_fi,dpb( n ) as specified in Equation C-13, shall obey the following constraint,

Dt_fi,dpb( n ) = = Dt_o,dpb( n ) DeltaTfiDivisor (E- )

where DeltaTfiDivisor is specified by Table E‑6 based on the value of pic_struct_present_flag, field_pic_flag, and pic_struct for picture n. Entries marked '-' in Table E‑6 indicate a lack of dependence of DeltaTfiDivisor on the corresponding syntax element.

The value computed for Dt_fi,dpb( n ) shall be the same for all n > 0. and equal to num_units_in_tick time_scale.

Table E‑ Divisor for computation of Dt_fi,dpb( n )

nal_hrd_parameters_present_flag equal to 1 specifies that NAL HRD parameters (pertaining to Type II bitstream conformance) are present. nal_hrd_parameters_present_flag equal to 0 specifies that NAL HRD parameters are not present.

NOTE When nal_hrd_parameters_present_flag is equal to 0, the conformance of the bitstream cannot be verified without provision of the NAL HRD parameters, including the NAL sequence HRD parameter information and all buffering period and picture timing SEI messages, by some means not specified in this Recommendation | International Standard

When nal_hrd_parameters_present_flag is equal to 1, NAL HRD parameters (subclauses E.10.2 and E.11.2) immediately follow the flag.

The variable NalHrdBpPresentFlag is derived as follows.

If any of the following is true, the value of NalHrdBpPresentFlag shall be set equal to 1.

nal_hrd_parameters_present_flag is present in the bitstream and is equal to 1

the need for presence of buffering periods for NAL HRD operation to be present in the bitstream in buffering period SEI messages is determined by the application, by some means not specified in this Recommendation | International Standard.

Otherwise, the value of NalHrdBpPresentFlag shall be set equal to 0.

vcl_hrd_parameters_present_flag equal to 1 specifies that VCL HRD parameters (pertaining to all bitstream conformance) are present. vcl_hrd_parameters_present_flag equal to 0 specifies that VCL HRD parameters are not present.

NOTE When vcl_hrd_parameters_present_flag is equal to 0, the conformance of the bitstream cannot be verified without provision of the VCL HRD parameters and all buffering period and picture timing SEI messages, by some means not specified in this Recommendation | International Standard

When vcl_hrd_parameters_present_flag is equal to 1, VCL HRD parameters (subclauses E.10.2 and E.11.2) immediately follow the flag.

The variable VclHrdBpPresentFlag is derived as follows.

If any of the following is true, the value of VclHrdBpPresentFlag shall be set equal to 1.

nal_hrd_parameters_present_flag is present in the bitstream and is equal to 1

the need for presence of buffering periods for VCL HRD operation to be present in the bitstream in buffering period SEI messages is determined by the application, by some means not specified in this Recommendation | International Standard.

Otherwise, the value of VclHrdBpPresentFlag shall be set equal to 0.

The variable CpbDpbDelaysPresentFlag is derived as follows.

If any of the following is true, the value of CpbDpbDelaysPresentFlag shall be set equal to 1.

nal_hrd_parameters_present_flag is present in the bitstream and is equal to 1

vcl_hrd_parameters_present_flag is present in the bitstream and is equal to 1

the need for presence of CPB and DPB output delays to be present in the bitstream in picture timing SEI messages is determined by the application, by some means not specified in this Recommendation | International Standard.

Otherwise, the value of CpbDpbDelaysPresentFlag shall be set equal to 0.

low_delay_hrd_flag specifies the HRD operational mode as specified in Annex C. When fixed_frame_rate_flag is equal to 1, low_delay_hrd_flag shall be equal to 0.

NOTE - When low_delay_hrd_flag is equal to 1, 'big pictures' that violate the nominal CPB removal times due to the number of bits used by an access unit are permitted. It is expected, but not required, that such 'big pictures' occur only occasionally.

pic_struct_present_flag equal to 1 specifies that picture timing SEI messages (subclause D.9.2) are present that include the pic_struct syntax element. pic_struct_present_flag equal to 0 specifies that the pic_struct syntax element is not present in picture timing SEI messages.

bitstream_restriction_flag equal to 1, specifies that the following sequence bitstream restriction parameters are present. bitstream_restriction_flag equal to 0, specifies that the following sequence bitstream restriction parameters are not present.

motion_vectors_over_pic_boundaries_flag equal to 0 indicates that no sample outside the picture boundaries and no sample at a fractional sample position whose value is derived using one or more samples outside the picture boundaries is used to inter predict any sample. motion_vectors_over_pic_boundaries_flag equal to 1 indicates that one or more samples outside picture boundaries may be used in inter prediction. When the motion_vectors_over_pic_boundaries_flag syntax element is not present, motion_vectors_over_pic_boundaries_flag value shall be inferred to be equal to 1.

max_bytes_per_pic_denom indicates a number of bytes not exceeded by the sum of the sizes of the VCL NAL units associated with any coded picture in the sequence.

The number of bytes that represent a picture in the NAL unit stream is specified for this purpose as the total number of bytes of VCL NAL unit data (i.e., the total of the NumBytesInNALunit variables for the VCL NAL units) for the picture. The value of max_bytes_per_pic_denom shall be in the range of 0 to 16, inclusive.

Depending on max_bytes_per_pic_denom the following applies.

If max_bytes_per_pic_denom is equal to 0, no limits are indicated.

Otherwise (max_bytes_per_pic_denom is not equal to 0), no coded picture shall be represented in the sequence by more than the following number of bytes.

( PicSizeInMbs * 256 * ChromaFormatFactor ) max_bytes_per_pic_denom (E- )

When the max_bytes_per_pic_denom syntax element is not present, the value of max_bytes_per_pic_denom shall be inferred to be equal to 2.

max_bits_per_mb_denom indicates the maximum number of coded bits of macroblock_layer( ) data for any macroblock in any picture of the sequence. The value of max_bits_per_mb_denom shall be in the range of 0 to 16, inclusive.

Depending on max_bits_per_mb_denom the following applies.

If max_bits_per_mb_denom is equal to 0, no limit is specified.

Otherwise (max_bits_per_mb_denom is not equal to 0), no coded macroblock_layer( ) shall be represented in the bitstream by more than the following number of bits.

( 2048 * ChromaFormatFactor + 128 ) max_bits_per_mb_denom  (E- )

Depending on entropy_coding_mode_flag, the bits of macroblock_layer( ) data are counted as follows.

If entropy_coding_mode_flag is equal to 0, the number of bits of macroblock_layer( ) data is given by the number of bits in the macroblock_layer( ) syntax structure for a macroblock.

Otherwise (entropy_coding_mode_flag is equal to 1), the number of bits of macroblock_layer( ) data for a macroblock is given by the number of times read_bits( 1 ) is called in subclauses 9.3.3.2.2 and when parsing the macroblock_layer( ) associated with the macroblock.

When the max_bits_per_mb_denom is not present, the value of max_bits_per_mb_denom shall be inferred to be equal to 1.

log2_max_mv_length_horizontal and log2_max_mv_length_vertical indicate the maximum absolute value of a decoded horizontal and vertical motion vector component, respectively, in luma sample units, for all pictures in the sequence. A value of n asserts that no absolute value of a motion vector component is greater than 2ⁿ units of luma sample displacement. The value of log2_max_mv_length_horizontal shall be in the range of 0 to 16, inclusive. The value of log2_max_mv_length_vertical shall be in the range of 0 to 16, inclusive. When log2_max_mv_length_horizontal is not present, the values of log2_max_mv_length_horizontal and log2_max_mv_length_vertical shall be inferred to be equal to 16.

NOTE - The maximum absolute value of a decoded vertical or horizontal motion vector component is also constrained by profile and level limits as specified in Annex A.

num_reorder_frames indicates the maximum number of frames, complementary field pairs, or non-paired fields that precede any frame, complementary field pair, or non-paired field in the sequence in decoding order and follow it in output order. The value of num_reorder_frames shall be in the range of 0 to max_dec_frame_buffering, inclusive. When the num_reorder_frames syntax element is not present, the value of num_reorder_frames value shall be inferred to be equal to max_dec_frame_buffering.

max_dec_frame_buffering specifies the required size of the HRD decoded picture buffer (DPB) in units of frame buffers. The sequence shall not require a decoded picture buffer with size of more than max_dec_frame_buffering frame buffers to enable the output of decoded pictures at the output times specified by dpb_output_delay of the picture timing SEI messages. The value of max_dec_frame_buffering shall be in the range of 0 to MaxDpbSize, inclusive (as specified in subclause A.3.1). When the max_dec_frame_buffering syntax element is not present, the value of max_dec_frame_buffering shall be inferred to be equal to MaxDpbSize.

E.8.2          HRD parameters semantics

cpb_cnt_minus1 plus 1 specifies the number of alternative CPB specifications in the bitstream. The value of cpb_cnt_minus1 shall be in the range of 0 to 31, inclusive. When low_delay_hrd_flag is equal to 1, cpb_cnt_minus1 shall be equal to 0. When cpb_cnt_minus1 is not present, it shall be inferred to be equal to 0.

bit_rate_scale together with bit_rate_value_minus1[ SchedSelIdx ]) specifies the maximum input bit rate of the SchedSelIdx-th CPB.

cpb_size_scale (together with cpb_size_value_minus1[ SchedSelIdx ]) specifies the CPB size of the SchedSelIdx-th CPB.

bit_rate_value_minus1[ SchedSelIdx ] (together with bit_rate_scale) specifies the maximum input bit rate for the SchedSelIdx‑th CPB.  bit_rate_value_minus1[ SchedSelIdx ] shall be in the range of 0 to 2³² ‑ 2, inclusive. For any SchedSelIdx > 0, bit_rate_value_minus1[ SchedSelIdx ] shall be greater than bit_rate_value_minus1[ SchedSelIdx ‑ 1 ]. The bit rate in bits per second is given by

BitRate[ SchedSelIdx ] = ( bit_rate_value_minus1[ SchedSelIdx ] + 1 ) * 2^{(6 + bit_rate_scale)}  (E- )

When the bit_rate_value_minus1[ SchedSelIdx ]  syntax element is not present, BitRate[ SchedSelIdx ] shall be inferred to be equal to 1000 * MaxBR for VCL HRD parameters.

When the bit_rate_value_minus1[ SchedSelIdx ]  syntax element is not present, BitRate[ SchedSelIdx ] shall be inferred to be equal to 1200 * MaxBR for NAL HRD parameters.

cpb_size_value_minus1[ SchedSelIdx ] is used together with cpb_size_scale to specify the SchedSelIdx-th CPB size. cpb_size_value_minus1[ SchedSelIdx ] shall be in the range of 0 to 2³² ‑ 2, inclusive. For any SchedSelIdx greater than 0, cpb_size_value_minus1[ SchedSelIdx ] shall be less than or equal to cpb_size_value_minus1[ SchedSelIdx ‑1 ].

The CPB size in bits is given by

CpbSize[ SchedSelIdx ] = ( cpb_size_value_minus1[ SchedSelIdx ] + 1 ) * 2^{(4 + cpb_size_scale)}  (E- )

When the cpb_size_value_minus1[ SchedSelIdx ] syntax element is not present, CpbSize[ SchedSelIdx ] shall be inferred to be equal to 1000 * MaxCPB for VCL HRD parameters.

When the cpb_size_value_minus1[ SchedSelIdx ] syntax element is not present, CpbSize[ SchedSelIdx ] shall be inferred to be equal to 1200 * MaxCPB for NAL HRD parameters.

For VCL HRD parameters, there shall be at least one value of SchedSelIdx for which BitRate[ SchedSelIdx ] <= 1000*MaxBR and CpbSize[ SchedSelIdx ] <= 1000 * MaxCPB (as specified in subclause A.3.1).

For NAL HRD parameters, there shall be at least one value of SchedSelIdx for which CpbSize[ SchedSelIdx ] <= 1200*MaxCPB and BitRate[ SchedSelIdx ] <= 1200*MaxBR.

cbr_flag[ SchedSelIdx ] equal to 0 specifies that to decode this bitstream by the HRD using the SchedSelIdx-th CPB specification, the hypothetical stream delivery scheduler (HSS) operates in an intermittent bit rate mode. cbr_flag[ SchedSelIdx ] equal to 1 specifies that the HSS operates in a constant bit rate (CBR) mode. When the cbr_flag[ SchedSelIdx ] syntax element is not present, the value of cbr_flag shall be inferred to be equal to 0.

initial_cpb_removal_delay_length_minus1 specifies the length in bits of the initial_cpb_removal_delay[ SchedSelIdx ] and initial_cpb_removal_delay_offset[ SchedSelIdx ] syntax elements of the buffering period SEI message. The length of initial_cpb_removal_delay[ SchedSelIdx ] and of initial_cpb_removal_delay_offset[ SchedSelIdx ] is initial_cpb_removal_delay_length_minus1 + 1. When the initial_cpb_removal_delay_length_minus1 syntax element is present in more than one hrd_parameters( ) syntax structure within the VUI parameters syntax structure, the value of the initial_cpb_removal_delay_length_minus1 parameters shall be equal in both hrd_parameters( ) syntax structures. When the initial_cpb_removal_delay_length_minus1 syntax element is not present, it shall be inferred to be equal to 23.

cpb_removal_delay_length_minus1 specifies the length in bits of the cpb_removal_delay syntax element. The length of the cpb_removal_delay syntax element of the picture timing SEI message is cpb_removal_delay_length_minus1 + 1. When the cpb_removal_delay_length_minus1 syntax element is present in more than one hrd_parameters( ) syntax structure within the VUI parameters syntax structure, the value of the cpb_removal_delay_length_minus1 parameters shall be equal in both hrd_parameters( ) syntax structures. When the cpb_removal_delay_length_minus1 syntax element is not present, it shall be inferred to be equal to 23.

dpb_output_delay_length_minus1 specifies the length in bits of the dpb_output_delay syntax element. The length of the dpb_output_delay syntax element of the picture timing SEI message is dpb_output_delay_length_minus1 + 1. When the dpb_output_delay_length_minus1 syntax element is present in more than one hrd_parameters( ) syntax structure within the VUI parameters syntax structure, the value of the dpb_output_delay_length_minus1 parameters shall be equal in both hrd_parameters( ) syntax structures. When the dpb_output_delay_length_minus1 syntax element is not present, it shall be inferred to be equal to 23.

time_offset_length greater than 0 specifies the length in bits of the time_offset syntax element. time_offset_length equal to 0 specifies that the time_offset syntax element is not present. When the time_offset_length syntax element is present in more than one hrd_parameters( ) syntax structure within the VUI parameters syntax structure, the value of the time_offset_length parameters shall be equal in both hrd_parameters( ) syntax structures. When the time_offset_length syntax element is not present, it shall be inferred to be equal to 24.