The encoder refused B pictures outright. Three things were missing.
The slice header writer marked every picture as a reference, nal_ref_idc
hardcoded to 1. A B picture nothing will reference has to say so, otherwise
a decoder keeps it in the DPB and its frame_num accounting drifts until the
DPB overflows; the client's is_reference flag decides now, an IDR aside.
The same writer emitted no B slice syntax at all, so direct_spatial_mv_pred_flag
and the L1 reference list modification flag were absent and the bitstream went
out of sync from that bit onwards. Spatial direct mode is chosen, which
derives motion from neighbours rather than the co-located picture, so no
direct MV buffer has to be read back.
Finally the L1 reference list was never built. It comes from RefPicList1 the
same way L0 comes from RefPicList0, and feeds MFX_AVC_REF_IDX_STATE list 1
and the L1 count in MFX_AVC_IMG_STATE.
Prediction itself is unchanged: a B macroblock is coded as B_L0_16x16, so it
predicts from L0 alone and costs about what a P macroblock costs. What a B
picture buys here is that it need not enter the DPB, not a cheaper prediction.
Bi-prediction would need the macroblock type raised to B_Bi_16x16 and motion
vectors for both lists.
dEQP-VK.video.encode.h264.i_p_b_13_* goes from NotSupported to 4/4 Pass.
NOT FULLY VERIFIED: the run over the whole encode family aborted partway on an
unrelated device loss, with two QualityWarnings in intra_refresh that were not
there before, so this needs re-running on a healthy GPU before being trusted.
Co-Authored-By: Claude Opus 5 <[email protected]>
Only constant QP was offered. Add the two bitrate modes, with the quantizer
derived once per picture from the requested bitrate and frame rate.
There is no closed loop and there cannot be one here: a picture's size is
known only after the GPU encodes it, and vkCmdEncodeVideoKHR merely records
commands. The VA-API encoder for this hardware runs its bit rate control
after each frame completes, which that API's synchronisation allows and this
one does not, and gen7 has no macroblock-level rate control the hardware
drives on its own -- the VA-API driver leaves those MFX_AVC_IMG_STATE fields
zero on this generation. Vulkan defines rate control as best effort, and
this is the effort.
The quantizer comes from a measured curve rather than a guess: sweeping QP 14
to 50 over a 176x144 clip gives 361 bits per macroblock at QP 26 for an I
picture and 166 for a P one, halving every 12.5 steps of the quantizer for I
and every 10.3 for P. Measured against the target across a 15x range of
bitrates, I pictures land between 0.82x and 1.30x, P pictures between 0.59x
and 1.56x -- the spread on P is content dependent and nothing open loop can
narrow it. A target below about 90 bits per macroblock cannot be met at all,
since that is where the curve flattens.
dEQP-VK.video.encode.h264 goes from 84 to 92 of 120 with no failures. The
remaining 28 are quantization maps, which gen7 has no hardware path for, and
B frames.
Co-Authored-By: Claude Opus 5 <[email protected]>
Splitting a picture into slices is the implementation's job. The encoder
instead read first_mb_in_slice out of each entry's slice header and used the
next entry's value as the end of the range, but clients leave that field at
zero on every entry and expect the driver to derive it and write it into the
emitted header. A multi-slice picture therefore came out as one slice
covering nothing, and vkCmdEncodeVideoKHR failed.
Divide the macroblock rows evenly across the NALU slice entries, which keeps
every slice row-aligned as required while
VK_VIDEO_ENCODE_H264_CAPABILITY_ROW_UNALIGNED_SLICE_BIT_KHR is not
advertised, and reject more slices than there are rows.
This also removes the workaround for ffmpeg passing first_mb_in_slice=1,
which was not a client bug at all: the field was never an input.
Per-picture-partition intra refresh needs nothing beyond this, since the
refreshed partition is just a slice the client marks as an I slice and slices
are already coded by their own slice_type, so offer that mode too.
dEQP-VK.video.encode.h264 goes from 76 to 84 of 120. The remaining 36 are
honest: quantization maps have no hardware path on gen7, where the only
per-macroblock QP is the one the CPU writes into the PAK object and the only
indirect data a PAK object fetches is motion vectors; B frames and rate
control are not implemented.
Co-Authored-By: Claude Opus 5 <[email protected]>
The encoder ignored VkVideoReferenceSlotInfoKHR entirely: it took
pReferenceSlots[0] as the one reference, wrote its address into frame store 0
and hardcoded the L0 list to that store. Two DPB slots were all it could
honestly advertise, which is fewer than a GOP of any length needs.
Record each slot the operation names under its own index, since a slot maps
onto the frame store of the same index and MFX_PIPE_BUF_ADDR_STATE holds 16
of those, and build the L0 entries from StdVideoEncodeH264ReferenceListsInfo,
whose RefPicList0 already carries slot indices. Clients that pass reference
slots without a list keep the old behaviour.
maxDpbSlots therefore goes to 16. maxActiveReferencePictures stays at 1:
every macroblock is still coded against reference index 0, so a picture may
be kept in the DPB but only the one it names first is predicted from.
dEQP-VK.video.encode.h264.intra_refresh_* goes from 12 Pass and 32 stopped at
the maxDpbSlots >= GOP frame count precondition to 36 Pass, the remaining 8
being per-picture-partition mode, which is not offered. The family as a whole
is 76 of 120 with no failures. ffmpeg's h264_vulkan encoder no longer trips
its slot_index assertion and produces a full GOP.
Co-Authored-By: Claude Opus 5 <[email protected]>
Each macroblock's intra/inter decision is already made on its own when the
PAK objects are emitted, so refreshing a region is a matter of coding the
macroblocks it covers the way an I slice codes them. Regions divide the
picture evenly along rows, columns or raster order, which leaves the block,
block-row and block-column modes supported and gives empty regions for free
when the cycle is longer than the split axis.
Two things needed care. PAK keeps the I slice numbering for MbType5Bits
inside a P slice, where H.264 itself offsets intra mb_types by 5; handing it
the offset value hangs the MFC. And under constrained intra prediction a
neighbour only counts as available if it is in the refresh region too,
because prediction may not cross into an inter macroblock.
Nothing constrains motion away from not-yet-refreshed regions, which is
sound only because there is no motion estimation: every vector is zero, so a
macroblock reads its own co-located block and never leaves its region.
Motion estimation will have to honour VkVideoReferenceIntraRefreshInfoKHR's
dirty region count.
Per-picture-partition mode is not offered. It wants the refreshed slice
coded as an I slice, header included, rather than as intra macroblocks
inside a P slice.
dEQP-VK.video.encode.h264.intra_refresh_* goes from 44 NotSupported to 12
Pass; all three modes pass in the variants that can run. The other 32 stop
at a CTS precondition, maxDpbSlots >= GOP frame count, which the driver's
two slots do not meet. Raising that needs real DPB slot tracking and is
what gates B frames and ffmpeg's encoder too.
Co-Authored-By: Claude Opus 5 <[email protected]>
The queue family properties claimed no support with the note that the VCS
"seemingly cannot do a posted memory write". It can: the encode feedback
query already writes its availability, buffer offset and byte count from a
video queue through exactly the same MI_FLUSH_DW post-sync write, and the
VK_QUERY_TYPE_RESULT_STATUS_ONLY_KHR paths in genX_query.c are complete.
Only the advertised capability was missing.
dEQP-VK.video.encode.h264.query_with_status_* goes from NotSupported, where
it failed device creation, to 4/4 Pass; the family is now 40 of 120 with no
failures. h264 decode is unaffected.
Co-Authored-By: Claude Opus 5 <[email protected]>
VK_IMAGE_LAYOUT_GENERAL already lands on the conservative aux choice for
every usage, since anv_layout_to_aux_state() derives that from the layout's
usage flags and GENERAL reports all of them, and video images carry no aux
at all. So the layout itself needed no work; what the extension does add is
a requirement that the host image copy layout lists name every layout it
declares interchangeable with GENERAL.
Those layouts can only be offered if a host copy is safe in each of them,
and the CPU cannot resolve an aux surface on its way to the image. Create
images that carry VK_IMAGE_USAGE_HOST_TRANSFER_BIT without one, which makes
every layout equivalent for them and vkTransitionImageLayoutEXT a genuine
no-op rather than one justified by only ever offering GENERAL.
dEQP-VK.video.encode.h264 goes from 18 to 36 passes out of 120, still with
no failures: the general_layout variants were the whole gap.
dEQP-VK.image.host_image_copy goes from 2037 passes and one failure to
19140 passes and none, the failure having been the properties test checking
exactly this list.
Co-Authored-By: Claude Opus 5 <[email protected]>
VK_KHR_video_maintenance1 lets a client chain VkVideoInlineQueryInfoKHR onto
vkCmdEncodeVideoKHR instead of bracketing it with vkCmdBeginQuery and
vkCmdEndQuery. The driver advertises the feature but only ever wrote the
feedback result to the address vkCmdBeginQuery had recorded, so with an
inline query nothing wrote the result or marked it available and the client
waited on a query that never became ready.
Derive the address from the chained pool when one is present.
dEQP-VK.video.encode.h264.inline_query_* went from DEVICE_LOST, which also
aborted the rest of the run, to Pass. With this and the missing 352x288
input clip generated, the whole h264 encode family now runs clean: 18 pass,
0 fail out of 120, no aborts.
Co-Authored-By: Claude Opus 5 <[email protected]>
MFX_QM_STATE and MFX_FQM_STATE hold separate register sets for the intra
and the inter scaling lists, but only the intra ones were programmed. The
hardware therefore quantized every inter residual with whatever those
registers happened to hold, so a coded P macroblock came back corrupted
while I frames were correct. That is why the P path shipped as skip-only.
With both sets programmed, P macroblocks can carry a real residual: encode
them as coded BP_L0_16x16 with a zero motion vector instead of forcing
P_Skip. A P frame now tracks changing content rather than repeating its
reference.
Measured on a 176x144 clip at QP 26, P frame luma PSNR against the true
source: 12.81 dB before, 43.87 dB after (the I frame scores 44.53 dB), at
2062 bytes versus the I frame's 4474. With an unchanged source the P frame
costs 67 bytes, down from 4862.
dEQP-VK.video.encode.h264.* goes from 12 to 14 passes with no new failures.
There is still no motion estimation, so a moving block costs a full residual
rather than a motion vector.
Co-Authored-By: Claude Opus 5 <[email protected]>
GPGPU_WALKER has no encoding for a zero group count, so passing one
through walks the full dimension and the command never retires, hanging
rcs0 (GPU HANG ecode 7:1:8ff8ffff, EIR and PGTBL_ER both clear). A zero
dispatch is legal and defined as a no-op, so drop it at record time.
through walks the full dimension and the command never retires, hanging
rcs0 (GPU HANG ecode 7:1:8ff8ffff, EIR and PGTBL_ER both clear). A zero
dispatch is legal and defined as a no-op, so drop it at record time.
Gen9+ uses COMPUTE_WALKER and handles this in hardware, which is why
upstream anv carries no equivalent guard.
Fixes dEQP-VK.compute.pipeline.basic.empty_workgroup_{all,x,y,z} and
empty_shader, which previously aborted the run and could take the whole
machine down. compute.pipeline.* now 20285 cases, 0 fail.
