Skip to content

[d3d11] Support planar outputs in video processor - #4872

Merged
doitsujin merged 7 commits into
doitsujin:masterfrom
LizzyFleckenstein03:planar_video
Apr 26, 2025
Merged

doitsujin merged 7 commits into
doitsujin:masterfrom
LizzyFleckenstein03:planar_video

Conversation

@LizzyFleckenstein03

@LizzyFleckenstein03 LizzyFleckenstein03 commented Apr 24, 2025 •

Copy link
Copy Markdown
Contributor

This aims to support the game JR East Train Simulator, which uses the video processor with input and output streams both in the NV12 format.

Vulkan does not (usually?) support rendering to a G8_B8R8_2PLANE_420_UNORM (=NV12) view (it lacks the COLOR_ATTACHMENT_BIT in features - tho I've only checked on a modern AMD gpu).

Two passes are necessary since the Y and CbCr views may have different dimensions (in the case of NV12, CbCr is half the size of the Y view).

Since the creation of the output view now significantly overlaps with the input view's code (both have to iterate over the aspectMask to create views), I've introduced a superclass VideoProcessorView to share the behavior.

In case of planar output, the output view still creates a view in its "native" format, to support being cleared by D3D11CommonContext::ClearView.

@doitsujin

doitsujin commented Apr 24, 2025 •

Copy link
Copy Markdown
Owner

Would it be possible to split this into multiple commits? I'm having a very hard time working out what's going on when there's a relatively large refactor + adding new functionality all in one massive diff.

Vulkan does not (usually?) support rendering to a G8_B8R8_2PLANE_420_UNORM (=NV12) view

Because rendering to planar images conceptually doesn't make much sense; especially when the chroma plane is subsampled.

As for ClearView, I don't know if this is supposed to do anything on planar views, would need some sort of test case to prove that it does, and if it does, work out how the color mapping actually works, and then clear the sub-views for each aspect one at a time. Views can only be created for one plane at a time.

The whole video thing in general is neither particularly well tested, nor particualrly feature-complete since games so far haven't really needed much of it, planar VOV is certainly a first.

@doitsujin

doitsujin commented Apr 24, 2025 •

Copy link
Copy Markdown
Owner

OK, looking at ClearView docs, the color mapping for yuv formats is straight-forward and well-defined, but we'd still need to deal with individual aspects, take subsampling into account (especially if there's clear rects involved) etc etc.

I do need to write some tests for this.

@LizzyFleckenstein03

LizzyFleckenstein03 commented Apr 24, 2025 •

Copy link
Copy Markdown
Contributor Author

sorry if it's a naive question, but are there tests located anywhere inside this repo or in a different one? I could contribute tests for this (I already wrote some amount of testing code for this), I'm just having trouble figuring out where to put them

@doitsujin

Copy link
Copy Markdown
Owner

Not really, no.

@LizzyFleckenstein03

Copy link
Copy Markdown
Contributor Author

I split it into different commits now - I hope it makes more sense like this.

@doitsujin doitsujin left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this is indeed much easier to reason about.

See comments; overall approach looks solid, but unless absolutely required I'd prefer to ignore the whole ClearView thing for now since there's going to be more refactoring needed.

Comment thread src/d3d11/d3d11_video.h Outdated
Comment thread src/d3d11/d3d11_video.h Outdated
Comment thread src/d3d11/shaders/d3d11_video_blit_frag.frag Outdated
Comment thread src/d3d11/d3d11_video.cpp Outdated
Comment thread src/d3d11/d3d11_video.cpp Outdated
Comment thread src/d3d11/d3d11_video.cpp Outdated
Comment thread src/d3d11/d3d11_video.cpp Outdated
@LizzyFleckenstein03

LizzyFleckenstein03 commented Apr 25, 2025 •

Copy link
Copy Markdown
Contributor Author

Updated the PR according to feedback.

I've also noticed that GetImageSubresources() isn't used anywhere, and since this PR touches it, I've decided to remove it in a separate commit before the other changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants