NANCY: Fix chroma-keyed Bink videos not being keyed

[fedossayenko]

Chroma-keyed Bink videos in the nancy engine are never keyed: their key colour is
drawn fully opaque. In Nancy Drew 11: Curse of Blackmoor Manor, Nigel's ambient loop
in the library (CDVideo/Nigel_Fidget.bik) renders as a solid green rectangle with
the character composited on top.

The cause is a pixel format mismatch. Video::BinkDecoder decodes to RGB565, while
nancy's surfaces and the transparent colour derived from the BSUM chunk use the
engine's input pixel format, which is RGB555 (_inputPixelFormat16) for games before
Nancy13. Frames reach the surface through GraphicsManager::copyToManaged(), which is
a memcpy and performs no conversion, so full green is stored as 0x07E0 while the
engine keys on 0x03E0. ManagedSurface keys on an exact match, so nothing is keyed.

BinkDecoder already implements setOutputPixelFormat(), but MoviePlayer::loadFile()
never called it. This asks the decoder for the engine's own input format instead.

Verified with temporary instrumentation in PlaySecondaryVideo::init():

before:  frame fmt=RGB565@2  cornerPix=000007e0  surfFmt=RGB555@2  trans=000003e0
after:   frame fmt=RGB555@2  cornerPix=000003e0  surfFmt=RGB555@2  trans=000003e0

Tested on nancy11 (English Windows retail, ciftree.dat md5
3c406d4f391b6536982c6081f2dd1f4e, 55960641 bytes) in a
--disable-all-engines --enable-engine=nancy build on macOS/arm64. The green
background is gone and the video composites correctly.

Scanning the first-frame corner pixel of all 907 .bik files in CDVideo/ and
HDVideo/ found exactly one chroma-keyed video in this game, which is probably why
this went unnoticed. Other game types are unaffected in practice: the call is a no-op
whenever the requested format already matches what the decoder produces.

One follow-up that this patch deliberately does not address: because Bink is lossy,
pixels along the subject's silhouette are blends of the key colour and the subject and
do not exactly equal the key, so a thin green fringe remains once the background is
correctly keyed. That needs tolerance somewhere in the keying path rather than a
format fix, and the right design is your call. I have a local workaround that snaps a
greenish pixel to the key colour only when it directly touches an already-keyed pixel,
confining erosion to the boundary; happy to submit it separately if that approach is
wanted.

[fedossayenko]

Before / after, plus the remaining fringe described above:

1. key colour drawn opaque

2. with this patch — background keyed correctly, lossy-codec fringe still visible 3. same scene with the local boundary-snap workaround, which is not part of this PR

[bluegr]

Thanks for your work! Interesting issue... that video definitely worked, and after checking the game files, this video exists as both an AVF and a Bink file.

Given the above, I think that the issue has been caused by a recent commit, where we prefer Bink videos over AVF ones, if both exist. This was done in commit a72a80d , so could you try swapping the two if branches for AVF and Bink, to check if that fixes the issue?

Unfortunately, I can't verify the above right now, as I'm on vacation.

[fedossayenko]

Thanks for the pointer, that was really helpful. I tried swapping the two branches and it does fix
Nigel_Fidget, so a72a80d is definitely what surfaced this.

Two things left me unsure whether the swap is the right fix, though I may be missing context.
BLU_FoodCU_OVL is also chroma-keyed and ships only as a .bik, so the preference order wouldn't
reach that one. And swapping would put Nancy12's EXT_PieSorting_PUZ back on its AVF, which I had a
look at and it does seem to be the earlier version.

For now I've left the pixel format change in since it covers both, but I'd be glad to switch to the
branch swap if you'd prefer to keep the fix narrower. Which would you rather have?

No rush at all, this can easily wait until you're back. Enjoy the rest of your holiday!

[bluegr]

After reviewing this, the changes here are definitely a cleaner way of what I've done in 3fe1886, so this PR can be merged.

I'll probably revert my changes above, after testing a bit more.
I'll investigate the issue with the AVF/Bink preference and commit a fix separately.

Thanks for your work!

[bluegr]

Update:

• My previous per-frame conversion implementation has been reverted in commit 42c9a39
• The AVF/Bink video preference has now been properly implemented in commit 447d50b

[fedossayenko]

Thanks for sorting this out.

Two things I got wrong. The BLU_FoodCU_OVL example in my commit message is wrong, those records are
image overlays, so they never load a .bik. And the AVF for Nigel_Fidget isn't older art after all,
it's the newer file, and the other fidget videos are AVF only, so AVF there looks right.

I rebuilt master and checked the library scene. Nigel looks fine, no green rectangle.