Skip to content

API Spec: Add zero-copy accessor ResourceCandidate.ValueAsMemoryBuffer for EmbeddedData - #6659

Merged
Mike Crider (codendone) merged 6 commits into
mainfrom
user/codendone/apiSpecMRTmemoryUsage
Oct 2, 2026
Merged

Mike Crider (codendone) merged 6 commits into
mainfrom
user/codendone/apiSpecMRTmemoryUsage

Conversation

@codendone

Copy link
Copy Markdown
Collaborator

Spec proposes new zero-copy accessors for embedded MRT Core resources. Both existing paths copy the bytes on every read even though the .pri is already memory-mapped. The new APIs hand out a read-only, non-owning view instead.

Covers two API layers:

  • WinRT: ResourceCandidate.ValueAsMemoryBuffer(), an IMemoryBuffer-based alternative to ValueAsBytes.
  • Flat-C (MRM.h): MrmResourceData2, MrmLoadStringOrEmbeddedResourceNoCopy, MrmLoadStringOrEmbeddedResourceByIndexNoCopy, and MrmFreeResourceData.

ResourceCandidate.ValueAsMemoryBuffer() is the important public API, expected to be used by WinUI and others. The flat-C APIs are support for ResourceCandidate.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Outdated
/// Zero-copy accessor for embedded/binary resources: returns a read-only IMemoryBuffer that
/// projects the resource bytes directly (no copy). The buffer (and any reference created
/// from it) pins the backing resource data for its lifetime; Close it when done. Throws for
/// Kind != EmbeddedData. Prefer this over ValueAsBytes to avoid a per-call copy.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why must this throw on "non-embedded data"? Maybe the app has downstream stuff that works better with IMemoryBuffer so "get an IMemoryBuffer" seems nicer. You have all the logic already - if somebody calls this on non-embedded-data value, conjure up a new MemoryBuffer filled with ValueAsBytes... Avoids more conditionals in caller code.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DISCUSSION: Current API returning as BYTEs requires it to be embedded and throws if it isn't. Docs don't say this but code does. Throwing here maintains consistency with existing behavior.

RECOMMEND: Doc bug; fix

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracking with https://task.ms/64372174

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess my point was that the consistency is needless. This is a new API, returning a view on data. Why can't it "just work" on all kinds of resource data that has a linear byte representation? Looser rules are usually good. More use cases are usually good.

Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved
Comment thread specs/mrtcore/ZeroCopyResourceData.md Resolved

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

API Review: APPROVED

@codendone

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
2 pipeline(s) were filtered out due to trigger conditions.

@codendone

Copy link
Copy Markdown
Collaborator Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
2 pipeline(s) were filtered out due to trigger conditions.

@codendone
Mike Crider (codendone) merged commit 888bfbd into main Oct 2, 2026
51 checks passed
@codendone
Mike Crider (codendone) deleted the user/codendone/apiSpecMRTmemoryUsage branch October 2, 2026 23:14
Mike Crider (codendone) added a commit that referenced this pull request Oct 10, 2026
…r for EmbeddedData (#6660)

Implements #6659.

# Summary

Loading an embedded (`EmbeddedData`) resource from a memory-mapped PRI previously copied the same blob up to three times: twice at load and once more on every read. This change removes all three copies so the bytes can be read directly from the read-only PRI mapping.

## The copies and how each is removed

### Copy 1: load-time heap duplication

The loaders (`MRM.cpp`) obtained a `BlobResult` that was already just a *view* into the mapped PRI, then forced a heap copy via `BlobResultReleaseOwnershipBuffer` purely to decouple the returned bytes from the mapping lifetime.

**Fix:** New opt-in loaders `MrmLoadStringOrEmbeddedResourceNoCopy` / `...ByIndexNoCopy` return the view directly (`MrmResourceData2.isView == TRUE`) instead of copying. Lifetime is preserved by holding a strong resource-manager reference so the mapping stays alive for the candidate's lifetime. Owned buffers (e.g. decompressed blobs) still come back with `isView == FALSE`.

### Copy 2: construction

`ResourceMap.cpp` wrapped the load-time buffer in an `array_view` which the `ResourceCandidate` ctor copied *again* into a `com_array m_blobData`. The original was then released, so it was a transient allocation with an unnecessary memcopy of the entire buffer.

**Fix:** Switch to an adopting constructor, where `ResourceCandidate` now takes ownership of the loader buffer (`embedded_resoure_ptr`) rather than re-copying it. For the no-copy (`isView==TRUE`) path, it simply stores the borrowed view.

### Copy 3: every read

`ValueAsBytes()` allocated a fresh `com_array` copy on each call.

**Fix:** Added `ValueAsMemoryBuffer()` WinRT API, which returns an `IMemoryBuffer` over the single backing buffer/view. This enables zero copies per read. WinUI will be updated to use this API. (`ValueAsBytes()` is unchanged for compatibility.)

## ABI safety

The existing APIs are unchanged. New flat-C APIs in `MRM.h` enable a non-copy view access to embedded data. The new `ResourceCandidate.ValueAsMemoryBuffer()` provides that access via the WinRT `ResourceCandidate` type.

## Testing

New TAEF unit tests cover view vs. owned results, strings-are-not-views, by-index loading, and `MrmFreeResourceData` ownership semantics. The changes have also been tested in a WinUI app on a private build using the new `ValueAsMemoryBuffer()` API.
Mike Crider (codendone) added a commit that referenced this pull request Oct 13, 2026
….ValueAsMemoryBuffer for EmbeddedData (#6857)

* Implement new zero-copy accessor ResourceCandidate.ValueAsMemoryBuffer for EmbeddedData (#6660)

Implements #6659.

# Summary

Loading an embedded (`EmbeddedData`) resource from a memory-mapped PRI previously copied the same blob up to three times: twice at load and once more on every read. This change removes all three copies so the bytes can be read directly from the read-only PRI mapping.

## The copies and how each is removed

### Copy 1: load-time heap duplication

The loaders (`MRM.cpp`) obtained a `BlobResult` that was already just a *view* into the mapped PRI, then forced a heap copy via `BlobResultReleaseOwnershipBuffer` purely to decouple the returned bytes from the mapping lifetime.

**Fix:** New opt-in loaders `MrmLoadStringOrEmbeddedResourceNoCopy` / `...ByIndexNoCopy` return the view directly (`MrmResourceData2.isView == TRUE`) instead of copying. Lifetime is preserved by holding a strong resource-manager reference so the mapping stays alive for the candidate's lifetime. Owned buffers (e.g. decompressed blobs) still come back with `isView == FALSE`.

### Copy 2: construction

`ResourceMap.cpp` wrapped the load-time buffer in an `array_view` which the `ResourceCandidate` ctor copied *again* into a `com_array m_blobData`. The original was then released, so it was a transient allocation with an unnecessary memcopy of the entire buffer.

**Fix:** Switch to an adopting constructor, where `ResourceCandidate` now takes ownership of the loader buffer (`embedded_resoure_ptr`) rather than re-copying it. For the no-copy (`isView==TRUE`) path, it simply stores the borrowed view.

### Copy 3: every read

`ValueAsBytes()` allocated a fresh `com_array` copy on each call.

**Fix:** Added `ValueAsMemoryBuffer()` WinRT API, which returns an `IMemoryBuffer` over the single backing buffer/view. This enables zero copies per read. WinUI will be updated to use this API. (`ValueAsBytes()` is unchanged for compatibility.)

## ABI safety

The existing APIs are unchanged. New flat-C APIs in `MRM.h` enable a non-copy view access to embedded data. The new `ResourceCandidate.ValueAsMemoryBuffer()` provides that access via the WinRT `ResourceCandidate` type.

## Testing

New TAEF unit tests cover view vs. owned results, strings-are-not-views, by-index loading, and `MrmFreeResourceData` ownership semantics. The changes have also been tested in a WinUI app on a private build using the new `ValueAsMemoryBuffer()` API.

* Added containment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants