Repository navigation
Auto-detect audio format in OpenAISpeechToTextClient - #7575
Merged
Merged
Conversation
…#7543) When the audio stream is not a FileStream, the client now peeks at the leading bytes to detect the format (wav, webm, m4a, mp3) and sets the multipart filename accordingly. This fixes HTTP 400 errors when sending non-MP3 audio (e.g. WAV) in a MemoryStream, since the OpenAI API uses the file extension to determine the audio format. - Add DetectAudioExtension using Span.SequenceEqual for readability - Add integration tests for all OpenAI-supported formats (mp3, wav, m4a, webm) - Add unit tests covering each magic-byte detection branch - Add ExpectedAudioFilename assertion to VerbatimMultiPartHttpHandler Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates OpenAISpeechToTextClient to auto-detect audio format (wav/webm/m4a/mp3) from leading “magic bytes” when the provided audio stream is not a FileStream, and uses the detected extension in the multipart filename so OpenAI can correctly infer the format (fixing 400s for non-MP3 MemoryStream inputs).
Changes:
- Add stream-header “magic byte” detection and filename resolution logic in
OpenAISpeechToTextClient. - Add unit tests validating filename selection for each supported format and branch.
- Add integration coverage for multiple embedded audio formats and enhance multipart handler assertions to validate the uploaded filename.
Reviewed changes
Copilot reviewed 5 out of 9 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAISpeechToTextClient.cs | Adds filename resolution with magic-byte detection for non-FileStream inputs. |
| test/Libraries/Microsoft.Extensions.AI.OpenAI.Tests/OpenAISpeechToTextClientTests.cs | Adds theory-based unit tests asserting detected multipart filenames for different headers. |
| test/Libraries/Microsoft.Extensions.AI.Integration.Tests/VerbatimMultiPartHttpHandler.cs | Adds optional filename assertion for multipart “file” fields. |
| test/Libraries/Microsoft.Extensions.AI.Integration.Tests/SpeechToTextClientIntegrationTests.cs | Adds integration test that exercises auto-detection across multiple audio formats. |
| test/Libraries/Microsoft.Extensions.AI.Integration.Tests/Microsoft.Extensions.AI.Integration.Tests.csproj | Embeds additional audio resource files used by the new integration test. |
Comments suppressed due to low confidence (1)
src/Libraries/Microsoft.Extensions.AI.OpenAI/OpenAISpeechToTextClient.cs:121
- In
GetStreamingTextAsync,ResolveFilename(audioSpeechStream)is executed unconditionally even for translation requests, but the translation branch immediately delegates toGetTextAsync(...)(which resolves the filename again). With the new magic-byte peek, this results in redundant header reads/rewinds for translation streaming.
_ = Throw.IfNull(audioSpeechStream);
string filename = ResolveFilename(audioSpeechStream);
if (IsTranslationRequest(options))
{
foreach (var update in (await GetTextAsync(audioSpeechStream, options, cancellationToken).ConfigureAwait(false)).ToSpeechToTextResponseUpdates())
jozkee
commented
Jun 16, 2026
Collaborator
🎉 Good job! The coverage increased 🎉
Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1467244&view=codecoverage-tab |
tarekgh
reviewed
Jun 17, 2026
jeffhandley
approved these changes
Jun 17, 2026
tarekgh
reviewed
Jun 17, 2026
tarekgh
approved these changes
Jun 18, 2026
Collaborator
🎉 Good job! The coverage increased 🎉
Full code coverage report: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1471017&view=codecoverage-tab |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When the audio stream is not a FileStream, the client now peeks at the leading bytes to detect the format (wav, webm, m4a, mp3) and sets the multipart filename accordingly. This fixes HTTP 400 errors when sending non-MP3 audio (e.g. WAV) in a MemoryStream, since the OpenAI API uses the file extension to determine the audio format.
Fixes #7543
Microsoft Reviewers: Open in CodeFlow