Skip to content

[ISSUE #7038] feat: clean AdminBrokerProcessor stub paths and admin parity gaps - #7039

Merged
mxsm merged 1 commit into
mainfrom
codex/admin-broker-processor-cleanup
Apr 4, 2026
Merged

mxsm merged 1 commit into
mainfrom
codex/admin-broker-processor-cleanup

Conversation

@mxsm

@mxsm mxsm commented Apr 4, 2026 •

Copy link
Copy Markdown
Owner

Which Issue(s) This PR Fixes(Closes)

Brief Description

This PR cleans a batch of low-dependency broker admin parity gaps that previously returned unknown response, exposed unfinished store paths, or serialized incompatible payloads.

Main changes:

  • implement broker admin handling for:
    • GetEarliestMsgStoreTime
    • QueryConsumeTimeSpan
    • CloneGroupOffset
    • CleanExpiredConsumequeue
    • DeleteExpiredCommitlog
    • CleanUnusedTopic
    • ResumeCheckHalfMessage
  • add missing remoting protocol types required by these admin requests:
    • CloneGroupOffsetRequestHeader
    • GetEarliestMsgStoretimeRequestHeader
    • QueryConsumeTimeSpanBody
    • ResumeCheckHalfMessageRequestHeader
  • align GetAllMessageRequestMode to return MessageRequestModeSerializeWrapper instead of the raw manager map
  • implement LocalFileMessageStore::clean_unused_topic
  • implement LocalFileMessageStore::clean_expired_consumer_queue
  • add targeted broker/store/remoting tests for the new paths

How Did You Test This Change?

  • cargo test -p rocketmq-store clean_unused_topic_deletes_only_non_retained_non_system_non_lmq_topics
  • cargo test -p rocketmq-broker clean_expired_consumequeue_returns_success
  • cargo test -p rocketmq-broker get_all_message_request_mode_returns_configured_modes
  • cargo test -p rocketmq-remoting resume_check_half_message_request_header_deserializes_from_map
  • cargo test -p rocketmq-broker resume_check_half_message_requeues_half_message
  • cargo fmt --all
  • cargo clippy --workspace --no-deps --all-targets --all-features -- -D warnings

Summary by CodeRabbit

Release Notes

  • New Features

    • Added admin operations for consumer group offset management (reset, clone, query).
    • Added subscription group management operations (create, update, delete).
    • Added broker maintenance operations for cleaning expired logs and unused topics.
    • Added transaction message recovery and requeue functionality.
  • Improvements

    • Enhanced message store cleanup efficiency for better resource management.

@rocketmq-rust-bot

Copy link
Copy Markdown
Collaborator

🔊@mxsm 🚀Thanks for your contribution🎉!

💡CodeRabbit(AI) will review your code first🔥!

Note

🚨The code review suggestions from CodeRabbit are to be used as a reference only, and the PR submitter can decide whether to make changes based on their own judgment. Ultimately, the project management personnel will conduct the final code review💥.

@coderabbitai

coderabbitai Bot commented Apr 4, 2026 •

Copy link
Copy Markdown
Contributor

Caution

Review failed

Pull request was closed or merged during review

Walkthrough

Implemented missing admin broker request handlers and protocol types to support consumer offset inspection, message request mode querying, consumer group operations, and transaction half-message recovery. Added corresponding store methods for topic cleanup and consumer queue maintenance.

Changes

Cohort / File(s) Summary
Protocol Headers & Bodies
rocketmq-remoting/src/protocol/header.rs, rocketmq-remoting/src/protocol/header/clone_group_offset_request_header.rs, rocketmq-remoting/src/protocol/header/get_earliest_msg_storetime_request_header.rs, rocketmq-remoting/src/protocol/header/resume_check_half_message_request_header.rs, rocketmq-remoting/src/protocol/body.rs, rocketmq-remoting/src/protocol/body/query_consume_time_span_body.rs, rocketmq-remoting/src/protocol/body/query_subscription_response_body.rs, rocketmq-remoting/src/protocol/body/subscription_group_list.rs
Added new request header types for clone group offset, earliest message store time, and resume check half-message operations. Added new response body types for query consume time span, subscription queries, and subscription group lists. All include serde support with camelCase naming and protocol codec traits.
Admin Broker Processor Routing
rocketmq-broker/src/processor/admin_broker_processor.rs
Updated process_request_inner to route 13 additional request codes from unknown responses to dedicated handler methods across consumer, subscription group, topic, offset, and message-related handlers.
Consumer Offset Management
rocketmq-broker/src/offset/manager/consumer_offset_manager.rs
Added clean_offset_by_group, clear_pull_offset, and clone_offset public methods. Refactored clean_offset_by_topic to use shared remove_keys_matching helper that synchronizes deletions across three internal offset tables.
Consumer Admin Handlers
rocketmq-broker/src/processor/admin_broker_processor/consumer_request_handler.rs
Implemented six new handlers: get_all_message_request_mode (returns wrapped mode map), invoke_broker_to_reset_offset (server-side or RPC reset), invoke_broker_to_get_consumer_status (delegates to Broker2Client), query_subscription_by_consumer (returns subscription data), query_consume_time_span (computes per-queue timestamps), and clone_group_offset (copies offsets between groups). Includes offset resolution by timestamp logic and unit/integration tests.
Subscription Group Admin Handlers
rocketmq-broker/src/processor/admin_broker_processor/subscription_group_handler.rs
Added update_and_create_subscription_group_list (validates and persists groups) and delete_subscription_group (removes groups, optionally cleans offsets based on lite-bind status). Includes group name validation enforcing non-blank, length, and character legality checks.
Topic Admin Handlers
rocketmq-broker/src/processor/admin_broker_processor/topic_request_handler.rs
Added clean_unused_topic handler that computes retained topic set and delegates cleanup to message store.
Offset Admin Handlers
rocketmq-broker/src/processor/admin_broker_processor/offset_request_handler.rs
Added get_earliest_msg_store_time (with static-topic rewrite logic for leader routing), clean_expired_consumequeue, and delete_expired_commitlog handlers. Includes RPC invocation for non-leader static topic queue mappings.
Message-Related Admin Handlers
rocketmq-broker/src/processor/admin_broker_processor/message_related_handler.rs
Added resume_check_half_message handler that validates message ID, retrieves stored half-message, resets check count, and re-submits to store. Includes helper to convert MessageExt to transactional half-topic representation.
Message Store Implementation
rocketmq-store/src/message_store/local_file_message_store.rs
Implemented clean_unused_topic (retains configured topics, filters out system and LMQ topics), clean_expired_consumer_queue (spawns or uses async runtime for cleanup), and refactored delete_topics to use shared delete_topics_inner helper. Includes test infrastructure and validation logic.
Broker Runtime Test Helper
rocketmq-broker/src/broker_runtime.rs
Added init_processor_for_test helper method (test-only, cfg-gated) to enable test-driven processor initialization without consuming return values.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

🐰 Hoppity-hop through protocol land,
New headers and bodies—oh isn't it grand!
Offset cloning and half-messages resume,
Admin APIs bloom throughout the broker's room!
From store to handler, clean logic takes flight,
whiskered paws wave—the parity looks right! 🌟

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.74% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title accurately describes the main objective: cleaning up AdminBrokerProcessor stub paths and addressing admin parity gaps by implementing missing handlers and protocol types.
Linked Issues check ✅ Passed The PR comprehensively addresses all objectives from issue #7038: implements all 7 missing admin request handlers, adds all 4 required protocol headers/bodies, implements store methods, and aligns GetAllMessageRequestMode.
Out of Scope Changes check ✅ Passed All changes are directly scoped to issue #7038 objectives. Helper methods (init_processor_for_test, offset management refactors, validator functions) are necessary supporting implementations for the main admin handlers and store operations.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/admin-broker-processor-cleanup

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@mxsm mxsm changed the title [7038] feat: clean AdminBrokerProcessor stub paths and admin parity gaps [ISSUE #7038] feat: clean AdminBrokerProcessor stub paths and admin parity gaps Apr 4, 2026
@mxsm
mxsm merged commit 5cf556e into main Apr 4, 2026
10 of 12 checks passed
@rocketmq-rust-bot rocketmq-rust-bot added approved PR has approved and removed ready to review waiting-review waiting review this PR labels Apr 4, 2026
@codecov

codecov Bot commented Apr 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 78.71256% with 334 lines in your changes missing coverage. Please review.
✅ Project coverage is 55.82%. Comparing base (10b0eea) to head (6800082).
⚠️ Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
...admin_broker_processor/consumer_request_handler.rs 85.05% 87 Missing ⚠️
...r/admin_broker_processor/offset_request_handler.rs 66.66% 67 Missing ⚠️
...eader/get_earliest_msg_storetime_request_header.rs 18.51% 66 Missing ⚠️
...tmq-broker/src/processor/admin_broker_processor.rs 0.00% 39 Missing ⚠️
...or/admin_broker_processor/topic_request_handler.rs 0.00% 22 Missing ⚠️
.../admin_broker_processor/message_related_handler.rs 86.00% 21 Missing ⚠️
...min_broker_processor/subscription_group_handler.rs 91.66% 16 Missing ⚠️
...tore/src/message_store/local_file_message_store.rs 92.10% 9 Missing ⚠️
...oker/src/offset/manager/consumer_offset_manager.rs 93.26% 7 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7039      +/-   ##
==========================================
+ Coverage   55.39%   55.82%   +0.43%     
==========================================
  Files        1061     1067       +6     
  Lines      174277   175897    +1620     
==========================================
+ Hits        96537    98193    +1656     
+ Misses      77740    77704      -36     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mxsm
mxsm deleted the codex/admin-broker-processor-cleanup branch April 20, 2026 02:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI review first Ai review pr first approved PR has approved auto merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement] Clean AdminBrokerProcessor stub paths and align broker admin parity

2 participants