Repository navigation
fix: cap organization history undo receipts (settings.json bloat hotfix) - #393
Conversation
… budget RecentOrganizationHistory capped only the entry count (24), never the receipt count inside an entry, so repeated multi-thousand-file batches grew settings.json without bound: a real 1.5.3 profile hit 19.8 MB (24 entries x ~520 KB of full source/destination/identity receipts), inflating the startup object graph and every debounced save. OrganizationHistoryPolicy now bounds retention two ways: - MaxUndoReceiptItemsPerEntry (500): larger batches keep their history entry as a summary - the new persisted TotalItemCount, no receipts, CanUndo=false. Undo receipts are all-or-nothing, so a truncated receipt list can never stay undoable. - MaxUndoReceiptItemBudget (2500): when the whole history exceeds the budget the oldest entries are downgraded to summaries first; entries are never deleted for budget reasons. The policy runs at both append sites (OrganizerService.AddHistoryEntryAsync and the desktop organization transaction, including the retry merge) and in the load normalizer, so an existing bloated profile shrinks on the first save after upgrade: the real 19.8 MB profile migrates to 90,984 bytes with all 24 entries preserved. Legacy entries without TotalItemCount fall back to Items.Count for display; retained entries keep their full DestinationIdentity receipts untouched. The real-profile migration is reproducible through the env-gated RealSettingsCopy_MigrationShrinksBloat test (DESKBOX_HISTORY_MIGRATION_FIXTURE).
Review of the first cut found the retention policy had crossed a transaction boundary: in the desktop organization flow, OrganizationHistory.Items is also the durable commit evidence that tells RecoverPendingAsync which moves already committed, the resume state of an interrupted undo, the retry merge base, and the execution result page data. Compacting it blindly broke three of those roles. - Commit order is now: save full receipts (commit point) -> clear the recovery journal -> compact -> save again. A crash anywhere in between costs at most a larger settings.json, never a wrongly restored file. Load-time normalization no longer drops receipts at all; compaction runs in RecoverPendingAsync once no journal is outstanding, which startup always reaches. - A discarded transaction is now marked with a persisted UndoReceiptsDiscarded flag. Retry merges inherit the marker, carry the real TotalItemCount forward, and can never regain CanUndo. - Entries with an interrupted undo (UndoStarted or any IsRestored receipt, not yet IsUndone) are never compacted: their receipts are resume state, and an interrupted ManagedDrop undo has no recovery journal to consult at all. - DesktopOrganizationExecutionResult.CompletedItems snapshots this run's receipts so the result page renders real counts even after the persisted entry becomes a summary; the page no longer reads the persisted Items. - ManagedDrop appends only cap the new entry; the global budget runs in the post-recovery compaction so it cannot touch entries that still back a pending journal. New integration tests cover the crash window (committed moves stay put while full receipts are on disk), the pending-undo journal case, the discarded-retry case, and a 601-file execution reporting full results with a compacted entry; the oldest-first budget test now uses real descending timestamps.
|
四个发现全部修复(ea4bbd8,3707/3707 绿),逐条对账: 🔴 F1 — commit 时序重排采纳你的建议顺序:完整 receipts 落盘(commit point)→ 清 recovery journal → compact → 再 Save。 🔴 F2 — load 不再 destructive + 中断 undo 保护
关于"只查 UndoStarted 不够稳":完全同意,而且光查 journal 也不够——ManagedDrop 的中断 undo 根本没有 journal,靠 🔴 F3 — 显式持久化 UndoReceiptsDiscarded新增 🟠 F4 — 执行结果与持久化 History 解耦
🟡 F5 — 测试时间方向
两个 Limitation均按你的判断保持现状:首次升级仍要完整反序列化一次 19.8MB(迁移后落盘 91,858 字节,下次启动彻底恢复);数量预算不做字节测量。真实档案迁移复测通过: 另外两处主动收紧
journal 拆分仍留后续架构 PR,本 PR 未动。 |
Second review round found the retention policy still leaked into two callers, and the commit point was not actually durable. - OrganizerService.OrganizeDropAsync / MoveItemsBackToDesktopAsync now return OrganizerOperationResult: CompletedItems carries this run's receipts for callers, History is the persisted entry (possibly a receipt-less summary). A 501-file import or move-back no longer reports zero completed items; WidgetViewModel and the auto organization watcher follow the new shape. - Every settings save that gates a recovery journal clear is now SaveCheckedAsync: ExecuteAsync's commit point throws on failure so the journal survives, the undo reconcile and forward recovery completion keep the journal on failure, and both abandon flows throw. A silent save failure can no longer delete the only durable record of a transaction. - Abandon is now an undo lifecycle endpoint: CanUndo=false plus UndoStarted=false, followed by the post-clear retention pass, so an abandoned entry can never stay protected from compaction forever. IsUndoLifecycleActive additionally requires CanUndo, making every non-undoable entry compactable. - ApplyRetentionPolicy takes a protected transaction id: the entry a pending recovery journal still references keeps its receipts and survives the entry cap, while the global 2500 budget and the 24-entry cap now also run on ordinary imports (AddHistoryEntryAsync consults the shared journal), so a long session cannot grow history without bound between compaction passes. - The crash-window test now reloads settings from disk as a real restart; new tests cover 501-file import/move-back result counts, save-failure journal survival, abandon-time compaction, and the protected-entry budget. The abandon contract test follows the new endpoint semantics. Real-profile migration unchanged: 19,785,160 -> 91,858 bytes.
|
第二轮 6 个发现全部处理(c73c538,3714/3714 绿),PR body 已同步重写。逐条对账: 🔴 B1 — 普通格子的运行结果与持久化 history 彻底解耦采纳你的建议做了统一:新增
回归测试: 🔴 B2 — commit point 做实原则按你说的统一了:凡"下一步删 journal、且删除依赖 settings 已 durable"的保存点,一律
compact 之后的第二笔 save 保持 best effort(失败最坏=文件大一点)。测试 🟠 H1 — Abandon = 生命周期终点按你倾向的第二种: 🟠 H2 — 2500 恢复为实时 invariant
🟡 M1 — 真 restart 模拟
🟡 M2 — Retry 统计口径PartialResult 文案改用 🔵 L — PR body 已重写按三段 commit 结构(预算与摘要 / transaction boundary / 全链路解耦与 commit 做实)重写,含设计约束、测试清单、已知 Limitation,与代码一致。 真实档案迁移复测不变: |
Third review round: two crash windows and a cap ordering bug around the retain/protect contract. - Abandon is now crash-safe via a durable journal terminal marker: DesktopOrganizationRecoveryJournal.IsAbandoned is persisted BEFORE the abandon touches settings, and RecoverPendingAsync refuses to act on an abandoned journal (a crash between marking and clearing can never re-execute what the user gave up). The undo-reconcile branch also refuses terminal entries (!CanUndo || IsUndone), so a crash between a finished undo and the journal clear cannot flip CanUndo back on, and UndoAsync rejects abandoned journals defensively. - The entry cap now runs after the protection contract, not before it: EnforceEntryCap keeps every protected entry (journal-referenced AND active undos) first and fills the remaining slots with newest normal history — if protected entries alone exceed the cap, the list grows past it rather than dropping resume state. DesktopOrganizationTransaction's own RemoveRange(24) is gone and the settings load no longer trims the count: the retention policy is the single source of the cap. - Ordinary imports no longer hard-depend on a journal parse: TryRunRetentionPolicyAsync swallows read failures and the caller caps only the brand-new entry (CapEntryReceipts), so a corrupt or mid-replacement journal can never fail an import whose files already moved; the next safe compaction pass picks up the rest. The shared store's journal path is injectable so tests never touch the real data directory. New tests: abandoned-undo journal must not revive, abandoned marker must not restore files, active undo survives an over-cap list, corrupt journal leaves an import successful. Real-profile migration unchanged: 19,785,160 -> 91,858 bytes.
|
第三轮 3 个发现全部修复(731df77,3718/3718 绿),PR body 预算表述已按 Medium 1 修正。逐条对账: 🔴 B1 — Abandon 的 durable 终态两个方向都按你的建议做了: Undo 方向(复活窗口): Forward 方向(反向恢复): 🔴 B2 — cap 优先于保护的顺序错误认账:
测试 🟠 High — 普通导入不再硬依赖 journal parse按你的语义做了 🟡 两个 Medium
真实档案迁移复测不变: |
The IsAbandoned marker made recovery refuse to execute an abandoned journal, but it cleared the WAL without completing the abandon's settings-side half: a crash between the WAL write and the settings save left the abandon permanently unfinished (an undo entry the user gave up on stayed resumable; a forward abandon's widget/rule/directory cleanup never ran). Recovery now runs TryFinalizeAbandonedJournalAsync: apply the terminal state (undo: CanUndo/UndoStarted flip; forward: RemoveUncommittedWidgets), persist with a checked save, and only then clear the journal. When the save fails the WAL survives and the next startup retries - the abandon eventually completes instead of being dropped. The finalize core is shared with AbandonPendingRecoveryAsync (and the terminal-state flip with AbandonUndoAsync) so the semantics cannot drift apart. Tests: abandoned undo WAL with pre-abandon settings finalizes the terminal state; abandoned forward WAL completes the created-widget/rule/ directory cleanup; a failed finalize keeps the WAL and a second recovery pass finishes it.
|
最后的 Blocker 已修(20e08bc,3721/3721 绿)——完全按你给的结构把 IsAbandoned 从"跳过标记"补成完整 WAL finalize。 实现
语义复用按你的建议做了单一来源:
整个 abandon 事务现在是:写 WAL → 应用 settings 终态 → durable SaveChecked → 删 WAL——任何一步崩溃,下次启动从 WAL 继续 2~4 步,abandon 最终一定完成。 三个测试(对应你点的 crash matrix)
真实档案迁移复测不变(19,785,160 → 91,858 bytes)。等你下一轮最终 regression sweep。 |
Two transaction boundary findings from the full regression sweep. - RecoverPendingAsync's terminal-undo branch cleared the journal on the strength of the in-memory entry state alone. After an undo whose checked save failed earlier in the same process (files restored, receipts reconciled in memory, disk still pre-undo), that branch would delete the journal and leave the stale disk state as the only record — a restart would then show a resumable undo for files that are already back on the desktop. The branch now runs a checked save first: durable confirmation before the journal clear, idempotent for the crash-after-save case, journal kept on failure. - ExecuteAsync kept its rollback catch active after the durable commit (settings save + journal clear). Any post-commit failure would restore the original in-memory graphs while the files are already moved and no journal remains to reconcile them — strictly worse than the failure itself. A committed flag now scopes the rollback to the pre-commit window; compaction and directory cleanup after the flag are best-effort post-commit maintenance. The new test walks the full durability matrix: undo with a failing checked save keeps the journal, a same-process recovery confirms the terminal state before clearing it, and a fresh settings reload proves the disk state is terminal.
|
两个事务边界都已修(a0fcd5b,3722/3722 绿),PR body 已同步到 6 commits / 3722 测试。逐条对账: 🔴 Blocker — 终态的 durable 确认完全按你给的修法: 新增测试
🟠 High — commit 后永不回滚
🟡 Low — PR body已同步:6 commits 结构(新增第 5 段 WAL finalize、第 6 段终态确认与 commit 边界)、测试数 3722/3722、undo durable 矩阵测试入清单。 五轮收敛下来的事务不变量现在完整闭环:receipts 是 transaction 状态时永不可动;删 journal 前对应 settings 状态必须 checked-durable(commit、终态、abandon finalize 三处同规);durable commit 之后永不回滚;WAL 标记配套 finalize 重放;宁不瘦身不让文件操作失败。等你下一轮最终合并审计。 |
问题
RecentOrganizationHistory只限制历史条目数(24),从不限制单条 entry 内的 receipt 数量。多文件大批量整理会无上界膨胀 settings.json——实测一个 1.5.3 真实档案达到 19.8MB(24 条 × 约 520KB 的完整 source/destination/identity receipt),放大两条链路:关键设计约束(两轮深审收敛)
OrganizationHistoryEntry.Items在 Desktop Organization 体系里四重角色:undo receipt、retry merge state、forward recovery 的 durable commit evidence(RecoverPendingAsync靠它识别已提交的移动)、结果页 UI 数据。retention 策略只能在 transaction boundary 之后动它。修复结构(6 commits,经五轮代码审阅迭代)
1. 预算与摘要(b98243b)
OrganizationHistoryPolicy:单条MaxUndoReceiptItemsPerEntry = 500+ 2500 条历史 receipt 预算(MaxUndoReceiptItemBudget——注意:active undo 与 journal 保护的 receipts 不计入预算,生命周期结束后由下一轮 compact 回收,极端情况总量可临时超过 2500);超限降级为摘要(持久化TotalItemCount记真实数量、清 Items、CanUndo=false)。Receipt 全有或全无:截断的 receipt 列表绝不保持可撤销。DesktopOrganizationTransaction与 settings load 不再各自裁剪。2. transaction boundary(ea4bbd8)
RecoverPendingAsync确认 journal 清除/不存在后执行(启动必经,老档案升级后首次启动自动瘦身)。IsUndoLifecycleActive:undo 进行中/中断的 entry(CanUndo && !IsUndone && (UndoStarted || 任何 IsRestored))一律不降级——ManagedDrop 的中断 undo 没有 journal,receipts 是唯一续命状态。UndoReceiptsDiscarded:retry merge 继承标记、TotalItemCount累加真实总数、永不重获CanUndo。DesktopOrganizationExecutionResult.CompletedItems快照供结果页。3. 全链路解耦与 commit 做实(c73c538)
OrganizerOperationResult:OrganizeDropAsync/MoveItemsBackToDesktopAsync返回{ History, CompletedItems }——普通文件格子 501+ 文件导入/移回的 UI 后处理(格子 upsert、排序插入、返回计数)不再读可能已被摘要化的History.Items;VM 两处 + 自动整理 watcher 已跟随。SaveCheckedAsync——ExecuteAsynccommit 失败即 throw(journal 保留,下次启动 recovery);undo 对账与 forward 恢复完成点失败保留 journal 重试(幂等);两个 abandon 流程失败 throw(调用方均有失败 UI)。静默保存失败不再可能删掉事务的唯一持久化记录。CanUndo=false+UndoStarted=false+ journal 清后立刻 retention,被放弃的 entry 永远不会卡在"active"状态逃过 compact。ApplyRetentionPolicy(history, protectedTransactionId)——pending journal 引用的 entry 保留 receipts 且永不成为 24 条裁剪的牺牲品;其余历史正常执行 500/2500/24 策略。AddHistoryEntryAsync(普通导入路径)现在每次执行全量策略(查共享 journal 获取保护 id),连续操作不再能在两次 compact 之间无限累积。4. durable 终态与保护顺序(731df77)
DesktopOrganizationRecoveryJournal.IsAbandoned持久终态标记——abandon 先把标记落盘再动 settings;RecoverPendingAsync拒绝执行 abandoned journal(forward 方向不会把用户选择保留的文件恢复回去),undo 对账分支拒绝终态 entry(!CanUndo || IsUndone,不会把已放弃/已完成的 undo 复活),UndoAsync防御性拒绝 abandoned journal。5. Abandon WAL finalize(20e08bc)
IsAbandoned从"跳过标记"补成完整 WAL:启动 recovery 跑TryFinalizeAbandonedJournalAsync——应用 settings 终态(undo 双 false / forward 的 widget/rule/目录 cleanup)→SaveCheckedAsync→ 失败 WAL 留存下次续做 → 成功才清 journal。ApplyAbandonedTerminalState/MarkUndoAbandoned被 Recover/AbandonPending/AbandonUndo 三入口共用。abandon 事务=写 WAL→应用终态→durable save→删 WAL,任何窗口崩溃下次从 WAL 续做。6. 终态 durable 确认与 commit 边界(a0fcd5b)
RecoverPendingAsync终态分支在清 journal 前先SaveCheckedAsync确认——undo 的 checked save 失败后内存终态不能冒充持久化终态(同进程点恢复也不再可能删掉 journal 留下旧磁盘态);已持久化场景幂等重存。ExecuteAsync加committed标志:durable commit(settings save + journal clear)之后任何 post-commit 失败永不回滚(compaction/目录清理为 best-effort 维护段),回滚严格限定在 commit 前窗口。真实数据验证
膨胀现场的 19.8MB settings.json 副本(env 门控测试
RealSettingsCopy_MigrationShrinksBloat,DESKBOX_HISTORY_MIGRATION_FIXTURE指向副本,不碰现场):测试(29 个新增/重写,全套 3722/3722 绿,x64)
LoadAsync从磁盘重载 → recovery 返回 0、文件原地不动、之后才 compactExecuteAsync抛IOException→ recovery journal 仍存在RecoverPendingAsync确认终态后才清 journal → 全新 SettingsService 从磁盘重载断言IsUndone=true/CanUndo=falseCompletedItems完整计数 + 持久化 entry 摘要化CompletedItems.Count==601+ 持久化摘要 + journal 清后 compactItemCountfallback;旧膨胀档案 load 不动 receipts / recovery 后瘦身 >10 倍不碰 / 后续
DestinationIdentity语义零改动;被保留 entry 完整保存 identity