ARTICLE DETAIL

资讯详情

深耕编程入门与网站建设的一线实战洞察。

Maka 托管依赖存储权威(Managed Dependency Storage Authority v1):一份以 SQLite 收据与 fail-closed 语义守护依赖树的实现契约

Maka 托管依赖存储权威(Managed Dependency Storage Authority v1):一份以 SQLite 收据与 fail-closed 语义守护依赖树的实现契约 Maka 托管依赖存储权威Managed Dependency Storage Authority v1一份以 SQLite 收据与 fail-closed 语义守护依赖树的实现契约【免费下载链接】makaApache Maka (Incubating) is a high-performance agent workspace that keeps a complete record of everything it did.项目地址: https://gitcode.com/GitHub_Trending/mak/makaApache Maka 将「托管依赖环境」managed dependency environment视为可重复物化、可崩溃收敛、可对抗篡改的受管资产。本文基于 docs/architecture/managed-dependency-storage-authority-v1.zh-CN.md 这份 implementation-contract 文档结合 packages/storage/src/managed-dependency-environment.ts 及其测试套件完整拆解ManagedDependencyEnvironmentAuthority的权威边界、发布顺序、崩溃恢复模型、身份与平台规则、Lease/配额/GC 语义以及它作为 M1.3 里程碑「堆叠地基」的平铺演进路径。读完本文你将掌握为什么 artifact 与 receipt 必须分离、为什么 rename 不能直接用作提交点、如何在进程崩溃与断电两种故障模型下各自收敛以及如何用真实 child-process 崩溃 failpoint 验证收敛契约。1. 核心不变量本模块只证明一件事文档开篇即声明整个模块只维护一个不变量同一个 canonical dependency environment identity 最多对应一棵由 Maka 发布、由 artifact 权限域之外的 durable receipt 证明、可在崩溃后收敛重开的依赖树任何身份、路径、内容或平台证据不可证明时 fail closed。翻译成工程约束就是同一 identity 最多一棵树多个消费者并发acquire同一个 environment只产生一份 authority-owned artifact测试「publishes one Maka-owned artifact for concurrent equivalent acquisitions」以provisionCalls 1验证。receipt 在 artifact 权限域之外证明依赖树内容的收据存放在专用 SQLite 数据库中而不是与node_modules/同目录从而杜绝「改 artifact 顺手改旁边 receipt」的协调式伪造。fail closed 是默认出口任何身份、路径、内容、平台证据无法自证一致时直接拒绝绝不自动接受或覆盖。该模块的 owner 是ManagedDependencyEnvironmentAuthority它独占以下能力environment identity 计算、artifact 发布、SQLite receipt、lease、pending reservation 与 GC。注入的 producer依赖安装器只获得一次性 staging 中的outputRoot与scratchRoot不接触 storage root、canonical artifact path 或 receipt database——这是「producer 是租客、authority 是房东」的职责切分。1.1 单实例所有权同进程与跨进程双重防重同一个 canonical storage root 在同一时刻只能存在一个 authority同进程activeAuthorityOwners是模块私有的Mapstring, object第二次createManagedDependencyEnvironmentAuthority会因「storage root already has an active owner」直接拒绝managed-dependency-environment.ts。跨进程在managed-workspaces/下打开稳定的 lock artifactdependency-environment-authority-v1.lock用fs-native-extensions的tryLock获取OS exclusive lock拿不到锁即拒绝managed-dependency-environment.ts。测试通过真实子进程持有 owner lock 验证第二进程被拒managed-dependency-environment-owner-child.ts。关键时序约束lock 在任何读取、修复、发布、租用、GC 之前取得只在 authority 完整关闭或初始化失败时释放若 close 因存在 active lease 被拒绝owner 不释放。1.2 本 PR 明确不包含什么文档逐项划清了边界避免过度承诺bundled npm 的实现与网络策略归 PR 2node_modules/.binsymlink 的 producer 配额扫描归 PR 2workspace execution owner/worker logical binding归 PR 4Desktop/CLI/Runtime Host 接线归 PR 4release packaging、audit 或 SBOM归 PR 3。因此该模块是stacked foundation在下一个生产 owner consumer 出现之前保持 Draft不得单独宣称 M1.3 用户能力闭环。2. 权威与原子性边界SQLite 收据 分阶段发布顺序2.1 存储布局权威事实receipt存放在 artifact 域之外的专用 SQLite databasemanaged-workspaces/dependency-environment-authority-v1.sqliteartifact 目录managed-workspaces/dependency-environments/64位sha256/只允许包含node_modules/不能携带同目录 receipt。测试「keeps the receipt in a constrained authority outside the producer-owned artifact domain」验证了environment-receipt.json在同目录不存在ENOENT而 SQLite 数据库可访问managed-dependency-environment.test.ts。数据库结构managed-dependency-environment.tsCREATE TABLE IF NOT EXISTS managed_dependency_environment_receipts ( environment_digest TEXT PRIMARY KEY NOT NULL CHECK ( length(environment_digest) 64 AND environment_digest NOT GLOB *[^0-9a-f]* ), receipt_json TEXT NOT NULL ) STRICT;打开数据库即执行PRAGMA synchronous FULLuser_version从 0 迁移到 1environment_digest用CHECK约束锁死为 64 位小写十六进制从 schema 层面拒绝畸形 digest。2.2 固定发布顺序发布顺序在文档中是一条不可打乱的操作链canonical identity validation - process owner claim cross-process OS lock - pending reservation - producer-owned random staging - producer provision resolves (PR 2 必须证明其进程树已退出) - deep-copy node_modules into new authority-owned inodes - delete the producer-owned tree - one authoritative tree seal: hash every entry fsync every regular file fsync directories bottom-up ADS/reparse validation - fsync artifact staging root - atomic rename artifact - fsync publication parent - SQLite receipt transaction (synchronousFULL) - reopen and revalidate artifact receipt - lease源码中的openOrPublishEnvironment忠实实现了这条链managed-dependency-environment.ts先尝试openPublishedEnvironment命中已发布环境则直接复用在stagingRoot下创建{digest}-{uuid}随机事务目录其中producer/project/node_modules是 producer 的outputRoot.maka-runtime是scratchRoot调用producer.provision(...)realpath校验 producer 输出未逃逸 staging authoritycp(producerOutputRoot, dependencyRoot, { recursive, dereference: false, verbatimSymlinks: true })深拷贝到 authority 新 inoderm(producerRoot)删除 producer 树hashDependencyTree(dependencyRoot, { durable: true })做一次性权威 sealhash fsync 合并于同一遍历原语syncDirectory(artifactStagingRoot)→rename(artifactStagingRoot, artifactRoot)→syncDirectory(environmentsRoot)receiptAuthority.write(receipt)SQLitesynchronousFULL事务删除事务目录requirePublishedEnvironment重开重验后返回。失败路径同样完整任何一步抛错都会先清理transactionRoot再竞态检查是否已有并发方发布成功raced若 receipt 不存在则连 artifact 一起删除后抛出——绝不留半成品。2.3 为什么不能把 rename 当作原子提交文档明确filesystem rename 与 SQLite 不是共同事务恢复依赖可收敛状态而非假装原子。四种组合的唯一合法结果artifactreceipt处理存在不存在删除 orphan artifact重新物化不存在存在删除 orphan receipt存在存在且一致重新验证后复用存在存在但不一致fail closed不自动接受或覆盖cleanupIncompletePublicationsmanaged-dependency-environment.ts在 authority 启动时执行前两种清理且对目录名不是 64 位 hex、或存在 symlink/reparse point 的缓存条目直接抛错——unowned cache entry 即启动失败。3. 身份、路径与平台规则3.1 identity 由权威重算调用者不能自证authority 根据 manifest/lockfile bytes、package manager、Node ABI、platform/arch、producer runtime/policy 重新计算 identity调用者传入的environmentId若不等于重算结果直接以identity is not canonical拒绝assertCanonicalIdentitymanaged-dependency-environment.ts。identity 计算使用域分离domain separation防串扰managed-dependency-environment.tsenvironmentId sha256(maka.managed_dependency_environment.v1\0 || canonicalIdentity)其中canonicalIdentity是 manifest/lockfile 路径与各自 SHA-256、包管理器名与版本、Node 版本与 ABI、平台、架构、producer runtime identity、producer policy identity、policy version 的规范化 JSON。测试验证了等价输入得到相同 identity改一个nodeAbi、platform、producerRuntimeIdentitySha256或producerPolicyIdentitySha256都会得到不同 identitymanaged-dependency-environment.test.ts。3.2 路径与 digest 规则publication digest 必须是固定 64 位小写十六进制 SHA-256所有join/realpath/rename之后重新检查 containmentisPathWithin。normalizeTrackedPath拒绝.、绝对路径、..、../前缀只接受 workspace 相对路径assertIdentityText拒绝空串与含 NUL 字节的输入。伪造environmentId/path traversal 在任何文件写入前即被拒绝测试验证sha256:../../escaped不产生任何逃逸文件。3.3 符号链接POSIX 宽容、Windows 零容忍Linux/macOS相对 symlink 只有在目标仍位于 dependency root 内时允许link path 与 target 都进入 tree digest绝对链接或逃逸链接拒绝hashDirectory中isPathWithin(resolve(dirname, target), root)校验。测试「accepts a POSIX package bin symlink」验证.bin/fixture-cli - ../fixture-package/cli.js可正常发布与读取managed-dependency-environment.test.ts。Windows拒绝 symlink/reparse point通过系统绝对路径\?\namespacedPowerShell 枚举检测 NTFS named streamADShashDependencyTree对 win32 额外调用assertNoWindowsAlternateStreams。3.4 平台持久性语义差异进程崩溃 vs 断电所有平台统一要求authority-owned 普通文件先完成平台可用同步内部目录按子目录到父目录顺序同步随后同步 staging rootrename 后再同步 publication parent最后才允许提交 SQLite receipt。tree hash 与 durable seal必须由同一个遍历原语完成不允许维护两套可能漂移的目录语义。Linux只有在文件系统与硬件兑现 file/directoryfsync合同时上述顺序才覆盖断电恢复。macOS普通fsync不等价于F_FULLFSYNC且本协议未同时为 artifact 与 receipt 启用并证明 full-sync ordering。WindowsNode 文件系统接口无法提供与 POSIX 等价的目录项持久性证明。因此macOS 与 Windows v1 只承诺进程崩溃收敛不承诺断电后 artifact publication 自动收敛断电后若 artifact/receipt 不一致必须 fail closed通过删除缓存后重新物化或人工清理恢复。文档特别提醒绿色的 child-process crash test 不能充当 power-loss 证明。3.5 Windows 只读依赖文件Windows 上只读 dependency file 是合法输入。authority 只在自己的 unpublished staging inode 上临时增加 owner-write 权限syncRegularFile中chmod(path, originalMode | 0o200)以完成文件同步并在继续 publication 前恢复原始 modemanaged-dependency-environment.ts。producer inode 与最终 published mode 都不得被永久改写任何同步或权限恢复失败都必须在 receipt 前 fail closed。测试「publishes Windows read-only dependency files without changing their final mode」验证发布后文件mode 0o222 0仍只读。3.6 POSIX 保留可写 fd 的隔离Linux/macOS 上 rename不会撤销producer 保留的 writable file descriptor因此 producer tree绝不直接 rename 成 artifact。authority 必须复制到新 inode、删除 producer tree、再做完整内容证明旧 descriptor 后续只能修改已 unlink 的 producer inode。测试「isolates published POSIX content from a producer-retained writable handle」在发布后通过保留句柄写入MALICIOUS\n并 truncate/sync断言 authority-owned artifact 内容仍为trusted\nmanaged-dependency-environment.test.ts。配套测试「publishes authority-owned file inodes instead of producer-owned inodes」用dev:ino对比证明发布后 inode 已切换。4. Lease、配额与失败状态4.1 生命周期open - draining - closedclose()先把状态置为draining拒绝新的 acquisition再等待所有已接纳 acquisition 完成 lease 安装或失败。drain 完成后若仍存在 active leaseleaseCounts.size 0close明确拒绝并恢复open不能先关闭 receipt owner 再返回活 leasemanaged-dependency-environment.ts。测试「close drains an acquisition through lease installation before deciding its outcome」用 failpoint 卡在before_environment_lease验证 close 会等待 lease 安装完成、发现活 lease 后报still has active leases并恢复 openmanaged-dependency-environment.test.ts。4.2 pending reservation 与 GC 保护acquire开始即安装 pending reservationpendingCounts直到正式 lease 建立或失败清理GC 不得删除 pending/inflight/leased digest。测试「does not collect a published environment while its acquisition is still pending」在maxCacheBytes: 0的极端配额下验证 pending 环境不被回收managed-dependency-environment.test.ts。同一 digest 的并发 acquisition 共享同一个 inflight 任务inflightMap发布只执行一次。4.3 配额与 GCcache cost 内容字节数 每个 entry 的固定治理成本源码中每个 entry 记账contentBytes contentEntries * 40964 KiB 保守元数据单位避免大量空文件绕过软配额managed-dependency-environment.ts。GC 按 LRUmtime排序、串行执行一次 GC rejection 可报告给当前调用者但不能永久毒化后续 GC task chaingcTask gcTask.catch(() undefined).then(...)。默认配额maxCacheBytes为 2 GiB2 * 1024 * 1024 * 1024非负安全整数校验在创建时完成。4.4 启动失败语义authority startup 遇到 malformed receiptdecodeReceipt严格校验 18 个收据键的完整集合与类型或 unowned cache entry 时整体 fail closedQuarantine 会引入新的 durable lifecycle本 PR 不在静默增加。5. 崩溃与对抗矩阵文档用一张矩阵锁定每个对抗场景的唯一合法结果场景唯一合法结果producer 中断staging 清理无 artifact/receipt完整 tree durable seal 前中断无 receipt重启清理 staging/orphantree seal 完成、artifact rename 前中断无 receipt重启清理 staging/orphanartifact rename 后进程退出重启删除无 receipt artifact再物化receipt commit 后进程退出重启重验并复用Linux receipt 前断电receipt 不得领先已同步的完整 artifact treemacOS/Windows publication 期间断电v1 不承诺自动收敛不一致时 fail closedWindows dependency file 为只读staging 临时提权同步后恢复 mode再继续发布artifact 与同目录伪 receipt 一起修改外部 SQLite receipt 不变acquire 拒绝伪造 environmentId/path traversal任何文件写入前拒绝Windows ADS/reparsepublish/reopen 拒绝acquisition pending 时 GCpending digest 保留cache 中任意已发布内容漂移新 acquisition fail closed同进程第二个 authority接触 receipt/artifact/GC 前拒绝另一进程持有同 root authorityOS lock 处拒绝不得观察或删除其 leaseclose 与 lease 安装并发drain 后因 active lease 拒绝 closePOSIX producer 保留 output writable fdlate write 不得改变 authority-owned artifact「协调式伪造」场景尤其值得注意测试同时改写 artifact 内容并放置同目录伪environment-receipt.json重开 authority 后 acquire 依然以artifact contains an unowned entry|content does not match its receipt拒绝——因为真 receipt 在 SQLite 里内容树 hash 对不上managed-dependency-environment.test.ts。6. 源码级实现证据核心实现集中在 packages/storage/src/managed-dependency-environment.ts约 1470 行关键构件Producer 封闭策略MANAGED_DEPENDENCY_PRODUCER_POLICY_V1冻结为hermetic_dependency_builder_v1固定network: registry_https_only、filesystem: maka_owned_staging_only、secrets: none、childProcess: verified_runtime_only、lifecycleScripts: disabledcreateManagedDependencyEnvironmentProducerCapability只接受 SHA-256 runtime identityassertProducerCapability逐键校验 producer 声明的能力与期望完全一致任何偏差如把 network 改成unrestricted在创建 authority 时即拒绝managed-dependency-environment.test.ts。单一遍历原语hashDependencyTree同时完成「排序遍历 内容 hash durable fsync」目录与文件类型以d\0/f\0/l\0前缀区分并计入 digest杜绝两套语义漂移managed-dependency-environment.ts。Windows ADS 检测Node 侧无 reparse follow 地收集路径清单通过 stdin 交给一次非递归 PowerShellFindFirstStreamW/FindNextStreamW的 C# 内联实现::$DATA之外的任何 stream 即拒绝脚本带 120 秒超时与 1 MiB 输出上限避免 GitHub-hosted runner 冷启动 csc 编译超时被误报managed-dependency-environment.ts。Failpoint 契约ManagedDependencyEnvironmentFailpoint暴露after_environment_tree_durable/after_environment_publish/after_environment_receipt_durable/before_environment_lease四个边界测试可精确验证 durable seal、publish、receipt 提交、lease 的先后顺序managed-dependency-environment.ts。测试证据位于managed-dependency-environment.test.tsidentity 确定性、能力校验、单/多进程单 owner、篡改与协调式伪造拒绝、identity 伪造拒绝、ADS 拒绝3 例、POSIX bin symlink 接受、inode 交接、durable seal 顺序、只读文件 mode 保持、保留 fd 隔离、并发去重、close drain、LRU GC、pending GC 保护。managed-dependency-environment-crash.test.ts跨进程 owner 拒绝 四个崩溃收敛场景。fixturemanaged-dependency-environment-crash-child.ts生产形态 child-process 崩溃、managed-dependency-environment-owner-child.ts真实跨进程 owner lock。7. Extraction ledger 与平铺 PR 顺序文档作为实现契约附带了从旧集成分支codex/managed-workspace-environment-provisioning-m1-3提取的提交归属表extraction ledger核心映射来源提交平铺归属处理方式afb798092feat(storage): add managed dependency artifact authorityPR 1迁移 authority、identity、receipt、crash fixture 与最小测试57aba6bb7fix(runtime): bind managed dependencies to producer authorityPR 1/4仅提取 storage authority 的 producer capability/identity 硬化cff6fc476fix(storage): harden managed dependency authorityPR 1提取外置 SQLite receipt、路径/平台校验、pending/GC 与对抗测试89c9e0a3cfeat(runtime-host): ship verified bundled npm environmentsPR 2/3/4producer 隔离归 PR 2供应链归 PR 3host 组装归 PR 48150e90a7fix(runtime-host): constrain bundled npm provisioningPR 2/3provision policy/quota 归 PR 2shipped runtime verification 归 PR 327f8f6b8efix(release): verify shipped bundled npm closurePR 3release audit、manifest、notice 与 packaged smoke16e10b3c5feat(storage): bind dependency environments to managed scopesPR 4owner-bound lease、logical binding 与 read-only worker consumer9a42a761cfix(runtime-host): transport packaged dependency authorityPR 4Desktop/CLI/Runtime Host 生命周期与关闭顺序980143a3bdocs(architecture): split M1.3 authority boundaries文档证据由每个平铺 PR 维护自己的 owner/rollback 合同后续平铺顺序固定为PR 1本文定义的 storage authority即本模块PR 2producer boundary必须证明 producer 进程树退出后provision()才 resolve并包含真实 npm.binsymlink 的配额与取消/超时测试PR 3bundled npm runtime 供应链、发布审计与许可证材料PR 4唯一生产 consumer把真实 baseline、environment lease 和 Read/Glob/Grep worker 串成闭环。两个关键约束值得重申PR 1–3 只是堆叠地基PR 4 合并前不能把 M1.3 描述为用户可用且PR 2 不得通过拒绝所有 POSIX symlink 来规避.bin——producer inventory 应计量合法 link最终 storage authority 与 worker 仍负责 target containmentWindows 继续拒绝 reparse point。8. 验证要求与不变量收尾文档规定验证时必须执行storage build、focused authority tests、真实 Windows ADS test以及三个 child-process crash failpoint。后续 PR不得通过放宽本 authority 的 fail-closed 规则来提高采用率——这是该实现契约最硬的一条底线。综合文档与源码可以总结出本模块的设计哲学把「持久化提交」从「文件系统原子性」中解放出来改用「可收敛状态机 域外收据 全量内容证明」换取跨平台、跨故障模型的确定性。authority 在发布链路上不依赖任何单一原子的 rename 或事务跨越文件系统与数据库而是确保任何一个崩溃点之后重启都能通过「清理 orphan / 重验复用 / fail closed」三条路径之一收敛到合法状态在 macOS/Windows 上主动降级承诺进程崩溃收敛而非断电收敛并用 failpoint 测试把每个边界都钉死在契约上。这正是 Maka 把依赖环境当作可审计资产而非一次性缓存目录的根本原因。参考路径速览契约文档docs/architecture/managed-dependency-storage-authority-v1.zh-CN.md核心实现packages/storage/src/managed-dependency-environment.ts功能测试packages/storage/src/tests/managed-dependency-environment.test.ts崩溃收敛测试packages/storage/src/tests/managed-dependency-environment-crash.test.ts崩溃 fixturepackages/storage/src/tests/fixtures/managed-dependency-environment-crash-child.tsowner fixturepackages/storage/src/tests/fixtures/managed-dependency-environment-owner-child.ts所属包packages/storage/package.json【免费下载链接】makaApache Maka (Incubating) is a high-performance agent workspace that keeps a complete record of everything it did.项目地址: https://gitcode.com/GitHub_Trending/mak/maka创作声明:本文部分内容由AI辅助生成(AIGC),仅供参考
返回列表