Skip to content

Vendored Maven jar rebuild is not byte-reproducible across OSes (Windows differs) #275

Closed
Closed
Vendored Maven jar rebuild is not byte-reproducible across OSes (Windows differs)#275
Labels
agent:triagedbugSomething isn't workingpriority:p3

Description

@mikolalysenko
Mikola Lysenko (mikolalysenko)
opened on Sep 26, 2026

[agent] Filed by Claude Code on behalf of Mikola Lysenko (@mikolalysenko) while adding Maven patch SBOM annotations to depscan.

Summary

The vendored Maven jar that socket-patch rebuilds locally is not byte-reproducible across operating systems. From the same inputs, Windows writes a jar with the same size but different bytes than Linux and macOS. The different jar sha256 lands in .socket/vendor/state.json (artifact.sha256) and in the committed .jar.sha1. A project vendored on Windows and re-vendored on Linux therefore churns its committed tree.

Evidence

depscan run https://github.com/SocketDev/depscan/actions/runs/36261631322 used the same socket-patch revision 3efdc31d built per OS, the same patch records, the same Maven Central upstream jars and a JDK 17 runner. It covered 13 Maven lines on ubuntu-latest, 9 on windows-latest and 3 on macos-latest. Each PASS cell's committed tree was compared with the macOS arm64 capture of the same (Maven version, shape, mode):

OS identical differs only in vendoredAt jar digest in state.json differs
Linux 432 628 0
macOS 102 148 0
Windows 300 109 327

All 327 Windows cells are in the local-rebuild vendored modes: get --mode vendored --vendor-source build, scan --mode vendored --detached --vendor-source build, and get --save-only followed by vendor --offline. The --vendor-source service cells, which use the prebuilt jar, are identical. Hosted trees are identical on every OS.

The rebuilt jar was not uploaded, so which zip fields differ is not verified. Likely candidates are the per-entry "version made by" host byte, the external file attributes (Unix mode bits versus DOS attributes), or entry-name normalization of \ separators.

Expected

vendor/maven_repo.rs describes the rebuild as a deterministic re-zip. The same inputs should give the same jar bytes on every OS, so that committed vendored trees are stable across a team's machines.

Repro

Run socket-patch get <uuid> --mode vendored --vendor-source build on the same Maven project, with the same ~/.m2 jar, on Windows and on Linux, then compare sha256sum .socket/vendor/maven/<uuid>/**/*.jar. The depscan harness workspaces/pipeline/src/test/integration/socket-patch-sbom/maven-compatibility/capture-maven.py on PR SocketDev/depscan#26855 automates this per OS.

Note: the open perf PR #257 replaces the on-disk rebuild with an in-memory repack (stage_local_jar / MemoryRepack), so a fix may belong there.

Activity

  1. mikolalysenko commented on Sep 26, 2026

    @mikolalysenko
    Mikola Lysenko (mikolalysenko)
    on Sep 26, 2026
    CollaboratorAuthor

    This is very likely the zip "version made by" host byte. PR #257 fixes it in f9fc676 ("fix(vendor): stamp rebuilt zips with a Unix host on every platform"). The Windows test job there caught the same symptom on rebuilt wheels and nupkgs.

    • Cause: zip 8.x stamps the central directory's host as DOS on Windows builds and as Unix everywhere else. A jar/wheel/nupkg rebuilt on Windows therefore has the same size but different bytes, and readers are told to ignore the Unix modes it carries.
    • Fix: vendor/common.rs write_zip_entries now pins .system(zip::System::Unix). On Speed up hosted and vendored scans: concurrent API requests, parallel crawl, single-pass rewriters #257, every production local rebuild (maven local_rebuild_jar, nuget local_rebuild, pypi wheel builds) writes through that one function.
    • Test: write_zip_entries_stamps_a_unix_host_on_every_platform pins the host byte and modes, and it runs on the Windows CI leg.
    • Scope: Unix output is byte-identical to before; only Windows-built archives change.

    To confirm, re-run the capture matrix against #257's head. The local-rebuild vendored cells should come out identical across windows-latest, Linux and macOS.

    🤖 Generated with Claude Code

  2. mikolalysenko commented on Sep 26, 2026

    @mikolalysenko
    Mikola Lysenko (mikolalysenko)
    on Sep 26, 2026
    CollaboratorAuthor

    [agent] Confirmed fixed on #257 head f9fc6765: depscan run https://github.com/SocketDev/depscan/actions/runs/36265269179 built the CLI per OS. It ran the full Maven capture matrix (25 jobs, Maven 2.2.1 to 4.0.0-rc-7 on Linux, Windows and macOS) and every job was green. At Maven 3.9.16, all 55 vendored cells now record identical jar sha256 values in .socket/vendor/state.json on ubuntu-latest, windows-latest and macos-latest. At 3efdc31d, 327 Windows cells differed.

  3. mikolalysenko commented on Sep 30, 2026

    @mikolalysenko
    Mikola Lysenko (mikolalysenko)
    on Sep 30, 2026
    CollaboratorAuthor

    [agent] Fixed on main by #257 (squash-merged as 8c381ec). crates/socket-patch-core/src/vendor/common.rs write_zip_entries now pins .system(zip::System::Unix) (line 252), so rebuilt jars/wheels/nupkgs get the same host byte on every OS. The regression test write_zip_entries_stamps_a_unix_host_on_every_platform (same file, line 2422) runs on the Windows CI leg. The cross-OS Maven capture matrix confirmation is in the comment above. Closing as completed.

    Generated by Claude Code

  4. Mikola Lysenko (mikolalysenko)
    added
    bugSomething isn't working
    priority:p3
    agent:triaged
    on Sep 30, 2026
  5. Mikola Lysenko (mikolalysenko)
    closed this as completedon Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent:triagedbugSomething isn't workingpriority:p3

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions