Add the virtual: module path namespace

[robhogan]

Currently all modules in a Graph are keyed by a module's absolute real path on disk, with the exception of context modules (our only "virtual" module at present), which use a distinguishing ?ctx=... suffix and some bespoke wiring.

The path is therefore the module's build-time identifier. We use it as keys in change sets in the delta calculator and HMR, and as the only input to module ID factories (serializer.createModuleIdFactory). So we need a way to guarantee virtual and real paths do not clash.

This reserves virtual:<absolute path> as that identifier, which maps to ?virtual=true for URLs referring to modules.

Exposure

A module path becomes user-visible in development module names, HMR source URLs, server-URL source paths and async bundle paths. Dev names (for visual inspection of the bundle) use virtual:<relative path>, and URLs are the ones a file at the virtual path would have, plus a virtual=true query parameter.

getAllFiles leaves virtual modules out, since they have no file.

The server can't serve a virtual module by URL yet, and the path in its URL may be a real file's. So bundle and map requests with the parameter fail, and source requests with it are passed on rather than served from disk.

Nothing produces a virtual: path until #2036, so existing bundles are unchanged.

Design

Why virtual: for file paths

• No potential conflict with a real posix path (in particular, /foo.js?virtual is a valid file name in posix)
• Stripping the prefix gives exactly the path the transformer and resolver see, with no lookup
• Preserves basename, extension and project-relative path - all inputs to the transformer, which does not need to know that a module is virtual.
• Legible (this is the main advantage over a \0 prefix, which has all of the same tradeoffs.)
• Extensible to other potential namespaces
• Maps cleanly to virtual:///…, mirroring file:///…, if module identities ever become URLs (which I'd like!)

Why a namespace rather than a virtual flag

• Module id factories, delta deleted lists and inverse dependencies only ever see the path string
• Watcher events are matched to modules by path, so a file created later at a virtual path mustn't hit the virtual module
• No API that takes a module path changes shape

Why ?virtual=true for URLs

• Readable, and ideally distinguishable as a virtual module, where source URLs appear eg in stack traces
• Friendly appearance in dev tools source listings
• Works with every existing pathname form ([metro-project], [metro-watchFolders]/N, server-root-relative) without stacking routes
• Keeps the extension at the end of the pathname
• The path says where the module notionally lives, the query says how to read it
• A virtual module beside a real file at the same path still gets a distinct HMR URL and source path
• Leaves a potential route to serving them (not yet planned): map the pathname back as for a file, then look up virtual:<path>

Tradeoffs

• Naive node:path API uses eg path.resolve interpret the virtual:/absolute/path as a relative path from the cwd (this is also true of \0 prefix, or any prefix).
• Doesn't align with either Expo's \0 prefix or Metro's current ?ctx= - but both can be aligned.

Changelog: Internal

Test plan

New unit tests for the helpers, plus HMR, dev name, async path, getAllFiles and url-server source map cases for a virtual module, and for the server's handling of virtual module URLs.

In particular, a virtual module at the same path as a real file gets distinct HMR URLs and source paths, and a request for its source isn't served from that file.

[vzaidman]

How far are we from migrating to virtual:/// ?

[robhogan]

How far are we from migrating to virtual:/// ?

I don't have any short term plans, but I think there'd be value in Metro moving to project-relative paths represented as URLs (giving us schemes, query parameters, uniform posix, portability), instead of absolute paths everywhere, and it'd follow from that.

I haven't gone with that in this PR because virtual:///<URL encoded path> alongside <unencoded absolute path> would be more confusing IMO. Better to be consistent with raw file paths everywhere and just add a distinguishing prefix that's easily stripped (virtual:<absolute path> is technically also a valid URL, just not file-like).

Changing absolute paths to URLs would be a pretty significant breaking API change though unfortunately - these paths leak out everywhere, which is what made even this change tricky.

[vzaidman]

Are these urls exposed to any plausible use outside of Metro? If so, adding "virtual:" is obviously a breaking change as well.