Vendoring

# Multi-file packages

A vendored package is one file under a digested asset path. Plenty of packages need more than that, and pin reads every download before it writes one.

## Why one file isn't always enough

A digested asset path is not the directory the package was published to.

[`pin`](https://importmap-plus.zoolutions.llc/docs/pinning) downloads a package to `vendor/javascript/<package>.js` and the asset pipeline serves it under a digested path. That works as long as the file is self-contained. Plenty of published packages aren't: the entry point imports a sibling by relative path, spawns a worker, reads `import.meta.url` to find its own directory, computes an `import()`, or fetches a `.wasm` binary beside itself.

Every one of those references is resolved by the browser against the file's own URL — which is the digested asset path, not the CDN directory the package was published to. Neither Propshaft nor Sprockets rewrites `import` statements, so the request goes out for a file that was never vendored and comes back 404. Nothing warns you before the page.

So `pin` reads what it downloaded before it writes anything to `vendor/javascript`, and takes one of three routes: vendor the entry alone, vendor it together with [the file graph it needs](#vendoring-the-file-graph), or [leave the package on its CDN](#packages-kept-remote) and record why.

## What the check looks for

Five patterns, all of them reported.

`relative imports` is the one a download can answer for itself, by bringing the files along. The other four have no answer: they name a path nobody can rewrite, so the package stays on its CDN.

| Reason | What was found |
| --- | --- |
| `relative imports` | An `import` or `export` from `"./x"` or `"../x"`. Vendored with the entry when the whole graph can be; the reason on a pin when it couldn't be. |
| `dynamic imports` | An `import()` of something other than a string literal, so what it loads isn't knowable here. |
| `workers` | `new Worker(…)` or `new SharedWorker(…)`. A worker is fetched as its own script and never goes through the import map. |
| `import.meta.url` | The file asking for its own URL, which is a digested asset path, not the directory the package was published to. |
| `wasm` | A string naming a `.wasm` binary, fetched at runtime from a path that isn't there. |

The check reads the source with regular expressions rather than parsing JavaScript. Block comments and regex literals are discounted first — a published bundle is full of `/** @typedef {import('./slide.js').Slide} Slide */`, naming files it never loads — but string literals are read as they stand. So a string containing `import "./x.js"` keeps a package remote, while a bare `const path = "./x.js"` matches nothing: the first four patterns are anchored on the keyword. Only `wasm` matches a plain string, since a `.wasm` path needs no keyword to be fetched.

That asymmetry is deliberate, because the two mistakes are not equal: a package wrongly kept remote still works, while one wrongly vendored 404s in the browser. [`--vendor`](#overriding-the-check) is the way to say you know better.

## Vendoring the file graph

The whole graph is downloaded, and one pin_all_from line maps it.

Everything an entry reaches through relative imports lives under the package's own version directory on the CDN, which is a closed, finite set. `pin` crawls it, and rewrites every relative specifier to the bare key `<package>/<path without the extension>`:

```console
$ ./bin/importmap pin @popperjs/core@2.11.8
Pinning "@popperjs/core" to vendor/javascript/@popperjs/core.js via download from https://ga.jspm.io/npm:@popperjs/core@2.11.8/lib/index.js (with 47 sibling files)
```

```ruby
pin "@popperjs/core", to: "@popperjs--core.js" # @2.11.8
pin_all_from "vendor/javascript/@popperjs--core", under: "@popperjs/core", to: "@popperjs--core" # @2.11.8 (graph of @popperjs/core)
```

The entry keeps the flat file and the plain pin it always had, so `update`, `outdated`, `lock` and `pristine` see it exactly as before. Its siblings go into a directory beside it, and the one `pin_all_from` line maps every file in that directory to the key its specifiers were rewritten to:

```
vendor/javascript/@popperjs--core.js               # the entry: "@popperjs/core"
vendor/javascript/@popperjs--core/lib/enums.js     # "@popperjs/core/lib/enums"
vendor/javascript/@popperjs--core/_/a0ba12d2.js    # "@popperjs/core/_/a0ba12d2"
```

Every file in that directory is an entry in the import map, which is inlined in every page: `@popperjs/core` contributes 48 entries — the 47 files plus its own — and `date-fns` about 250. A directory also wins over a `pin` of the same key, so pinning a subpath of a package you have vendored this way has no effect — `pin` says so when it happens. `pin <package> --remote` is the way back to a single CDN URL.

The keys are written under the package the CDN *URL* names, which is not always the package your pin names: jspm resolves Node's `buffer` and `crypto` into files in `@jspm/core`, so two such pins each keep their own directory and map the files they share to the same key — the browser evaluates each module once. If those two pins ever sat at different versions of that package, one of them would win for every file they share, and `pin` says so when it writes the second line.

Bare specifiers are left alone: a chunk that imports `"react"` still resolves through your import map to your pin of react. A file that is another pin's own entry isn't copied either — the specifier is rewritten to that pin's key, so the browser evaluates the module once. Two pins of one package do each keep their own copy of a chunk they share; both map it to the same key, so one copy is what every importer gets and the other is dead weight on disk.

The directory belongs to the pin that wrote it — the `pin_all_from` line with the `(graph of …)` comment is what says so — and is replaced whole on every `pin`, `update` and `pristine`, so a file the package dropped between versions goes with it. A directory of that name the import map doesn't map that way is yours: `pin` skips the package, says so, writes nothing and exits non-zero rather than renaming it away — the same as when a CDN fails partway through a crawl, where the pin and whatever files it had are left exactly as they were. `unpin` takes the directory and its line along with the pin, and [`--vendor`](#overriding-the-check) drops both.

Only jspm, jsDelivr and unpkg are crawled: their URLs say which package and version a file belongs to, and so where the package's directory ends. A download from esm.sh or skypack, or from a URL you wrote yourself, stays remote. So does a package whose graph can't be taken over whole — a relative path that climbs out of the package, a sibling the CDN hasn't got, a sibling that isn't JavaScript, or two files that would collapse to one key. Refusing the whole package is deliberate: a remote pin works, where a graph with a hole in it is a page that doesn't load.

## Packages kept remote

Pinned to the CDN instead of downloaded, with the reason on the pin.

A download the file graph can't answer for — any reason but `relative imports`, or a graph that couldn't be taken over whole — is pinned to its CDN URL instead, and the pin comment records why:

```console
$ ./bin/importmap pin fflate@0.8.2
Pinning "fflate" to https://ga.jspm.io/npm:fflate@0.8.2/esm/browser.js (kept remote: workers) (integrity sha384-LHsueLoV1K9WgSy1O7/1wS6dZkUWIrvJ0c//VrNmLohwlHd7TN2mAzZZJbBUdU29)
```

```ruby
pin "fflate", to: "https://ga.jspm.io/npm:fflate@0.8.2/esm/browser.js", integrity: "sha384-LHsueLoV1K9WgSy1O7/1wS6dZkUWIrvJ0c//VrNmLohwlHd7TN2mAzZZJbBUdU29" # @0.8.2 (remote: workers)
```

The URL is fetched once to hash it, so the pin carries a [subresource-integrity](https://importmap-plus.zoolutions.llc/docs/integrity) hash of the bytes the browser will get. The `remote:` detail and the reason are part of the [provenance grammar](https://importmap-plus.zoolutions.llc/docs/provenance), which means later commands can read them back.

From then on the pin behaves like any other [remote pin](https://importmap-plus.zoolutions.llc/docs/pinning#pinning-to-a-remote-url): `pin` and `update` re-resolve it from the same CDN and keep the reason, and `pristine` skips it. The vendored file an app already has is never removed by a refusal — the download is inspected before anything on disk is touched.

## Packages the CDN hands back as CommonJS

A UMD or CommonJS bundle exports nothing through an import map.

jspm and `esm.run` build an ES module out of whatever a package ships. The CDNs that serve a package's own `dist` file do not, and plenty of packages still publish a UMD bundle there. Loaded through an import map it runs and exports nothing, so `import x from "pkg"` fails to link — the browser reports that the module provides no export named `default` and refuses to run anything that imported it. Nothing warns you before the page.

A download that says it is CommonJS is pinned to its CDN URL instead, with the reason on the pin, the same way a file that can't stand alone is:

```console
$ ./bin/importmap pin google-libphonenumber@3.2.42 --from jsdelivr
Pinning "google-libphonenumber" to https://cdn.jsdelivr.net/npm/google-libphonenumber@3.2.42/dist/libphonenumber.js (kept remote: not an ES module) (integrity sha384-5MZuONtIM2yEzrksaOeJteNW8bmvGovwqxtzk2kVpXG+CWbh7A7KXOaZmBFdxadr)
```

```ruby
pin "google-libphonenumber", to: "https://cdn.jsdelivr.net/npm/google-libphonenumber@3.2.42/dist/libphonenumber.js", integrity: "sha384-5MZuONtIM2yEzrksaOeJteNW8bmvGovwqxtzk2kVpXG+CWbh7A7KXOaZmBFdxadr" # @3.2.42 (remote: not an ES module)
```

A file counts as an ES module when it has a top-level `import` or `export` statement, or when it never says it is CommonJS or AMD — no `module.exports`, no `exports.x =`, no `require(`, no `define(`, no `typeof exports == "object"` loader sniff. The two halves read the source differently on purpose: an `import` statement is believed only outside a string literal or a line comment, because a bundle shipping a usage example in either is still CommonJS, while a `module.exports` is believed wherever it appears.

The default `pin` never reaches this, since jspm converts the package for you. `--vendor` downloads it anyway when you know better, and dropping `--from` lets jspm do the conversion.

## Overriding the check

--vendor downloads it anyway; --remote skips the download entirely.

```console
$ ./bin/importmap pin fflate@0.8.2 --vendor
Pinning "fflate" to vendor/javascript/fflate.js via download from https://ga.jspm.io/npm:fflate@0.8.2/esm/browser.js
```

```ruby
pin "fflate" # @0.8.2 (vendored)
```

`--vendor` downloads the entry file and nothing else: a package that had a graph directory loses it, line and all, because overriding the check is saying you want that one file.

It also converts a pin that was kept remote back to a download. The `vendored` mark is what makes the override stick: without it the next `update` would inspect the new download, refuse it again and quietly undo your decision.

It applies only to the packages you name, not to the dependencies a CDN resolves alongside them: `pin pdfjs-dist --vendor` vendors pdfjs-dist, while a dependency of it that can't stand alone is still checked and kept remote, so the override can't quietly vendor something you never asked about. Pass `--remote` and `--vendor` together and [`--remote`](https://importmap-plus.zoolutions.llc/docs/pinning#pinning-to-a-remote-url) wins — it names a destination, where `--vendor` only overrides a check. For the same reason it leaves a pin on a host no CDN answers for alone, reporting it as [skipped](https://importmap-plus.zoolutions.llc/docs/pinning#custom-urls) rather than vendoring whatever the package spec resolved to.

Packages an app already vendored before this check existed are not rewritten on their own, and `bin/importmap pristine` downloads them again exactly as their pins say. The next `pin` or `update` that touches one does re-resolve it, though, and converts it if its file can't stand alone — which is the fix arriving rather than a surprise, since that vendored file was already 404ing for the siblings it wanted.

## Proving it before the browser does

doctor reads the map and the files, offline.

The check runs when a package is pinned. What it can't see is an app that was assembled over years — a package vendored before the check existed, a file hand-edited, a pin whose sibling directory somebody deleted. [`doctor`](https://importmap-plus.zoolutions.llc/docs/updating#doctor) is that half: it boots the app and checks the import map against the files that are actually there, and a vendored file still importing a sibling by relative path is one of the things it reports — with the command that fixes it.

```console
$ ./bin/importmap doctor
error    vendor/javascript/popper.js imports "./enums.js" by relative path — run bin/importmap pin @popperjs/core to vendor its files
1 error, 0 warnings
```

It exits non-zero on an error, so it belongs in [CI](https://importmap-plus.zoolutions.llc/docs/updating#checking-the-app-in-ci) — which is where a 404 that only happens on one page is cheapest to find.