This PR converts `generate-lavamoat-policies.sh` to `.js` using Yargs. This makes it easier to only generate policy files for a specific build type (using the `-t` flag), which is often useful during Flask development. In addition, the `lavamoat:background:auto` scripts are renamed, and the main readme is updated with some useful tips.
Note that `lavamoat:background:auto:dev` is removed and `lavamoat:background:auto` should be used during local development.
* Automate the Flask release
A Flask release will now be published alongside each main extension
release. The version of each Flask release will be the same as the
extension version except it will have the suffix `-flask.0`.
* Programmatically remove build prefix
The create GH release Bash script derives the Flask version from the
Flask build filename by removing the build prefix, leaving just the
version. Rather than hard-coding the prefix size to remove, it is now
calculated programmatically so that it is easier to read and update.
* Fix tag publishing
The tab publishing step used the wrong credentials, and didn't properly
identify the commit author. This has now been fixed.
`improved-yarn-audit` has been updated so that it supports GitHub
advisories. Two new GitHub advisories have been ignored, as they are
both moderate RegExp DoS vulnerabilities that don't affect us, and they
are embedded deep within our dependency graph and are difficult to
update.
There are a few issues encountered when running `yarn setup` on new
Apple Silicon (aka M1, aka arm64) Macs:
* The script halts when attempting to run the install step for
the `chromedriver` package with the message "Only Mac 64 bits
supported". This is somewhat misleading as it seems to indicate that
chromedriver can only be installed on a 64-bit Mac. However, what I
think is happening is that the installation script for `chromedriver`
is not able to detect that an arm64 CPU *is* a 64-bit CPU. After
looking through the `chromedriver` repo, it appears that 87.0.1 is the
first version that adds a proper check ([1]).
Note that upgrading chromedriver caused the Chrome-specific tests to
fail intermittently on CI. I was not able to 100% work out the reason
for this, but ensuring that X (which provides a way for Chrome to run
in a GUI setting from the command line) is available seems to fix
these issues.
* The script also halts when attempting to run the install step for
the `electron` package. This happens because for the version of
`electron` we are using (9.4.2), there is no available binary for
arm64. It appears that Electron 11.x was the first version to support
arm64 Macs ([2]). This is a bit trickier to resolve because we don't
explicitly rely on `electron` — that's brought in by `react-devtools`.
The first version of `react-devtools` that relies on `electron` 11.x
is 4.11.0 ([3]).
[1]: 469dd0a6ee
[2]: https://www.electronjs.org/blog/apple-silicon
[3]: https://github.com/facebook/react/blob/main/packages/react-devtools/CHANGELOG.md#4110-april-9-2021
The version field is now stored in the main `package.json` file rather
than in the base manifest. It is built into the final manifest during
the build script.
This makes it easier to communicate what the current version should be
to our `auto-changelog` script. It's also generally a more conventional
place to keep track of the version, even considering that we're not
publishing to npm.
The check for the GITHUB_TOKEN environment variable was being done in
the wrong release script. It has been migrated to the relevant script.
A second check for the username has also been added, as it is also
required.
The changelog will now be automatically updated when a release branch
is created. A new release header along with changelog entries for any
new commits will be added.
Note that this changelog will still need to be manually cleaned up, but
it's one less manual step at least.
The old Bash script for adding a new release header to the changelog
has been removed, as that functionality is now built into the changelog
update script.
A new script has been added to commit any changes made to the manifest
and changelog. This step used to happen at the end of the bump manifest
version script, but now the changelog update relies upon the manifest
version bump happening first, so it needed to be re-ordered. The
changes should only be committed on the first run of the branch, as
it's contingent upon the manifest changing (due to the version bump).
Further changelog updates won't trigger new automatic commits.
The script responsible for creating the "Sync `master` with `develop`"
PR has been removed. We will soon be eliminating the need for a
`master` branch altogether, so we don't need this anymore. Also, this
script hasn't been running correctly in a long time. We've been
creating this PR manually.
New changelog release headers now omit the date. These headers are
added automatically when a new release branch is created, and that
rarely ends up being the actual date of the release, so these dates
have all been inaccurate anyway.
The date will be re-added to the changelog later as part of a new
script, after a release has been published.
The CI script to ensure no LavaMoat policy changes are required has
been failing despite there being no changes. It turns out that the
command used to check for changes (`git diff-index`) was failing
despite the lack of changes because the file was written again by
`yarn lavamoat:auto` but git hadn't gotten around to updating its index
since the write occurred, so it was considering it as changed until it
verified it wasn't [1].
The command has been replaced by `git diff --exit-code --quiet`, which
should do exactly the same thing except that it forces git to update
its internal cache to verify whether changes are present.
[1]: https://stackoverflow.com/questions/34807971/why-does-git-diff-index-head-result-change-for-touched-files-after-git-diff-or-g
A CI job has been added to ensure the `allow-scripts` config and the
LavaMoat auto-generated policy is up-to-date. This will only run on
release branches and the `master` branch, because it's too difficult a
requirement to meet for each PR for contributors on macOS, due to
differences in the dependency graph caused by optional dependencies.
The `allow-scripts` and LavaMoat policy have both been updated using
`yarn allow-scripts auto` and `yarn lavamoat:auto`.
The `.sh` file extension is now used for all Bash scripts. This ensures
the files are recognized as Bash scripts by the ShellCheck CI job, and
by editors/IDEs for improved syntax highlighting.
* use @lavamoat/allow-scripts for package postinstall allow list
* dnode: set "weak" to false
Co-authored-by: kumavis <kumavis@users.noreply.github.com>
Co-authored-by: Erik Marks <25517051+rekmarks@users.noreply.github.com>
Dependencies are now cached between builds, using a checksum of the
`yarn.lock` file as the cache key. The `node_modules` directory and the
`.har` file from the install are cached and restored, so that we ensure
the record of the install is always preserved alongside the
dependencies.
The consolidation of the `collect-har-artifact` script was to make it
easier to cache the `.har` file along with the dependencies.
The version bump script referenced the old file path for the manifest.
It was stored as a single file called `manifest.json`, but it was split
into `_base.json` and platform-specific manifests in #8140.
The manifest version bump script has been updated to reference
`_base.json`, which is the piece that has the version property.
