As we convert parts of the codebase to TypeScript, we will want a way to
track progress. This commit adds a dashboard which displays all of the
files that we wish to convert to TypeScript and which files we've
already converted.
The list of all possible files to convert is predetermined by walking
the dependency graph of each entrypoint the build system uses to compile
the extension (the files that the entrypoint imports, the files that the
imports import, etc). The list should not need to be regenerated, but
you can do it by running:
yarn ts-migration:enumerate
The dashboard is implemented as a separate React app. The CircleCI
configuration has been updated so that when a new commit is pushed, the
React app is built and stored in the CircleCI artifacts. When a PR is
merged, the built files will be pushed to a separate repo whose sole
purpose is to serve the dashboard via GitHub Pages (this is the same
way that the Storybook works). All of the app code and script to build
the app are self-contained under
`development/ts-migration-dashboard`. To build this app yourself, you
can run:
yarn ts-migration:dashboard:build
or if you want to build automatically as you change files, run:
yarn ts-migration:dashboard:watch
Then open the following file in your browser (there is no server
component):
development/ts-migration-dashboard/build/index.html
Finally, although you shouldn't have to do this, to manually deploy the
dashboard once built, you can run:
git remote add ts-migration-dashboard git@github.com:MetaMask/metamask-extension-ts-migration-dashboard.git
yarn ts-migration:dashboard:deploy
* Automate the Flask release
A Flask release will now be published alongside each main extension
release. The version of each Flask release will be the same as the
extension version except it will have the suffix `-flask.0`.
* Programmatically remove build prefix
The create GH release Bash script derives the Flask version from the
Flask build filename by removing the build prefix, leaving just the
version. Rather than hard-coding the prefix size to remove, it is now
calculated programmatically so that it is easier to read and update.
* Fix tag publishing
The tab publishing step used the wrong credentials, and didn't properly
identify the commit author. This has now been fixed.
* Changed registryUrl for snaps only in firefox
Fixed getPlatform to only be imported into metamask-controller in flask
Removed snaps specific testrunner script and use run-all with a cli option
* Fixed flakey tests
* Removed unneeded await
* Added delay
* Fixed linting
# Permission System 2.0
## Background
This PR migrates the extension permission system to [the new `PermissionController`](https://github.com/MetaMask/snaps-skunkworks/tree/main/packages/controllers/src/permissions).
The original permission system, based on [`rpc-cap`](https://github.com/MetaMask/rpc-cap), introduced [`ZCAP-LD`](https://w3c-ccg.github.io/zcap-ld/)-like permissions to our JSON-RPC stack.
We used it to [implement](https://github.com/MetaMask/metamask-extension/pull/7004) what we called "LoginPerSite" in [version 7.7.0](https://github.com/MetaMask/metamask-extension/releases/tag/v7.7.0) of the extension, which enabled the user to choose which accounts, if any, should be exposed to each dapp.
While that was a worthwhile feature in and of itself, we wanted a permission _system_ in order to enable everything we are going to with Snaps.
Unfortunately, the original permission system was difficult to use, and necessitated the creation of the original `PermissionsController` (note the "s"), which was more or less a wrapper for `rpc-cap`.
With this PR, we shake off the yoke of the original permission system, in favor of the modular, self-contained, ergonomic, and more mature permission system 2.0.
Note that [the `PermissionController` readme](https://github.com/MetaMask/snaps-skunkworks/tree/main/packages/controllers/src/permissions/README.md) explains how the new permission system works.
The `PermissionController` and `SubjectMetadataController` are currently shipped via `@metamask/snap-controllers`. This is a temporary state of affairs, and we'll move them to `@metamask/controllers` once they've landed in prod.
## Changes in Detail
First, the changes in this PR are not as big as they seem. Roughly half of the additions in this PR are fixtures in the test for the new migration (number 68), and a significant portion of the remaining ~2500 lines are due to find-and-replace changes in other test fixtures and UI files.
- The extension `PermissionsController` has been deleted, and completely replaced with the new `PermissionController` from [`@metamask/snap-controllers`](https://www.npmjs.com/package/@metamask/snap-controllers).
- The original `PermissionsController` "domain metadata" functionality is now managed by the new `SubjectMetadataController`, also from [`@metamask/snap-controllers`](https://www.npmjs.com/package/@metamask/snap-controllers).
- The permission activity and history log controller has been renamed `PermissionLogController` and has its own top-level state key, but is otherwise functionally equivalent to the existing implementation.
- Migration number 68 has been added to account for the new state changes.
- The tests in `app/scripts/controllers/permissions` have been migrated from `mocha` to `jest`.
Reviewers should focus their attention on the following files:
- `app/scripts/`
- `metamask-controller.js`
- This is where most of the integration work for the new `PermissionController` occurs.
Some functions that were internal to the original controller were moved here.
- `controllers/permissions/`
- `selectors.js`
- These selectors are for `ControllerMessenger` selector subscriptions. The actual subscriptions occur in `metamask-controller.js`. See the `ControllerMessenger` implementation for details.
- `specifications.js`
- The caveat and permission specifications are required by the new `PermissionController`, and are used to specify the `eth_accounts` permission and its JSON-RPC method implementation.
See the `PermissionController` readme for details.
- `migrations/068.js`
- The new state should be cross-referenced with the controllers that manage it.
The accompanying tests should also be thoroughly reviewed.
Some files may appear new but have just moved and/or been renamed:
- `app/scripts/lib/rpc-method-middleware/handlers/request-accounts.js`
- This was previously implemented in `controllers/permissions/permissionsMethodMiddleware.js`.
- `test/mocks/permissions.js`
- A truncated version of `test/mocks/permission-controller.js`.
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* add storybook unit tests with CI integration
* fix command and fix casing for test
* change ci ordering for storybook tasks
* fix syntax error
* fix jest
* lint
* Add transaction-total-banner render test to Storybook (#12517)
* transaction-total-banner
* lint
* confirm to spec
* lint
* fix jest ocnfig for snapshot test failure
`improved-yarn-audit` has been updated so that it supports GitHub
advisories. Two new GitHub advisories have been ignored, as they are
both moderate RegExp DoS vulnerabilities that don't affect us, and they
are embedded deep within our dependency graph and are difficult to
update.
The beta and Flask builds are now built on CI and included in the
metamask bot comment alongside the main builds. The same sourcemap
linter and mozilla linter used for the prod builds is also run on the
beta and Flask builds.
Closes#12426
There are a few issues encountered when running `yarn setup` on new
Apple Silicon (aka M1, aka arm64) Macs:
* The script halts when attempting to run the install step for
the `chromedriver` package with the message "Only Mac 64 bits
supported". This is somewhat misleading as it seems to indicate that
chromedriver can only be installed on a 64-bit Mac. However, what I
think is happening is that the installation script for `chromedriver`
is not able to detect that an arm64 CPU *is* a 64-bit CPU. After
looking through the `chromedriver` repo, it appears that 87.0.1 is the
first version that adds a proper check ([1]).
Note that upgrading chromedriver caused the Chrome-specific tests to
fail intermittently on CI. I was not able to 100% work out the reason
for this, but ensuring that X (which provides a way for Chrome to run
in a GUI setting from the command line) is available seems to fix
these issues.
* The script also halts when attempting to run the install step for
the `electron` package. This happens because for the version of
`electron` we are using (9.4.2), there is no available binary for
arm64. It appears that Electron 11.x was the first version to support
arm64 Macs ([2]). This is a bit trickier to resolve because we don't
explicitly rely on `electron` — that's brought in by `react-devtools`.
The first version of `react-devtools` that relies on `electron` 11.x
is 4.11.0 ([3]).
[1]: 469dd0a6ee
[2]: https://www.electronjs.org/blog/apple-silicon
[3]: https://github.com/facebook/react/blob/main/packages/react-devtools/CHANGELOG.md#4110-april-9-2021
The benchmark script can now be set to retry upon failure, like the E2E
tests do. The default is zero, just as with the E2E tests. A retry of 2
has been set in CI to match the E2E tests as well.
The `retry` module had to be adjusted to throw an error in the case of
failure. Previously it just set the exit code, but that only worked
because it was the last thing called before the process ended. That is
no longer the case.
This script makes it easier to run an individual E2E test. In the past
I've run individual scripts by editing `run-all.sh` manually, but now
that can be done more easily with this script. It also allows setting
the number of retries to use and the browser to use from the CLI.
This script has been added as an npm script as well, called
'test:e2e:single'.
The `run-all.sh` script was rewritten in JavaScript to make it easier
to pass through a `--retries` argument.
The default number of retries has been set to zero to make local
testing easier. It has been set to 2 on CI.
This was mainly done to consolidate the code used to run an E2E test in
one place, to make later improvements easier.
Sentry is now configured with environment variables, rather than with
hard-coded values. This makes it easier to test Sentry functionality
using a different Sentry account, as we did recently during QA of
v9.5.1.
The only change for the normal build process is the introduction of the
`SENTRY_DSN_DEV` variable, which can be set via `.metamaskrc` or via an
environment variable. This determines where error reports are sent. It
still defaults to our team Sentry account's `metamask-testing` project.
The `sentry:publish` script now requires SENTRY_ORG and SENTRY_PROJECT
to be set in order to publish release artifacts. The CircleCI
configuration has been updated with these values, so it should act the
same as it did before. Previously we had used a CLI flag to specify the
organization and project, but Sentry already natively supports these
environment variables [1].
[1]: https://docs.sentry.io/product/cli/configuration/#configuration-values
The version field is now stored in the main `package.json` file rather
than in the base manifest. It is built into the final manifest during
the build script.
This makes it easier to communicate what the current version should be
to our `auto-changelog` script. It's also generally a more conventional
place to keep track of the version, even considering that we're not
publishing to npm.
Add changelog validation to CI, ensuring that the changelog follows the
KeepAChangelog format. Additionally, this will also ensure that all of
the auto-generated "Uncategorized" changelog entries on release
branches get categorized.
The check for the GITHUB_TOKEN environment variable was being done in
the wrong release script. It has been migrated to the relevant script.
A second check for the username has also been added, as it is also
required.
The SSH key used for Storybook deployments to `metamask-storybook` has
been updated. This new key is associated with `metamaskbot` rather than
a specific team member.
* Setup jest config
* Adjust test for jest.
* Adjust lint config
* Omit swaps ui folder for unit testing
* Omit swaps from test:unit:lax
* Add jest.config.js to script files
* Restore mocks rather than clearing them.
* Update jest config and adjust lint to include subdirs
* Convert view-quote-price-difference test to jest
* Add jest ci and ci coverage scripts. Add jest unit test to general test command
* Add test coverage to ci
* Use --ignore flag
* Fixup
* Add @metamask/eslint-config-jest
* Update .eslintrc.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Adds jest-coverage/
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
