* Automate the Flask release
A Flask release will now be published alongside each main extension
release. The version of each Flask release will be the same as the
extension version except it will have the suffix `-flask.0`.
* Programmatically remove build prefix
The create GH release Bash script derives the Flask version from the
Flask build filename by removing the build prefix, leaving just the
version. Rather than hard-coding the prefix size to remove, it is now
calculated programmatically so that it is easier to read and update.
* Fix tag publishing
The tab publishing step used the wrong credentials, and didn't properly
identify the commit author. This has now been fixed.
This commit allows developers to write TypeScript files and lint them
(either via a language server in their editor of choice or through the
`yarn lint` command).
The new TypeScript configuration as well as the updated ESLint
configuration not only includes support for parsing TypeScript files,
but also provides some compatibility between JavaScript and TypeScript.
That is, it makes it possible for a TypeScript file that imports a
JavaScript file or a JavaScript file that imports a TypeScript file to
be linted.
Note that this commit does not integrate TypeScript into the build
system yet, so we cannot start converting files to TypeScript and
pushing them to the repo until that final step is complete.
* Automate the Flask release
A Flask release will now be published alongside each main extension
release. The version of each Flask release will be the same as the
extension version except it will have the suffix `-flask.0`.
* Programmatically remove build prefix
The create GH release Bash script derives the Flask version from the
Flask build filename by removing the build prefix, leaving just the
version. Rather than hard-coding the prefix size to remove, it is now
calculated programmatically so that it is easier to read and update.
* Fix tag publishing
The tab publishing step used the wrong credentials, and didn't properly
identify the commit author. This has now been fixed.
The version of a build is now derived from both the `version` field in
`package.json` and the requested build type and version. The build type
and version are added onto the manifest version as a suffix, according
to the SemVer prerelease format.
We already have support in the extension for versions of this format,
but to apply a Flask or Beta version required manual updates to
`package.json`. Now it can be done just with build arguments.
A `get-version` module was created to make it easier to generate the
version in the various places we do that during the build. It was
created in the `development/lib` directory because it will be used by
other non-build development scripts in a future PR.
The `BuildType` constant was extracted to its own module as well, and
moved to the `development/lib` directory. This was to make it clear
that it's used by various different development scripts, not just the
build.
We would like to insert TypeScript into the ESLint configuration, and
because of the way that the current config is organized, that is not
easy to do.
Most files are assumed to be files that are suited for running in a
browser context. This isn't correct, as we should expect most files to
work in a Node context instead. This is because all browser-based files
will be run through a transpiler that is able to make use of
Node-specific variables anyway.
There are a couple of important ways we can categories files which our
ESLint config should be capable of handling well:
* Is the file a script or a module? In other words, does the file run
procedurally or is the file intended to be brought into an existing
file?
* If the file is a module, does it use the CommonJS syntax (`require()`)
or does it use the ES syntax (`import`/`export`)?
When we introduce TypeScript, this set of questions will become:
* Is the file a script or a module?
* If the file is a module, is it a JavaScript module or a TypeScript
module?
* If the file is a JavaScript module, does it use the CommonJS syntax
(`require()`) or does it use the ES syntax (`import`/`export`)?
To represent these divisions, this commit removes global rules — so now
all of the rules are kept in `overrides` for explicitness — and sets up
rules for CommonJS- and ES-module-compatible files that intentionally do
not overlap with each other. This way TypeScript (which has its own set
of rules independent from JavaScript and therefore shouldn't overlap
with the other rules either) can be easily added later.
Finally, this commit splits up the ESLint config into separate files and
adds documentation to each section. This way sets of rules which are
connected to a particular plugin (`jsdoc`, `@babel`, etc.) can be easily
understood instead of being obscured.
If an error occurs while running Browserify, the stream that Browserify
creates will emit an `error` event. However, this event is not being
handled, so Node will catch it instead. But the error message it
produces is very nebulous, as it merely spits out the stream object and
completely ignores the actual error that occurred. So this commit
listens for the `error` event and outputs the error.
One note here is that when we are outputting the error, we must get
around a bug that exists in Endo where if you pass an Error object to
`console.{log,error,info,debug}` then you will just see `{}` on-screen.
We get around this by printing `err.stack`.
* mock gas price api
* fix error
* full url
* remove duplicated packages
* full url
* customise mock per test
* customise mock per test
* enable mocking
* enable mocking
* enable mocking by default
* duplicated packages
* update mockttp
* pass through
* pass through
This PR adds `snaps` under Flask build flags to the extension. This branch is mostly equivalent to the current production version of Flask, excepting some bug fixes and tweaks.
Closes#11626
* added fix for snaps devx issue
* reordered lines
* updated comment
* added test that ensures removeFencedCode detects a file with sourceMap inclusion
* fixed test
* Update development/build/transforms/remove-fenced-code.test.js
Co-authored-by: Erik Marks <25517051+rekmarks@users.noreply.github.com>
ESLint rules have been added to enforce our JSDoc conventions. These
rules were introduced by updating `@metamask/eslint-config` to v9.
Some of the rules have been disabled because the effort to fix all lint
errors was too high. It might be easiest to enable these rules one
directory at a time, or one rule at a time.
Most of the changes in this PR were a result of running
`yarn lint:fix`. There were a handful of manual changes that seemed
obvious and simple to make. Anything beyond that and the rule was left
disabled.
The ESLint config for the extension explicitly includes support for
Prettier. However, this is already being provided by our global ESLint
config (`@metamask/eslint-config`). Therefore there is no need to
include it here. In fact, this is causing weird issues where the `curly`
option is getting overridden somehow. After this change, these syntaxes
are invalid:
``` javascript
if (foo) return;
```
``` javascript
if (foo) return 'bar';
```
* Update support links for Flask
* Disable 'prefer-const' in code fence linting
* Add bespoke home footer for Flask and update logic
* fixup! Add bespoke home footer for Flask and update logic
* Fix code fence lint failure
* Fix support request link in account menu
* Fix unit test failure
The Firefox extension version format does not support the version
format we use (SemVer), so we have to specially format the extension
version to be compatible. The format we chose was
`[major].[minor].[patch].[buildType][buildVersion]`. But when we tried
to submit a build with a version in that format, it was rejected as
invalid for unknown reasons.
The Firefox extension format has been updated to
`[major].[minor].[patch][buildType][buildVersion]`. This seems to pass
validation.
The `version_name` manifest field was being used on Chrome to store the
build type. However, Chrome intended this field to be a full
representation of the version, for display purposes. This was evident
when uploading this version to the Chrome Web Store, because it used
`flask` as the entire version.
Instead the `version_name` field now includes the full SemVer version
string. The version parsing code within the build script and in the
wallet itself have been updated accordingly.
The build script only allowed prerelease versions for the "beta" build
type (e.g. `X.Y.Z-beta.0`). Now it allows Flask prerelease versions as
well.
This is required for the Flask release, where the prerelease version
helps distinguish the Flask error reports and metrics.
The ESLint config has been updated to v8. The breaking changes are:
* The Prettier rule `quoteProps` has been changed from `consistent` to
`as-needed`, meaning that if one key requires quoting, only that key is
quoted rather than all keys.
* The ESLint rule `no-shadow` has been made more strict. It now
prevents globals from being shadowed as well.
Most of these changes were applied with `yarn lint:fix`. Only the
shadowing changes required manual fixing (shadowing variable names were
either replaced with destructuring or renamed).
The dependency `globalThis` was added to the list of dynamic
dependencies in the build system, where it should have been already.
This was causing `depcheck` to fail because the new lint rules required
removing the one place where `globalThis` had been erroneously imported
previously.
A rule requiring a newline between multiline blocks and expressions has
been disabled temporarily to make this PR smaller and to avoid
introducing conflicts with other PRs.
`remote-redux-devtools` is now explicitly excluded and disabled in non-
dev builds, and in the `testDev` build. This was causing console errors
in the `testDev` build during e2e tests, which would cause certain
tests to fail.
This was already only supposed to be enabled for development builds,
but this library used the `NODE_ENV` environment variable to make that
determination. This gives us more control over when it's disabled.
The React dev tools can result in console errors if dev tools is not
open during the test. Some of our e2e tests fail if there are any
console errors, so these errors break those tests.
`react-devtools` has been completely disabled for `testDev` builds to
make debugging e2e tests easier. The React dev tools can still be used
from development builds.
A propType error was showing up during e2e tests with a `testDev`
build. It was caused by `process.env.IN_TEST` being treated as a
boolean, when in fact it is either the string `'true'` or a boolean.
`IN_TEST` has been updated to always be a boolean. `loose-envify` has
no trouble injecting boolean values, so there's no reason to treat this
as a string.
The coverage reporter for the console has been changed from `text` to
`text-summary` because `text` was too long. It exceeded the maximum
length of the CircleCI terminal output shown on their jobs page, making
it very difficult to see why the unit test coverage job failed.
The text summary only displays overall coverage metrics, but that is
usually enough to indicate when the test fails due to a drop in
coverage. The more detailed breakdown is still available as a HTML
report linked in the `metamaskbot` comment, and in the `jest-coverage`
directory when run locally.
The three on-disk coverage reports used previously (`lcov`, `json`, and
`clover`) have been replaced with just `html`. The HTML report was part
of the `lcov` report, and it was the only one we were using.
The LavaMoat policy generation script would sporadically fail because
it ran the build concurrently three times, and the build includes
steps that delete the `dist` directory and write to it. So if one build
process tried to write to the directory after another deleted it, it
would fail.
This was solved by adding a new `--policy-only` flag to the build
script, and a new `scripts:prod` task. The `scripts:prod` task only
runs the script tasks for prod, rather than the entire build process.
The `--policy-only` flag stops the script tasks once the policy has
been written, and stops any other files from being written to disk.
This prevents the three concurrent build processes from getting in each
others way, and it dramatically speeds up the process.
The environment variables used for test builds was wrong for certain
bundles because the `testing` flag wasn't passed through to the
function that determines which environment variables to inject.
Effectively this means that test builds on `master` were going to the
production `metamask` Sentry project rather than the `test-metamask`
project. This has been the case since #11080.
The `testing` flag is now included for all bundles, and test builds now
use the `test-metamask` Sentry project in all cases.
This PR improves the error handling of the code fence removal transform stream by catching errors thrown by the `removeFencedCode` function and passing them to the `end` callback. This appears to resolve a problem where watched builds would blow up whenever a file with fences was reloaded.
This PR adds one LavaMoat background script policy or each build type. It also renames the build system policy directory from `node` to `build-system` to make its purpose more clear. Each build type has the original `policy-override.json` for `main` builds. The `.prettierignore` file has been updated to match the locations of the new auto-generated policy files.
We need to maintain separate policies for each build type because each type will produce different bundles with different internal and external modules.
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
The build system now supports platform-specific modifications to the
manifest for each build type. The need to customize the `id` on Firefox
motivated this change.
To support this, a new directory was made in each build type directory
for manifest changes. The images currently in this directory were moved
into an `images` subdirectory.
This new `manifest` directory can include each manifest file currently
in `app/manifest`. The `_base.json` file is assumed to exist, but the
platform manifest modifications are optional.
* GridPlus: Adds support for GridPlus Lattice1 hardware wallet
* Fixes issue with switching hardware HD path
The main `Select HD Path` piece of the account selection component was not
properly hooked up to the state manager (`onPathChange`) and the extra
`Popover` component was being used instead.
I'm not sure what the origin of this is, but I don't see why the Popover
is needed at all. I have remove it and hooked `onPathChange` directly into
the HD path selector dropdown.
This was an issue that nearly every Lattice user who had come from Ledger
has contacted us about.
* GridPlus: Addresses QA issues
* Adds Lattice tutorial + image
* Cleans up connectivity issues (see: https://github.com/GridPlus/eth-lattice-keyring/pull/16)
* GridPlus: Adds Firefox support
To connect to the Lattice you need to open a new tab/window and get
login data from it. We were not able to do this for Firefox because
we relied on the `window` API. This is now fixed.
See corresponding changes:
* `eth-lattice-keyring`: https://github.com/GridPlus/eth-lattice-keyring/pull/17
* Lattice connector: https://github.com/GridPlus/wallet-web/pull/152
* GridPlus: Adds missing error path for Firefox
See: 242a93f559
The beta and Flask builds are now built on CI and included in the
metamask bot comment alongside the main builds. The same sourcemap
linter and mozilla linter used for the prod builds is also run on the
beta and Flask builds.
Closes#12426
This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions.
The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details.
### Code Fencing and ESLint
When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor.
### File Exclusion
For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`.
As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice.
Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
The code fence transform was including contents after the final END directive twice. That was not covered by the tests, because none of the examples contained any content after the final END directive, and concatenating the empty string twice does not produce an observable difference in the test results.
This bug was due to an off-by-one error in the loop of the multiSplice function. The error has been fixed, and more test cases have been added.
Static files have been added for the Flask build. This includes logos
of each size and variety that we use, and it includes the 3D model JSON
file.
Closes#12427
The production build was accidentally broken in #12440 because of a
merge conflict with a #12441 that wasn't initially noticed. The
conflict was the renaming of the `BuildTypes` variable to `BuildType`.
This variable is used to check the current build type, but only for
production builds. `BuildTypes` is `undefined`, so this would result in
a crash when that enum was used.
The build script has been updated to embed the correct Infura project
ID and Segment write key for beta and Flask builds. These are set via
environment variable or config file. They have already been added in CI
as environment variables.
The Segment production write key has also been moved into the set of
environment variables that can be set in the configuration file. This
was to make the way we reference it more consistent.
The new project IDs and keys are only used in the "production"
environment, which right now is the merge step into the `master`
branch. This is appropriate for Flask, but it doesn't match our plan
for how the beta release would get created. In a future PR, when the
beta release automation work is completed, the conditions for when
the beta secrets are used should be updated to ensure they're used only
for the beta builds.
Closes#11896
Recently validation was added for our build configuration as part of
the PR #12438. This had the unintended consequence of making all builds
from forks fail because they don't get secrets injected. Specifically
it was the missing `INFURA_PROJECT_ID` that made the builds fail.
The Infura project ID is no longer required for building. In practice
it's still required for doing anything with a build but running e2e
tests, but that's all we need to do in CI anyway.
The build type (i.e. the distribution) is now included in the Sentry
environment during setup, for all builds except the "main" build. This
will allow us to track Flask and beta errors separately from other
errors.
A constant was created for the build types. The equivalent constant in
our build scripts was updated to match it more closely, for
consistency. We can't use the same constant in both places because our
shared constants are in modules that use ES6 exports, and our build
script does not yet support ES6 exports.
The singular `BuildType` was used rather than `BuildTypes` to match our
naming conventions elsewhere for enums. We name them like classes or
types, rather than like a collection.
Relates to #11896
We now use two separate Infura project IDs for production builds, and
for all other builds. Previously all CI builds used the production
Infura project ID. Separating them will make our Infura dashboard
metrics more representative of real production usage.
The new environment variable for production has been setup in CI
already, but the old environment variable will remain set to the
production project ID until this commit is included in a release.
We can't switch the old environment variable out until we're confident
that it won't get used for a production build.
We now use constants for the various different build environments. This
was done to improve the JSDoc types of the `getInfuraProjectId` helper
method.
The `getConfigValue` function was added to make it easier to validate
that required config values are set. This should ensure builds fail
early with an informative error message when they are missing the
necessary configuration.
The main `version` field in `package.json` will now include the beta
version (if present) rather than it being passed in via the CLI when
building. The `version` field is now a fully SemVer-compatible version,
with the added restriction that any prerelease portion of the version
must match the format `<build type>.<build version>`.
This brings the build in-line with the future release process we will
be using for the beta version. The plan is for each future release to
enter a "beta phase" where the version would get updated to reflect
that it's a beta, and we would increment this beta version over time as
we update the beta. The manifest gives us a place to store this beta
version. It was also important to replace the automatic minor bump
logic that was being used previously, because the version in beta might
not be a minor bump.
Additionally, the filename logic used for beta builds was updated to
be generic across all build types rather than beta-specific. This will
be useful for Flask builds in the future.
* lavamoat - add lavamoat to webapp background
* test:e2e - add delay to resolve failure
* test:e2e - add delay to resolve failure
* build - add a switch for applying lavamoat, currently off for all
* test/e2e - remove delays added for lavamoat
* Revert "test/e2e - remove delays added for lavamoat"
This reverts commit 79c3479f15c072ed362ba1d4f1af41ea11a17d63.
* lockdown - breakout making globalThis properties non-writable into lockdown-more.js
* Update app/scripts/lockdown-more.js
Co-authored-by: David Walsh <davidwalsh83@gmail.com>
* Update app/scripts/lockdown-more.js
Co-authored-by: Erik Marks <25517051+rekmarks@users.noreply.github.com>
Co-authored-by: David Walsh <davidwalsh83@gmail.com>
Co-authored-by: Erik Marks <25517051+rekmarks@users.noreply.github.com>
The MetaMask logo used for beta development builds was wrong. The lock
screen (and any other place using the `@metamask/logo` logo) showed the
correct logo, but all of our static assets used the "regular" logo.
Now the beta logo should be used everywhere for beta development
builds.
This is a refactor to replace the `isBeta` boolean with `buildType`
throughout the build system. This will allow us to modify the behaviour
of each step of the build process for Flask as well.
This should result in no functional changes.
This PR adds build-time code exclusion by means of code fencing. For details, please see the README in `./development/build/transforms`. Note that linting of transformed files as a form of validation is added in a follow-up, #12075.
Hopefully exhaustive tests are added to ensure that the transform works according to its specification. Since these tests are Node-only, they required their own Jest config. The recommended way to work with multiple Jest configs is using the `projects` field in the Jest config, however [that feature breaks coverage collection](https://github.com/facebook/jest/issues/9628). That being the case, I had to set up two separate Jest configs. In order to get both test suites to run in parallel, Jest is now invoked via a script, `./test/run-jest.sh`.
By way of example, this build system feature allows us to add fences like this:
```javascript
this.store.updateStructure({
...,
GasFeeController: this.gasFeeController,
TokenListController: this.tokenListController,
///: BEGIN:ONLY_INCLUDE_IN(beta)
PluginController: this.pluginController,
///: END:ONLY_INCLUDE_IN
});
```
Which at build time are transformed to the following if the build type is not `beta`:
```javascript
this.store.updateStructure({
...,
GasFeeController: this.gasFeeController,
TokenListController: this.tokenListController,
});
```
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
This rationalizes how arguments are passed to and parsed by the build system. To accomplish this, everything that isn't an environment variable from `.metamaskrc` or our CI environment is now passed as an argument on the command line.
Of such arguments, the `entryTask` is still expected as a positional argument in the first position (i.e. `process.argv[2]`), but everything else must be passed as a named argument. We use `minimist` to parse the arguments, and set defaults to preserve existing behavior.
Arguments are parsed in a new function, `parseArgv`, in `development/build/index.js`. They are assigned to environment variables where convenient, and otherwise returned from `parseArgv` to be passed to other functions invoked in the same file.
This change is motivated by our previous inconsistent handling of arguments to the build system, which will grow increasingly problematic as the build system grows in complexity. (Which it will very shortly, as we introduce Flask builds.)
Miscellaneous changes:
- Adds a build system readme at `development/build/README.md`
- Removes the `beta` package script. Now, we can instead call: `yarn dist --build-type beta`
- Fixes the casing of some log messages and reorders some parameters in the build system
There are a few issues encountered when running `yarn setup` on new
Apple Silicon (aka M1, aka arm64) Macs:
* The script halts when attempting to run the install step for
the `chromedriver` package with the message "Only Mac 64 bits
supported". This is somewhat misleading as it seems to indicate that
chromedriver can only be installed on a 64-bit Mac. However, what I
think is happening is that the installation script for `chromedriver`
is not able to detect that an arm64 CPU *is* a 64-bit CPU. After
looking through the `chromedriver` repo, it appears that 87.0.1 is the
first version that adds a proper check ([1]).
Note that upgrading chromedriver caused the Chrome-specific tests to
fail intermittently on CI. I was not able to 100% work out the reason
for this, but ensuring that X (which provides a way for Chrome to run
in a GUI setting from the command line) is available seems to fix
these issues.
* The script also halts when attempting to run the install step for
the `electron` package. This happens because for the version of
`electron` we are using (9.4.2), there is no available binary for
arm64. It appears that Electron 11.x was the first version to support
arm64 Macs ([2]). This is a bit trickier to resolve because we don't
explicitly rely on `electron` — that's brought in by `react-devtools`.
The first version of `react-devtools` that relies on `electron` 11.x
is 4.11.0 ([3]).
[1]: 469dd0a6ee
[2]: https://www.electronjs.org/blog/apple-silicon
[3]: https://github.com/facebook/react/blob/main/packages/react-devtools/CHANGELOG.md#4110-april-9-2021
This adds an `--omit-lockdown` flag to our build script, which will cause SES `lockdown` to be omitted from the resulting bundle. Useful for development when we don't want the environment to be locked down.
Thanks to @kumavis for the suggestion.
This PR makes ~all named intrinsics in all of our JavaScript processes non-modifiable. A named intrinsic is any property specified by the ECMAScript specification that exists on `globalThis` when the JavaScript process starts. We say that a property is non-modifiable if it is non-configurable and non-writable. We make exceptions for properties that meet any of the following criteria:
1. Properties that are non-configurable by the time `lockdown-run.js` is executed are not modified, because they can't be.
2. Properties that have accessor properties (`get` or `set`) are made non-configurable, but their writability cannot be modified, and is therefore left unchanged. It's unclear how many of the named intrinsics this applies to, if any, but it's good defensive programming, regardless.
The Sentry `release` was not being configured correctly. It was being
left blank. This is because the location of the extension version was
moved in #11029. The build script was correctly updated in that PR, but
that work was accidentally undone in a merge error that was included
in #11080.
Fixing up tests and add back old custom gas modal for non-eip1559 compliant networks
Remove unnecessary props from send-gas-row.component
fix breaking test
Fix primary and secondary title overrides
fix rebase issue
Fix rebase conflict
Co-authored-by: David Walsh <davidwalsh83@gmail.com>
The benchmark script can now be set to retry upon failure, like the E2E
tests do. The default is zero, just as with the E2E tests. A retry of 2
has been set in CI to match the E2E tests as well.
The `retry` module had to be adjusted to throw an error in the case of
failure. Previously it just set the exit code, but that only worked
because it was the last thing called before the process ended. That is
no longer the case.
This script makes it easier to run an individual E2E test. In the past
I've run individual scripts by editing `run-all.sh` manually, but now
that can be done more easily with this script. It also allows setting
the number of retries to use and the browser to use from the CLI.
This script has been added as an npm script as well, called
'test:e2e:single'.
The `run-all.sh` script was rewritten in JavaScript to make it easier
to pass through a `--retries` argument.
The default number of retries has been set to zero to make local
testing easier. It has been set to 2 on CI.
This was mainly done to consolidate the code used to run an E2E test in
one place, to make later improvements easier.
The function we were using to run shell commands during the
`sentry:publish` script were swallowing the CLI output. We also weren't
correctly detecting the process exit in some cases.
The `run-command` module originally written for `auto-changelog`
(introduced in #10782 and replaced in #10993) has been resurrected for
running commands where we don't care about the output, or where we want
to use the output for something. A second function (`runInShell`) has
been added for running commands with the same STDOUT and STDERR
streams, so that the output is sent directly to the CLI. This ensures
that the console output from the shell script we run gets correctly
output to the CLI.
Sentry is now configured with environment variables, rather than with
hard-coded values. This makes it easier to test Sentry functionality
using a different Sentry account, as we did recently during QA of
v9.5.1.
The only change for the normal build process is the introduction of the
`SENTRY_DSN_DEV` variable, which can be set via `.metamaskrc` or via an
environment variable. This determines where error reports are sent. It
still defaults to our team Sentry account's `metamask-testing` project.
The `sentry:publish` script now requires SENTRY_ORG and SENTRY_PROJECT
to be set in order to publish release artifacts. The CircleCI
configuration has been updated with these values, so it should act the
same as it did before. Previously we had used a CLI flag to specify the
organization and project, but Sentry already natively supports these
environment variables [1].
[1]: https://docs.sentry.io/product/cli/configuration/#configuration-values
The method used for uploading release artifacts to Sentry has been
updated to allow `sentry-cli` to associate our minified bundles with
the corresponding source map file. This should help Sentry display rich
stack traces.
Previously Sentry had used the `sourceMappingURL` to associate source
maps with bundles, but we recently removed this in #10695. The hope is
that this change to the upload process will ensure the mapping works
correctly without the `sourceMappingURL` comment.
The `upload_bundles` function was removed because the later
`upload_sourcemaps` function actually uploaded both the bundles and
source maps.
The `--rewrite` flag was added to enable a newer "rewrite" feature of
the Sentry CLI that they recommend using [1]. This rewrite is where
they associate source maps with bundles.
The `url-prefix` has been updated to be `metamask` rather than
`sourcemaps`. I don't think `sourcemaps` was ever the correct prefix.
We normalize our errors to have the path `metamask/` before sending any
reports to Sentry.
[1]: https://docs.sentry.io/product/cli/releases/#sentry-cli-sourcemaps
The version field is now stored in the main `package.json` file rather
than in the base manifest. It is built into the final manifest during
the build script.
This makes it easier to communicate what the current version should be
to our `auto-changelog` script. It's also generally a more conventional
place to keep track of the version, even considering that we're not
publishing to npm.
The `auto-changelog` script has been replaced with the package
`@metamask/auto-changelog`. This package includes a script that has
an `update` command that is roughly equivalent to the old
`auto-changelog.js` script, except better. The script also has a
`validate` command.
The `repository` field was added to `package.json` because it's
utilized by the `auto-changelog` script, and this was easier than
specifying the repository URL with a CLI argument.
The `auto-changelog.js` script crashes when trying to add a new release
header. This bug was introduced in #10847. The cause was a simple
misnamed parameter.
When updating the changelog for a release candidate, any unreleased
changes are now migrated to the release header.
Generally we don't make a habit of adding changes to the changelog
prior to creating a release candidate, but if any are there we
certainly don't want them duplicated.
The `auto-changelog.js` script has been refactoring into various
different modules. This was done in preparation for migrating this to
a separate repository, where it can be used in our libraries as well.
Functionally this should act _mostly_ the same way, but there have been
some changes. It was difficult to make this a pure refactor because of
the strategy used to validate the changelog and ensure each addition
remained valid. Instead of being updated in-place, the changelog is now
parsed upfront and stored as a "Changelog" instance, which is a new
class that was written to allow only valid changes. The new changelog
is then stringified and completely overwrites the old one.
The parsing had to be much more strict, as any unanticipated content
would otherwise be erased unintentionally. This script now also
normalizes the formatting of the changelog (though the individual
change descriptions are still unformatted).
The changelog stringification now accommodates non-linear releases as
well. For example, you can now release v1.0.1 *after* v2.0.0, and it
will be listed in chronological order while also correctly constructing
the `compare` URLs for each release.
The changelog script now accepts an `--rc` flag to tell it whether to
add new changes to `Unreleased` or to the header for the current
version.
Previously this was inferred from whether the current version matched
the most recent tag. However this method only works for the first
update. Using a flag simplifies this logic, and makes it possible to
manually re-run this for further updates to a release candidate.
Each changelog release now has category headers. The standard "keep a
changelog" [1] categories are used, along with the addition of
"Uncategorized" for any changes that have not yet been categorized.
The changelog script has been updated to add this "Uncategorized"
header if it isn't already present, and to place any new commits under
this header.
The changelog has been updated to property categorize each change in
recent releases, and to place changes in older releases under the
header "Uncategorized".
[1]: https://keepachangelog.com/en/1.0.0/
Each release header now includes a link to the range of commits
included with that release. These links are at the end of the document,
in accordance with the "keep a changelog" [1] format.
For the purpose of this changelog, the "previous release" is the most
recent release mentioned in the changelogs. The diffs ignore any
releases that were omitted from the changelog. This is mainly an issue
with older releases, so it seemed acceptable. All releases have been
documented for a couple of years now, and will be going forward as
well.
The name of the "Current Develop Branch" section was changed to
"Unreleased" to confirm with "keep a changelog".
The `auto-changelog.js` script has been updated to update/add these
links whenever adding a new release header as well.
[1]: https://keepachangelog.com/en/1.0.0/
The changelog release header format has been updated to match the "keep
a changelog" [1] format. Each header is now the bracketed version
number followed by a dash, then the release date in ISO-8601 format.
The release dates in each header were also updated to match the date of
the corresponding GitHub Release [2]. Many of these dates were
incorrect because they were set on the day we created the release
candidate, rather than on the day of release.
Any changelog release entries without a corresponding GitHub release
was left with the date already specified.
The three oldest release headers were missing dates. For the first two,
I used the date of the version bump commit. For the third, I removed it
since no changes were listed anyway, and it represented a range of
releases rather than a single one.
The `auto-changelog.js` script has been updated to account for this new
format as well.
[1]: https://keepachangelog.com/en/1.0.0/
[2]: https://github.com/MetaMask/metamask-extension/releases
The changelog update script now prevents duplicate entries from being
added. Specifically, it will ensure that if a PR has been referenced
already in an entry, it will not add it again.
This should prevent it from adding duplicate entries for changes that
were cherry-picked into hotfix releases.
Note that this duplication prevention only works for entries containing
a PR number. We don't have any way to prevent duplicate entries yet in
cases where we don't know the associated PR. We will be preventing this
possibility entirely pretty soon in some upcoming release automation
changes though, so I'm not concerned about this omission.
Instead of always placing new changelog entries under the "Current
Develop Branch" header, the changelog script now places them under the
header for the current release if that release has not yet been tagged.
This eliminates one manual step from the release process.
Relates to #10752
Our build script waits for the `close` event to determine whether the
task has exited. The `exit` event is a better representation of this,
because if a stream is shared between multiple processes, the process
may exit without the `close` event being emitted.
We aren't sharing streams between processes, so this edge case doesn't
apply to us. This just seemed like a more suitable event to listen to,
since we care about the process exiting not the stream ending.
See this description of the `close` event from the Node.js
documentation [1]:
>The `'close'` event is emitted when the stdio streams of a child
>process have been closed. This is distinct from the `'exit'` event,
>since multiple processes might share the same stdio streams.
And see this description of the `exit` event:
>The `'exit'` event is emitted after the child process ends.
[1]: https://nodejs.org/docs/latest-v14.x/api/child_process.html#child_process_event_exit
* build - declare background as html
* build - fill in empty file when a missing file is expected
* lint - fix
* Update development/build/manifest.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Update development/build/manifest.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Excluding sourcemaps comment in production builds
FixesMetaMask/metamask-extension#7077
* Fix source map explorer script
The source map explorer script now re-adds the source map comment to
each file to ensure the source map visualization still works. Each
module with a sourcemap is copied to a temporary directory along with
the module it corresponds to, and from there it's passed into
`source-map-explorer`. This should ensure the resulting visualization
matches what it was before.
Everything has been moved inside of functions to generally improve
readability, and to allow the use of local variables.
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
The source map explorer script will now use `yarn` instead of `npx` to
create the visualizations, to ensure that it's using the exact version
of `source-map-explorer` that we have in our dependencies.
The standard set of Bash flags have been set as well, and the standard
Bash shebang we use. This ensures the script will fail if an error is
encountered.
Build warnings related to Sass have been reduced by dynamically
importing `gulp-sass` and `sass-compiler` at the point where it's first
used. This ensures that the four Dart-related build warnings are only
emitted for the build process that is actually running Sass, rather
than by _every_ build process.
The bundle visualizations for the library bundles has been fixed.
Previously it was trying to generate a visualization for the non-
existent 'libs.js' module. Now it correctly generates a visualization
for the 'ui-libs.js` and 'bg-libs.js' modules.
The `.sh` file extension is now used for all Bash scripts. This ensures
the files are recognized as Bash scripts by the ShellCheck CI job, and
by editors/IDEs for improved syntax highlighting.
The `verify-locale-strings` script now ignores unit tests. This ensures
the use of a string literal in a unit test won't mistakenly make this
script believe that a message is used in the extension.
This came up recently in #10396, where the deletion of unit tests for
dead code triggered an unused message lint failure. This was then fixed
in #10395.
* ci - run storybook and add to build-artifacts
* ci/storybook - rename storybook build path and fix artifact upload
* ci/storybook - rename link text
* clean - remove accidently committed storybook build dir
* storybook - fix image path to relative (#10364)
The `verify-locale-strings.js` script now validates that the
descriptions from the `en` locale are also present in all other
locales.
These descriptions are intended to help with translation, and are not
meant to be translated. This check will ensure that translators don't
accidentally translate these. It also ensures they're present alongside
each translated message, which might be helpful for understanding
context.
The report on missing messages has been removed from the verify
locales script. This report was making the console output of this
command unreasonably long, and would obscure the reports on any invalid
entries.
A new script was written to report on missing localized messages.
This can be run with the command `yarn locale-coverage`. This will
print a report to the console on the coverage for each locale.
The environment variables `METAMETRICS_PROJECT_ID` and
`ETH_GAS_STATION_API_KEY` were still being injected into the JavaScript
build, despite being unused. The MetaMetrics project ID was made
obsolete in #9646, and the ETH Gas Station API key was made obsolete in
PR #9867
* Maintain console logging in dev mode
Co-authored-by: kumavis <aaron@kumavis.me>
Co-authored-by: Erik Marks <rekmarks@protonmail.com>
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
The Firefox e2e tests now use the `.zip` file for testing the
extension. We've found this to produce more similar results to
production, compared to the old method of loading the unzipped
directory.
Passing in a `.zip` file to the Chrome driver didn't seem to work. I
didn't investigate this further to see if it was possible, but I'm not
sure it makes a difference on Chrome anyway.
When you load an extension `.zip` file in Firefox, it fails to load
scripts with the `.cjs` file extension. However, it works if you load
the extension via the `manifest.json` file instead.
After renaming the `lockdown.cjs` file to `lockdown.js`, it works in
Firefox in all cases, regardless whether it's loaded by manifest or by
`.zip`.
* Remove use of ethgassthat; use metaswap /gasPrices api for gas price estimates
* Remove references to ethgasstation
* Pass base to BigNumber constructor in fetchExternalBasicGasEstimates
* Update ui/app/hooks/useTokenTracker.js
Co-authored-by: Erik Marks <25517051+rekmarks@users.noreply.github.com>
* Delete gas price chart
* Remove price chart css import
* Delete additional fee chart code
* Lint fix
* Delete more code no longer used after ethgasstation removal
Co-authored-by: Erik Marks <25517051+rekmarks@users.noreply.github.com>
* Freezeglobals: remove Promise freezing, add lockdown
* background & UI: temp disable sentry
* add loose-envify, dedupe symbol-observable
* use loose envify
* add symbol-observable patch
* run freezeGlobals after sentry init
* use require instead of import
* add lockdown to contentscript
* add error code in message
* try increasing node env heap size to 2048
* change back circe CI option
* make freezeGlobals an exported function
* make freezeGlobals an exported function
* use freezeIntrinsics
* pass down env to child process
* fix unknown module
* fix tests
* change back to 2048
* fix import error
* attempt to fix memory error
* fix lint
* fix lint
* fix mem gain
* use lockdown in phishing detect
* fix lint
* move sentry init into freezeIntrinsics to run lockdown before other imports
* lint fix
* custom lockdown modules per context
* lint fix
* fix global test
* remove run in child process
* remove lavamoat-core, use ses, require lockdown directly
* revert childprocess
* patch package postinstall
* revert back child process
* add postinstall to ci
* revert node max space size to 1024
* put back loose-envify
* Disable sentry to see if e2e tetss pass
* use runLockdown, add as script in manifest
* remove global and require from runlockdown
* add more memory to tests
* upgrade resource class for prep-build & prep-build-test
* fix lint
* lint fix
* upgrade remote-redux-devtools
* skillfully re-add sentry
* lintfix
* fix lint
* put back beep
* remove envify, add loose-envify and patch-package in dev deps
* Replace patch with Yarn resolution (#9923)
Instead of patching `symbol-observable`, this ensures that all
versions of `symbol-observable` are resolved to the given range, even
if it contradicts the requested range.
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
The `sentry:publish` script now exits with an exit code of `1` upon
failure, indicating that something went wrong. Previously it would exit
with a code of `0`, indicating to CI that everything worked correctly.
The script will now also exit early if the authentication check fails.
This mock Segment server can be used to test our extension metrics. It
will respond to all request with HTTP 200, and will print the requests
to the console. It also has parsing built-in for Segment request
payloads.
Right now only the event name is printed, but we can enhance this in
the future to print more event information. We can also enhance the
mock to be a more realistic representation of the API.
The extension has been modified to allow the Segment host to be
overwritten with the `SEGMENT_HOST` environment variable. This will
ensure that all Segment events are redirected to that host.
So for example, to create a dev build that uses this server, you could
set the `SEGMENT_WRITE_KEY` and `SEGMENT_LEGACY_WRITE_KEY` values to
any non-empty string, and set `SEGMENT_HOST` to
`http://localhost:9090`.
This was created originally to test PR #9768
It was getting rather complicated to keep track of which Segment keys
were set where, and under which name.
The build script now injects a key even in test environments, but it is
unused if `IN_TEST` is truthy. This should be functionally equivalent
to the old logic. I find this simpler mainly for two reasons: there is
one less intermediate variable to keep track of now, and the `IN_TEST`
check is now directly in the module where we're constructing the
`segment` instance, rather than being referenced at a distance in a
comment.
The old setup made it difficult to turn on metrics for specific e2e
tests as well, which will be done in a subsequent PR.
The `devMode` parameter being passed to the `buildScss` function was
not being used. The `buildScss` function was declared _inside_ the
function in which it is invoked, so the `devMode` variable is already
in scope - it doesn't need to be passed in.
The `styles:dev` step of the development build was broken in #9568 when
the `lockdown()` function from SES was introduced. One of the effects
of `lockdown()` is preventing the use of `RegExp.lastMatch`, which is
relied upon by `postcss` for parsing inline sourcemaps.
This problem has been worked around by re-arranging the build pipeline
for styles.
Firstly, the `autoprefixer` stage was moved _before_ sourcemaps were
generated. Its placement _after_ sourcemaps was a mistake in the first
place - it should have always been before they were written, so that
the sourcemaps could keep track of the changes made by `autoprefixer`.
Second, the `-rtl` sourcemap generation was moved to a separate build
pipeline. The `rtl` stage also uses `postcss`, and it must come after
the sourcemaps are written for the non-RTL styles, so splitting it into
a separate build was the only way to avoid passing in inline
sourcemaps. This does make the build slightly slower, but it also
decreases the size of the RTL sourcemaps dramatically.
* Add ses lockdown to build system using lavamoat-core
* use proper object.assign version
* disable lint rules for ses lockdown
* deps - update rtlcss
Co-authored-by: kumavis <aaron@kumavis.me>
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
`@metamask/eslint-config` has been updated to v4.1.0. This update
requires that we update `eslint` to v7 as well, which in turn requires
updating most `eslint`-related packages.
Most notably, `babel-eslint` was replaced with `@babel/eslint-parser`,
and `babel-eslint-plugin` was replaced by `@babel/eslint-plugin`. This
required renaming all the `babel/*` rules to `@babel/*`.
Most new or updated rules that resulted in lint errors have been
temporarily disabled. They will be fixed and re-enabled in subsequent
PRs.
On Windows, spawn fails if the exact filename
of a binary isn't passed. e.g. `spawn('yarn')` fails
because the binary is named `yarn.cmd`.
Instead, we depend on `cross-spawn` which handles differences
in `spawn` across platforms.
* Delete localhost provider type
* Use ganache-cli default chain ID for tests
* Delete unused test firstTimeState variable
* Migrate default ganache-cli network to frequentRpcListDetail
* Add default test provider state
* Add test functionality to createJsonRpcClient
* Lint locales
* Update test middleware creation
* fixup! Update test middleware creation
* Fix require-unicode-regexp issues
See [`require-unicode-regexp`](https://eslint.org/docs/rules/require-unicode-regexp) for more information.
This change enables `require-unicode-regexp` and fixes the issues raised by the rule.
* Remove case-insensitive flag from regexps
All JavaScript files included in the final bundle are now listed as
target files. The `phishing-detect.js` file is the only new file to be
validated that was not validated before.
Any files that are expected to fail validation are commented out, with
a note explaining why they're expected to fail.
The report printed to the console for invalid source map samples has
been improved in a few ways:
* The entire message is now printed using `console.error`, so the
contents aren't split between STDERR and STDOUT
* The code fence is now guaranteed to be a set length, rather than it
varying depending on the filename
* The code fence is no longer padded on the inside with newlines, which
results in a more compact output that is (in my opinion) just as
readable.
Our source maps were being corrupted during minification, because the
`gulp-terser-js` plugin we were using didn't account for the existence
of sourcemaps in the input. A configuration option to allow the input
of sourcemaps was added in v5.2.0. The plugin has been updated, and we
now use this option.
Previously the generated sourcemaps had an invalid entry in the
"sources" array, with the filename of the bundle itself. This was not a
real source. After this change, this invalid source is no longer
present.
The Sentry DSN is now expected to be provided via environment variable
for production builds. The build script will fail if it is missing, and
an error will be thrown at runtime if it is missing.
The `SENTRY_DSN` environment variable has been set in CI to the old
value for `SENTRY_PROD_DSN`. We can migrate to a new DSN at some point
in the future.
The MetaMetrics project ID can now be set via environment variable. It
has not been set yet in practice, so for now the old project IDs will
still be used. This is in preparation for migrating to a new project.
The remaining integration tests are all covered by e2e tests, so
they're no longer needed.
All associated scripts, fixtures, and dependencies have also been
removed.
The `createStaticServer` function was split from the `static-server.js`
script, so that the static server could be constructed
programmatically. `static-server.js` remains responsible for the CLI.
This was done to make it easier to programmatically start the test dapp
from e2e tests.
The `static-server` script now exits with a code of `1` upon failure.
Previously it would print the error to the console but exit with a code
of `0`, indicating success.
We were including the polyfill for the `Intl.RelativeTimeFormat` API,
but we weren't including any locale data. This polyfill doesn't work
without the locale data for whichever locale you're formatting.
The data for all locales we support is now included. The locale data
is loaded from disk as-needed (during app startup, and upon each change
in locale).
* update connected accounts appearance
* consolidate account alerts
* UnconnectedAccountAlert: use ConnectedAccountsList
* move switch account action out of menu in all views
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
A new page has been created for viewing assets. This replaces the old
`selectedToken` state, which previously would augment the home page
to show token-specific information.
The new asset page shows the standard token overview as seen previously
on the home page, plus a history filtered to show just transactions
relevant to that token.
The actions that were available in the old token list menu have been
moved to a "Token Options" menu that mirrors the "Account Options"
menu.
The `selectedTokenAddress` state has been removed, as it is no longer
being used for anything.
`getMetaMetricState` has been renamed to `getBackgroundMetaMetricState`
because its sole purpose is extracting data from the background state
to send metrics from the background. It's not really a selector, but
it was convenient for it to use the same selectors the UI uses to
extract background data, so I left it there for now.
A new Redux store has been added to track state related to browser history.
The most recent "overview" page (i.e. the home page or the asset page) is
currently being tracked, so that actions taken from the asset page can return
the user back to the asset page when the action has finished.
Add alert suggesting that the user switch to a connected account. This
alert is displayed when the popup is opened over an active tab that is
connected to some account, but not the current selected account. The
user can choose to switch to a connected account, or dismiss the alert.
This alert is only shown once per account switch. So if the user
repeatedly opens the popup on a dapp without switching accounts, it'll
only be shown the first time. The alert also won't be shown if the user
has just dismissed an "Unconnected account" alert on this same dapp
and account, as that would be redundant.
The alert has a "Don't show me this again" checkbox that allows the
user to disable the alert. It can be re-enabled again on the Alerts
settings page.
* Remove `estimatedGas` property from `txMeta`
The `estimatedGas` property was a cache of the gas value estimated for
a transaction when the default gas limit was set. This property wasn't
used anywhere. It may have been useful for debugging purposes, but the
same gas estimate is already stored on the `history` property so it
should be present in state logs regardless.
* Remove `gasLimitSpecified` txMeta property
The `gasLimitSpecified` property of `txMeta` wasn't used for anything.
It might have been useful for debugging purposes, but whether or not
the gas limit was specified can also be determined from looking at the
transaction history, so it's not a huge loss.
* Remove `gasPriceSpecified` txMeta property
The `gasPriceSpecified` property of `txMeta` wasn't used for anything.
It might have been useful for debugging purposes, but whether or not
the gas price was specified can also be determined from looking at the
transaction history, so it's not a huge loss.
* Remove `simpleSend` txMeta property
The `simpleSend` property of `txMeta` was used to ensure a buffer was
not added to the gas limit during gas estimation for simple send
transactions. It was made redundant by #8484, which accomplishes this
without the use of this property.
An alert is now shown when the user switches from an account that is
connected to the active tab to an account that is not connected. The
alert prompts the user to dismiss the alert or connect the account
they're switching to.
The "loading" state is handled by disabling the buttons, and the error
state is handled by displaying a generic error message and disabling
the connect button.
The new reducer for this alert has been created with `createSlice` from
the Redux Toolkit. This utility is recommended by the Redux team, and
represents a new style of writing reducers that I hope we will use more
in the future (or at least something similar). `createSlice` constructs
a reducer, actions, and action creators automatically. The reducer is
constructed using their `createReducer` helper, which uses Immer to
allow directly mutating the state in the reducer but exposing these
changes as immutable.
The page tested by the benchmark was changed from `notification` to
`home` in #8358, but the announce script was still expecting the
`notification` page to be in the results. It does collect results for
all pages, but the `notification` page was hard-coded to be used for
the benchmark summary.
The announce script now correctly looks for the `home` page results for
the benchmark summary. Variable names have been updated to make it more
clear what's going on here as well.
The `shift-list-item` component for displaying ShapeShift transactions
has been removed, along with three other components that were used
solely by that component (`copyButton`, `eth-balance`, and
`fiat-value`).
This component hasn't been used in some time, as ShapeShift
transactions no longer exist to display. The controller that ShapeShift
transactions originated from was removed in #8118, and it became
impossible to create new ShapeShift transactions from within MetaMask
in #6746
The `transForward` app state is no longer used, so it has been removed.
Associated actions have been removed as well.
This state dates back a few years, so I was unable to determine when it
was made obsolete.
These state files were snapshots of the Redux state used for
integration tests that have since been removed. Only three of these
states were still used - the three that correspond with the remaining
integration tests.
The default state used when the test environment is spun up was changed
to be the first of these three (`confirm sig requests`) so that the old
default state could safely be removed as well.
Previously all browser globals were allowed to be used anywhere by
ESLint because we had set the `env` property to `browser` in the ESLint
config. This has made it easy to accidentally use browser globals
(e.g. #8338), so it has been removed. Instead we now have a short list
of allowed globals.
All browser globals are now accessed as properties on `window`.
Unfortunately this change resulted in a few different confusing unit
test errors, as some of our unit tests setup assumed that a particular
global would be used via `window` or `global`. In particular,
`window.fetch` didn't work correctly because it wasn't patched by the
AbortController polyfill (only `global.fetch` was being patched).
The `jsdom-global` package we were using complicated matters by setting
all of the JSDOM `window` properties directly on `global`, overwriting
the `AbortController` for example.
The `helpers.js` test setup module has been simplified somewhat by
removing `jsdom-global` and constructing the JSDOM instance manually.
The JSDOM window is set on `window`, and a few properties are set on
`global` as well as needed by various dependencies. `node-fetch` and
the AbortController polyfill/patch now work as expected as well,
though `fetch` is only available on `window` now.
The official npm package for Font Awesome Free is now used instead of
the vendored styles. Previously we had been using v4.4.0, now we're
using v5.13.0.
We're now importing the Font Awesome SCSS modules instead of using the
minified CSS bundle. This integrates more cleanly into our build
system, and it lets us use their mixins directly in the future if we
need to.
The variable `fa-font-path` has been set to reference our font
directory, as instructed here:
https://fontawesome.com/how-to-use/on-the-web/using-with/sass#compile
This was done to reduce the number of direct dependencies we have. It
should be functionally equivalent. The bundle size should not change,
as we use `clone` as a transitive dependency in a number of places.
Our JSDoc documentation has not been updated in a very long time, and
we don't use JSDoc in enough places for the docs to have been
especially useful. The tools and scripts used to generate and publish
these docs have been removed.
References to this documentation have also been removed from the
README.
Hopefully once the TypeScript migration has made substantial progress,
we can generate more useful documentation using something like TypeDoc.
`node-fetch` is a smaller and simpler project than `request-promise`,
and we already have it as a transitive dependency.
`request-promise` was also incorrectly listed as a production
dependency. `node-fetch` has been added as a `devDependency` to replace
it, as it was only used in one CI script.
* Add benchmark to CI
The page load benchmark for Chrome is now run during CI, and the
results are collected and summarized in the `metamaskbot` comment.
Closes#6881
* Double default number of samples
The number of default samples was changed from 10 to 20. The results
from 10 samples would show statistically significant changes in page
load times between builds, so weren't a sufficiently useful metric.
* Add lint:shellcheck:package
* Add double quote as per jq suggestion
* Use single quotes
* Ignore SC2016
* Use shellcheck script
* Set some flags
* Put shellcheck --version on new line
* Disable SC2016 (singe quotes is actually what we want here)
This rule prevents using an async function as a Promise executor (e.g.
as the argument to a `Promise` constructor). This pattern is usually a
mistake because it implies that the Promise constructor was not
necessary in the first place. It also makes error handling difficult,
as any errors thrown would be uncaught unless you wrap the body in a
try/catch block.
* Add support for one-click onboarding
MetaMask now allows sites to register as onboarding the user, so that
the user is redirected back to the initiating site after onboarding.
This is accomplished through the use of the `metamask-onboarding`
library and the MetaMask forwarder.
At the end of onboarding, a 'snackbar'-stype component will explain to the
user they are about to be moved back to the originating dapp, and it will
show the origin of that dapp. This is intended to help prevent phishing
attempts, as it highlights that a redirect is taking place to an untrusted
third party.
If the onboarding initiator tab is closed when onboarding is finished,
the user is redirected to the onboarding originator as a fallback.
Closes#6161
* Add onboarding button to contract test dapp
The `contract-test` dapp (run with `yarn dapp`, used in e2e tests) now
uses a `Connect` button instead of connecting automatically. This
button also serves as an onboarding button when a MetaMask installation
is not detected.
* Add new static server for test dapp
The `static-server` library we were using for the `contract-test` dapp
didn't allow referencing files outside the server root. This should
have been possible to work around using symlinks, but there was a bug
that resulted in symlinks crashing the server.
Instead it has been replaced with a simple static file server that
will serve paths starting with `node_modules` from the project root.
This will be useful in testing the onboarding library without vendoring
it.
* Add `@metamask/onboarding` and `@metamask/forwarder`
Both libraries used to test onboarding are now included as dev
dependencies, to help with testing. A few convenience scripts
were added to help with this (`yarn forwarder` and `yarn dapp-forwarder`)
* Add shellcheck lint script
* Add to build
* Add shellcheck lint to main lint task
* Put shellcheck in the right place, hopefully?
* Fix declared multiple executor types
* Add sudo
* Address shellcheck warnings
* Add test-lint-shellcheck
* Add test-lint-shellcheck to workflow
* Use correct lint task
* output version which could be helpful for debugging
* Address PR feedback
* consistency++
* Delete unused code
* Run threebox imports through migrations
* Remove 3box feature flag
* Remove unnecessary use of 'type' in threebox._updatePlugin
* Fix threebox controller getLastUpdated
* Turn off threebox by default
* Rename restoredFromThreeBox to showRestorePrompt
* Remove accientally added method from threebox controller
* Restore from threebox on import from unlock screen
* Throw on non 404 errors from Box.getconfig in new3Box
* ci - install deps - limit install scripts to those needed for build
* Update .circleci/scripts/deps-install.sh
Co-Authored-By: Mark Stacey <markjstacey@gmail.com>
* ci - install deps - expand install scripts needed for tests
* ci - install deps - expand install scripts needed for integration tests
* ci - install deps - fix node-sass script ref
* github - set codeowners for scripts/deps-install
* development - add utility to show deps with install scripts
* lint fix
* deps - move read-installed to devDeps
* Adds ab test controller with a fullScreenVsPopup test
* Add migration for fullScreenVsPopup state
* Move abtest state under an 'abtests' object.
* MetaMask shows fullScreen group of a/b test unapproved txs in a full browser tab
* Ensure cancel metrics event in confirm-transaction-base.component.js is sent in all cases
* Switch to existing tab for unapproved tx if it exists when opening in full screen
* Send metrics event for entering a/b test from confirm screen
* Fix lint, unit and integration tests related to a/b test code
* Remove unnecessary tabs.query call in triggerUiInNewTab
* Add '--quiet' flag to verify locales script
The `--quiet` flag reduces the console output to just the essential
information for running in a CI environment. For each locale, it will
print the number of unused messages (if any).
* Add `verify-locales` script to lint CI job
The locales are now verified as part of the lint CI job. Any unused
messages detected will result in the job failing.
* Switch to using string literals for locale keys
Various message keys were being specified with a string template
instead of a string literal. They have been switched to use string
literals so that the script for detecting unused messages can find
them.
* Remove unused locale messages
A number of unused locale messages have been removed - probably
leftover from old UI elements that have since been removed.
The `verify_locale_strings` script has been augmented to search the UI
for string literals, and match those against the locale message keys in
the `en` locale. Any messages without a corresponding string literal
are assumed to be unused.
The script has also been updated with an optional `--fix` parameter,
which will automatically delete any unused messages from locales.
148 unused messages were found in this case, out of a total of about
650 messages. Another 70 messages are _potentially_ unused and require
further investigation, but weren't as easy to rule out because they
were found in string literals.
* Remove additional unused locale messages
The following messages were more difficult to rule out because they
were present as string literals in the UI. They do appear to be
unused as locale keys though.
* Adds threebox controller
* Adds threebox approval modal
* Fix unit tests and lint after addition of threebox
* Correct threebox behaviour after rejecting request for backup; fixes e2e tests.
* Update threebox controller for automatic syncing
* Ensure frontend locale updates when preferences are changed via direct update within controller
* Add toggle in settings for 3box syncing
* Update threebox controller for latest 3box version
* Delete unnecessary frontend changes for threebox integration
* Backing up address book contacts with threebox
* Update unit tests for 3box-integration additions
* Only enable threebox by default for new wallets
* Mock globals for correct unit tests
* 3box '1.10.2' -> '^1.10.2'
* Correct capilalization on 3Box
* Use log.debug instead of console.log in threebox controller
* Update yarn.lock
* Remove edge build
* Split 3box module into background deps js file
* extra bundle opts for bg-libs
* sync yarn.lock
* new3Box logic
* Show confirm threebox restore after import
* Remove bg-libs.js from manifest file for dev builds
* Switch 3Box controller to using the spaces api (instead of the profile api)
* Finalize switching to spaces api and only restoring from 3box after import
* Update metamask-controller-test.js for threebox controller changes
* Make threebox modal style consistent with others and update success button wording
* Use mock 3box when in test
* Correct 3box modal header
* Remove unnecessary property of threebox controller provider
* Remove unnecessary method calls after restoration from 3box in the threebox-restore-confirm modal.
* Replace setThreeBoxSyncingPermission calls in routes/index.js with turnThreeBoxSyncingOn
* Replace erroneous use of with
* Replace erroneous use of threeboxSyncing with threeBoxSyncingAllowed in advancted-tab directory
* Lint fixes for 3box changes
* Log errors encountered when updating 3Box
* Remove unnecessary parameter from state update
* Add timeout to initial 3Box sync
The initial 3Box sync will now timeout after 1 minute. If the timeout
is triggered, 3Box is disabled and cannot be re-enabled unless the
initial sync does finally finish. If it never finishes, 3Box cannot
be enabled unless the extension is reinstalled.
The Advanced Settings page was updated to show this option as disabled
in that circumstance, with a new discription explaining why it's
disabled. The UI here could certainly be improved.
Additionally, "on" and "off" labels were added to the toggle to match
the other toggles on the Advanced Settings page.
* Use non-minified 3Box module
We had previously used the minified 3Box module to avoid a build error
encountered when `envify` was processing the `libp2p` module (which is
used by 3Box). The build would fail because `esprima` (used by `envify`)
is incompatible with the object spread/rest operator (which is used in
`libp2p`).
That issue has been solved by adding a global Babelify transformation
specifically for transpiling out the object rest/spread operator from
dependencies. It has been targetted to only affect `libp2p` to avoid
extending the build time too much. This workaround can be used until
a new version of `esprima` is released that includes this bug fix.
* Use app key addresses for threebox
* Replace use of modal for confirming 3box restoration with a home notification
* Adds e2e tests for restoring from threebox
* Update eth-keyring-controller to 5.1.0
* Correct parameters passed to getAppKeyAddress in threebox.js
* Add prefix to origin passed to getAppKeyAddress in threebox.js
* Remove unused locale message.
* Prevent CORS errors in firefox e2e tests
* Ensure extraneous scripts are excluded from the local test dev build
* Move threeBoxLastUpdate state from home.component to redux
* Threebox PR code cleanup
* Always use first address when initializing threebox
* Replace setRestoredFromThreeBox api with setRestoredFromThreeBoxToFalse and setRestoredFromThreeBoxToTrue
* Update development/metamaskbot-build-announce.js to include ui-libs and bg-libs in hard coded bundle list
* Update test/e2e/threebox.spec.js to use new helpers added with pull #7144
* Make setFeatureFlag available on the ui window during testing
* Hide threebox feature behind a feature flag that can only be activated via dev console
* Remove unnecessary migration of threebox feature flag
* Prevent this.init() call in threebox constructor if feature flag is not turned on
* Prevent threebox notification from showing if feature flag is falsy
* http://localhost/8889 -> http://localhost/* in gulp manifest:testing tasks
* ci - create source-map-explorer build-artifacts
* ci - add source-map-explorer builds to metamaskbot comment
* lint fix
* ci - source-map-explorer - include all bundles
* Add extra locales to index.json
* Fix verify locale script bugs
- All locales didn't work because it was putting an object inside of an object.
- Individual checks didn't work because of how the destructuring was done.
- Extra items were being printed as missing items.
* Sync up string changes w/ Brave wallet
* Add 33 new locales from Brave wallet
* Merge tml into ta code
As a solution to the constant lockfile churn issues we've had with
`npm`, the project now uses `yarn` to manage dependencies.
The `package-lock.json` file has been replaced with `yarn.lock`, which
was created using `yarn import`. It should approximate the contents of
`package-lock.json` fairly well, though there may be some changes due to
deduplication. The codeowners file has been updated to reference this
new lockfile.
All documentation and npm scripts have been updated to reference `yarn`
rather than `npm`. Note that running scripts using `npm run` still works
fine, but it seemed better to switch those to `yarn` as well to avoid
confusion.
The `npm-audit` Bash script has been replaced with `yarn-audit`. The
output of `yarn audit` is a bit different than `npm audit` in that it
returns a bitmask to describe which severity issues were found. This
made it simpler to check the results directly from the Bash script, so
the associated `npm-audit-check.js` script was no longer required. The
output should be exactly the same, and the information is still sourced
from the same place (the npm registry).
The new `yarn-audit` script does have an external dependency: `jq`.
However, `jq` is already assumed to be present by another CI script, and
is present on all CI images we use. `jq` was not added to `package.json`
as a dependency because there is no official package on the npm
registry, just wrapper scripts. We don't need it anywhere exept on CI
anyway.
The section in `CONTRIBUTING` about how to develop inside the
`node_modules` folder was removed, as the advice was a bit dated, and
wasn't specific to this project anyway.
The CSS is now served as an external file instead of being injected.
This was done to improve performance. Ideally we would come to a middle
ground between this and the former behaviour by injecting only the CSS
that was required for the initial page load, then lazily loading the
rest. However that change would be more complex. The hope was that
making all CSS external would at least be a slight improvement.
Performance metrics were collected before and after this change to
determine whether this change actually helped. The metrics collected
were the timing events provided by Chrome DevTools:
* DOM Content Loaded (DCL) [1]
* Load (L) [2]
* First Paint (FP) [3]
* First Contentful Paint (FCP) [3]
* First Meaningful Paint (FMP) [3]
Here are the results (units in milliseconds):
Injected CSS:
| Run | DCL | L | FP | FCP | FMP |
| :--- | ---: | ---: | ---: | ---: | ---: |
| 1 | 1569.45 | 1570.97 | 1700.36 | 1700.36 | 1700.36 |
| 2 | 1517.37 | 1518.84 | 1630.98 | 1630.98 | 1630.98 |
| 3 | 1603.71 | 1605.31 | 1712.56 | 1712.56 | 1712.56 |
| 4 | 1522.15 | 1523.72 | 1629.3 | 1629.3 | 1629.3 |
| **Min** | 1517.37 | 1518.84 | 1629.3 | 1629.3 | 1629.3 |
| **Max** | 1603.71 | 1605.31 | 1712.56 | 1712.56 | 1712.56 |
| **Mean** | 1553.17 | 1554.71 | 1668.3 | 1668.3 | 1668.3 |
| **Std. dev.** | 33.41 | 33.43 | 38.16 | 38.16 | 38.16 |
External CSS:
| Run | DCL | L | FP | FCP | FMP |
| :--- | ---: | ---: | ---: | ---: | ---: |
| 1 | 1595.4 | 1598.91 | 284.97 | 1712.86 | 1712.86 |
| 2 | 1537.55 | 1538.99 | 199.38 | 1633.5 | 1633.5 |
| 3 | 1571.28 | 1572.74 | 268.65 | 1677.03 | 1677.03 |
| 4 | 1510.98 | 1512.33 | 206.72 | 1607.03 | 1607.03 |
| **Min** | 1510.98 | 1512.33 | 199.38 | 1607.03 | 1607.03 |
| **Max** | 1595.4 | 1598.91 | 284.97 | 1712.86 | 1712.86 |
| **Mean** | 1553.8025 | 1555.7425 | 239.93 | 1657.605 | 1657.605 |
| **Std. dev.** | 29.5375 | 30.0825 | 36.88 | 37.34 | 37.34 |
Unfortunately, using an external CSS file made no discernible improvement
to the overall page load time. DCM and L were practically identical, and
FCP and FMP were marginally better (well within error margins).
However, the first paint time was _dramatically_ improved. This change
seems worthwhile for the first paint time improvement alone. It also
allows us to delete some code and remove a dependency.
The old `css.js` module included two third-party CSS files as well, so
those have been imported into the main Sass file. This was easier than
bundling them in the gulpfile.
The resulting CSS bundle needs to be served from the root because we're
using a few `@include` rules that make this assumption. We could move
this under `/css/` if desired, but we'd need to update each of these
`@include` rules.
Relates to #6646
[1]: https://developer.mozilla.org/en-US/docs/Web/Events/DOMContentLoaded
[2]: https://developer.mozilla.org/en-US/docs/Web/Events/load
[3]: https://developers.google.com/web/fundamentals/performance/user-centric-performance-metrics
These files were referencing npm scripts that no longer existed. Notices
appear to no longer exist, and the `ui-dev.js` module is no longer
actively used.
The `mock-dev.js` module is still used for certain integration tests, so
I've just removed the reference to the non-existent script.
We've been using the `eslint-plugin-json` plugin for some time, but we
haven't been visiting `.json` files in the lint script. The lint script
has now been updated to incude `.json` files, which means any invalid
JSON will result in a lint error.
Unfortunately this JSON plugin doesn't seem to apply the other eslint
rules (such as `key-spacing`) to the JSON files. I wasn't able to find a
way to get that to work. Instead I manually auto-formatted each of the
locale `message.json` files, which fixed many whitespace
inconsistencies.
The `states.json` file was deleted completely, as it appears to be
unused. It wasn't a valid JSON file anyway, it was JavaScript. It looks
like a `states.js` file is automatically generated, but an old copy was
accidentally saved as `states.json` and included in the repo.
Many duplicate key errors were found and fixed in the
`development/states/` JSON files.
`package-lock.json` was added to `.eslintignore` because it was very
slow to lint, and linting it doesn't provide much value.
The AppVeyor configuration appears to be unused - this project is not
connected to AppVeyor.
After deleting the AppVeyor config, the JSDoc config was the last thing
in the `development/tools` directory. That felt a little silly, so I
moved it up to `development`.
* Add React and Redux DevTools
* Conditionally load react-devtools
* Add start:dev npm script to run the app with devtools
Co-Authored-By: Mark Stacey <markjstacey@gmail.com>
* Improve auto changelog script
The auto changelog script was creating empty or invalid entries in a
number of different cases, such as when the body of a commit spanned
multiple lines. This has been fixed, and the following additional
improvements have been made:
- Error handling (it will now crash upon encountering an error)
- Commits without a PR number in the subject are listed without the PR
prefix
- Invalid shellcheck warnings ignored
- Only the first line of the commit body is shown
- Carriage returns are stripped (some commits contain them)
This script should be more reliable for helping to manually update the
changelog. It's still not sufficiently robust to use as part of an
automated process - I don't think that's feasible without maintaining
stricter control over commit messages conventions and/or merge
strategies.
* Remove unused state 'gas.basicPriceAndTimeEstimates'
* Check for invalid estimates from local storage
Gas estimates were being cached in local storage then later retrieved,
but the retrieved values were not being checked. If the data failed to
save, failed to load, or was cleared since being saved, it would result
in the gas estimates being set to undefined.
The estimates retrieved from local storage are now checked before they
are used. If they are falsy, the estimates are retrieved from the
network instead.
This should fix this Sentry issue:
[METAMASK-6W0T](https://sentry.io/share/issue/cfe470314a5741768b19050815322aa4/)
A few additional changes were made to the gas-duck tests to accommodate
the use of `sinon.restore`. `restore` is strongly recommended by the
`sinon` team, as neglecting to use it can result in memory leaks. It has
the additional benefit of ensuring you create fresh stubs/spies for each
test, which means they no longer need to be reset between tests.
* Introduce delay for eth_estimateGas calls with in test
* Add test that fails when gas estimates of contract method calls without gas are too high.
* Get transaction gas data from unApprovedTxs instead of confirmTransaction
* Fix selection of gas data in gas-modal-page-container.container
* Lint changes related to Version-6.7.2-gasLimitFix
* Fix e2e tests on Version-6.7.2-gasLimitFix
* Fix unit and integration tests for changes from Version-6.7.2-gasLimitFix
* more e2e fixes
* Add assertions for transaction values on confirm screen
* Fix display of transaction amount on confirm screen.
* Introduce delay for eth_estimateGas calls with in test
* Add test that fails when gas estimates of contract method calls without gas are too high.
* Get transaction gas data from unApprovedTxs instead of confirmTransaction
* Fix selection of gas data in gas-modal-page-container.container
* Lint changes related to Version-6.7.2-gasLimitFix
* Fix e2e tests on Version-6.7.2-gasLimitFix
* Fix unit and integration tests for changes from Version-6.7.2-gasLimitFix
* more e2e fixes
* Add assertions for transaction values on confirm screen
* Fix display of transaction amount on confirm screen.
* Add networks tab to settings, with header.
* Adds network list to settings network tab.
* Adds form to settings networks tab and connects it to network list.
* Network tab: form adding and editing working
* Settings network form properly handles input errors
* Add translations for settings network form
* Clean up styles of settings network tab.
* Add popup-view styles and behaviour to settings network tab.
* Fix save button on settings network form
* Adds 'Add Network' button and addMode to settings networks tab
* Lint fix for settings networks tab addition
* Fix navigation in settings networks tab.
* Editing an rpcurl in networks tab does not create new network, just changes rpc of old
* Fix layout of settings tabs other than network
* Networks dropdown 'Custom Rpc' item links to networks tab in settings.
* Update settings sidebar networks subheader.
* Make networks tab buttons width consistent with input widths in extension view.
* Fix settings screen subheader height in popup view
* Fix height of add networks button in popup view
* Add optional label to chainId and symbol form labels in networks setting tab
* Style fixes for networks tab headers
* Add ability to customize block explorer used by custom rpc
* Stylistic improvements+fixes to custom rpc form.
* Hide cancel button.
* Highlight and show network form of provider by default.
* Standardize network subheader name to 'Networks'
* Update e2e tests for new settings network form
* Update unit tests for new rpcPrefs prop
* Extract blockexplorer url construction into method.
* Fix broken styles on non-network tabs in popup mode
* Fix block explorer url links for cases when provider in state has not been updated.
* Fix vertical spacing of network form
* Don't allow click of save button on network form if nothing has changed
* Ensure add network button is shown in popup view
* Lint fix for networks tab
* Fix block explorer url preference setting.
* Fix e2e tests for custom blockexplorer in account details modal changes.
* Update integration test states to include frequentRpcList property
* Fix some capitalizations in en/messages.json
* Remove some console.logs added during custom rpc form work
* Fix external account link text and url for modal and dropdown.
* Documentation, url validation, proptype required additions and lint fixes on network tab and form.
