1
0
mirror of https://github.com/kremalicious/metamask-extension.git synced 2024-12-23 09:52:26 +01:00
Commit Graph

294 Commits

Author SHA1 Message Date
PeterYinusa
c07e477c13
E2e metrics (#13904)
* remove metrics build

* remove segment server from tests

* enable cors

* mock segment globally

* metrics e2e test

* running test builds

* move file

* destructuring
2022-03-15 13:17:48 -03:00
Mark Stacey
6aaeab2f24
Automate the Flask release process (#13898)
* Automate the Flask release

A Flask release will now be published alongside each main extension
release. The version of each Flask release will be the same as the
extension version except it will have the suffix `-flask.0`.

* Programmatically remove build prefix

The create GH release Bash script derives the Flask version from the
Flask build filename by removing the build prefix, leaving just the
version. Rather than hard-coding the prefix size to remove, it is now
calculated programmatically so that it is easier to read and update.

* Fix tag publishing

The tab publishing step used the wrong credentials, and didn't properly
identify the commit author. This has now been fixed.
2022-03-15 08:54:37 -02:30
Shane
1f55af3151
Fixed firefox Snaps + add initial end-to-end tests (#13671)
* Changed registryUrl for snaps only in firefox
Fixed getPlatform to only be imported into metamask-controller in flask
Removed snaps specific testrunner script and use run-all with a cli option

* Fixed flakey tests

* Removed unneeded await

* Added delay

* Fixed linting
2022-03-10 14:01:55 -08:00
PeterYinusa
4f6eb02854
update chromedriver (#13854) 2022-03-07 19:05:58 +00:00
PeterYinusa
2b1256faf4
Update to latest ChromeDriver (#13616) 2022-02-14 17:06:10 +00:00
PeterYinusa
2b010710ae
update chromedriver to v97 (#13502) 2022-02-03 15:02:56 +00:00
Brad Decker
022f0efcd7
Update vulnerability ignore list (#13390) 2022-01-25 14:34:50 -06:00
Erik Marks
31cf7c10a4
Permission System 2.0 (#12243)
# Permission System 2.0

## Background

This PR migrates the extension permission system to [the new `PermissionController`](https://github.com/MetaMask/snaps-skunkworks/tree/main/packages/controllers/src/permissions).
The original permission system, based on [`rpc-cap`](https://github.com/MetaMask/rpc-cap), introduced [`ZCAP-LD`](https://w3c-ccg.github.io/zcap-ld/)-like permissions to our JSON-RPC stack.
We used it to [implement](https://github.com/MetaMask/metamask-extension/pull/7004) what we called "LoginPerSite" in [version 7.7.0](https://github.com/MetaMask/metamask-extension/releases/tag/v7.7.0) of the extension, which enabled the user to choose which accounts, if any, should be exposed to each dapp.
While that was a worthwhile feature in and of itself, we wanted a permission _system_ in order to enable everything we are going to with Snaps.
Unfortunately, the original permission system was difficult to use, and necessitated the creation of the original `PermissionsController` (note the "s"), which was more or less a wrapper for `rpc-cap`.

With this PR, we shake off the yoke of the original permission system, in favor of the modular, self-contained, ergonomic, and more mature permission system 2.0.

Note that [the `PermissionController` readme](https://github.com/MetaMask/snaps-skunkworks/tree/main/packages/controllers/src/permissions/README.md) explains how the new permission system works.

The `PermissionController` and `SubjectMetadataController` are currently shipped via `@metamask/snap-controllers`. This is a temporary state of affairs, and we'll move them to `@metamask/controllers` once they've landed in prod.

## Changes in Detail

First, the changes in this PR are not as big as they seem. Roughly half of the additions in this PR are fixtures in the test for the new migration (number 68), and a significant portion of the remaining ~2500 lines are due to find-and-replace changes in other test fixtures and UI files.

- The extension `PermissionsController` has been deleted, and completely replaced with the new `PermissionController` from [`@metamask/snap-controllers`](https://www.npmjs.com/package/@metamask/snap-controllers).
- The original `PermissionsController` "domain metadata" functionality is now managed by the new `SubjectMetadataController`, also from [`@metamask/snap-controllers`](https://www.npmjs.com/package/@metamask/snap-controllers).
- The permission activity and history log controller has been renamed `PermissionLogController` and has its own top-level state key, but is otherwise functionally equivalent to the existing implementation.
- Migration number 68 has been added to account for the new state changes.
- The tests in `app/scripts/controllers/permissions` have been migrated from `mocha` to `jest`.

Reviewers should focus their attention on the following files:

- `app/scripts/`
  - `metamask-controller.js`
    - This is where most of the integration work for the new `PermissionController` occurs.
      Some functions that were internal to the original controller were moved here.
  - `controllers/permissions/`
    - `selectors.js`
      - These selectors are for `ControllerMessenger` selector subscriptions. The actual subscriptions occur in `metamask-controller.js`. See the `ControllerMessenger` implementation for details.
    - `specifications.js`
      - The caveat and permission specifications are required by the new `PermissionController`, and are used to specify the `eth_accounts` permission and its JSON-RPC method implementation.
        See the `PermissionController` readme for details.
  - `migrations/068.js`
    - The new state should be cross-referenced with the controllers that manage it.
      The accompanying tests should also be thoroughly reviewed.

Some files may appear new but have just moved and/or been renamed:

- `app/scripts/lib/rpc-method-middleware/handlers/request-accounts.js`
  - This was previously implemented in `controllers/permissions/permissionsMethodMiddleware.js`.
- `test/mocks/permissions.js`
  - A truncated version of `test/mocks/permission-controller.js`.

Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2021-12-06 19:16:49 -08:00
PeterYinusa
9babc8b804
update chromedriver to v96 (#12921) 2021-12-01 18:21:29 +00:00
Dan J Miller
76158bf26b
Use node-browsers-medium-plus executor for validate-lavamoat-config job (#12884) 2021-11-29 17:34:51 -03:30
Etienne Dusseault
5a14a1a54a
Add storybook render tests with CI integration (#12477)
* add storybook unit tests with CI integration

* fix command and fix casing for test

* change ci ordering for storybook tasks

* fix syntax error

* fix jest

* lint

* Add transaction-total-banner render test to Storybook (#12517)

* transaction-total-banner

* lint

* confirm to spec

* lint

* fix jest ocnfig for snapshot test failure
2021-11-23 16:41:30 -08:00
kumavis
d9d1a831a6
ci - enforce yarn lock deduplications (#12737)
* ci - test for yarn lock deduplications

* deps - update yarn.lock and patches

* lavamoat - update policy

* test - ui/helpers/utils/optimism/buildUnserializedTransaction - test against json obj

* lint fix

* patch-package - patch @babel/runtime for lavamoat support

* patch-package - fix additional @babel/runtime lockdown incompats

* patch-package - cleanup sass patch
2021-11-23 09:17:27 -10:00
Mark Stacey
fb6375472e
Update improved-yarn-audit and ignore 2 advisories (#12765)
`improved-yarn-audit` has been updated so that it supports GitHub
advisories. Two new GitHub advisories have been ignored, as they are
both moderate RegExp DoS vulnerabilities that don't affect us, and they
are embedded deep within our dependency graph and are difficult to
update.
2021-11-19 11:53:19 -03:30
kumavis
1ec2af0337
ci - update shellcheck (#12739) 2021-11-18 14:58:31 -10:00
kumavis
f5f2f38ea5
ci - fix yarn-audit (#12738)
* ci/yarn-audit - ensure error propagation

* ci/yarn-audit - ensure dependencies are installed first
2021-11-18 10:57:47 -10:00
PeterYinusa
c0f0e1cfc4
update chromedriver to v95 (#12603) 2021-11-05 15:52:24 +00:00
Mark Stacey
690144a480
Add beta and Flask builds to CI (#12572)
The beta and Flask builds are now built on CI and included in the
metamask bot comment alongside the main builds. The same sourcemap
linter and mozilla linter used for the prod builds is also run on the
beta and Flask builds.

Closes #12426
2021-11-04 16:14:48 -02:30
Alex Donesky
3b5e33bc4c
use improved-yarn-audit and exclude 1002401 and 1002581 (#12310)
* use improved-yarn-audit and exclude 1002401
2021-10-08 12:18:38 -05:00
kumavis
a174d50ba5
ci - improve lavamoat validation debug info (#12259)
* ci - improve lavamoat validation debug info

* Update validate-allow-scripts.sh

* Update validate-lavamoat-policy.sh
2021-10-01 09:39:28 -10:00
kumavis
d9d20160d6
LavaMoat Node update and various small enhancements (#12239)
* lavamoat - update lavamoat-node and relevant policy + two handy patches

* test/e2e - add timeout known to be flaky

* lavamoat-viz - rename npm script
2021-09-28 20:56:08 -10:00
kumavis
d3f7464333
Ci test config improvement (#12223)
* ci - use same resource class for chrome and ff e2e tests

* e2e:chrome - enable logging by default

* lint fix
2021-09-27 12:39:31 -10:00
Elliot Winkler
bbe972ca46
Upgrade chromedriver to 93 (#11990)
This allows developers to run the Chrome e2e tests locally without
having to have a custom version of Chrome installed.
2021-09-08 14:54:31 -06:00
Elliot Winkler
8ffebb294b
Fix 'yarn setup' on M1 Macs (#11887)
There are a few issues encountered when running `yarn setup` on new
Apple Silicon (aka M1, aka arm64) Macs:

* The script halts when attempting to run the install step for
  the `chromedriver` package with the message "Only Mac 64 bits
  supported". This is somewhat misleading as it seems to indicate that
  chromedriver can only be installed on a 64-bit Mac. However, what I
  think is happening is that the installation script for `chromedriver`
  is not able to detect that an arm64 CPU *is* a 64-bit CPU. After
  looking through the `chromedriver` repo, it appears that 87.0.1 is the
  first version that adds a proper check ([1]).

  Note that upgrading chromedriver caused the Chrome-specific tests to
  fail intermittently on CI. I was not able to 100% work out the reason
  for this, but ensuring that X (which provides a way for Chrome to run
  in a GUI setting from the command line) is available seems to fix
  these issues.

* The script also halts when attempting to run the install step for
  the `electron` package. This happens because for the version of
  `electron` we are using (9.4.2), there is no available binary for
  arm64. It appears that Electron 11.x was the first version to support
  arm64 Macs ([2]). This is a bit trickier to resolve because we don't
  explicitly rely on `electron` — that's brought in by `react-devtools`.
  The first version of `react-devtools` that relies on `electron` 11.x
  is 4.11.0 ([3]).

[1]: 469dd0a6ee
[2]: https://www.electronjs.org/blog/apple-silicon
[3]: https://github.com/facebook/react/blob/main/packages/react-devtools/CHANGELOG.md#4110-april-9-2021
2021-09-01 10:40:40 -06:00
ryanml
a44d863093
Adding yarn scripts for changelog validation (#11868) 2021-08-18 08:59:48 -07:00
kumavis
984b78730f
CI - enforce dep usage with depcheck (#11518)
* deps - validate with depcheck

* ci - add depcheck + fixes for missing/extra deps

* ci - run depcheck after deps prep install

* deps - add yarn-deduplicate development tool

* Update .circleci/config.yml

Co-authored-by: Mark Stacey <markjstacey@gmail.com>

* Update .circleci/config.yml

Co-authored-by: Mark Stacey <markjstacey@gmail.com>

* Removing @lavamoat/preinstall-always-fail

* [depcheck] removing unused dependencies

* Update .depcheckrc.yml

Co-authored-by: Mark Stacey <markjstacey@gmail.com>
Co-authored-by: ryanml <ryanlanese@gmail.com>
2021-07-27 20:03:03 -07:00
Mark Stacey
6416a1ef98
Add logs for Google Chrome binary checksum verification (#11366)
The Chrome installation script now has logs explaining whether the
checksum verification succeeded or failed.
2021-06-23 17:17:16 -02:30
Mark Stacey
cc90fca2f6
Add retries to the benchmark script (#11319)
The benchmark script can now be set to retry upon failure, like the E2E
tests do. The default is zero, just as with the E2E tests. A retry of 2
has been set in CI to match the E2E tests as well.

The `retry` module had to be adjusted to throw an error in the case of
failure. Previously it just set the exit code, but that only worked
because it was the last thing called before the process ended. That is
no longer the case.
2021-06-21 12:46:18 -02:30
Brad Decker
7cc747c792
change download location of chrome file (#11346)
* change download location of chrome file

* Update .circleci/scripts/chrome-install.sh

Co-authored-by: Mark Stacey <markjstacey@gmail.com>

Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2021-06-18 11:45:58 -05:00
Mark Stacey
7535d63466
Add run-e2e-test.js script (#11301)
This script makes it easier to run an individual E2E test. In the past
I've run individual scripts by editing `run-all.sh` manually, but now
that can be done more easily with this script. It also allows setting
the number of retries to use and the browser to use from the CLI.

This script has been added as an npm script as well, called
'test:e2e:single'.

The `run-all.sh` script was rewritten in JavaScript to make it easier
to pass through a `--retries` argument.

The default number of retries has been set to zero to make local
testing easier. It has been set to 2 on CI.

This was mainly done to consolidate the code used to run an E2E test in
one place, to make later improvements easier.
2021-06-15 15:21:25 -02:30
Mark Stacey
37dc19a352
Migrate Sentry settings to environment variables (#11085)
Sentry is now configured with environment variables, rather than with
hard-coded values. This makes it easier to test Sentry functionality
using a different Sentry account, as we did recently during QA of
v9.5.1.

The only change for the normal build process is the introduction of the
`SENTRY_DSN_DEV` variable, which can be set via `.metamaskrc` or via an
environment variable. This determines where error reports are sent. It
still defaults to our team Sentry account's `metamask-testing` project.

The `sentry:publish` script now requires SENTRY_ORG and SENTRY_PROJECT
to be set in order to publish release artifacts. The CircleCI
configuration has been updated with these values, so it should act the
same as it did before. Previously we had used a CLI flag to specify the
organization and project, but Sentry already natively supports these
environment variables [1].

[1]: https://docs.sentry.io/product/cli/configuration/#configuration-values
2021-05-18 13:56:22 -02:30
Mark Stacey
36869a4350
Migrate version from _base manifest to package.json (#11029)
The version field is now stored in the main `package.json` file rather
than in the base manifest. It is built into the final manifest during
the build script.

This makes it easier to communicate what the current version should be
to our `auto-changelog` script. It's also generally a more conventional
place to keep track of the version, even considering that we're not
publishing to npm.
2021-05-10 14:16:03 -07:00
Mark Stacey
efb0d12404
Add changelog validation (#10999)
Add changelog validation to CI, ensuring that the changelog follows the
KeepAChangelog format. Additionally, this will also ensure that all of
the auto-generated "Uncategorized" changelog entries on release
branches get categorized.
2021-05-06 16:25:50 -02:30
Mark Stacey
3d952536d8
Migrate check for GITHUB_TOKEN (#10958)
The check for the GITHUB_TOKEN environment variable was being done in
the wrong release script. It has been migrated to the relevant script.

A second check for the username has also been added, as it is also
required.
2021-05-03 11:54:39 -02:30
Mark Stacey
7c602266cc
Update deploy key used for Storybook (#10966)
The SSH key used for Storybook deployments to `metamask-storybook` has
been updated. This new key is associated with `metamaskbot` rather than
a specific team member.
2021-04-30 18:49:07 -02:30
Thomas Huang
253efc6f8c
Jest config (#10855)
* Setup jest config

* Adjust test for jest.

* Adjust lint config

* Omit swaps ui folder for unit testing

* Omit swaps from test:unit:lax

* Add jest.config.js to script files

* Restore mocks rather than clearing them.

* Update jest config and adjust lint to include subdirs

* Convert view-quote-price-difference test to jest

* Add jest ci and ci coverage scripts. Add jest unit test to general test command

* Add test coverage to ci

* Use --ignore flag

* Fixup

* Add @metamask/eslint-config-jest

* Update .eslintrc.js

Co-authored-by: Mark Stacey <markjstacey@gmail.com>

* Adds jest-coverage/

Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2021-04-09 10:20:32 -07:00
Mark Stacey
482cbfe929
Add --rc flag to changelog script (#10839)
The changelog script now accepts an `--rc` flag to tell it whether to
add new changes to `Unreleased` or to the header for the current
version.

Previously this was inferred from whether the current version matched
the most recent tag. However this method only works for the first
update. Using a flag simplifies this logic, and makes it possible to
manually re-run this for further updates to a release candidate.
2021-04-07 16:47:41 -02:30
Mark Stacey
b18161c066
Remove unused CI script (#10840)
This script has not been used since #10795. It is now gone.
2021-04-07 15:46:01 -02:30
Mark Stacey
687803539f
Update the changelog when creating an RC (#10795)
The changelog will now be automatically updated when a release branch
is created. A new release header along with changelog entries for any
new commits will be added.

Note that this changelog will still need to be manually cleaned up, but
it's one less manual step at least.

The old Bash script for adding a new release header to the changelog
has been removed, as that functionality is now built into the changelog
update script.

A new script has been added to commit any changes made to the manifest
and changelog. This step used to happen at the end of the bump manifest
version script, but now the changelog update relies upon the manifest
version bump happening first, so it needed to be re-ordered. The
changes should only be committed on the first run of the branch, as
it's contingent upon the manifest changing (due to the version bump).
Further changelog updates won't trigger new automatic commits.
2021-04-02 18:41:51 -02:30
Mark Stacey
358fec049b
Remove script for creating master sync PR (#10791)
The script responsible for creating the "Sync `master` with `develop`"
PR has been removed. We will soon be eliminating the need for a
`master` branch altogether, so we don't need this anymore. Also, this
script hasn't been running correctly in a long time. We've been
creating this PR manually.
2021-03-31 19:54:52 -02:30
Mark Stacey
6aee658ca2
Remove date from changelog release header (#10790)
New changelog release headers now omit the date. These headers are
added automatically when a new release branch is created, and that
rarely ends up being the actual date of the release, so these dates
have all been inaccurate anyway.

The date will be re-added to the changelog later as part of a new
script, after a release has been published.
2021-03-31 19:54:29 -02:30
kumavis
b668a90303
ci - cache deps before patch-package (#10735)
* ci - cache deps before patch-package

* ci - bump dep cache number (cache break)
2021-03-26 11:55:03 +08:00
Mark Stacey
20b2c5f004
Fix LavaMoat config check (#10514)
The CI script to ensure no LavaMoat policy changes are required has
been failing despite there being no changes. It turns out that the
command used to check for changes (`git diff-index`) was failing
despite the lack of changes because the file was written again by
`yarn lavamoat:auto` but git hadn't gotten around to updating its index
since the write occurred, so it was considering it as changed until it
verified it wasn't [1].

The command has been replaced by `git diff --exit-code --quiet`, which
should do exactly the same thing except that it forces git to update
its internal cache to verify whether changes are present.

[1]: https://stackoverflow.com/questions/34807971/why-does-git-diff-index-head-result-change-for-touched-files-after-git-diff-or-g
2021-02-25 11:13:47 -03:30
Mark Stacey
681db78ff8
Add CI check to ensure LavaMoat policy is updated (#10493)
A CI job has been added to ensure the `allow-scripts` config and the
LavaMoat auto-generated policy is up-to-date. This will only run on
release branches and the `master` branch, because it's too difficult a
requirement to meet for each PR for contributors on macOS, due to
differences in the dependency graph caused by optional dependencies.

The `allow-scripts` and LavaMoat policy have both been updated using
`yarn allow-scripts auto` and `yarn lavamoat:auto`.
2021-02-24 12:22:28 -03:30
Mark Stacey
73e0fdddd0
Use .sh file extension for all Bash scripts (#10499)
The `.sh` file extension is now used for all Bash scripts. This ensures
the files are recognized as Bash scripts by the ShellCheck CI job, and
by editors/IDEs for improved syntax highlighting.
2021-02-23 12:56:41 -03:30
Etienne Dusseault
f196c9feb8
Add Lavamoat to build system (#9939)
* lavamoat - run build system in lavamoat

* lavamoat/allow-scripts - add missing policy entry

* update viz and lavvamoat

* trim policy file

* bump viz

* prue policy override

* regen policy file

* Update package.json

* Update package.json

* Apply suggestions from code review

Co-authored-by: kumavis <kumavis@users.noreply.github.com>

* update policy, remove redundant patches

* use yarn setup in CI

Co-authored-by: kumavis <aaron@kumavis.me>
Co-authored-by: kumavis <kumavis@users.noreply.github.com>
2021-02-22 22:43:29 +08:00
kumavis
efd280172f
ci - run storybook and add to build-artifacts (#10360)
* ci - run storybook and add to build-artifacts

* ci/storybook - rename storybook build path and fix artifact upload

* ci/storybook - rename link text

* clean - remove accidently committed storybook build dir

* storybook - fix image path to relative (#10364)
2021-02-04 22:30:22 +08:00
kumavis
b52d82ac27
ci/benchmark - increase executor size to medium+ (#10361) 2021-02-04 21:13:00 +08:00
Etienne Dusseault
fc409a103b
Add .yarnrc to disable scripts (#10354)
* add yarn rc file to disable scripts

* remove ignore scripts in CI

* re-add entry

* add lavamoat preinstall always fail

* allow-scripts - add missing package to denylist

Co-authored-by: kumavis <kumavis@users.noreply.github.com>
2021-02-03 21:53:12 -03:30
kumavis
b98cef16af
Update to Node v14 (#9514)
* manual rebase against develop

* Update .nvmrc
2021-02-03 13:45:38 +08:00
Etienne Dusseault
6b34fb4184
Use @lavamoat/allow-scripts (#10009)
* use @lavamoat/allow-scripts for package postinstall allow list
* dnode: set "weak" to false

Co-authored-by: kumavis <kumavis@users.noreply.github.com>
Co-authored-by: Erik Marks <25517051+rekmarks@users.noreply.github.com>
2021-02-01 20:08:42 -08:00
Mark Stacey
5e01602a01
Cache dependencies (#10065)
Dependencies are now cached between builds, using a checksum of the
`yarn.lock` file as the cache key. The `node_modules` directory and the
`.har` file from the install are cached and restored, so that we ensure
the record of the install is always preserved alongside the
dependencies.

The consolidation of the `collect-har-artifact` script was to make it
easier to cache the `.har` file along with the dependencies.
2020-12-14 11:46:42 -03:30
Mark Stacey
94f0588f0e
Add HTML coverage report to MetaMask bot comment (#10061)
The HTML code coverage report generated by `nyc` is now included in the
MetaMask bot comment. It has been saved as an artifact on CircleCI.
2020-12-11 16:21:00 -03:30
Mark Stacey
da1aae772b
Remove coveralls (#10063)
We don't look at coveralls very much. We might occasionally consult it
to see a report on our code coverage, but that report is already
generated entirely locally, and has been added to the MetaMask bot
comment in #10061.
2020-12-11 16:20:45 -03:30
Mark Stacey
caa5c73697
Use CircleCI executors to simplify CI config (#10055)
The CI config has been updated to use CircleCI executors. This allows
us to define the container environments used in one place, and reuse
these environment definitions between jobs.

This should result in no functional changes.
2020-12-11 14:07:13 -03:30
Mark Stacey
d8ec5f19f6
Use .zip for Firefox e2e tests (#10056)
The Firefox e2e tests now use the `.zip` file for testing the
extension. We've found this to produce more similar results to
production, compared to the old method of loading the unzipped
directory.

Passing in a `.zip` file to the Chrome driver didn't seem to work. I
didn't investigate this further to see if it was possible, but I'm not
sure it makes a difference on Chrome anyway.
2020-12-11 12:54:17 -03:30
Mark Stacey
0b7579b5d8
Update Firefox version used on CI for e2e tests (#10058)
The Firefox version has been updated to the latest stable version: v83.
This was required to replicate production Firefox errors we saw
recently.
2020-12-11 12:03:00 -03:30
Mark Stacey
cc1161a52a
Add metrics e2e test (#9784)
An e2e test has been added that uses the new mock Segment server to
verify that the three initial page metric events are sent correctly.

Using the mock Segment server requires a special build with this mock
Segment server hostname embedded, so a distinct job for building and
running this test was required. As such, it was left out of the
`run-all.sh` script.
2020-12-01 17:54:56 -03:30
Etienne Dusseault
63f759e14e
build - catch build pipeline errors properly (#9840) 2020-11-20 11:56:40 -03:30
Mark Stacey
5434d8d3a5
Temporarily disable sesify viz step (#9841)
The sesify viz step of the build was broken in #9838 when
`eth-rpc-errors@4` was introduced to the project. `eth-rpc-errors@4`
uses inline sourcemaps without including the full source in the
sourcemap, which breaks `sesify`.

`sesify` has been fixed[1] (under its new name, `lavamoat-browserify`),
but it has been disabled temporarily until this fix is included in a
new release, and until we can update to use it.

[1]: https://github.com/LavaMoat/LavaMoat/pull/121
2020-11-10 12:48:04 -03:30
Whymarrh Whitby
5e5d9e6c0a
Set NODE_OPTIONS=--max_old_space_size=1024 for builds (#9731)
This reduces the footprint of each Node process in an attempt to try
and lower the failure rate.

> Expected behavior with –max-old-space-size within container constraints
>
> By default, Node.js (up to 11.x) uses a maximum heap size of 700MB and 1400MB
> on 32-bit and 64-bit platforms, respectively. For current defaults, see the
> reference mentioned at the end of blog.

  [1]:https://developer.ibm.com/languages/node-js/articles/nodejs-memory-management-in-container-environments/
2020-10-27 12:44:24 -02:30
Whymarrh Whitby
7e5e697f89
Create release PRs as a draft, initially (#9660)
This change updates the command used to create RC pull requests to create [draft PRs][1].

  [1]:https://docs.github.com/en/free-pro-team@latest/github/collaborating-with-issues-and-pull-requests/about-pull-requests#draft-pull-requests
2020-10-20 09:15:53 -07:00
Whymarrh Whitby
baa2350604
Skip running e2e tests when commit is tagged with '[skip e2e]' (#9486)
[skip e2e]
2020-10-07 15:12:52 -02:30
Whymarrh Whitby
dd3c2cc3ec
Fix create_release_pull_request build step (#9484) 2020-10-06 16:39:53 -02:30
Whymarrh Whitby
92314cc3ed
Format manifest file after applying new version (#9480) 2020-10-06 15:27:59 -02:30
Whymarrh Whitby
4a5a8e40aa
Use the latest ShellCheck (#9446) 2020-09-21 17:38:15 -02:30
Mark Stacey
f7edc83a4e
Add source map validator to CI (#9135)
Source maps are now validated during CI. This is done during a new job
called "validate-source-maps`, and it is required to pass for tests to
pass.
2020-08-04 14:21:46 -03:00
Whymarrh Whitby
b378e57b27
Drop unused reviewers from CI scripts (#9068) 2020-07-23 21:15:09 -02:30
Mark Stacey
2856af2336
Remove integration tests (#8959)
The remaining integration tests are all covered by e2e tests, so
they're no longer needed.

All associated scripts, fixtures, and dependencies have also been
removed.
2020-07-10 12:22:36 -03:00
Mark Stacey
991ef366d7
Fix manifest version bump script (#8715)
The version bump script referenced the old file path for the manifest.
It was stored as a single file called `manifest.json`, but it was split
into `_base.json` and platform-specific manifests in #8140.

The manifest version bump script has been updated to reference
`_base.json`, which is the piece that has the version property.
2020-06-01 18:07:27 -03:00
kumavis
bc99728e84
ci/scripts/deps-install - exit if any command fails (#8368) 2020-04-20 10:18:16 -03:00
kumavis
7686edadb0
Build system refactor (#8140)
* build - start static asset task cleanup

* build - simplify manifest tasks

* build - refactor + rename some tasks

* build - various cleanups

* manifest - fix ref from controller

* build - drop gulp for simple async tasks

* build - breakout gulpfile into multiple files

* build - rename some tasks

* build - use task fn refs instead of string names

* build - bundle all scripts first, except for contentscript

* build - improve task timeline

* deps - update lock

* build - improve task time printout

* build/scripts - remove intermediate named task

* build - use 'yarn build' for task entry points

* build - properly run tasks via runTask for timeline display

* development/announcer - fix manifest path + clean

* build - lint fix

* build - make all defined tasks possible entry points

* build/task - properly report errors during task

* ci - fix sesify/lavamoat-viz build command

* build/scripts - run each bundle in separate processes

* lint fix

* build - forward childProcess logs to console

* build/task - fix parallel/series stream end event

* build/scripts refactor contentscript+inpage into a single task

* build/static - use the fs for 150x speedup zomg

* lint fix

* build/static - fix css copy

* Update development/build/scripts.js

Co-Authored-By: Mark Stacey <markjstacey@gmail.com>

* Update development/build/scripts.js

Co-Authored-By: Mark Stacey <markjstacey@gmail.com>

* Update development/build/index.js

Co-Authored-By: Mark Stacey <markjstacey@gmail.com>

* deps - remove redundant mkdirp

* deps - remove unused pumpify

* deps - remove redundant merge-deep

* deps - prefer is-stream of isstream

* deps - remove clone for lodash.cloneDeep

* clean - remove commented code

* build/static - use fs.copy + fast-glob instead of linux cp for better platform support

* build/manifest - standardize task naming

* build/display - clean - remove unused code

* bugfix - fix fs.promises import

* build - create "clean" as named task for use as entrypoint

* build/static - fix for copying dirs

* Update development/build/task.js

Co-Authored-By: Mark Stacey <markjstacey@gmail.com>

* Update development/build/display.js

Co-Authored-By: Mark Stacey <markjstacey@gmail.com>

* Update development/build/display.js

Co-Authored-By: Mark Stacey <markjstacey@gmail.com>

* Update development/build/display.js

Co-Authored-By: Mark Stacey <markjstacey@gmail.com>

* build - use task refs, tasks only return promises not streams, etc

* lint fi bad merge + lint

* build - one last cleanup + refactor

* build - add comments introducing file

* build/manifest - fix bug + subtasks dont beed to be named

* Update package.json

Co-Authored-By: Mark Stacey <markjstacey@gmail.com>

* build/task - remove unused fn

* Update package.json

Co-Authored-By: Mark Stacey <markjstacey@gmail.com>

* Update development/build/styles.js

Co-Authored-By: Mark Stacey <markjstacey@gmail.com>

* Update development/build/styles.js

Co-Authored-By: Mark Stacey <markjstacey@gmail.com>

Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 08:55:02 +08:00
Whymarrh Whitby
1ae0933897
Use image hash for ShellCheck Docker image (#8022) 2020-02-10 14:17:31 -03:30
Whymarrh Whitby
5b28fb3a1c
Enable Storybook deploy on CI (#8009) 2020-02-08 16:26:33 -03:30
Whymarrh Whitby
667f4fa5f8
Revert CircleCI Node image (#7996)
This changeset "reverts" the image version of the CircleCI node image, to
work around a chromedriver/Chrome/WebDriver error. We were seeing the
following errors:

```
WebDriverError: unknown error: Chrome failed to start: exited abnormally
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
```

I can surmise that the content changes in the Docker image, from (1) to (2),
below did result in this breakage:

1. https://hub.docker.com/layers/circleci/node/10.18-browsers/images/sha256-e16740707de2ebed45c05d507f33ef204902349c7356d720610b5ec6a35d3d88
2. https://hub.docker.com/layers/circleci/node/10.18-browsers/images/sha256-14d76b9e092ca03543328409223a2d2fba29e3ee5b8274c90b0c98b80005ecf2

Alternative links:

1. circleci/node@sha256:e16740707de2ebed45c05d507f33ef204902349c7356d720610b5ec6a35d3d88
2. circleci/node@sha256:14d76b9e092ca03543328409223a2d2fba29e3ee5b8274c90b0c98b80005ecf2
2020-02-05 19:50:06 -04:00
Whymarrh Whitby
9a692d721d
ci: Update shellcheck version used in CI (#7951) 2020-01-30 10:29:16 -03:30
Whymarrh Whitby
4bb01a36eb
Bump Node version to 10.18 (#7925) 2020-01-29 13:36:10 -03:30
Mark Stacey
ffd24a2854
Remove JSDoc tools (#7897)
Our JSDoc documentation has not been updated in a very long time, and
we don't use JSDoc in enough places for the docs to have been
especially useful. The tools and scripts  used to generate and publish
these docs have been removed.

References to this documentation have also been removed from the
README.

Hopefully once the TypeScript migration has made substantial progress,
we can generate more useful documentation using something like TypeDoc.
2020-01-28 13:02:49 -04:00
Mark Stacey
6ae93b1ec0 Comment out storybook-deploy step (#7913)
The Storybook deploy step is currently broken because it's using the
wrong source branch (`master` instead of `develop`), and because the
key that CircleCI is setup with doesn't have write access to the repo.

While I expect we'll get these two problems fixed soon, this ensures
that we at least have passing builds on `develop` in the meantime.
2020-01-27 11:31:55 -05:00
ricky
245952f236
Minimum changes to get storybook working (#7884)
* Minimum changes to get storybook working

Undo path changes

* Add build:storybook scripts to package.json

* Add storybook deployer

* Add storybook:deploy to package.json

* Update circle ci config

* Update yarn.lock

* Remove addon-info

* Update yarn.lock file to reflect removing of addon-info

Co-authored-by: Dan J Miller <danjm.com@gmail.com>
2020-01-26 18:44:58 -05:00
Mark Stacey
550fba2466
Add benchmark to CI (#7871)
* Add benchmark to CI

The page load benchmark for Chrome is now run during CI, and the
results are collected and summarized in the `metamaskbot` comment.

Closes #6881

* Double default number of samples

The number of default samples was changed from 10 to 20. The results
from 10 samples would show statistically significant changes in page
load times between builds, so weren't a sufficiently useful metric.
2020-01-23 16:55:17 -04:00
Mark Stacey
523121209f
Revert the revert of "Use common test build during CI (#7196)" (#7404) (#7870)
This reverts commit 4b4c00e94f. The
original change was a possible optimization of CI, though it ended up
not having a huge impact. It was thought that it may have broken source
maps, because the test build overwrote the `dist` directory that the
source maps were written to. However this turned out not to be the
case, as the changes to `dist` are never persisted to the workspace.

This is being re-introduced because the test build is needed for an
additional job (the page load benchmark), and sharing the same build
for all three jobs would surely be faster than building it separately
three times.
2020-01-20 13:03:21 -04:00
Whymarrh Whitby
bcfe58d59b
Add lockfile-lint to CI (#7727) 2020-01-06 15:28:43 -03:30
ricky
476274474f
Add shellcheck lint (#7392)
* Add shellcheck lint script

* Add to build

* Add shellcheck lint to main lint task

* Put shellcheck in the right place, hopefully?

* Fix declared multiple executor types

* Add sudo

* Address shellcheck warnings

* Add test-lint-shellcheck

* Add test-lint-shellcheck to workflow

* Use correct lint task

* output version which could be helpful for debugging

* Address PR feedback

* consistency++
2019-11-19 10:46:10 -05:00
Whymarrh Whitby
a271e7f456
Update Node version to 10.17 (#7447) 2019-11-17 15:32:26 -03:30
Dan Finlay
3ffda6c684
Merge pull request #7410 from MetaMask/fix-sourcemaps
Fix sourcemaps
2019-11-15 09:40:51 -08:00
Whymarrh Whitby
f2e3fa58b6
circleci: v2.1 (#7421) 2019-11-14 13:17:32 -03:30
Mark Stacey
1c6e09b1ce Fix sourcemaps
The `install` script of `@sentry/cli` is required for the Sentry CLI to
work correctly. Without this step, the sourcemap upload fails silently.
2019-11-13 16:54:04 -04:00
Mark Stacey
4b4c00e94f
Revert "Use common test build during CI (#7196)" (#7404)
This reverts commit b0ec610908, which was
introduced in #7196. This change was preventing the sourcemaps from
being uploaded correctly.
2019-11-13 11:02:59 -04:00
Mark Stacey
fe28e0d134
Cleanup beforeunload handler after transaction is resolved (#7333)
* Cleanup beforeunload handler after transaction is resolved

The notification window was updated to reject transactions upon close
in #6340. A handler that rejects the transaction was added to
`window.onbeforeunload`, and it was cleared in `actions.js` if it was
confirmed or rejected.

However, the `onbeforeunload` handler remained uncleared if the
transaction was resolved in another window. This results in the
transaction being rejected when the notification window closes, even
long after the transaction is submitted and confirmed. This has been
the cause of many problems with the Firefox e2e tests.

Instead the `onbeforeunload` handler is cleared in the
`componentWillUnmount` lifecycle function, alongside where it's set in
the first place. This ensures that it's correctly unset regardless
of how the transaction was resolved, and it better matches user
expectations.

* Fix indentation and remove redundant export

The `run-all.sh` Bash script now uses consistent indentation, and is
consistent about only re-exporting the Ganache arguments when they
change.

* Ensure transactions are completed before checking balance

Various intermittent e2e test failures appear to be caused by React
re-rendering the transaction list during the test, as the transaction
goes from pending to confirmed. To avoid this race condition, the
transaction is now explicitly looked for in the confirmed transaction
list in each of the tests using this pattern.

* Enable all e2e tests on Firefox

The remaining tests that were disabled on Firefox now work correctly.
Only a few timing adjustments were needed.

* Update Firefox used in CI

Firefox v70 is now used on CI instead of v68. This necessitated
rewriting the function where the extension ID was obtained because the
Firefox extensions page was redesigned.
2019-10-31 13:27:22 -03:00
Erik Marks
478d6563f2
Freeze Promise global on boot (#7309)
* freeze background and UI Promise globals on boot

* add new tests

* remove tape
2019-10-24 06:54:32 -07:00
kumavis
e1efb4d7ac
ci - install deps - limit install scripts to whitelist (#7208)
* ci - install deps - limit install scripts to those needed for build

* Update .circleci/scripts/deps-install.sh

Co-Authored-By: Mark Stacey <markjstacey@gmail.com>

* ci - install deps - expand install scripts needed for tests

* ci - install deps - expand install scripts needed for integration tests

* ci - install deps - fix node-sass script ref

* github - set codeowners for scripts/deps-install

* development - add utility to show deps with install scripts

* lint fix

* deps - move read-installed to devDeps
2019-09-25 20:01:10 +08:00
Mark Stacey
b0ec610908
Use common test build during CI (#7196)
* Use common test build during CI

Previously both e2e test jobs were running `test:build`. Now there is a
separate job that runs `test:build` that runs before each e2e test job,
so that `test:build` is only run once instead of twice.

* Move test builds to separate directory

This prevents the test build from conflicting with the production build
in later jobs.
2019-09-21 13:32:07 -03:00
Mark Stacey
9f21317a30
Verify locales on CI (#7199)
* Add '--quiet' flag to verify locales script

The `--quiet` flag reduces the console output to just the essential
information for running in a CI environment. For each locale, it will
print the number of unused messages (if any).

* Add `verify-locales` script to lint CI job

The locales are now verified as part of the lint CI job. Any unused
messages detected will result in the job failing.
2019-09-21 13:31:33 -03:00
kumavis
dbbf6986d2
ci - install deps with "--har" flag to capture network activity (#7143)
* ci - install deps with "--har" flag to capture network activity

* ci - add yarn install HAR logs to build-artifacts

* ci - yarn har - fix typo

* Update .circleci/scripts/collect-har-artifact.sh

Co-Authored-By: Mark Stacey <markjstacey@gmail.com>
2019-09-12 00:03:47 +08:00
kumavis
629f5ef221
ci - create source-map-explorer build-artifacts (#7141)
* ci - create source-map-explorer build-artifacts

* ci - add source-map-explorer builds to metamaskbot comment

* lint fix

* ci - source-map-explorer - include all bundles
2019-09-11 23:35:30 +08:00
kumavis
0985e8f012
ci - build-artifacts - generate sesify-viz for inspecting deps (#7151)
* ci - build-artifacts - generate sesify-viz for inspecting deps

* lint fix
2019-09-11 22:47:21 +08:00
Mark Stacey
c9fffafd9a
Publish GitHub release from master branch (#7136)
* Publish GitHub release from master branch

This ensures that changes made on `develop` since branching for the
release are not included. It also ensures that the final release
sourcemaps line-up correctly (they were always build on master)`.

* Consolidate publish jobs

The jobs `job-publish-release` and `create_github_release` both handle
different parts of publishing a release. They have been consolidated
into a single `job-publish-release` job.

* Update release script to expect a merge commit

The release script was originally written to be run on `develop`, so it
expected the current commit to be a result of `Squash & Merge`. Now
that it's run on `master`, it will generally be run against a merge
commit.

The version is now extracted from the commit message using a regular
expression that should work on all version of Bash v3+, and should be
tolerant of both merge commits and `Squash & Merge` commits.

* Target `master` with release PR

`master` is now targeted by the release PR instead of `develop`, as
the release has to be created from the master branch.

The update to `develop` is handled after the release by a PR from
`master` to `develop`, which is created automatically after the
release.
2019-09-11 11:44:36 -03:00
Whymarrh Whitby
d3eafc3703
Fix Coveralls CI job (#7008) 2019-08-14 10:41:01 -02:30
Whymarrh Whitby
6bc1077880
Make Coveralls its own optional CI job (#7002)
* ci: Move Coveralls into optional step

* Split Coveralls step out of test:coverage npm script
2019-08-13 13:29:10 -02:30
Whymarrh Whitby
e321feb92d
Fix Firefox e2e test command on CI (#7000)
* Skip a few Firefox tests to get them passing again
* ci: Fix Firefox e2e test command

This is my bad, introduced in 7fc84f3cc0
2019-08-13 10:27:18 -02:30