This PR adds build-time code exclusion by means of code fencing. For details, please see the README in `./development/build/transforms`. Note that linting of transformed files as a form of validation is added in a follow-up, #12075.
Hopefully exhaustive tests are added to ensure that the transform works according to its specification. Since these tests are Node-only, they required their own Jest config. The recommended way to work with multiple Jest configs is using the `projects` field in the Jest config, however [that feature breaks coverage collection](https://github.com/facebook/jest/issues/9628). That being the case, I had to set up two separate Jest configs. In order to get both test suites to run in parallel, Jest is now invoked via a script, `./test/run-jest.sh`.
By way of example, this build system feature allows us to add fences like this:
```javascript
this.store.updateStructure({
...,
GasFeeController: this.gasFeeController,
TokenListController: this.tokenListController,
///: BEGIN:ONLY_INCLUDE_IN(beta)
PluginController: this.pluginController,
///: END:ONLY_INCLUDE_IN
});
```
Which at build time are transformed to the following if the build type is not `beta`:
```javascript
this.store.updateStructure({
...,
GasFeeController: this.gasFeeController,
TokenListController: this.tokenListController,
});
```
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
This rationalizes how arguments are passed to and parsed by the build system. To accomplish this, everything that isn't an environment variable from `.metamaskrc` or our CI environment is now passed as an argument on the command line.
Of such arguments, the `entryTask` is still expected as a positional argument in the first position (i.e. `process.argv[2]`), but everything else must be passed as a named argument. We use `minimist` to parse the arguments, and set defaults to preserve existing behavior.
Arguments are parsed in a new function, `parseArgv`, in `development/build/index.js`. They are assigned to environment variables where convenient, and otherwise returned from `parseArgv` to be passed to other functions invoked in the same file.
This change is motivated by our previous inconsistent handling of arguments to the build system, which will grow increasingly problematic as the build system grows in complexity. (Which it will very shortly, as we introduce Flask builds.)
Miscellaneous changes:
- Adds a build system readme at `development/build/README.md`
- Removes the `beta` package script. Now, we can instead call: `yarn dist --build-type beta`
- Fixes the casing of some log messages and reorders some parameters in the build system
This adds an `--omit-lockdown` flag to our build script, which will cause SES `lockdown` to be omitted from the resulting bundle. Useful for development when we don't want the environment to be locked down.
Thanks to @kumavis for the suggestion.
This PR makes ~all named intrinsics in all of our JavaScript processes non-modifiable. A named intrinsic is any property specified by the ECMAScript specification that exists on `globalThis` when the JavaScript process starts. We say that a property is non-modifiable if it is non-configurable and non-writable. We make exceptions for properties that meet any of the following criteria:
1. Properties that are non-configurable by the time `lockdown-run.js` is executed are not modified, because they can't be.
2. Properties that have accessor properties (`get` or `set`) are made non-configurable, but their writability cannot be modified, and is therefore left unchanged. It's unclear how many of the named intrinsics this applies to, if any, but it's good defensive programming, regardless.
The Sentry `release` was not being configured correctly. It was being
left blank. This is because the location of the extension version was
moved in #11029. The build script was correctly updated in that PR, but
that work was accidentally undone in a merge error that was included
in #11080.
Fixing up tests and add back old custom gas modal for non-eip1559 compliant networks
Remove unnecessary props from send-gas-row.component
fix breaking test
Fix primary and secondary title overrides
fix rebase issue
Fix rebase conflict
Co-authored-by: David Walsh <davidwalsh83@gmail.com>
Sentry is now configured with environment variables, rather than with
hard-coded values. This makes it easier to test Sentry functionality
using a different Sentry account, as we did recently during QA of
v9.5.1.
The only change for the normal build process is the introduction of the
`SENTRY_DSN_DEV` variable, which can be set via `.metamaskrc` or via an
environment variable. This determines where error reports are sent. It
still defaults to our team Sentry account's `metamask-testing` project.
The `sentry:publish` script now requires SENTRY_ORG and SENTRY_PROJECT
to be set in order to publish release artifacts. The CircleCI
configuration has been updated with these values, so it should act the
same as it did before. Previously we had used a CLI flag to specify the
organization and project, but Sentry already natively supports these
environment variables [1].
[1]: https://docs.sentry.io/product/cli/configuration/#configuration-values
The version field is now stored in the main `package.json` file rather
than in the base manifest. It is built into the final manifest during
the build script.
This makes it easier to communicate what the current version should be
to our `auto-changelog` script. It's also generally a more conventional
place to keep track of the version, even considering that we're not
publishing to npm.
Our build script waits for the `close` event to determine whether the
task has exited. The `exit` event is a better representation of this,
because if a stream is shared between multiple processes, the process
may exit without the `close` event being emitted.
We aren't sharing streams between processes, so this edge case doesn't
apply to us. This just seemed like a more suitable event to listen to,
since we care about the process exiting not the stream ending.
See this description of the `close` event from the Node.js
documentation [1]:
>The `'close'` event is emitted when the stdio streams of a child
>process have been closed. This is distinct from the `'exit'` event,
>since multiple processes might share the same stdio streams.
And see this description of the `exit` event:
>The `'exit'` event is emitted after the child process ends.
[1]: https://nodejs.org/docs/latest-v14.x/api/child_process.html#child_process_event_exit
* build - declare background as html
* build - fill in empty file when a missing file is expected
* lint - fix
* Update development/build/manifest.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Update development/build/manifest.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Excluding sourcemaps comment in production builds
FixesMetaMask/metamask-extension#7077
* Fix source map explorer script
The source map explorer script now re-adds the source map comment to
each file to ensure the source map visualization still works. Each
module with a sourcemap is copied to a temporary directory along with
the module it corresponds to, and from there it's passed into
`source-map-explorer`. This should ensure the resulting visualization
matches what it was before.
Everything has been moved inside of functions to generally improve
readability, and to allow the use of local variables.
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
Build warnings related to Sass have been reduced by dynamically
importing `gulp-sass` and `sass-compiler` at the point where it's first
used. This ensures that the four Dart-related build warnings are only
emitted for the build process that is actually running Sass, rather
than by _every_ build process.
The environment variables `METAMETRICS_PROJECT_ID` and
`ETH_GAS_STATION_API_KEY` were still being injected into the JavaScript
build, despite being unused. The MetaMetrics project ID was made
obsolete in #9646, and the ETH Gas Station API key was made obsolete in
PR #9867
* Maintain console logging in dev mode
Co-authored-by: kumavis <aaron@kumavis.me>
Co-authored-by: Erik Marks <rekmarks@protonmail.com>
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
The Firefox e2e tests now use the `.zip` file for testing the
extension. We've found this to produce more similar results to
production, compared to the old method of loading the unzipped
directory.
Passing in a `.zip` file to the Chrome driver didn't seem to work. I
didn't investigate this further to see if it was possible, but I'm not
sure it makes a difference on Chrome anyway.
When you load an extension `.zip` file in Firefox, it fails to load
scripts with the `.cjs` file extension. However, it works if you load
the extension via the `manifest.json` file instead.
After renaming the `lockdown.cjs` file to `lockdown.js`, it works in
Firefox in all cases, regardless whether it's loaded by manifest or by
`.zip`.
* Remove use of ethgassthat; use metaswap /gasPrices api for gas price estimates
* Remove references to ethgasstation
* Pass base to BigNumber constructor in fetchExternalBasicGasEstimates
* Update ui/app/hooks/useTokenTracker.js
Co-authored-by: Erik Marks <25517051+rekmarks@users.noreply.github.com>
* Delete gas price chart
* Remove price chart css import
* Delete additional fee chart code
* Lint fix
* Delete more code no longer used after ethgasstation removal
Co-authored-by: Erik Marks <25517051+rekmarks@users.noreply.github.com>
* Freezeglobals: remove Promise freezing, add lockdown
* background & UI: temp disable sentry
* add loose-envify, dedupe symbol-observable
* use loose envify
* add symbol-observable patch
* run freezeGlobals after sentry init
* use require instead of import
* add lockdown to contentscript
* add error code in message
* try increasing node env heap size to 2048
* change back circe CI option
* make freezeGlobals an exported function
* make freezeGlobals an exported function
* use freezeIntrinsics
* pass down env to child process
* fix unknown module
* fix tests
* change back to 2048
* fix import error
* attempt to fix memory error
* fix lint
* fix lint
* fix mem gain
* use lockdown in phishing detect
* fix lint
* move sentry init into freezeIntrinsics to run lockdown before other imports
* lint fix
* custom lockdown modules per context
* lint fix
* fix global test
* remove run in child process
* remove lavamoat-core, use ses, require lockdown directly
* revert childprocess
* patch package postinstall
* revert back child process
* add postinstall to ci
* revert node max space size to 1024
* put back loose-envify
* Disable sentry to see if e2e tetss pass
* use runLockdown, add as script in manifest
* remove global and require from runlockdown
* add more memory to tests
* upgrade resource class for prep-build & prep-build-test
* fix lint
* lint fix
* upgrade remote-redux-devtools
* skillfully re-add sentry
* lintfix
* fix lint
* put back beep
* remove envify, add loose-envify and patch-package in dev deps
* Replace patch with Yarn resolution (#9923)
Instead of patching `symbol-observable`, this ensures that all
versions of `symbol-observable` are resolved to the given range, even
if it contradicts the requested range.
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
This mock Segment server can be used to test our extension metrics. It
will respond to all request with HTTP 200, and will print the requests
to the console. It also has parsing built-in for Segment request
payloads.
Right now only the event name is printed, but we can enhance this in
the future to print more event information. We can also enhance the
mock to be a more realistic representation of the API.
The extension has been modified to allow the Segment host to be
overwritten with the `SEGMENT_HOST` environment variable. This will
ensure that all Segment events are redirected to that host.
So for example, to create a dev build that uses this server, you could
set the `SEGMENT_WRITE_KEY` and `SEGMENT_LEGACY_WRITE_KEY` values to
any non-empty string, and set `SEGMENT_HOST` to
`http://localhost:9090`.
This was created originally to test PR #9768
It was getting rather complicated to keep track of which Segment keys
were set where, and under which name.
The build script now injects a key even in test environments, but it is
unused if `IN_TEST` is truthy. This should be functionally equivalent
to the old logic. I find this simpler mainly for two reasons: there is
one less intermediate variable to keep track of now, and the `IN_TEST`
check is now directly in the module where we're constructing the
`segment` instance, rather than being referenced at a distance in a
comment.
The old setup made it difficult to turn on metrics for specific e2e
tests as well, which will be done in a subsequent PR.
The `devMode` parameter being passed to the `buildScss` function was
not being used. The `buildScss` function was declared _inside_ the
function in which it is invoked, so the `devMode` variable is already
in scope - it doesn't need to be passed in.
The `styles:dev` step of the development build was broken in #9568 when
the `lockdown()` function from SES was introduced. One of the effects
of `lockdown()` is preventing the use of `RegExp.lastMatch`, which is
relied upon by `postcss` for parsing inline sourcemaps.
This problem has been worked around by re-arranging the build pipeline
for styles.
Firstly, the `autoprefixer` stage was moved _before_ sourcemaps were
generated. Its placement _after_ sourcemaps was a mistake in the first
place - it should have always been before they were written, so that
the sourcemaps could keep track of the changes made by `autoprefixer`.
Second, the `-rtl` sourcemap generation was moved to a separate build
pipeline. The `rtl` stage also uses `postcss`, and it must come after
the sourcemaps are written for the non-RTL styles, so splitting it into
a separate build was the only way to avoid passing in inline
sourcemaps. This does make the build slightly slower, but it also
decreases the size of the RTL sourcemaps dramatically.
* Add ses lockdown to build system using lavamoat-core
* use proper object.assign version
* disable lint rules for ses lockdown
* deps - update rtlcss
Co-authored-by: kumavis <aaron@kumavis.me>
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
