1
0
mirror of https://github.com/kremalicious/metamask-extension.git synced 2024-10-22 19:26:13 +02:00
metamask-extension/development/build/transforms
Erik Marks a2d3d942ec
Exclude files from builds by build type (#12521)
This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions.

The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details.

### Code Fencing and ESLint

When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor.

### File Exclusion

For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`.

As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice.

Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-01 20:20:31 -07:00
..
README.md Build: Lint files after removing their code fences (#12075) 2021-09-15 17:18:28 -10:00
remove-fenced-code.js Fix off-by-one error in code fence transform (#12540) 2021-10-29 16:45:27 -07:00
remove-fenced-code.test.js Fix off-by-one error in code fence transform (#12540) 2021-10-29 16:45:27 -07:00
utils.js Exclude files from builds by build type (#12521) 2021-11-01 20:20:31 -07:00
utils.test.js Build: Lint files after removing their code fences (#12075) 2021-09-15 17:18:28 -10:00

Local Browserify Transforms

This directory contains home-grown Browserify transforms. Each file listed here exports a transform function factory.

Removing Fenced Code

./remove-fenced-code.js

When creating builds that support different features, it is desirable to exclude unsupported features, files, and dependencies at build time. Undesired files and dependencies can be excluded wholesale, but the use of undesired modules in files that should otherwise be included i.e. import statements and references to those imports cannot.

To support the exclusion of the use of undesired modules at build time, we introduce the concept of code fencing to our build system. Our code fencing syntax amounts to a tiny DSL, which is specified below.

The transform concatenates each file into a single string, and a string parser identifies any fences in the file. If any fences that should not be included in the current build are found, the fences and the lines that they wrap are deleted. The transform errors if a malformed fence line is identified.

For example, the following fenced code:

this.store.updateStructure({
  ...,
  GasFeeController: this.gasFeeController,
  TokenListController: this.tokenListController,
  ///: BEGIN:ONLY_INCLUDE_IN(beta)
  PluginController: this.pluginController,
  ///: END:ONLY_INCLUDE_IN
});

Is transformed to the following if the build type is not beta:

this.store.updateStructure({
  ...,
  GasFeeController: this.gasFeeController,
  TokenListController: this.tokenListController,
});

Note that multiple build types can be specified by separating them with commands inside the parameter parentheses:

///: BEGIN:ONLY_INCLUDE_IN(beta,flask)

Gotchas

By default, the transform will invoke ESLint on files that are modified by the transform. This is our first line of defense against creating unsyntactic code using code fences, and the transform will error if linting fails. (Live reloading will continue to work if enabled.) To toggle this behavior via build system arguments, see the build system readme.

Code Fencing Syntax

In the specification, angle brackets, < >, indicate required tokens, while straight brackets, [ ], indicate optional tokens.

Alphabetical characters identify the name and purpose of a token. All other characters, including parentheses, ( ), are literals.

A fence line is a single-line JavaScript comment, optionally surrounded by whitespace, in the following format:

///: <terminus>:<command>[(parameters)]

|__| |________________________________|
  |                  |
  |                  |
sentinel         directive

The first part of a fence line is the sentinel, which is always the string "///:". If the first four non-whitespace characters of a line are not the sentinel, the line will be ignored by the parser. The sentinel must be succeeded by a single space character, or parsing will fail.

The remainder of the fence line is called the directive. The directive consists of a terminus, command, and (optionally) parameters.

  • The terminus is one of the strings BEGIN and END. It must be followed by a single colon, :.
  • The command is a string of uppercase alphabetical characters, optionally including underscores, _. The possible commands are listed later in this specification.
  • The parameters are a comma-separated list of RegEx \w strings. They must be parenthesized, only specified for BEGIN directives, and valid for the command of the directive.

A valid code fence consists of two fence lines surrounding one or more lines of non-fence lines. The first fence line must consist of a BEGIN directive, and the second an END directive. The command of both directives must be the same, and the parameters (if any) must be valid for the command.

If an invalid fence is detected, parsing will fail, and the transform stream will end with an error.

Commands

ONLY_INCLUDE_IN

This, the only command defined so far, is used to exclude lines of code depending on the type of the current build. If a particular set of lines should only be included in a particular build type, say beta, they should be wrapped as follows:

///: BEGIN:ONLY_INCLUDE_IN(beta)
console.log('I am only included in beta builds.');
///: END:ONLY_INCLUDE_IN

At build time, the fences and the fenced lines will be removed if the build is not beta.

Parameters are required for this command, and they must be provided as a comma-separated list of one or more of:

  • main (the build system default build type)
  • beta
  • flask