1
0
mirror of https://github.com/kremalicious/metamask-extension.git synced 2024-12-02 06:07:06 +01:00
metamask-extension/development/build/scripts.js

817 lines
24 KiB
JavaScript
Raw Normal View History

Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
const { callbackify } = require('util');
const path = require('path');
const { writeFileSync, readFileSync } = require('fs');
const EventEmitter = require('events');
const gulp = require('gulp');
const watch = require('gulp-watch');
const Vinyl = require('vinyl');
const source = require('vinyl-source-stream');
const buffer = require('vinyl-buffer');
const log = require('fancy-log');
const browserify = require('browserify');
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
const watchify = require('watchify');
const babelify = require('babelify');
const brfs = require('brfs');
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
const envify = require('loose-envify/custom');
const sourcemaps = require('gulp-sourcemaps');
const applySourceMap = require('vinyl-sourcemaps-apply');
const pify = require('pify');
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
const through = require('through2');
const endOfStream = pify(require('end-of-stream'));
const labeledStreamSplicer = require('labeled-stream-splicer').obj;
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
const wrapInStream = require('pumpify').obj;
const Sqrl = require('squirrelly');
const lavapack = require('@lavamoat/lavapack');
const lavamoatBrowserify = require('lavamoat-browserify');
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
const terser = require('terser');
const bifyModuleGroups = require('bify-module-groups');
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
const metamaskrc = require('rc')('metamask', {
INFURA_PROJECT_ID: process.env.INFURA_PROJECT_ID,
INFURA_BETA_PROJECT_ID: process.env.INFURA_BETA_PROJECT_ID,
INFURA_FLASK_PROJECT_ID: process.env.INFURA_FLASK_PROJECT_ID,
INFURA_PROD_PROJECT_ID: process.env.INFURA_PROD_PROJECT_ID,
ONBOARDING_V2: process.env.ONBOARDING_V2,
COLLECTIBLES_V1: process.env.COLLECTIBLES_V1,
SEGMENT_HOST: process.env.SEGMENT_HOST,
SEGMENT_WRITE_KEY: process.env.SEGMENT_WRITE_KEY,
SEGMENT_BETA_WRITE_KEY: process.env.SEGMENT_BETA_WRITE_KEY,
SEGMENT_FLASK_WRITE_KEY: process.env.SEGMENT_FLASK_WRITE_KEY,
SEGMENT_PROD_WRITE_KEY: process.env.SEGMENT_PROD_WRITE_KEY,
SENTRY_DSN_DEV:
process.env.SENTRY_DSN_DEV ||
'https://f59f3dd640d2429d9d0e2445a87ea8e1@sentry.io/273496',
});
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
const { streamFlatMap } = require('../stream-flat-map.js');
const { version } = require('../../package.json');
2021-09-08 22:08:23 +02:00
2020-11-03 00:41:28 +01:00
const {
createTask,
composeParallel,
composeSeries,
runInChildProcess,
} = require('./task');
const {
createRemoveFencedCodeTransform,
} = require('./transforms/remove-fenced-code');
const { BuildType } = require('./utils');
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
/**
* The build environment. This describes the environment this build was produced in.
*/
const ENVIRONMENT = {
DEVELOPMENT: 'development',
PRODUCTION: 'production',
OTHER: 'other',
PULL_REQUEST: 'pull-request',
RELEASE_CANDIDATE: 'release-candidate',
STAGING: 'staging',
TESTING: 'testing',
};
/**
* Get a value from the configuration, and confirm that it is set.
*
* @param {string} key - The configuration key to retrieve.
* @returns {string} The config entry requested.
* @throws {Error} Throws if the requested key is missing.
*/
function getConfigValue(key) {
const value = metamaskrc[key];
if (!value) {
throw new Error(`Missing config entry for '${key}'`);
}
return value;
}
/**
* Get the appropriate Infura project ID.
*
* @param {object} options - The Infura project ID options.
* @param {BuildType} options.buildType - The current build type.
* @param {ENVIRONMENT[keyof ENVIRONMENT]} options.environment - The build environment.
* @param {boolean} options.testing - Whether the current build is a test build or not.
* @returns {string} The Infura project ID.
*/
function getInfuraProjectId({ buildType, environment, testing }) {
if (testing) {
return '00000000000000000000000000000000';
} else if (environment !== ENVIRONMENT.PRODUCTION) {
// Skip validation because this is unset on PRs from forks.
return metamaskrc.INFURA_PROJECT_ID;
} else if (buildType === BuildType.main) {
return getConfigValue('INFURA_PROD_PROJECT_ID');
} else if (buildType === BuildType.beta) {
return getConfigValue('INFURA_BETA_PROJECT_ID');
} else if (buildType === BuildType.flask) {
return getConfigValue('INFURA_FLASK_PROJECT_ID');
}
throw new Error(`Invalid build type: '${buildType}'`);
}
/**
* Get the appropriate Segment write key.
*
* @param {object} options - The Segment write key options.
* @param {BuildType} options.buildType - The current build type.
* @param {keyof ENVIRONMENT} options.enviroment - The current build environment.
* @returns {string} The Segment write key.
*/
function getSegmentWriteKey({ buildType, environment }) {
if (environment !== ENVIRONMENT.PRODUCTION) {
// Skip validation because this is unset on PRs from forks, and isn't necessary for development builds.
return metamaskrc.SEGMENT_WRITE_KEY;
} else if (buildType === BuildType.main) {
return getConfigValue('SEGMENT_PROD_WRITE_KEY');
} else if (buildType === BuildType.beta) {
return getConfigValue('SEGMENT_BETA_WRITE_KEY');
} else if (buildType === BuildType.flask) {
return getConfigValue('SEGMENT_FLASK_WRITE_KEY');
}
throw new Error(`Invalid build type: '${buildType}'`);
}
module.exports = createScriptTasks;
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
function createScriptTasks({
browserPlatforms,
buildType,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
ignoredFiles,
isLavaMoat,
livereload,
shouldLintFenceFiles,
}) {
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
// internal tasks
const core = {
// dev tasks (live reload)
2020-11-03 00:41:28 +01:00
dev: createTasksForBuildJsExtension({
taskPrefix: 'scripts:core:dev',
devMode: true,
}),
testDev: createTasksForBuildJsExtension({
taskPrefix: 'scripts:core:test-live',
devMode: true,
testing: true,
}),
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
// built for CI tests
2020-11-03 00:41:28 +01:00
test: createTasksForBuildJsExtension({
taskPrefix: 'scripts:core:test',
testing: true,
}),
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
// production
prod: createTasksForBuildJsExtension({ taskPrefix: 'scripts:core:prod' }),
};
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
// high level tasks
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
const { dev, test, testDev, prod } = core;
return { dev, test, testDev, prod };
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
function createTasksForBuildJsExtension({ taskPrefix, devMode, testing }) {
const standardEntryPoints = ['background', 'ui', 'content-script'];
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
const standardSubtask = createTask(
`${taskPrefix}:standardEntryPoints`,
createFactoredBuild({
browserPlatforms,
buildType,
devMode,
entryFiles: standardEntryPoints.map((label) => {
if (label === 'content-script') {
return './app/vendor/trezor/content-script.js';
}
return `./app/scripts/${label}.js`;
}),
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
ignoredFiles,
shouldLintFenceFiles,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
testing,
2020-11-03 00:41:28 +01:00
}),
);
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
// inpage must be built before contentscript
// because inpage bundle result is included inside contentscript
2020-11-03 00:41:28 +01:00
const contentscriptSubtask = createTask(
`${taskPrefix}:contentscript`,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
createTaskForBundleContentscript({ devMode, testing }),
);
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
// this can run whenever
const disableConsoleSubtask = createTask(
`${taskPrefix}:disable-console`,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
createTaskForBundleDisableConsole({ devMode }),
);
// this can run whenever
const installSentrySubtask = createTask(
`${taskPrefix}:sentry`,
createTaskForBundleSentry({ devMode }),
);
const phishingDetectSubtask = createTask(
`${taskPrefix}:phishing-detect`,
createTaskForBundlePhishingDetect({ devMode }),
);
// task for initiating browser livereload
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
const initiateLiveReload = async () => {
if (devMode) {
// trigger live reload when the bundles are updated
// this is not ideal, but overcomes the limitations:
// - run from the main process (not child process tasks)
// - after the first build has completed (thus the timeout)
// - build tasks never "complete" when run with livereload + child process
setTimeout(() => {
watch('./dist/*/*.js', (event) => {
livereload.changed(event.path);
});
}, 75e3);
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
}
};
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
// make each bundle run in a separate process
2020-11-03 00:41:28 +01:00
const allSubtasks = [
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
standardSubtask,
2020-11-03 00:41:28 +01:00
contentscriptSubtask,
disableConsoleSubtask,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
installSentrySubtask,
phishingDetectSubtask,
].map((subtask) =>
runInChildProcess(subtask, {
buildType,
isLavaMoat,
shouldLintFenceFiles,
}),
);
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
// make a parent task that runs each task in a child thread
return composeParallel(initiateLiveReload, ...allSubtasks);
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
}
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
function createTaskForBundleDisableConsole({ devMode }) {
const label = 'disable-console';
return createNormalBundle({
browserPlatforms,
buildType,
destFilepath: `${label}.js`,
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
devMode,
entryFilepath: `./app/scripts/${label}.js`,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
ignoredFiles,
label,
shouldLintFenceFiles,
});
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
}
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
function createTaskForBundleSentry({ devMode }) {
const label = 'sentry-install';
return createNormalBundle({
browserPlatforms,
buildType,
destFilepath: `${label}.js`,
devMode,
entryFilepath: `./app/scripts/${label}.js`,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
ignoredFiles,
label,
shouldLintFenceFiles,
});
}
function createTaskForBundlePhishingDetect({ devMode }) {
const label = 'phishing-detect';
return createNormalBundle({
buildType,
browserPlatforms,
destFilepath: `${label}.js`,
devMode,
entryFilepath: `./app/scripts/${label}.js`,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
ignoredFiles,
label,
shouldLintFenceFiles,
});
}
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
// the "contentscript" bundle contains the "inpage" bundle
function createTaskForBundleContentscript({ devMode, testing }) {
const inpage = 'inpage';
const contentscript = 'contentscript';
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
return composeSeries(
createNormalBundle({
buildType,
browserPlatforms,
destFilepath: `${inpage}.js`,
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
devMode,
entryFilepath: `./app/scripts/${inpage}.js`,
label: inpage,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
ignoredFiles,
shouldLintFenceFiles,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
testing,
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
}),
createNormalBundle({
buildType,
browserPlatforms,
destFilepath: `${contentscript}.js`,
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
devMode,
entryFilepath: `./app/scripts/${contentscript}.js`,
label: contentscript,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
ignoredFiles,
shouldLintFenceFiles,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
testing,
}),
);
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
}
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
function createFactoredBuild({
browserPlatforms,
buildType,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
devMode,
entryFiles,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
ignoredFiles,
shouldLintFenceFiles,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
testing,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
}) {
return async function () {
// create bundler setup and apply defaults
const buildConfiguration = createBuildConfiguration();
buildConfiguration.label = 'primary';
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
const { bundlerOpts, events } = buildConfiguration;
// devMode options
const reloadOnChange = Boolean(devMode);
const minify = Boolean(devMode) === false;
const envVars = getEnvironmentVariables({ buildType, devMode, testing });
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
setupBundlerDefaults(buildConfiguration, {
buildType,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
devMode,
envVars,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
ignoredFiles,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
minify,
reloadOnChange,
shouldLintFenceFiles,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
});
// set bundle entries
bundlerOpts.entries = [...entryFiles];
// setup lavamoat
// lavamoat will add lavapack but it will be removed by bify-module-groups
// we will re-add it later by installing a lavapack runtime
const lavamoatOpts = {
policy: path.resolve(__dirname, '../../lavamoat/browserify/policy.json'),
policyOverride: path.resolve(
__dirname,
'../../lavamoat/browserify/policy-override.json',
),
writeAutoPolicy: process.env.WRITE_AUTO_POLICY,
};
Object.assign(bundlerOpts, lavamoatBrowserify.args);
bundlerOpts.plugin.push([lavamoatBrowserify, lavamoatOpts]);
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
// setup bundle factoring with bify-module-groups plugin
// note: this will remove lavapack, but its ok bc we manually readd it later
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
Object.assign(bundlerOpts, bifyModuleGroups.plugin.args);
bundlerOpts.plugin = [...bundlerOpts.plugin, [bifyModuleGroups.plugin]];
// instrument pipeline
let sizeGroupMap;
events.on('configurePipeline', ({ pipeline }) => {
// to be populated by the group-by-size transform
sizeGroupMap = new Map();
pipeline.get('groups').unshift(
// factor modules
bifyModuleGroups.groupByFactor({
entryFileToLabel(filepath) {
return path.parse(filepath).name;
},
}),
// cap files at 2 mb
bifyModuleGroups.groupBySize({
sizeLimit: 2e6,
groupingMap: sizeGroupMap,
}),
);
// converts each module group into a single vinyl file containing its bundle
const moduleGroupPackerStream = streamFlatMap((moduleGroup) => {
const filename = `${moduleGroup.label}.js`;
const childStream = wrapInStream(
moduleGroup.stream,
// we manually readd lavapack here bc bify-module-groups removes it
lavapack({ raw: true, hasExports: true, includePrelude: false }),
source(filename),
);
return childStream;
});
pipeline.get('vinyl').unshift(moduleGroupPackerStream, buffer());
// add lavamoat policy loader file to packer output
moduleGroupPackerStream.push(
new Vinyl({
path: 'policy-load.js',
contents: lavapack.makePolicyLoaderStream(lavamoatOpts),
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
}),
);
// setup bundle destination
browserPlatforms.forEach((platform) => {
const dest = `./dist/${platform}/`;
pipeline.get('dest').push(gulp.dest(dest));
});
});
// wait for bundle completion for postprocessing
events.on('bundleDone', () => {
const commonSet = sizeGroupMap.get('common');
// create entry points for each file
for (const [groupLabel, groupSet] of sizeGroupMap.entries()) {
// skip "common" group, they are added to all other groups
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
if (groupSet === commonSet) continue;
switch (groupLabel) {
case 'ui': {
renderHtmlFile({
htmlName: 'popup',
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
groupSet,
commonSet,
browserPlatforms,
useLavamoat: false,
});
renderHtmlFile({
htmlName: 'notification',
groupSet,
commonSet,
browserPlatforms,
useLavamoat: false,
});
renderHtmlFile({
htmlName: 'home',
groupSet,
commonSet,
browserPlatforms,
useLavamoat: false,
});
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
break;
}
case 'background': {
renderHtmlFile({
htmlName: 'background',
groupSet,
commonSet,
browserPlatforms,
useLavamoat: false,
});
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
break;
}
case 'content-script': {
renderHtmlFile({
htmlName: 'trezor-usb-permissions',
groupSet,
commonSet,
browserPlatforms,
useLavamoat: false,
});
break;
}
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
default: {
throw new Error(
`build/scripts - unknown groupLabel "${groupLabel}"`,
);
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
}
}
}
});
await bundleIt(buildConfiguration);
};
}
function createNormalBundle({
browserPlatforms,
buildType,
destFilepath,
devMode,
entryFilepath,
extraEntries = [],
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
ignoredFiles,
label,
modulesToExpose,
shouldLintFenceFiles,
testing,
}) {
return async function () {
// create bundler setup and apply defaults
const buildConfiguration = createBuildConfiguration();
buildConfiguration.label = label;
const { bundlerOpts, events } = buildConfiguration;
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
// devMode options
const reloadOnChange = Boolean(devMode);
const minify = Boolean(devMode) === false;
const envVars = getEnvironmentVariables({ buildType, devMode, testing });
setupBundlerDefaults(buildConfiguration, {
buildType,
devMode,
envVars,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
ignoredFiles,
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
minify,
reloadOnChange,
shouldLintFenceFiles,
});
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
// set bundle entries
bundlerOpts.entries = [...extraEntries];
if (entryFilepath) {
bundlerOpts.entries.push(entryFilepath);
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
if (modulesToExpose) {
bundlerOpts.require = bundlerOpts.require.concat(modulesToExpose);
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
// instrument pipeline
events.on('configurePipeline', ({ pipeline }) => {
// convert bundle stream to gulp vinyl stream
// and ensure file contents are buffered
pipeline.get('vinyl').push(source(destFilepath));
pipeline.get('vinyl').push(buffer());
// setup bundle destination
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
browserPlatforms.forEach((platform) => {
const dest = `./dist/${platform}/`;
pipeline.get('dest').push(gulp.dest(dest));
});
});
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
await bundleIt(buildConfiguration);
};
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
function createBuildConfiguration() {
const label = '(unnamed bundle)';
const events = new EventEmitter();
const bundlerOpts = {
entries: [],
transform: [],
plugin: [],
require: [],
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
// non-standard bify options
manualExternal: [],
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
manualIgnore: [],
};
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
return { bundlerOpts, events, label };
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
function setupBundlerDefaults(
buildConfiguration,
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
{
buildType,
devMode,
envVars,
ignoredFiles,
minify,
reloadOnChange,
shouldLintFenceFiles,
},
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
) {
const { bundlerOpts } = buildConfiguration;
Object.assign(bundlerOpts, {
// Source transforms
transform: [
// Remove code that should be excluded from builds of the current type
createRemoveFencedCodeTransform(buildType, shouldLintFenceFiles),
// Transpile top-level code
babelify,
// Inline `fs.readFileSync` files
brfs,
],
// Use entryFilepath for moduleIds, easier to determine origin file
fullPaths: devMode,
// For sourcemaps
debug: true,
});
// Ensure react-devtools are not included in non-dev builds
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
if (!devMode) {
bundlerOpts.manualIgnore.push('react-devtools');
}
// Inject environment variables via node-style `process.env`
if (envVars) {
bundlerOpts.transform.push([envify(envVars), { global: true }]);
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
// Ensure that any files that should be ignored are excluded from the build
if (ignoredFiles) {
bundlerOpts.manualExclude = ignoredFiles;
}
// Setup reload on change
if (reloadOnChange) {
setupReloadOnChange(buildConfiguration);
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
if (minify) {
setupMinification(buildConfiguration);
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
// Setup source maps
setupSourcemaps(buildConfiguration, { devMode });
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
function setupReloadOnChange({ bundlerOpts, events }) {
// Add plugin to options
Object.assign(bundlerOpts, {
plugin: [...bundlerOpts.plugin, watchify],
// Required by watchify
cache: {},
packageCache: {},
});
// Instrument pipeline
events.on('configurePipeline', ({ bundleStream }) => {
// Handle build error to avoid breaking build process
// (eg on syntax error)
bundleStream.on('error', (err) => {
gracefulError(err);
});
});
}
function setupMinification(buildConfiguration) {
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
const minifyOpts = {
mangle: {
reserved: ['MetamaskInpageProvider'],
},
};
const { events } = buildConfiguration;
events.on('configurePipeline', ({ pipeline }) => {
pipeline.get('minify').push(
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
// this is the "gulp-terser-js" wrapper around the latest version of terser
through.obj(
callbackify(async (file, _enc) => {
const input = {
[file.sourceMap.file]: file.contents.toString(),
};
const opts = {
sourceMap: {
filename: file.sourceMap.file,
content: file.sourceMap,
},
...minifyOpts,
};
const res = await terser.minify(input, opts);
file.contents = Buffer.from(res.code);
applySourceMap(file, res.map);
return file;
}),
),
);
});
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
function setupSourcemaps(buildConfiguration, { devMode }) {
const { events } = buildConfiguration;
events.on('configurePipeline', ({ pipeline }) => {
pipeline.get('sourcemaps:init').push(sourcemaps.init({ loadMaps: true }));
pipeline
.get('sourcemaps:write')
// Use inline source maps for development due to Chrome DevTools bug
// https://bugs.chromium.org/p/chromium/issues/detail?id=931675
.push(
devMode
? sourcemaps.write()
: sourcemaps.write('../sourcemaps', { addComment: false }),
);
});
}
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
async function bundleIt(buildConfiguration) {
const { label, bundlerOpts, events } = buildConfiguration;
const bundler = browserify(bundlerOpts);
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
// manually apply non-standard options
bundler.external(bundlerOpts.manualExternal);
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
bundler.ignore(bundlerOpts.manualIgnore);
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
if (Array.isArray(bundlerOpts.manualExclude)) {
bundler.exclude(bundlerOpts.manualExclude);
}
// output build logs to terminal
bundler.on('log', log);
Exclude files from builds by build type (#12521) This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions. The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details. ### Code Fencing and ESLint When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor. ### File Exclusion For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`. As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice. Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
2021-11-02 04:20:31 +01:00
// forward update event (used by watchify)
bundler.on('update', () => performBundle());
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
console.log(`Bundle start: "${label}"`);
await performBundle();
console.log(`Bundle end: "${label}"`);
async function performBundle() {
// this pipeline is created for every bundle
// the labels are all the steps you can hook into
const pipeline = labeledStreamSplicer([
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
'groups',
[],
'vinyl',
[],
'sourcemaps:init',
[],
'minify',
[],
'sourcemaps:write',
[],
'dest',
[],
]);
const bundleStream = bundler.bundle();
// trigger build pipeline instrumentations
events.emit('configurePipeline', { pipeline, bundleStream });
// start bundle, send into pipeline
bundleStream.pipe(pipeline);
// nothing will consume pipeline, so let it flow
pipeline.resume();
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
await endOfStream(pipeline);
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
// call the completion event to handle any post-processing
events.emit('bundleDone');
Build system refactor (#8140) * build - start static asset task cleanup * build - simplify manifest tasks * build - refactor + rename some tasks * build - various cleanups * manifest - fix ref from controller * build - drop gulp for simple async tasks * build - breakout gulpfile into multiple files * build - rename some tasks * build - use task fn refs instead of string names * build - bundle all scripts first, except for contentscript * build - improve task timeline * deps - update lock * build - improve task time printout * build/scripts - remove intermediate named task * build - use 'yarn build' for task entry points * build - properly run tasks via runTask for timeline display * development/announcer - fix manifest path + clean * build - lint fix * build - make all defined tasks possible entry points * build/task - properly report errors during task * ci - fix sesify/lavamoat-viz build command * build/scripts - run each bundle in separate processes * lint fix * build - forward childProcess logs to console * build/task - fix parallel/series stream end event * build/scripts refactor contentscript+inpage into a single task * build/static - use the fs for 150x speedup zomg * lint fix * build/static - fix css copy * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/scripts.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/index.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * deps - remove redundant mkdirp * deps - remove unused pumpify * deps - remove redundant merge-deep * deps - prefer is-stream of isstream * deps - remove clone for lodash.cloneDeep * clean - remove commented code * build/static - use fs.copy + fast-glob instead of linux cp for better platform support * build/manifest - standardize task naming * build/display - clean - remove unused code * bugfix - fix fs.promises import * build - create "clean" as named task for use as entrypoint * build/static - fix for copying dirs * Update development/build/task.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/display.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build - use task refs, tasks only return promises not streams, etc * lint fi bad merge + lint * build - one last cleanup + refactor * build - add comments introducing file * build/manifest - fix bug + subtasks dont beed to be named * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * build/task - remove unused fn * Update package.json Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> * Update development/build/styles.js Co-Authored-By: Mark Stacey <markjstacey@gmail.com> Co-authored-by: Mark Stacey <markjstacey@gmail.com>
2020-03-09 01:55:02 +01:00
}
}
function getEnvironmentVariables({ buildType, devMode, testing }) {
const environment = getEnvironment({ devMode, testing });
if (environment === ENVIRONMENT.PRODUCTION && !process.env.SENTRY_DSN) {
throw new Error('Missing SENTRY_DSN environment variable');
}
return {
METAMASK_DEBUG: devMode,
METAMASK_ENVIRONMENT: environment,
METAMASK_VERSION: version,
METAMASK_BUILD_TYPE: buildType,
NODE_ENV: devMode ? ENVIRONMENT.DEVELOPMENT : ENVIRONMENT.PRODUCTION,
IN_TEST: testing ? 'true' : false,
PUBNUB_SUB_KEY: process.env.PUBNUB_SUB_KEY || '',
PUBNUB_PUB_KEY: process.env.PUBNUB_PUB_KEY || '',
CONF: devMode ? metamaskrc : {},
SENTRY_DSN: process.env.SENTRY_DSN,
SENTRY_DSN_DEV: metamaskrc.SENTRY_DSN_DEV,
INFURA_PROJECT_ID: getInfuraProjectId({ buildType, environment, testing }),
SEGMENT_HOST: metamaskrc.SEGMENT_HOST,
SEGMENT_WRITE_KEY: getSegmentWriteKey({ buildType, environment }),
SWAPS_USE_DEV_APIS: process.env.SWAPS_USE_DEV_APIS === '1',
ONBOARDING_V2: metamaskrc.ONBOARDING_V2 === '1',
COLLECTIBLES_V1: metamaskrc.COLLECTIBLES_V1 === '1',
};
}
function getEnvironment({ devMode, testing }) {
// get environment slug
if (devMode) {
return ENVIRONMENT.DEVELOPMENT;
} else if (testing) {
return ENVIRONMENT.TESTING;
} else if (process.env.CIRCLE_BRANCH === 'master') {
return ENVIRONMENT.PRODUCTION;
2020-11-03 00:41:28 +01:00
} else if (
/^Version-v(\d+)[.](\d+)[.](\d+)/u.test(process.env.CIRCLE_BRANCH)
) {
return ENVIRONMENT.RELEASE_CANDIDATE;
} else if (process.env.CIRCLE_BRANCH === 'develop') {
return ENVIRONMENT.STAGING;
} else if (process.env.CIRCLE_PULL_REQUEST) {
return ENVIRONMENT.PULL_REQUEST;
}
return ENVIRONMENT.OTHER;
}
function renderHtmlFile({
htmlName,
groupSet,
commonSet,
browserPlatforms,
useLavamoat,
}) {
if (useLavamoat === undefined) {
throw new Error(
'build/scripts/renderHtmlFile - must specify "useLavamoat" option',
);
}
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
const htmlFilePath = `./app/${htmlName}.html`;
const htmlTemplate = readFileSync(htmlFilePath, 'utf8');
const jsBundles = [...commonSet.values(), ...groupSet.values()].map(
(label) => `./${label}.js`,
);
const htmlOutput = Sqrl.render(htmlTemplate, { jsBundles, useLavamoat });
Build - refactor for bundle factoring and swappable runtime (#11080) * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * wip * build - breakout sentry-install bundle * deps - move new build sys deps to published versions * chore: lint fix * clean - remove unused file * clean - remove unsused package script * lavamoat - update build system policy * build - render html to all platforms * development - improve sourcemap debugger output * deps - update lavapack * lint - fix * deps - update lavapack for bugfix * deps - update lavapack for bugfix * deps - bump lavapack for line ending normalization * sourcemap explorer - disable boundary validation * ci - reset normal ci flow * build - re-enable minification on prod * build - remove noisy log about html dest * build - update terser and remove gulp wrapper for sourcemap fix * Revert "sourcemap explorer - disable boundary validation" This reverts commit 94112209ed880a6ebf4ee2ded411e59db6908162. * build - reenable react-devtools in dev mode * Updating lockfile * lint fix * build/dev - patch watchifys incompatible binary stats output * ui - add comment about conditional import * build - improve comment * Update development/stream-flat-map.js Co-authored-by: Brad Decker <git@braddecker.dev> * Outputting all bundle file links (metamaskbot) Co-authored-by: ryanml <ryanlanese@gmail.com> Co-authored-by: Brad Decker <git@braddecker.dev>
2021-07-15 19:59:34 +02:00
browserPlatforms.forEach((platform) => {
const dest = `./dist/${platform}/${htmlName}.html`;
// we dont have a way of creating async events atm
writeFileSync(dest, htmlOutput);
});
}
function beep() {
process.stdout.write('\x07');
}
function gracefulError(err) {
console.warn(err);
beep();
}