This PR adds `snaps` under Flask build flags to the extension. This branch is mostly equivalent to the current production version of Flask, excepting some bug fixes and tweaks.
Closes#11626
Adds a resolution for `follow-redirects` to resolve https://github.com/advisories/GHSA-74fj-2j2h-c42q. The dependency graph is `analytics-node#axios#follow-redirects`, and neither parent package has published a fix for this vulnerability.
This resolution was used to force a transitive dependency to be updated
to a specific version. But this target version was within the range
that was already requested, so a resolution is not needed. Yarn
resolutions are used for forcing a package to update to something
_outside_ of the requested range. For in-range updates, a Yarn lockfile
update is all we need, and it leaves us with less of a maintenance
burden (the resolution can clobber future updates).
Adds a resolution for `follow-redirects` to resolve https://github.com/advisories/GHSA-74fj-2j2h-c42q. The dependency graph is `analytics-node#axios#follow-redirects`, and neither parent package has published a fix for this vulnerability.
ESLint rules have been added to enforce our JSDoc conventions. These
rules were introduced by updating `@metamask/eslint-config` to v9.
Some of the rules have been disabled because the effort to fix all lint
errors was too high. It might be easiest to enable these rules one
directory at a time, or one rule at a time.
Most of the changes in this PR were a result of running
`yarn lint:fix`. There were a handful of manual changes that seemed
obvious and simple to make. Anything beyond that and the rule was left
disabled.
The `lint:fix` script now also calls `yarn stylelint --fix`. This step
was omitted previously, despite `stylelint` being part of the `lint`
npm script.
* origin/develop: (106 commits)
Add more missing `appName` messages (#13140)
Add missing `appName` localized messages for Flask and Beta (#13138)
EIP-1559 V2: Adding default settings to advanced gas modal (#12911)
EIP-1559 V2: Advanced gas fee modal - base fee and priority fee trends (#13025)
Wire up priority fee range in new gas modal (#13044)
EIP-1559 V2: Advanced gas fee modal - getting 12 hour ranges of base fee and priority fee (#12996)
Switch Flask horizontal logos (#13113)
Update `@babel/runtime` patch to fix lockdown error (#13109)
Use promisified background in setUseNonceField (#13107)
Fix account name duplicates (#12867)
Choose accounts refactor (#13039)
Fix permissions-connect-footer "learn more" link (#13092)
Feat/collectibles the return (#12970)
Subject metadata cleanup (#13090)
Fix merge conflict typo
Bump just-safe-set from 2.1.0 to 2.2.3 (#13049)
Fix typo in German translation (#13040)
Using EIP-1559 V2 for swaps (#12966)
Make restore vault a form so an user can submit via keyboard (#12989)
Remove legacy node parent detection (#12814)
...
* origin/develop: (100 commits)
Switch Flask horizontal logos (#13113)
Update `@babel/runtime` patch to fix lockdown error (#13109)
Use promisified background in setUseNonceField (#13107)
Fix account name duplicates (#12867)
Choose accounts refactor (#13039)
Fix permissions-connect-footer "learn more" link (#13092)
Feat/collectibles the return (#12970)
Subject metadata cleanup (#13090)
Fix merge conflict typo
Bump just-safe-set from 2.1.0 to 2.2.3 (#13049)
Fix typo in German translation (#13040)
Using EIP-1559 V2 for swaps (#12966)
Make restore vault a form so an user can submit via keyboard (#12989)
Remove legacy node parent detection (#12814)
Add stories for Home notification component (#13035)
Update Redux DevTools README instructions (#13038)
Jestify app/scripts/controller/network/**/*.test.js (#12985)
Fix order of account list (#12999)
Changes in gas loading animation in EIP-1559 V2 (#13016)
Add crowdin configuration and github action (#12552)
...
The ESLint config has been updated to v8. The breaking changes are:
* The Prettier rule `quoteProps` has been changed from `consistent` to
`as-needed`, meaning that if one key requires quoting, only that key is
quoted rather than all keys.
* The ESLint rule `no-shadow` has been made more strict. It now
prevents globals from being shadowed as well.
Most of these changes were applied with `yarn lint:fix`. Only the
shadowing changes required manual fixing (shadowing variable names were
either replaced with destructuring or renamed).
The dependency `globalThis` was added to the list of dynamic
dependencies in the build system, where it should have been already.
This was causing `depcheck` to fail because the new lint rules required
removing the one place where `globalThis` had been erroneously imported
previously.
A rule requiring a newline between multiline blocks and expressions has
been disabled temporarily to make this PR smaller and to avoid
introducing conflicts with other PRs.
The `selectHooks` function has been replaced with the equivalent
function from the `@metamask/rpc-methods` package, which is
functionally equivalent.
The function was included in that package so that it could be used
elsewhere in the `snaps-skunkworks` repo. Eventually the goal is to
migrate much of our RPC logic into this package so that it can be
shared across products, and by our libraries as needed.
# Permission System 2.0
## Background
This PR migrates the extension permission system to [the new `PermissionController`](https://github.com/MetaMask/snaps-skunkworks/tree/main/packages/controllers/src/permissions).
The original permission system, based on [`rpc-cap`](https://github.com/MetaMask/rpc-cap), introduced [`ZCAP-LD`](https://w3c-ccg.github.io/zcap-ld/)-like permissions to our JSON-RPC stack.
We used it to [implement](https://github.com/MetaMask/metamask-extension/pull/7004) what we called "LoginPerSite" in [version 7.7.0](https://github.com/MetaMask/metamask-extension/releases/tag/v7.7.0) of the extension, which enabled the user to choose which accounts, if any, should be exposed to each dapp.
While that was a worthwhile feature in and of itself, we wanted a permission _system_ in order to enable everything we are going to with Snaps.
Unfortunately, the original permission system was difficult to use, and necessitated the creation of the original `PermissionsController` (note the "s"), which was more or less a wrapper for `rpc-cap`.
With this PR, we shake off the yoke of the original permission system, in favor of the modular, self-contained, ergonomic, and more mature permission system 2.0.
Note that [the `PermissionController` readme](https://github.com/MetaMask/snaps-skunkworks/tree/main/packages/controllers/src/permissions/README.md) explains how the new permission system works.
The `PermissionController` and `SubjectMetadataController` are currently shipped via `@metamask/snap-controllers`. This is a temporary state of affairs, and we'll move them to `@metamask/controllers` once they've landed in prod.
## Changes in Detail
First, the changes in this PR are not as big as they seem. Roughly half of the additions in this PR are fixtures in the test for the new migration (number 68), and a significant portion of the remaining ~2500 lines are due to find-and-replace changes in other test fixtures and UI files.
- The extension `PermissionsController` has been deleted, and completely replaced with the new `PermissionController` from [`@metamask/snap-controllers`](https://www.npmjs.com/package/@metamask/snap-controllers).
- The original `PermissionsController` "domain metadata" functionality is now managed by the new `SubjectMetadataController`, also from [`@metamask/snap-controllers`](https://www.npmjs.com/package/@metamask/snap-controllers).
- The permission activity and history log controller has been renamed `PermissionLogController` and has its own top-level state key, but is otherwise functionally equivalent to the existing implementation.
- Migration number 68 has been added to account for the new state changes.
- The tests in `app/scripts/controllers/permissions` have been migrated from `mocha` to `jest`.
Reviewers should focus their attention on the following files:
- `app/scripts/`
- `metamask-controller.js`
- This is where most of the integration work for the new `PermissionController` occurs.
Some functions that were internal to the original controller were moved here.
- `controllers/permissions/`
- `selectors.js`
- These selectors are for `ControllerMessenger` selector subscriptions. The actual subscriptions occur in `metamask-controller.js`. See the `ControllerMessenger` implementation for details.
- `specifications.js`
- The caveat and permission specifications are required by the new `PermissionController`, and are used to specify the `eth_accounts` permission and its JSON-RPC method implementation.
See the `PermissionController` readme for details.
- `migrations/068.js`
- The new state should be cross-referenced with the controllers that manage it.
The accompanying tests should also be thoroughly reviewed.
Some files may appear new but have just moved and/or been renamed:
- `app/scripts/lib/rpc-method-middleware/handlers/request-accounts.js`
- This was previously implemented in `controllers/permissions/permissionsMethodMiddleware.js`.
- `test/mocks/permissions.js`
- A truncated version of `test/mocks/permission-controller.js`.
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* integration for tx decoding confirmation and history view
* upgrading @truffle/decoder to latest release 5.1.0
* Update acorn and colors patches
* feat: remove redundant styling
* feat: basic integration for nickname components
* feat: wiring functionality of adding new nickname
* feat: wire functionality of showing nickname modal
* feat: link the nickname popover with add/update popover
* feat: moving forward with address nicknames integration
* feat: fixing a bug related to passing chainId in addressBook
* feat: populating memo prop in addressbook entry
* feat: add explorer link
* feat: bug fixing update nickname component
* feat: fix proptypes
* feat: adding tooltip for copying nickname address
* featL fix styling for tx-details page
* feat: optimize code for error handling
* feat: limiting transaction decoding to tx with data
* feat: remove tree UI component
* feat: adding request to check for tx decoding supported networks
* feat: showing data hex component
* feat: fix react warnings
* feat: remove extra margin in tx decoding
* Remove unused package @truffle/source-map-utils
* Ensure messages get translated
* feat: link tx-decoding addresses with nicknames
* Omit value for boolean attributes
* Fix props reading in CopyRawData
* fix: fixing issue with transaltion
* Fix lint errors in TransactionDecoding
- Remove unused import
- Reorder imports
- Address conflict between caught `error` and error state flag by
renaming state flag to `hasError`
- Fix requestUrl identifier casing and use of template string
- Ensure `useEffect` gets passed the deps it needs
- Add scope braces around case statement where it's needed
- Omit literal `true` for boolean jsx attribute
- Refactor nested ternary as `if` statements
* fix: revert fetchWithCache modifications
* Fix linting for TransactionListItemDetails
- Remove unused import
- Fix import spacing
- Remove unused prop dereference
- Fix string interpolation for translated From/To
* Moving to popover pattern
* fix: sass color variable
* Omit value for boolean attribute
* Remove changes from modal.js
* fix: refactor nickname popovers
* Ensure const gets declared before it's used
* Fix linting for ConfirmTransactionBase
- Remove unused prop chainId
- Stop destructuring an unused field
* fix: refactor usage of nicknames popovers in send-content-container
* fix: remove extra prop updateAccountNicknameModal
* fix: refactor code for address.component
* fix: remove extra tooltip
* Ensure NicknamePopovers always returns component
* Fix linting for NicknamePopover component
- Fix useCallback deps
- Switch ternary to logical-or
* Fix linting for SenderToRecipient
... by fixing import order
* Remove unused addressCopied state
* Delete empty file
* fix: remove sender-to-recipient.container
* fix: refactor usage of nickname popovers in confirm-page-container
* fix: bug related to state variable
* Stylelint fix
* Lint fix
* Change "Total Amount" to "Total"
* Lint fix locales
* Update address-book.spec.js
* e2e test update
* Update e2e tests
* Fix issue where absence of function params in data hex tab would result in rendering a string
* Fix border radius, and width and height in small notification windows, of the update-nickname-popover
* Remove fake await
* Clean up
* Clean up
Co-authored-by: Alaa Hadad <alaahd@Alaas-MacBook-M1-Pro-14-inch.local>
Co-authored-by: Dan Miller <danjm.com@gmail.com>
Co-authored-by: g. nicholas d'andrea <gnidan@trufflesuite.com>
* Add support for eip-1559 on Trezor
* temp
* Lint fix
* Store trezor model type in background state instead attempting to get it in the frontend
* code simplification
* Temp update to eth-trezor-keyring version
* Tempory update to eth-trezor-keyring version
* Temp update to eth-trezor-keyring version
* Fix display of hdpath selector in connect hardware flow for trezor
* Updating the package version but we still need to run yarn setup and update the lockfile, once the package is updated
* Update yarn.lock
* Fix unit tests
* Version v10.6.3
* Fix "BigNumber Error: times() number type has more than 15 significant digits:" (#12802)
* Bump @metmamask/logo to v3.1.1 (#12822)
* Update changelog for v10.6.3
* We shouldn't be changing decimal places as user type, we should do that on blur. (#12631)
* We shouldn't be changing decimal places as user type, we should do that
on blur.
Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>
* Lint fixes.
Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>
* Refactor code.
Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>
* Linter fixes.
Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>
* Convert token input to BigNumber to handle decimals. (#12773)
* Fixes#12762
Adds a decimal length check for inputs and drops excess fractional part.
Another edgecase not accounted for is when a token's decimal precision is 0 and attempting sending decimals will result in omitting the fractional part.
* Change spies from sinon to jest and change onChange value to string.
* Adjust
* Remove sinon
* Add test for issue case
* DRY
* Simplify logic by using BigNumber
Co-authored-by: Dan Miller <danjm.com@gmail.com>
* Update changelog
* Ensure that handleChange in token-input.component handles empty values (#12835)
* Version v10.6.4
* update scams and network security risks link (#12752)
* Ignore sentry server errors in e2e tests (#12843)
* Ignore sentry server errors in e2e tests
* Update test/e2e/webdriver/driver.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
Co-authored-by: Dan J Miller <danjm.com@gmail.com>
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Update changelog for v10.6.4
Co-authored-by: MetaMask Bot <metamaskbot@users.noreply.github.com>
Co-authored-by: Daniel <80175477+dan437@users.noreply.github.com>
Co-authored-by: Olusegun Akintayo <akintayo.segun@gmail.com>
Co-authored-by: Thomas Huang <tmashuang@users.noreply.github.com>
Co-authored-by: Mingliang Liu <brightliu77@gmail.com>
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Version v10.6.3
* Fix "BigNumber Error: times() number type has more than 15 significant digits:" (#12802)
* Bump @metmamask/logo to v3.1.1 (#12822)
* Update changelog for v10.6.3
* We shouldn't be changing decimal places as user type, we should do that on blur. (#12631)
* We shouldn't be changing decimal places as user type, we should do that
on blur.
Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>
* Lint fixes.
Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>
* Refactor code.
Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>
* Linter fixes.
Signed-off-by: Akintayo A. Olusegun <akintayo.segun@gmail.com>
* Convert token input to BigNumber to handle decimals. (#12773)
* Fixes#12762
Adds a decimal length check for inputs and drops excess fractional part.
Another edgecase not accounted for is when a token's decimal precision is 0 and attempting sending decimals will result in omitting the fractional part.
* Change spies from sinon to jest and change onChange value to string.
* Adjust
* Remove sinon
* Add test for issue case
* DRY
* Simplify logic by using BigNumber
Co-authored-by: Dan Miller <danjm.com@gmail.com>
* Update changelog
* Ensure that handleChange in token-input.component handles empty values (#12835)
Co-authored-by: MetaMask Bot <metamaskbot@users.noreply.github.com>
Co-authored-by: Daniel <80175477+dan437@users.noreply.github.com>
Co-authored-by: Olusegun Akintayo <akintayo.segun@gmail.com>
Co-authored-by: Thomas Huang <tmashuang@users.noreply.github.com>
* add storybook unit tests with CI integration
* fix command and fix casing for test
* change ci ordering for storybook tasks
* fix syntax error
* fix jest
* lint
* Add transaction-total-banner render test to Storybook (#12517)
* transaction-total-banner
* lint
* confirm to spec
* lint
* fix jest ocnfig for snapshot test failure
* support qr based signer
* add CSP for fire fox
* get QR Hardware wallet name from device
* fix qrHardware state missing in runtime
* support qr based signer sign transaction
* refine Request Signature modal ui
* remove feature toggle
* refine ui
* fix notification is closing even there is a pending qr hardware transaction
* add chinese translation, refine ui, fix qr process was breaking in some case
* support import accounts by pubkeys
* refine qr-based wallet ui and fix bugs
* update @keystonehq/metamask-airgapped-keyring to fix that the signing hd path was inconsistent in some edge case
* fix: avoid unnecessay navigation, fix ci
* refactor qr-hardware-popover with @zxing/browser
* update lavamoat policy, remove firefox CSP
* refine qr reader ui, ignore unnecessary warning display
* code refactor, use async functions insteads promise
Co-authored-by: Soralit <soralitria@gmail.com>
`improved-yarn-audit` has been updated so that it supports GitHub
advisories. Two new GitHub advisories have been ignored, as they are
both moderate RegExp DoS vulnerabilities that don't affect us, and they
are embedded deep within our dependency graph and are difficult to
update.
* Add CollectiblesController
* bump controllers version
* add CollectibleDetectionController
* adapt to ERC1155 support changes in CollectiblesController
* update @metamask/controllers to v20.0.0
* update lavamoat policy files
* put collectibleDetectionController instantiation behind feature flag
`improved-yarn-audit` has been updated so that it supports GitHub
advisories. Two new GitHub advisories have been ignored, as they are
both moderate RegExp DoS vulnerabilities that don't affect us, and they
are embedded deep within our dependency graph and are difficult to
update.
Most notably this adds the ability to manage multiple Lattice/SafeCard
wallets simultaneously. If a user makes a request from an address not
associated with the device's active wallet, an error will display.
See: https://github.com/GridPlus/eth-lattice-keyring/pull/19
This PR adds one LavaMoat background script policy or each build type. It also renames the build system policy directory from `node` to `build-system` to make its purpose more clear. Each build type has the original `policy-override.json` for `main` builds. The `.prettierignore` file has been updated to match the locations of the new auto-generated policy files.
We need to maintain separate policies for each build type because each type will produce different bundles with different internal and external modules.
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
Most notably this adds the ability to manage multiple Lattice/SafeCard
wallets simultaneously. If a user makes a request from an address not
associated with the device's active wallet, an error will display.
See: https://github.com/GridPlus/eth-lattice-keyring/pull/19
We're bumping from `^6` to `^8`. All imports are now named, and they have been updated. This is a breaking change, in that support for `eth_signTransaction` is added in `^8.0.0`. We do not support this method in our UI, so our middleware stack has been instrumented to reject.
In addition, there are some non-breaking behavioral changes in this version that reviewers should be aware of, see the [7.0.0 release](https://github.com/MetaMask/eth-json-rpc-middleware/releases).
* Support for Layer 2 networks with transaction fees on both layers
* Use variable name in transaction-breakdown
* Add comment on code source to ui/helpers/utils/optimism/fetchEstimatedL1Fee.js
* Fix unit tests
* Ensure values passed to are defined
* Fix activity log
* Support for Layer 2 networks with transaction fees on both layers
* Use variable name in transaction-breakdown
* Add comment on code source to ui/helpers/utils/optimism/fetchEstimatedL1Fee.js
* Fix unit tests
* Ensure values passed to are defined
* Fix activity log
* GridPlus: Adds support for GridPlus Lattice1 hardware wallet
* Fixes issue with switching hardware HD path
The main `Select HD Path` piece of the account selection component was not
properly hooked up to the state manager (`onPathChange`) and the extra
`Popover` component was being used instead.
I'm not sure what the origin of this is, but I don't see why the Popover
is needed at all. I have remove it and hooked `onPathChange` directly into
the HD path selector dropdown.
This was an issue that nearly every Lattice user who had come from Ledger
has contacted us about.
* GridPlus: Addresses QA issues
* Adds Lattice tutorial + image
* Cleans up connectivity issues (see: https://github.com/GridPlus/eth-lattice-keyring/pull/16)
* GridPlus: Adds Firefox support
To connect to the Lattice you need to open a new tab/window and get
login data from it. We were not able to do this for Firefox because
we relied on the `window` API. This is now fixed.
See corresponding changes:
* `eth-lattice-keyring`: https://github.com/GridPlus/eth-lattice-keyring/pull/17
* Lattice connector: https://github.com/GridPlus/wallet-web/pull/152
* GridPlus: Adds missing error path for Firefox
See: 242a93f559
* Check if ledger was successfully able to establish transport on mount of confirm screens
* Update ledger message/action if transport creation was blocked by existing connection
* TEMP: point eth-ledger-bridge-keyring to commite, REMOVE BEFORE MERGE
* Update eth-ledger-bridge-keyring to v0.10.0
* Connect ledger via webhid if that option is available
* Explicitly setting preference for webhid
* Use ledgerTransportType enum instead of booleans for ledger live and webhid preferences
* Use single setLEdgerTransport preference methods and property
* Temp
* Lint fix
* Unit test fix
* Remove async keyword from setLedgerTransportPreference function definition in preferences controller
* Fix ledgelive setting toggle logic
* Migrate useLedgerLive preference property to ledgerTransportType
* Use shared constants for ledger transport type enums
* Use constant for ledger usb vendor id
* Use correct property to check if ledgerLive preference is set when deciding whether to ask for webhid connection
* Update eth-ledger-bridge-keyring to v0.9.0
* Only show ledger live transaction helper messages if using ledger live
* Only show ledger live part of tutorial if ledger live setting is on
* Fix ledger related prop type errors
* Explicitly use u2f enum instead of empty string as a transport type; default transport type to webhid if available; use constants for u2f and webhid
* Cleanup
* Wrap ledger webhid device request in try/catch
* Clean up
* Lint fix
* Ensure user can easily connect their ledger wallet when they need to.
* Fix locales
* Fix/improve locales changes
* Remove unused isFirefox property from confirm-transaction-base.container.js
* Disable transaction and message signing confirmation if ledger webhid requires connection
* Ensure translation keys for ledger connection options in settings dropdown can be properly detected by verify-locales
* Drop .component from ledger-instruction-field file name
* Move renderLedgerLiveStep to module scope
* Remove ledgerLive from function and message names in ledger-instruction-field
* Wrap ledger connection logic in ledger-instruction-field in try catch
* Clean up signature-request.component.js
* Check whether the signing address, and not the selected address, is a ledger account in singature-request.container
* Ensure ledger instructions and webhid connection button are shown on signature-request-original signatures
* Improve webhid selection handling in select-ledger-transport-type onChange handler
* Move metamask redux focused ledger selectors to metamask duck
* Lint fix
* Use async await in checkWebHidStatusRef.current
* Remove unnecessary use of ref in ledger-instruction-field.js
* Lint fix
* Remove unnecessary try/catch in ledger-instruction-field.js
* Check if from address, not selected address, is from a ledger account in confirm-approve
* Move findKeyringForAddress to metamask duck
* Fix typo in function name
* Ensure isEqualCaseInsensitive handles possible differences in address casing
* Fix Learn More link size in advanced settings tab
* Update app/scripts/migrations/066.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Update ui/pages/settings/advanced-tab/advanced-tab.component.test.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Add jsdoc comments for new selectors
* Use jest.spyOn for mocking navigator in ledger webhid migration tests
* Use LEDGER_TRANSPORT_TYPES values to set proptype of ledgerTransportType
* Use LEDGER_TRANSPORT_TYPES values to set proptype of ledgerTransportType
* Fix font size of link in ledger connection description in advanced settings
* Fix return type in setLedgerTransportPreference comment
* Clean up connectHardware code for webhid connection in actions.js
* Update app/scripts/migrations/066.test.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Update ui/ducks/metamask/metamask.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Add migration test for when useLedgerLive is true in a browser that supports webhid
* Lint fix
* Fix inline-link size
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Check if ledger was successfully able to establish transport on mount of confirm screens
* Update ledger message/action if transport creation was blocked by existing connection
* TEMP: point eth-ledger-bridge-keyring to commite, REMOVE BEFORE MERGE
* Update eth-ledger-bridge-keyring to v0.10.0
This PR enables the exclusion of JavaScript and JSON source by `buildType`, and enables the running of `eslint` under LavaMoat. 80-90% of the changes in this PR are `.patch` files and LavaMoat policy additions.
The file exclusion is designed to work in conjunction with our code fencing. If you forget to fence an import statement of an excluded file, the application will now error on boot. **This PR commits us to a particular naming convention for files intended only for certain builds.** Continue reading for details.
### Code Fencing and ESLint
When a file is modified by the code fencing transform, we run ESLint on it to ensure that we fail early for syntax-related issues. This PR adds the first code fences that will be actually be removed in production builds. As a consequence, this was also the first time we attempted to run ESLint under LavaMoat. Making that work required a lot of manual labor because of ESLint's use of dynamic imports, but the manual changes necessary were ultimately quite minor.
### File Exclusion
For all builds, any file in `app/`, `shared/` or `ui/` in a sub-directory matching `**/${otherBuildType}/**` (where `otherBuildType` is any build type except `main`) will be added to the list of excluded files, regardless of its file extension. For example, if we want to add one or more pages to the UI settings in Flask, we'd create the folder `ui/pages/settings/flask`, add any necessary files or sub-folders there, and fence the import statements for anything in that folder. If we wanted the same thing for Beta, we would name the directory `ui/pages/settings/beta`.
As it happens, we already organize some of our source files in this way, namely the logo JSON for Beta and Flask builds. See `ui/helpers/utils/build-types.js` to see how this works in practice.
Because the list of ignored filed is only passed to `browserify.exclude()`, any files not bundled by `browserify` will be ignored. For our purposes, this is mostly relevant for `.scss`. Since we don't have anything like code fencing for SCSS, we'll have to consider how to handle our styles separately.
Static files have been added for the Flask build. This includes logos
of each size and variety that we use, and it includes the 3D model JSON
file.
Closes#12427
On an M1 Mac, when running `yarn start`, CPU can spike to 100% CPU, and
sometimes a bunch of `mdworker` instances will get spawned. This seems
to be caused by the file-watching mechanism used in dev to automatically
regenerate the build when something is changed. More specifically, we
are using an older version of `watchify`, which uses an older version of
`chokidar`, which is the package that actually does the watching. v4.0.0
of `watchify` upgrades `chokidar` to v3.x ([1]), which comes with
"massive CPU & RAM consumption improvements" ([2]). After the upgrade,
CPU usage decreases to 20-40%.
[1]: https://github.com/browserify/watchify/blob/master/CHANGELOG.md#400
[2]: https://github.com/paulmillr/chokidar/releases/tag/3.0.0
* Connect ledger via webhid if that option is available
* Explicitly setting preference for webhid
* Use ledgerTransportType enum instead of booleans for ledger live and webhid preferences
* Use single setLEdgerTransport preference methods and property
* Temp
* Lint fix
* Unit test fix
* Remove async keyword from setLedgerTransportPreference function definition in preferences controller
* Fix ledgelive setting toggle logic
* Migrate useLedgerLive preference property to ledgerTransportType
* Use shared constants for ledger transport type enums
* Use constant for ledger usb vendor id
* Use correct property to check if ledgerLive preference is set when deciding whether to ask for webhid connection
* Update eth-ledger-bridge-keyring to v0.9.0
* Only show ledger live transaction helper messages if using ledger live
* Only show ledger live part of tutorial if ledger live setting is on
* Fix ledger related prop type errors
* Explicitly use u2f enum instead of empty string as a transport type; default transport type to webhid if available; use constants for u2f and webhid
* Cleanup
* Wrap ledger webhid device request in try/catch
* Clean up
* Lint fix
* Ensure user can easily connect their ledger wallet when they need to.
* Fix locales
* Fix/improve locales changes
* Remove unused isFirefox property from confirm-transaction-base.container.js
* Disable transaction and message signing confirmation if ledger webhid requires connection
* Ensure translation keys for ledger connection options in settings dropdown can be properly detected by verify-locales
* Drop .component from ledger-instruction-field file name
* Move renderLedgerLiveStep to module scope
* Remove ledgerLive from function and message names in ledger-instruction-field
* Wrap ledger connection logic in ledger-instruction-field in try catch
* Clean up signature-request.component.js
* Check whether the signing address, and not the selected address, is a ledger account in singature-request.container
* Ensure ledger instructions and webhid connection button are shown on signature-request-original signatures
* Improve webhid selection handling in select-ledger-transport-type onChange handler
* Move metamask redux focused ledger selectors to metamask duck
* Lint fix
* Use async await in checkWebHidStatusRef.current
* Remove unnecessary use of ref in ledger-instruction-field.js
* Lint fix
* Remove unnecessary try/catch in ledger-instruction-field.js
* Check if from address, not selected address, is from a ledger account in confirm-approve
* Move findKeyringForAddress to metamask duck
* Fix typo in function name
* Ensure isEqualCaseInsensitive handles possible differences in address casing
* Fix Learn More link size in advanced settings tab
* Update app/scripts/migrations/066.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Update ui/pages/settings/advanced-tab/advanced-tab.component.test.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Add jsdoc comments for new selectors
* Use jest.spyOn for mocking navigator in ledger webhid migration tests
* Use LEDGER_TRANSPORT_TYPES values to set proptype of ledgerTransportType
* Use LEDGER_TRANSPORT_TYPES values to set proptype of ledgerTransportType
* Fix font size of link in ledger connection description in advanced settings
* Fix return type in setLedgerTransportPreference comment
* Clean up connectHardware code for webhid connection in actions.js
* Update app/scripts/migrations/066.test.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Update ui/ducks/metamask/metamask.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Add migration test for when useLedgerLive is true in a browser that supports webhid
* Lint fix
* Fix inline-link size
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
The npm scripts used to run Mocha scripts have been greatly simplified.
As we transition more tests from Mocha to Jest it was becoming
increasingly difficult to update the CLI arguments to keep all of these
scripts working correctly. This reorganization should make that process
much simpler.
The base Mocha options are in `.mocharc.js` - all except for the target
tests to run. Those are still given via the CLI. There is a second
config file specifically for the `test:unit:lax` tests (i.e. the Mocha
tests that have no coverage requirements) because it requires a change
to the `ignored` configuration property. We can create an additional
configuration file for each test script we add that needs further
configuration changes.
The `test:unit:path` script used to be used to run Mocha tests at a
given path. Now that can be done using `yarn mocha` instead, so this
script has been removed.
The `yarn watch` command has been broken for some time now, so it has
been removed as well. Mocha tests can still be run with a file watcher
using `yarn mocha --watch <path>` or `yarn test:unit:mocha --watch`.
The README has been updated to remove references about the `watch`
command that was removed. I considered explaining the other test
scripts there as well, but they were difficult to explain I will
attempt to update the README after making further simplifications
instead.
`immer` has been updated to v9. This didn't require any changes on our
part; the only breaking changes are to the TypeScript types [1].
The `@reduxjs/toolkit` library has been updated as well, to ensure that
it's using the updated version of Immer internally as well. This update
makes our patch of that package obsolete, as the problematic pattern
that were were patching out is no longer present.
[1]: https://github.com/immerjs/immer/releases/tag/v9.0.0
`immer` has been updated to v9. This didn't require any changes on our
part; the only breaking changes are to the TypeScript types [1].
The `@reduxjs/toolkit` library has been updated as well, to ensure that
it's using the updated version of Immer internally as well. This update
makes our patch of that package obsolete, as the problematic pattern
that were were patching out is no longer present.
[1]: https://github.com/immerjs/immer/releases/tag/v9.0.0
The main `version` field in `package.json` will now include the beta
version (if present) rather than it being passed in via the CLI when
building. The `version` field is now a fully SemVer-compatible version,
with the added restriction that any prerelease portion of the version
must match the format `<build type>.<build version>`.
This brings the build in-line with the future release process we will
be using for the beta version. The plan is for each future release to
enter a "beta phase" where the version would get updated to reflect
that it's a beta, and we would increment this beta version over time as
we update the beta. The manifest gives us a place to store this beta
version. It was also important to replace the automatic minor bump
logic that was being used previously, because the version in beta might
not be a minor bump.
Additionally, the filename logic used for beta builds was updated to
be generic across all build types rather than beta-specific. This will
be useful for Flask builds in the future.
This PR fixes our local unit test package scripts. When the state migration unit tests were migrated to Jest in #12106, it left the `test:unit` script in a broken state, because it didn't tell `mocha` to ignore the state migration tests.
Arguably, that script was already broken, since the most reasonably expectation from its name is that it runs _all_ unit tests. The PR makes it so that it does just that, by means of `concurrently`.
Unfortunately, `concurrently` only outputs errors from child processes once (at the time when they exit, https://github.com/open-cli-tools/concurrently/issues/134). This means that we have to search/navigate the output for this combined script to identify the failure. That said, it's better than the status quo.
* lavamoat - add lavamoat to webapp background
* test:e2e - add delay to resolve failure
* test:e2e - add delay to resolve failure
* build - add a switch for applying lavamoat, currently off for all
* test/e2e - remove delays added for lavamoat
* Revert "test/e2e - remove delays added for lavamoat"
This reverts commit 79c3479f15c072ed362ba1d4f1af41ea11a17d63.
The warnings about use of the unsafe Buffer constructor have been
addressed by package updates and patches.
The updates were:
* `gulp-sourcemaps` was updated from v2 to v3, and was patched to
replace remaining uses of the `Buffer` constructor
* Upstream PR: https://github.com/gulp-sourcemaps/gulp-sourcemaps/pull/388
* The transitive dependency `yazl` was updated from v2.4.3 to v2.5.1
in the lockfile.
* The abandoned packages `combine-source-map` and `inline-source-map`
were patched.
The dependency `caniuse-lite` has been updated using a Yarn resolution,
because it was pinned to a specific version by some dependencies. All
versions requested in our dependency tree are 1.x so this did not
introduce any breaking changes.
This resolves a frequent console warning that shows up during builds,
and when running tests and the linter.
* Jestify migrations/
* Lint exclude migrations from mocha config, and add inclusion to jest config
* Add migration tests to jest config
* Exclude/ignore migration tests
* Set process.env.IN_TEST to true when running tests locally
This PR adds build-time code exclusion by means of code fencing. For details, please see the README in `./development/build/transforms`. Note that linting of transformed files as a form of validation is added in a follow-up, #12075.
Hopefully exhaustive tests are added to ensure that the transform works according to its specification. Since these tests are Node-only, they required their own Jest config. The recommended way to work with multiple Jest configs is using the `projects` field in the Jest config, however [that feature breaks coverage collection](https://github.com/facebook/jest/issues/9628). That being the case, I had to set up two separate Jest configs. In order to get both test suites to run in parallel, Jest is now invoked via a script, `./test/run-jest.sh`.
By way of example, this build system feature allows us to add fences like this:
```javascript
this.store.updateStructure({
...,
GasFeeController: this.gasFeeController,
TokenListController: this.tokenListController,
///: BEGIN:ONLY_INCLUDE_IN(beta)
PluginController: this.pluginController,
///: END:ONLY_INCLUDE_IN
});
```
Which at build time are transformed to the following if the build type is not `beta`:
```javascript
this.store.updateStructure({
...,
GasFeeController: this.gasFeeController,
TokenListController: this.tokenListController,
});
```
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
This rationalizes how arguments are passed to and parsed by the build system. To accomplish this, everything that isn't an environment variable from `.metamaskrc` or our CI environment is now passed as an argument on the command line.
Of such arguments, the `entryTask` is still expected as a positional argument in the first position (i.e. `process.argv[2]`), but everything else must be passed as a named argument. We use `minimist` to parse the arguments, and set defaults to preserve existing behavior.
Arguments are parsed in a new function, `parseArgv`, in `development/build/index.js`. They are assigned to environment variables where convenient, and otherwise returned from `parseArgv` to be passed to other functions invoked in the same file.
This change is motivated by our previous inconsistent handling of arguments to the build system, which will grow increasingly problematic as the build system grows in complexity. (Which it will very shortly, as we introduce Flask builds.)
Miscellaneous changes:
- Adds a build system readme at `development/build/README.md`
- Removes the `beta` package script. Now, we can instead call: `yarn dist --build-type beta`
- Fixes the casing of some log messages and reorders some parameters in the build system
* bump @metamask/controllers to v15.0.1 and remove AbortController workaround in e2e tests
* remove old abortcontroller polyfill
* bump @metamask/controllers to v15.0.2
There are a few issues encountered when running `yarn setup` on new
Apple Silicon (aka M1, aka arm64) Macs:
* The script halts when attempting to run the install step for
the `chromedriver` package with the message "Only Mac 64 bits
supported". This is somewhat misleading as it seems to indicate that
chromedriver can only be installed on a 64-bit Mac. However, what I
think is happening is that the installation script for `chromedriver`
is not able to detect that an arm64 CPU *is* a 64-bit CPU. After
looking through the `chromedriver` repo, it appears that 87.0.1 is the
first version that adds a proper check ([1]).
Note that upgrading chromedriver caused the Chrome-specific tests to
fail intermittently on CI. I was not able to 100% work out the reason
for this, but ensuring that X (which provides a way for Chrome to run
in a GUI setting from the command line) is available seems to fix
these issues.
* The script also halts when attempting to run the install step for
the `electron` package. This happens because for the version of
`electron` we are using (9.4.2), there is no available binary for
arm64. It appears that Electron 11.x was the first version to support
arm64 Macs ([2]). This is a bit trickier to resolve because we don't
explicitly rely on `electron` — that's brought in by `react-devtools`.
The first version of `react-devtools` that relies on `electron` 11.x
is 4.11.0 ([3]).
[1]: 469dd0a6ee
[2]: https://www.electronjs.org/blog/apple-silicon
[3]: https://github.com/facebook/react/blob/main/packages/react-devtools/CHANGELOG.md#4110-april-9-2021
* EIP-1559 - Provide support for Ledger
* Update ui/selectors/selectors.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Add shared constants for hw types
* bump eth-ledger-bridge-keyring to v0.7.0
Co-authored-by: David Walsh <davidwalsh83@gmail.com>
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
Co-authored-by: Alex <adonesky@gmail.com>
There are a few issues encountered when running `yarn setup` on new
Apple Silicon (aka M1, aka arm64) Macs:
* The script halts when attempting to run the install step for
the `chromedriver` package with the message "Only Mac 64 bits
supported". This is somewhat misleading as it seems to indicate that
chromedriver can only be installed on a 64-bit Mac. However, what I
think is happening is that the installation script for `chromedriver`
is not able to detect that an arm64 CPU *is* a 64-bit CPU. After
looking through the `chromedriver` repo, it appears that 87.0.1 is the
first version that adds a proper check ([1]).
Note that upgrading chromedriver caused the Chrome-specific tests to
fail intermittently on CI. I was not able to 100% work out the reason
for this, but ensuring that X (which provides a way for Chrome to run
in a GUI setting from the command line) is available seems to fix
these issues.
* The script also halts when attempting to run the install step for
the `electron` package. This happens because for the version of
`electron` we are using (9.4.2), there is no available binary for
arm64. It appears that Electron 11.x was the first version to support
arm64 Macs ([2]). This is a bit trickier to resolve because we don't
explicitly rely on `electron` — that's brought in by `react-devtools`.
The first version of `react-devtools` that relies on `electron` 11.x
is 4.11.0 ([3]).
[1]: 469dd0a6ee
[2]: https://www.electronjs.org/blog/apple-silicon
[3]: https://github.com/facebook/react/blob/main/packages/react-devtools/CHANGELOG.md#4110-april-9-2021
* EIP-1559 - Provide support for Ledger
* Update ui/selectors/selectors.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Add shared constants for hw types
* bump eth-ledger-bridge-keyring to v0.7.0
Co-authored-by: David Walsh <davidwalsh83@gmail.com>
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
Co-authored-by: Alex <adonesky@gmail.com>
Adds the latest version of `@metamask/controllers`, and updates our usage of the `ApprovalController`, which has been migrated to `BaseControllerV2`. Of [the new `controllers` release](https://github.com/MetaMask/controllers/releases/tag/v15.0.0), only the `ApprovalController` migration should be breaking.
This is the first time we use events on the `ControllerMessenger` to update the badge, so I turned the messenger into a property on the main `MetaMaskController` in order to subscribe to events on it in `background.js`. I confirmed that the badge does indeed update during local QA.
As it turns out, [MetaMask/controllers#571](https://github.com/MetaMask/controllers/pull/571) was breaking for a single unit test case, which is now handled during setup and teardown for the related test suite (`metamask-controller.test.js`).
* Use current block gas limit as the limit passed eth_estimateGas (#11658)
* Revert "Use current block gas limit as the limit passed eth_estimateGas (#11658)" (#11660)
This reverts commit aee79fd44d.
* Use current block gas limit as the limit passed eth_estimateGas (#11658)
* Version v9.8.4
* Allow higher precision gas prices in the send flow (#11652)
* Allow higher precision gas prices in the send flow
* Fix gas duck test
* Allow more decimals in transaction breakdown gas price
* [skip e2e] Update changelog for v9.8.4 (#11665)
Co-authored-by: ryanml <ryanlanese@gmail.com>
Co-authored-by: MetaMask Bot <metamaskbot@users.noreply.github.com>
* Use current block gas limit as the limit passed eth_estimateGas (#11658)
* Revert "Use current block gas limit as the limit passed eth_estimateGas (#11658)" (#11660)
This reverts commit aee79fd44d.
* Use current block gas limit as the limit passed eth_estimateGas (#11658)
* Version v9.8.4
* Allow higher precision gas prices in the send flow (#11652)
* Allow higher precision gas prices in the send flow
* Fix gas duck test
* Allow more decimals in transaction breakdown gas price
* [skip e2e] Update changelog for v9.8.4 (#11665)
Co-authored-by: Dan J Miller <dmiller@kyokan.io>
Co-authored-by: MetaMask Bot <metamaskbot@users.noreply.github.com>
* add erc-721 token detection and flag to disable sending
* addressing feedback
* remove redundant provider instantiation
* fix issue caused by unprotected destructuring
* add tests and documentation
* move add isERC721 flag to useTokenTracker hook
* Update and unit tests
* use memoizedTokens in useTokenTracker
Co-authored-by: Dan Miller <danjm.com@gmail.com>
* add erc-721 token detection and flag to disable sending
* addressing feedback
* remove redundant provider instantiation
* fix issue caused by unprotected destructuring
* add tests and documentation
* move add isERC721 flag to useTokenTracker hook
* Update and unit tests
* use memoizedTokens in useTokenTracker
Co-authored-by: Dan Miller <danjm.com@gmail.com>
