* Split out advanced gas fees by network and delete old values
* use arrow functions in preferences test
* changes
* added back priorityFeeProperCase to en messages
* update types
* remove case change
* feat(878): implement new incoming transaction toggle networks for setting and onboarding
* Update state snapshots
* feat(878): change gaps, migration types based on comment
---------
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
Remove the IncomingTransactionController and replace it with an internal helper class.
Move incoming transactions into the central transactions object.
Create a new RemoteTransactionSource interface to decouple incoming transaction support from Etherscan.
Split the incoming transaction logic into multiple files for easier maintenance.
Migration #77 would set the `TokenListController.tokensChainsCache`
state to `undefined` if it wasn't already set to anything when that
migration was run. This is probably harmless except that it results
in Sentry errors during migrations, and it results in that property
having a value (at least temporarily) that doesn't match its type.
Migration #77 has been updated to prevent this property from being
set to `undefined` going forward. A new migration has been added to
delete this value for any users already affected by this problem. The
new migration was named "92.1" so that it could run after 92 but before
93, to make backporting this to v10.34.x easier (v10.34.x is currently
on migration 92). "92.1" is still a valid number so this should work
just as well as a whole number.
* capture exception for sentry when invariant conditions are met in migration 82
* Code cleanup
* Capture exceptions in invariant conditions for migrations 83,84,85,86,89,91,93,94
* Update app/scripts/migrations/082.test.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Code cleanup
* Fix SentryObject type declaration
* Stop throwing error if preferences controller is undefined
* Refactor 084 and 086 to remove double negative
* Capture exceptions for invariant states in in migrations 87,88,90 and 92
* lint fix
* log warning in migration 82 when preferences controller is undefined
---------
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
The "last fetched" state for the `PhishingController` has been deleted
to force an immediate full update of the phishing configuration state.
We're doing this because the state was cleared in v10.34.2 because the
format of that state had changed.
This has been implemented in migration 92. The previous migration 92
has been renamed to 93 because it won't be included until a future
release. We need the migrations to remain sequential, and this will
save us from having to resolve a complex conflict when releasing this.
* Fix migration 88 to handle the case where chainId keys can be undefined
* Add migration 91 to delete network configurations that have no chainId
* Lint fix
* Update migration number
* Update migration 91 description
* Update version numbers in 091.test.js
* Fix 088.test.ts typescript problem
* Fix 088.test.ts typescript problem
* Update app/scripts/migrations/091.ts
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Change app/scripts/migrations/091.test.js to typescript
* clone oldstorage for test result comparisons in 091.test.js
* Lint fix
* Add missing test case
---------
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Handle the case where tokensChainsCache data is undefined in migration 77
* Delete parts of state that should have been removed in migrations 82,84,86 and 88
* Create 077-supplements.md
* Update 077-supplements.md
* Update 077-supplements/*.js code comments
* Fix types and jsdoc
* Type/lint fix
* Cleanup
* Add 'should set data to an empty object if it is null' test case to 077.test.js
* Update app/scripts/migrations/077.test.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Modify deletion criteria so that all decimal chain id proprties are deleted in migration 88 supplement
* Readme.md
* Update app/scripts/migrations/077.test.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Update app/scripts/migrations/077.test.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Update app/scripts/migrations/077.test.js
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Lint fix
* Only delete decimal chain id keyed-entries in migration 88 supplement if there are hexadecimal keyed entries as well
* Remove redundant test
* Add 'does not delete' cases for nftcontroller related tests in 077.test.js
---------
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Update phishing controller to v4.0.0
* Move phishing e2e test utilities into its own helper.js
* Update phishing detection e2e test
* Update MetaMask Controller test mocks
* Update mv3 phishing tests
* Fix test for 500 error on warning page
* Allow for directories in test folder
* Update migration number
* Linting fixes
* Remove fail on console error
* Separate mocks from helpers
* Have migration delete PhishingController state entirely
* Remove phishing detection directory
* Only delete the listState in migration
* Bump migration version
* Migration 89: ensure providerConfig in state has an id property
* Exit transformState function early if providerConfig already has an id
* Update migrations/index.js
* Code cleanup
* Rename `provider` to `providerConfig`
The network controller `provider` state has been renamed to
`providerConfig`. This better reflects what this state is, and makes
this controller more closely aligned with the core network controller.
All references to the provider configuration have been updated to
prefer `providerConfig` over `provider`, to make the distinction clear
between a provider and provider config.
Closes#18902
* Add migration
* Remove the network controller `previousProviderStore`
The `previousProvider` network controller state has been replaced with
a private internal property. This was only used internally, it did not
need to be part of state.
This relates to https://github.com/MetaMask/metamask-extension/issues/18303
* Remove redundant tests
* Ensure that all networkConfiguration object in networkController state have an id
* Lint fix
* Update app/scripts/migrations/084.ts
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Add unit tests for error cases
* Simplify code
* Remove unnecessary any typing
* Fix network controller type checking
* Lint fix
* Improve typing
---------
Co-authored-by: legobeat <109787230+legobeat@users.noreply.github.com>
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
The `network` store of the network controller crams two types of data
into one place. It roughly tracks whether we have enough information to
make requests to the network and whether the network is capable of
receiving requests, but it also stores the ID of the network (as
obtained via `net_version`).
Generally we shouldn't be using the network ID for anything, as it has
been completely replaced by chain ID, which all custom RPC endpoints
have been required to support for over a year now. However, as the
network ID is used in various places within the extension codebase,
removing it entirely would be a non-trivial effort. So, minimally, this
commit splits `network` into two stores: `networkId` and
`networkStatus`. But it also expands the concept of network status.
Previously, the network was in one of two states: "loading" and
"not-loading". But now it can be in one of four states:
- `available`: The network is able to receive and respond to requests.
- `unavailable`: The network is not able to receive and respond to
requests for unknown reasons.
- `blocked`: The network is actively blocking requests based on the
user's geolocation. (This is specific to Infura.)
- `unknown`: We don't know whether the network can receive and respond
to requests, either because we haven't checked or we tried to check
and were unsuccessful.
This commit also changes how the network status is determined —
specifically, how many requests are used to determine that status, when
they occur, and whether they are awaited. Previously, the network
controller would make 2 to 3 requests during the course of running
`lookupNetwork`.
* First, if it was an Infura network, it would make a request for
`eth_blockNumber` to determine whether Infura was blocking requests or
not, then emit an appropriate event. This operation was not awaited.
* Then, regardless of the network, it would fetch the network ID via
`net_version`. This operation was awaited.
* Finally, regardless of the network, it would fetch the latest block
via `eth_getBlockByNumber`, then use the result to determine whether
the network supported EIP-1559. This operation was awaited.
Now:
* One fewer request is made, specifically `eth_blockNumber`, as we don't
need to make an extra request to determine whether Infura is blocking
requests; we can reuse `eth_getBlockByNumber`;
* All requests are awaited, which makes `lookupNetwork` run fully
in-band instead of partially out-of-band; and
* Both requests for `net_version` and `eth_getBlockByNumber` are
performed in parallel to make `lookupNetwork` run slightly faster.
The migration template has been converted to TypeScript. We can start
writing migrations in TypeScript today; they have no dependencies on
JavaScript modules.
The PhishingController has been updated to v2. This release should
dramatically reduce network traffic and double the update speed of the
phishing list.
This was accomplished by combining both of our phishing configurations
into one list (the "stalelist"), then creating a separate list of the
changes just the past few days (the "hotlist"). Now users will download
a smaller list more frequently (every 30 minutes rather than every
hour), whereas the full list is only updated every 4 days.
The combined configuration means that we no longer know which list was
responsible for each block. The phishing warning page has been updated
to dynamically look this information up, to ensure users are still
directed to the correct place to dispute a block. This update to the
phishing warning page also includes the recent redesign.
* Migrate to new controller packages
`@metamask/controllers` is deprecated, and most of the controllers that
lived here are now located in their own package ([1]). This commit
replaces `@metamask/controllers` in `package.json` with references to
these packages and updates `import` lines to match.
[1]: https://github.com/MetaMask/controllers/pull/831
* Support GitHub registry for draft PRs (#16549)
* Add additional allowed host to lockfile linter
* Update LavaMoat policies
* Add policy exception for nanoid
* Add additional nanoid overrides
* Update LavaMoat policies again
* Bump controller packages
* Update lavamoat
* Bump controller packages
* Update packages to v1.0.0
* Expand gitignore comment
* Unpin controller dependencies, using ^ range instead
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
The controllers package has been updated to v33. The only breaking
change in this release was to rename the term "collectible" to "NFT"
wherever it appeared in the API.
Changes in this PR have been kept minimal; additional renaming can be
done in separate PRs. This PR only updates the controller names,
controller state, controller methods, and any direct references to
these things. NFTs are still called "collectibles" in most places.
* Remove 3box feature and delete ThreeBoxController
Lint locale messages
lavamoat policy updates
* Restore 3Box user trait with value `false`
The 3Box user trait has been restored and hard-coded as `false`. This
ensures that users don't get stuck in our metrics as having this trait.
A deprecation comment has been left in various places for this trait.
* Remove unused state
* Remove additional 3box-related things
* Run `yarn-deduplicate`
* Restore migration that was lost while rebasing
* Remove obsolete override
* Remove additional unused resolutions/dependencies
* Update LavaMoat policies
* Remove obsolete security advisory ignore entries
* Remove 3Box fixture builder method
* Update unit tests
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Deprecating Rinkeby, setting default debug network to Goerli
* Deprecating Ropsten and Kovan
* Conflict fix
* Remove unused localization, test fixes
* Add migration for moving used deprecated testnets to custom networks
* Fix migrator test
* Add more unit tests
* Migration updates provider type to rpc if deprecated network is selected
* Migration fully and correctly updates the provider if selected network is a deprecated testnet
* Continue to show deprecation warning on each of rinkeby, ropsten and kovan
* Add rpcUrl deprecation message to loading screen
* Removing mayBeFauceting prop
Co-authored-by: Dan Miller <danjm.com@gmail.com>
* remove decentralized 4byte function signature registry since it is griefed and we can't algorithmically check for best option when 4byte is down
* add migration
* remove nock of on chain registry call in getMethodDataAsync test
* Increase likelyhood of valid method signatures being returned by getMethodData
* Update coverage
* Update coverage
* Update coverage
* add a migration to clear knownMethodData
* Small typo changes
Co-authored-by: Alex <adonesky@gmail.com>
* Rename NotificationController to AnnouncementController
* Fix test
* Add test for missing NotificationController state
* Bump controllers
* Move test to correct file
* Rename config key
* Add migration 71 to list of migrations
* Fix selector after migration
We currently store the JSON-RPC request and response objects in the permission activity log. The utility of doing this was always rather dubious, but never problematic. Until now.
In Flask, as the restricted methods have expanded in number, user secrets may be included on JSON-RPC message objects. This PR removes these properties from the permission activity log, and adds a migration which does the same to existing log objects. We don't interact with the log objects anywhere in our codebase, but we don't want unexpected properties to cause errors in the future should any log objects be retained.
This PR also updates relevant tests and test data. It makes a minor functional change to how a request is designated as a success or failure, but this should not change any behavior in practice.
ESLint rules have been added to enforce our JSDoc conventions. These
rules were introduced by updating `@metamask/eslint-config` to v9.
Some of the rules have been disabled because the effort to fix all lint
errors was too high. It might be easiest to enable these rules one
directory at a time, or one rule at a time.
Most of the changes in this PR were a result of running
`yarn lint:fix`. There were a handful of manual changes that seemed
obvious and simple to make. Anything beyond that and the rule was left
disabled.
The ESLint config has been updated to v8. The breaking changes are:
* The Prettier rule `quoteProps` has been changed from `consistent` to
`as-needed`, meaning that if one key requires quoting, only that key is
quoted rather than all keys.
* The ESLint rule `no-shadow` has been made more strict. It now
prevents globals from being shadowed as well.
Most of these changes were applied with `yarn lint:fix`. Only the
shadowing changes required manual fixing (shadowing variable names were
either replaced with destructuring or renamed).
The dependency `globalThis` was added to the list of dynamic
dependencies in the build system, where it should have been already.
This was causing `depcheck` to fail because the new lint rules required
removing the one place where `globalThis` had been erroneously imported
previously.
A rule requiring a newline between multiline blocks and expressions has
been disabled temporarily to make this PR smaller and to avoid
introducing conflicts with other PRs.
This PR introduces the concept of subject _types_ to be associated with each subject in the `SubjectMetadataController`, and used for control flow in our RPC stack (`setupProviderEngine` and so forth).
We already differentiate between "types" of subjects in various places on an ad hoc basis via boolean flags (e.g. `isInternal` in our RPC stack) or the presence/absence of certain values in the subject's metadata (specifically `metadata.extensionId`). The status quo is manageable if not ideal, but will start to become untenable with the introduction of Snaps in the near future.
Therefore, this PR establishes a `SUBJECT_TYPES` enum and adds the `subjectType` property to the metadata of each subject. A new migration is added to accomplish this. Finally, we specify and `INTERNAL` subject type to distinguish internal from external requests.
# Permission System 2.0
## Background
This PR migrates the extension permission system to [the new `PermissionController`](https://github.com/MetaMask/snaps-skunkworks/tree/main/packages/controllers/src/permissions).
The original permission system, based on [`rpc-cap`](https://github.com/MetaMask/rpc-cap), introduced [`ZCAP-LD`](https://w3c-ccg.github.io/zcap-ld/)-like permissions to our JSON-RPC stack.
We used it to [implement](https://github.com/MetaMask/metamask-extension/pull/7004) what we called "LoginPerSite" in [version 7.7.0](https://github.com/MetaMask/metamask-extension/releases/tag/v7.7.0) of the extension, which enabled the user to choose which accounts, if any, should be exposed to each dapp.
While that was a worthwhile feature in and of itself, we wanted a permission _system_ in order to enable everything we are going to with Snaps.
Unfortunately, the original permission system was difficult to use, and necessitated the creation of the original `PermissionsController` (note the "s"), which was more or less a wrapper for `rpc-cap`.
With this PR, we shake off the yoke of the original permission system, in favor of the modular, self-contained, ergonomic, and more mature permission system 2.0.
Note that [the `PermissionController` readme](https://github.com/MetaMask/snaps-skunkworks/tree/main/packages/controllers/src/permissions/README.md) explains how the new permission system works.
The `PermissionController` and `SubjectMetadataController` are currently shipped via `@metamask/snap-controllers`. This is a temporary state of affairs, and we'll move them to `@metamask/controllers` once they've landed in prod.
## Changes in Detail
First, the changes in this PR are not as big as they seem. Roughly half of the additions in this PR are fixtures in the test for the new migration (number 68), and a significant portion of the remaining ~2500 lines are due to find-and-replace changes in other test fixtures and UI files.
- The extension `PermissionsController` has been deleted, and completely replaced with the new `PermissionController` from [`@metamask/snap-controllers`](https://www.npmjs.com/package/@metamask/snap-controllers).
- The original `PermissionsController` "domain metadata" functionality is now managed by the new `SubjectMetadataController`, also from [`@metamask/snap-controllers`](https://www.npmjs.com/package/@metamask/snap-controllers).
- The permission activity and history log controller has been renamed `PermissionLogController` and has its own top-level state key, but is otherwise functionally equivalent to the existing implementation.
- Migration number 68 has been added to account for the new state changes.
- The tests in `app/scripts/controllers/permissions` have been migrated from `mocha` to `jest`.
Reviewers should focus their attention on the following files:
- `app/scripts/`
- `metamask-controller.js`
- This is where most of the integration work for the new `PermissionController` occurs.
Some functions that were internal to the original controller were moved here.
- `controllers/permissions/`
- `selectors.js`
- These selectors are for `ControllerMessenger` selector subscriptions. The actual subscriptions occur in `metamask-controller.js`. See the `ControllerMessenger` implementation for details.
- `specifications.js`
- The caveat and permission specifications are required by the new `PermissionController`, and are used to specify the `eth_accounts` permission and its JSON-RPC method implementation.
See the `PermissionController` readme for details.
- `migrations/068.js`
- The new state should be cross-referenced with the controllers that manage it.
The accompanying tests should also be thoroughly reviewed.
Some files may appear new but have just moved and/or been renamed:
- `app/scripts/lib/rpc-method-middleware/handlers/request-accounts.js`
- This was previously implemented in `controllers/permissions/permissionsMethodMiddleware.js`.
- `test/mocks/permissions.js`
- A truncated version of `test/mocks/permission-controller.js`.
Co-authored-by: Mark Stacey <markjstacey@gmail.com>
* Add migration to set showTestNetworks to true if there is evidence the user has used a test network
* Add migration to index file
* Remove console.log
* Clean up conditional structure in migration 67
