Problem statement:
BigchainDB v2.0.0b9 has been around for quite a while. Recently we have updated
Tendermint supported version to v0.31.5 which has incompatible blockchain.
Despite the fact that we have defined instructions on chain migration, no one
expected to migrate to incompatible chain within patch version range. So there
is a demand for Tendermint v0.22.8 compatibility among BigchainDB users.
Work has been done:
bigchaindb-abci package was upgraded to support multiple API versions.
New configuration field stating tendermint version was added.
Signed-off-by: David Dashyan <mail@davie.li>
1. Save pre-commit state in the beginning of end block.
2. Provide an interface for custom elections to rollback on crash recovery.
3. Simplify pre-commit management.
4. Add crash recovery for updert-validator and chain migration elecitons.
* Separate pending and effective validator updates.
- Pending validator updates do not prevent elections from concluding.
- ValidatorElection overrides has_conclude to not conclude when there is a pending update for the matching height.
- Effective validator updates deem past elections inconclusive.
* Problem: Looking for election block is inefficient.
Solution: Record placed elections, update the records upon election conclusion.
* Clarify the conclusion order in Election.process_blocks.
* Insert election records in bulk.
Otherwise, one can significantly slow nodes down by posting a whole bunch of unique elections.
* Change get_election to use find_one.
* Calculate total votes without making extra query.
* Fix the pending valset check.
* Fix election test setup.
* Problem: We need a way to synchronize a halt to block production to allow for upgrades across breaking changes.
* Solution: Created `MigrationElection`.
* Problem: Need documentation for `migration` elections.
* Solution: Updated the docs.
* Problem: `MigrationElection` needs 'new' CLI method.
* Solution: Updated the definition of `election` to include the new `migration` type.
* Problem: The way `end_block` checks for concluded elections assumes there is only one type of election (so we can't conclude an `upsert-validator` and a `chain-migration` at the same height).
* Solution: Re-engineered the code in `Elections` to conclude multiple elections in the same block. If more than one election change the validator set, only one of them is applied.
* Problem: Tendermint change to store validator changes at height h+2 will break `Election.get_status`.
* Solution: Reworked `get_validator_change` to look at only the latest block height or less.
- Hard-wire the supported Tendermint version(s) right in the code of BigchainDB Server. Check the version of Tendermint and disconnect if Tendermint version is an unsupported one.
- Expose a CLI command bigchaindb tendermint-version to display the supported Tendermint versions.
- PR also takes care the long list of warnings we get when we run tests.
- Updated deprecated pymongo methods
- Do not call pytest fixtures directly.
- Also added the doc for the new cli command
Solution: Stop getting events from the Tendermint events API. Get the
event right before returning from the COMMIT phase of Tendermint, and
publish it to the events queue.
* Problem: `ValidatorElection` and `MigrationElection` need to inherit from a common `Election` class.
Solution: Factored the common logic out of `ValidatorElection` and moved it to `Election` parent class.
* Problem: No need to store different types of elections in their own tables
Solution: Remove `DB_TABLE` property from `Election` class.
Solution: Created the `elections` table with secondary_index `election_id`.
* Problem: `UpsertValidatorVote` can be generalized to just be `Vote`
Solution: Renamed, refactored and moved the `Vote` class to tie in with the more general `Election` base class.
* Problem: `election_id` is not unique if two elections have the same properties.
Solution: Added a random `uuid4` seed to enforce uniqueness.
* Problem: No good way to check for val set absence.
Solution: Make get_validator_set/get_validators return None/[] when there are no validators yet.
* Problem: Incompatible ABCI chain upgrades.
Solution: Record known chains and sync through InitChain. Triggering the migration and adjusting other ABCI endpoints will follow.
* Problem: We need to store the `election_id` as part of the `validator_update` so we can efficiently check which election was resposible for the change
Solution: Added the parameter to `store_validator_set` and aligned the tests
* Problem: Logic for `upsert-validator show` is convoluted
Solution: Rewrote the function to be much simpler
* Problem: Need a uniqueness constraint for election_id wrt validator changes
Solution: Added a new key to the db schema
* Problem: Source files contain no license info
Solution: Add comments with SPDX license info to source files
* Python 3 files don't need # -*- coding: utf-8 -*-
* Problem: Validator set not tracked by BigchainDB
Solution: BigchainDB depends on tendermint's RPC API to get the validator set
which is not avaiable during replay so the validators set should be tracked
inside BigchainDB
* Problem: Unclear code and documentation
Solution: Fix decode_validator and docs strings
* Problem: Doc strings missing
Solution: Add doc string for store_validato_set
* Problem: core.py contains an unused class, `Bigchain`
Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.
* Problem: core.py contains an unused class, `Bigchain`
Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.
* Fixed flake8 complaint about too many blank lines
* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...
Sorry in advance!
* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...
Sorry in advance!
* Updating from master changed BigchainDB.process_post_response to a private method, so I had to align with that.
* Fixed a couple stale references to bigchaindb.Bigchain in docstrings
* Missed a reference to `Bigchain` in a patch call...
* Problem: BigchainDB class should be part of project root
Solution: Removed the /tendermint directory and moved its contents to project root
* Problem: Flake8 complained that imports were not at the top of the file
Solution: Had to play around with the order of imports to avoid cyclic dependencies, but its working and style compliant now
* Problem: Stale reference to /tendermint directory in the index
Solution: Removed the references to /tendermint
* Problem: Flake8 complaining of unused import in __init__.py
Solution: The import is there so I can import App directly from bigchaindb, rather than from bigchaindb.core (which I think improves code readability. I added a # noqa to silence Flake8.
* Problem: Stale references to `bigchaindb.tendermint.BigchainDB` in the rst files cause Sphinx to fail
Solution: Updated the autodoc files to use `bigchaindb.BigchainDB` instead
* Problem: Stale reference to the `tendermint` directory in an @patch in a disabled test
Solution: Updated the @patch for completeness
* Problem: BigchainDB class should be part of project root
Solution: Removed the /tendermint directory and moved its contents to project root
* Problem: Flake8 complained that imports were not at the top of the file
Solution: Had to play around with the order of imports to avoid cyclic dependencies, but its working and style compliant now
* Problem: Stale reference to /tendermint directory in the index
Solution: Removed the references to /tendermint
* Problem: Flake8 complaining of unused import in __init__.py
Solution: The import is there so I can import App directly from bigchaindb, rather than from bigchaindb.core (which I think improves code readability. I added a # noqa to silence Flake8.
* Problem: Stale references to `bigchaindb.tendermint.BigchainDB` in the rst files cause Sphinx to fail
Solution: Updated the autodoc files to use `bigchaindb.BigchainDB` instead
* Problem: Stale reference to the `tendermint` directory in an @patch in a disabled test
Solution: Updated the @patch for completeness
* Problem: core.py contains an unused class, `Bigchain`
Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.
* Problem: core.py contains an unused class, `Bigchain`
Solution: Remove core.py. Refactor BigchainDB Class to remove inheritance from Bigchain.
* Fixed flake8 complaint about too many blank lines
* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...
Sorry in advance!
* Attempting to fix Sphinx docs. This may result in some redundant commits, as I don't know what I'm doing, and I can't experiment without running the CI...
Sorry in advance!
* Updating from master changed BigchainDB.process_post_response to a private method, so I had to align with that.
* Fixed a couple stale references to bigchaindb.Bigchain in docstrings
* Missed a reference to `Bigchain` in a patch call...
* Merge with master and re-do some changes
* Problem: Tendermint configuration not present in BigchainDB config
* Handle tendermint configurations properly
- Update docs
* Nitpick
* Missed some conflicts
* Make changes in tendermint/lib.py instead of deprecated core.py
- Also remove redundant info from docs about default values
* Fix docsserver automethod
* Problem: voting code is not used anymore
Solution: remove all voting related code
* Problem: Some voting functionality is still present.
Solution: Remove it. Update some of the related tests.
* Problem: some skipped tests are now running
Solution: remove pytest mark to not run them
* Problem: fastquery is not related to votes
Solution: remove it in another PR
* Problem: BigchainDB config has stale/deprecated parameters
- Remove `keyring` and `keypair` from config
- Update tests
- Add `tendermint` config
* Fix flake8
* Update some naming conventions and redundant changes
* Remove redundant routine `fast_query`
* Remove deprecated parameters and descriptions
* remove some more unwanted code
* Problem: Two flake8 errors made Travis fail
Solution: Fix the two flake8 errors
* Address comments
- Remove reference of nodes_except_me and me_private and me
as attributes of BigchainDB instances
- Update and re-add test(s)
- Do not introduce `tendermint` in configuration instead handle that
in a separate PR along with docs
* Address comments
- Remove tests that are already covered with 2.0
- Remove tests that are no longer relevant
- Add TODO for more cleanup
* Remove tendermint config from configure command
* Fixed 2 PEP257 compliance errors
* Upgrade Tendermint to 0.19.7 except for Docker image, which is still at 0.19.2
* Problem: backlog_reassign_delay is defunct but still listed
Solution: remove all mentions of & calls for backlog_reassign_delay
* Testing tendermint docker compose for travis
* Typo tendermint*
* Re-do migrate cli changes
* don't skip event stream test
* Fix flake8 and unskip a test skipped during dev
* Revert styling changes made from editor
* skip a failing test case which was already disabled
* Remove reference to 'keypair' in commands
- If spent=None return all outputs
- If spent=True return only spent outputs
- If spent=False return only unspent outputs
- Updated documentation
- Add the ability to return only spent outputs in core
- Added and update tests
* Add assets text search endpoint
* Filter out assets from invalid transactions.
- Added the limit argument to limit the returned results
- Created and updated tests
* Added documentation for the assets endpoint.
- Added some docstrings
* Removed unnecessary fixtures
David Dashyan
Jürgen Eckel
Lev Berman
Troy McConaghy
Muawia Khan
Zachary Bowen
vrde
Vanshdeep Singh
codegeschrei
Ketan Bhatt
Rodolphe Marques
libscott
Tim Daubenschütz
Scott Sadler
Sylvain Bellemare