1. Save pre-commit state in the beginning of end block.
2. Provide an interface for custom elections to rollback on crash recovery.
3. Simplify pre-commit management.
4. Add crash recovery for updert-validator and chain migration elecitons.
* Separate pending and effective validator updates.
- Pending validator updates do not prevent elections from concluding.
- ValidatorElection overrides has_conclude to not conclude when there is a pending update for the matching height.
- Effective validator updates deem past elections inconclusive.
* Problem: Looking for election block is inefficient.
Solution: Record placed elections, update the records upon election conclusion.
* Clarify the conclusion order in Election.process_blocks.
* Insert election records in bulk.
Otherwise, one can significantly slow nodes down by posting a whole bunch of unique elections.
* Change get_election to use find_one.
* Calculate total votes without making extra query.
* Fix the pending valset check.
* Fix election test setup.
- Do not conclude migration election if there is a migration in progress.
- Rewrite election tests to not use mocks and assert many different things.
- Record concluded elections in the `election` collection.
* Problem: We need a way to synchronize a halt to block production to allow for upgrades across breaking changes.
* Solution: Created `MigrationElection`.
* Problem: Need documentation for `migration` elections.
* Solution: Updated the docs.
* Problem: `MigrationElection` needs 'new' CLI method.
* Solution: Updated the definition of `election` to include the new `migration` type.
* Problem: The way `end_block` checks for concluded elections assumes there is only one type of election (so we can't conclude an `upsert-validator` and a `chain-migration` at the same height).
* Solution: Re-engineered the code in `Elections` to conclude multiple elections in the same block. If more than one election change the validator set, only one of them is applied.
* Problem: Tendermint change to store validator changes at height h+2 will break `Election.get_status`.
* Solution: Reworked `get_validator_change` to look at only the latest block height or less.
