Commit Graph

1369 Commits

Author SHA1 Message Date
Tim Gates 0961aa6b26
docs: fix simple typo, neeeded -> needed (#2717)
There is a small typo in bigchaindb/lib.py, tests/tendermint/test_lib.py.

Should read `needed` rather than `neeeded`.
2021-12-06 14:11:29 +01:00
David Dashyan 8630e74f77 Remove @asyncio.coroutine decorators
Signed-off-by: David Dashyan <mail@davie.li>
2020-09-29 12:17:46 +00:00
David Dashyan 8895d50f5b Version bump
Signed-off-by: David Dashyan <mail@davie.li>
2020-09-29 12:17:46 +00:00
Amartya Gaur b272fb342a
owners_before error message correction (#2698)
* owners_before error message correction
Signed-off-by: Amartya Gaur amarkaushik1999@gmail.com

* Changed some formatting
Signed-off-by: Amartya Gaur amarkaushik1999@gmail.com

* Changes essential to pass tests
Signed-off-by: Amartya Gaur amarkaushik1999@gmail.com

* Resolve conflicts with master
Signed-off-by: Amartya Gaur amarkaushik1999@gmail.com
2020-08-12 12:32:33 +00:00
Karthikeyan Singaravelan e86666b6b3
Import ABC from collections.abc (#2704)
Signed-off-by: Karthikeyan Singaravelan <tir.karthi@gmail.com>
2020-07-22 08:51:58 +00:00
lana-shanghai 4a008e51e3
[WIP] Documentation re-org (#2694)
* Reorganized docs

* Fixed internal links in basic usage

* fixed the docker-compose command and volume for docs

* fixed docs tests

* fix travis docs test

* tox ini file

* fixed readme localhost links

* edited tox and test docs to previous state

* Fix tests errors related to docs reorganization

Signed-off-by: David Dashyan <mail@davie.li>

* Added ansible script installation option

Signed-off-by: Lana Ivina <lana@ipdb.io>

* Added ansible script to network setup guide

Signed-off-by: Lana Ivina <lana@ipdb.io>

* Hid the non-working button for now. 

Signed-off: Lana Ivina <lana@ipdb.io>

* Try now button

Co-authored-by: David Dashyan <mail@davie.li>
2020-05-18 20:22:26 +07:00
David Dashyan 186cd87444
Updated bigchaindb-abci (#2689)
* Updated bigchaindb-abci

Reason: gevent API change breaks BigchainDB installation
Signed-off-by: David Dashyan <mail@davie.li>

* Version bump

Signed-off-by: David Dashyan <mail@davie.li>

* Updated changelog
2020-04-14 23:59:51 +02:00
David Dashyan 9e99c024d3
Replace headers (#2683)
Signed-off-by: David Dashyan <mail@davie.li>
2020-04-06 11:52:18 +02:00
David Dashyan 09391351a4
Version bump to 2.2.0 (#2682)
Signed-off-by: David Dashyan <mail@davie.li>
2020-02-20 16:30:37 +01:00
David Dashyan 7df59994e9 Tendermint v0.22.8 backward compatibility (#2677)
Problem statement:

BigchainDB v2.0.0b9 has been around for quite a while. Recently we have updated
Tendermint supported version to v0.31.5 which has incompatible blockchain.
Despite the fact that we have defined instructions on chain migration, no one
expected to migrate to incompatible chain within patch version range. So there
is a demand for Tendermint v0.22.8 compatibility among BigchainDB users.

Work has been done:

bigchaindb-abci package was upgraded to support multiple API versions.
New configuration field stating tendermint version was added.

Signed-off-by: David Dashyan <mail@davie.li>
2020-01-23 23:17:27 +01:00
Jürgen Eckel b54c31539f
Transcation API extension: last_tx parameter (#2667)
* added last_tx switch to the TX get query to only get the latest TX for a given asset.
* extended test case to contain last_tx field
* Add last_tx option in query module
* Fix natural descending cursor sort argument
2019-11-06 23:23:14 +01:00
Jürgen Eckel 9bcefdf3e2
Tendermint upgrade (#2666)
* migrated to tendermint 0.35.1

* increased bigchaindb_abci version

* fixed API adjustments due to Tendermint upgrade

Signed-off-by: Juergen Eckel <juergen@riddleandcode.com>

* migrated to bigchaindb-abci == py-abci 0.7.1

Signed-off-by: Juergen Eckel <juergen@riddleandcode.com>
2019-10-07 16:32:35 +02:00
Nikita Chernyi 2975c372c8 Remove pysha3 for python >= 3.6 (#2652)
* Init pypy support

* Fixed setup
2019-09-25 22:22:51 +02:00
Mike Klimontov c801c833fc Add unit transaction_mode_types (#2648)
Signed-off-by: Mike Klimontov <klimontovm@gmail.com>
2019-06-12 16:28:24 +02:00
Troy McConaghy f8191b0d8e
Last commit before v2.0.0b9 (#2608) 2018-11-27 19:50:05 +01:00
Troy McConaghy dcfe23f292
Account for values that are arrays/lists (#2607)
when checking if keys are valid.
2018-11-25 20:24:03 +01:00
Troy McConaghy 5de2fef284
Problem: "language" values are not being checked in metadata (#2606)
Check "language" values in metadata too, not just in `asset.data`.
2018-11-25 19:11:35 +01:00
Troy McConaghy d85b9df615
Last PR before the release of v2.0.0b8 (#2596) 2018-11-03 18:37:27 +01:00
Troy McConaghy 20e6c6a686
Update acceptance Dockerfile to use latest Python driver (#2594)
* Update acceptance Dockerfile to use latest Python driver

* Fixed four Flake8 errors
2018-11-03 17:34:36 +01:00
Troy McConaghy 3da13eda3d
Last PR before releasing 2.0.0b7 (#2576)
* Updated changelog for 2.0.0b7

* Change 2.0.0b3 to b7 in network-setup.md

* Updated k8s yamls for 2.0.0b7 release

* Updated version.py for 2.0.0b7 release
2018-09-28 13:23:04 +02:00
Lev Berman 0c4d6ec906 Implement crash recovery for elections. (#2566)
1. Save pre-commit state in the beginning of end block. 
2. Provide an interface for custom elections to rollback on crash recovery. 
3. Simplify pre-commit management. 
4. Add crash recovery for updert-validator and chain migration elecitons.
2018-09-28 12:32:45 +02:00
Vanshdeep Singh 78dafce146 Problem: Outputs API responds with incorrect outputs (#2567).
* Problem: Outputs API doesn't respond with correct unspent/spent outputs

Solution: Fix fastquery such that embedded document is queried properly

* Problem: key order agnostic queries not implemented

Solution: get_spent queries embedded documents which respect key order. This is
not expected by the application hence the query should be altered to query any
kind of key order

* Problem: Mongo query for get_spent too complicated

Solution: Simplify query using $elemMatch

* Problem: No test for checking mixed spent outputs

Solution: Add test for `get_spending_transactions` to check that correct
matching is done when querying documents with multiple inputs

* Problem: tranasction ids not assert when getting spent outputs

Solution: assert tranasction ids
2018-09-26 10:17:39 +02:00
Lev Berman cb74cb43d2 Extend chain migration show command (#2558)
* Extend `election show` with migration data.

* Test chain-migration show.
2018-09-25 11:38:29 +02:00
Lev Berman 24ca0b32a9 Separate pending and effective validator updates. (#2556)
* Separate pending and effective validator updates.

- Pending validator updates do not prevent elections from concluding.
- ValidatorElection overrides has_conclude to not conclude when there is a pending update for the matching height.
- Effective validator updates deem past elections inconclusive.

* Problem: Looking for election block is inefficient.

Solution: Record placed elections, update the records upon election conclusion.

* Clarify the conclusion order in Election.process_blocks.

* Insert election records in bulk.

Otherwise, one can significantly slow nodes down by posting a whole bunch of unique elections.

* Change get_election to use find_one.

* Calculate total votes without making extra query.

* Fix the pending valset check.

* Fix election test setup.
2018-09-21 10:51:57 +02:00
Troy McConaghy 39be7a2fdf
Renamed "consensus" to "validation" where relevant (#2561) 2018-09-20 17:17:33 +02:00
Muawia Khan eb139fba00 Remove benchmark logging (#2565).
* Problem: Benchmark logging is not needed.

* Solution: Clean it up.
2018-09-20 14:30:30 +02:00
Lev Berman 126e90e732 Adjust multiple elections conclusion. (#2553)
- Do not conclude migration election if there is a migration in progress.
- Rewrite election tests to not use mocks and assert many different things.
- Record concluded elections in the `election` collection.
2018-09-18 12:32:16 +02:00
Vanshdeep Singh cf6fa6b553 Problem: Election type not casted for runnign command (#2551)
Solution: Replace '-' with '_' before running election sub-command
2018-09-17 15:36:38 +02:00
Troy McConaghy 528ba07c68
Last PR before 2.0.0b6 (#2550)
* Update k8s files for 2.0.0b6

* Update version.py for 2.0.0b6
2018-09-17 15:00:26 +02:00
Vanshdeep Singh 16355748dd Problem: Docs build fails (#2549)
Solution: Create __init__.py so that migrations can be imported properly
2018-09-17 14:39:00 +02:00
Troy McConaghy 8e5f770707 List the "blocks" endpoint in response to HTTP info requests (#2545)
* Problem: HTTP info endpoints don't include "blocks"

Solution: Add "blocks" to the response to the HTTP info endpoints

* Removed "blocks" endpoints from "Advanced Usage"
because they are really not that advanced
2018-09-17 14:06:48 +02:00
Zachary Bowen e6893632dc Create migration election class (#2535).
* Problem: We need a way to synchronize a halt to block production to allow for upgrades across breaking changes.

* Solution: Created `MigrationElection`.

* Problem: Need documentation for `migration` elections.

* Solution: Updated the docs.

* Problem: `MigrationElection` needs 'new' CLI method.

* Solution: Updated the definition of `election` to include the new `migration` type.

* Problem: The way `end_block` checks for concluded elections assumes there is only one type of election (so we can't conclude an `upsert-validator` and a `chain-migration` at the same height).

* Solution: Re-engineered the code in `Elections` to conclude multiple elections in the same block. If more than one election change the validator set, only one of them is applied.

* Problem: Tendermint change to store validator changes at height h+2 will break `Election.get_status`.

* Solution: Reworked `get_validator_change` to look at only the latest block height or less.
2018-09-17 13:59:57 +02:00
Muawia Khan bd39076522 Problem: Users trying to use an incompatible version of Tendermint (#2541)
-  Hard-wire the supported Tendermint version(s) right in the code of BigchainDB Server. Check the version of Tendermint and disconnect if Tendermint version is an unsupported one.
- Expose a CLI command bigchaindb tendermint-version to display the supported Tendermint versions.
- PR also takes care the long list of warnings we get when we run tests.
    - Updated deprecated pymongo methods
    - Do not call pytest fixtures directly.
    - Also added the doc for the new cli command
2018-09-14 15:37:18 +02:00
Shahbaz Nazir c68fab6c31 Problem: A duplicate tx may result in HTTP 500.
* Improve parsing of errors received from Tendermint RPC.
2018-09-13 16:03:22 +02:00
Lev Berman f55f22144f Problem: the skip_id param is redundant. (#2531)
Solution: remove the param; only call validate_id where it is needed.
2018-09-13 15:35:24 +02:00
vrde 241aae335d Problem: validation is slow (#2489)
Solution: parallelize the validation of transactions. This patch adds a
new flag for the `start` command, namely
`--experimental-parallel-validation`, that can be used to enable this
experimental feature.
2018-09-13 12:16:45 +02:00
vrde 4b54e702f8 Problem: Events API is unreliable (#2529)
Solution: Stop getting events from the Tendermint events API. Get the
event right before returning from the COMMIT phase of Tendermint, and
publish it to the events queue.
2018-09-12 16:55:23 +02:00
Lev Berman 35e35ecd57 Problem: New collections are not created in existing DB. (#2520)
Solution: Do not abort the initialisation if a collection exists. Unify the index creation.
2018-09-12 14:39:07 +02:00
Zachary Bowen 8a7650c13a Generalize election management commands (#2515)
* Problem: `ValidatorElection` and `MigrationElection` need to inherit from a common `Election` class

Solution: Factored the common logic out of `ValidatorElection` and moved it to `Election` parent class

* Problem: Adding base58 as a requirement seems to break the build...

Solution: Reverting the changes

* Problem: Responding to a suggestion for improved method naming

Solution: Refactored `get_result_by_election_id` to `get_election_result_by_id`

* Problem: No need to store different types of elections in their own tables

Solution: Remove `DB_TABLE` property from `Election` class

* Revert "Problem: No need to store different types of elections in their own tables"

This reverts commit db45374d3c.

* Problem: Missed a method in `Bigchain` class when updating the naming for an election method

Solution: Finished the refactoring

* Problem: Need a table to store data for all elections

Solution: Created the `elections` table with secondary_index `election_id`

* Problem: `Election` class needs to be updated to store concluded elections in the `elections` table

Solution: Updated the class to use the new table

* Problem: `UpsertValidatorVote` can be generalized to just be `Vote`

Solution: Renamed, refactored and moved the `Vote` class to tie in with the more general `Election` base class

* Problem: Error in docstring return signature

Solution: Fixed the docstring

* Problem: Hardcoded reference to the `VOTE_TYPE` in `Election` base class

Solution: Pointed the reference to the class variable

* Problem: Schema still refers to `ValidatorElectionVote` instead of `Vote`

Solution:  Renamed `TX_SCHEMA_VALIDATOR_ELECTION_VOTE` as `TX_SCHEMA_VOTE`

* Problem: `Election` class variable `ELECTION_TYPE` is overly specific

Solution: Renamed `ELECTION_TYPE` to `OPERATION`

* Problem: Command line options for `upsert-validator` can be generalized to manage any type of election

Solution: Refactored the CLI to manage generalized elections

* Problem: Default for `show_election` not implemented for `Election` class

Solution: Create a default method that work if all fields in the 'asset[data]' can be displayed without additional formatting

* Problem: Multiple small issues with style etc.

Solution: Addressed comments from PR

* Problem: `Election` class variable to `VOTE_TYPE` unnecessary

Solution: Removed the variable and hardcoded everything to use the `Vote` class

* Problem: Minor style issues with PR

Solution: Addressing comments

* Problem: Changes to format for validator keys broke some tests

Solution: Aligned the tests to reflect the changed key format

* Problem: `election show` command displaying the base56 public key

Solution: Cast any public key to base64

* Problem: `election_parser` help message still refers to upsert-validator

Solution: Updated the help message
2018-09-12 13:41:16 +02:00
Zachary Bowen 0fe749d830 Create abstract election class (#2498).
* Problem: `ValidatorElection` and `MigrationElection` need to inherit from a common `Election` class.

Solution: Factored the common logic out of `ValidatorElection` and moved it to `Election` parent class.

* Problem: No need to store different types of elections in their own tables

Solution: Remove `DB_TABLE` property from `Election` class.

Solution: Created the `elections` table with secondary_index `election_id`.

* Problem: `UpsertValidatorVote` can be generalized to just be `Vote`

Solution: Renamed, refactored and moved the `Vote` class to tie in with the more general `Election` base class.

* Problem: `election_id` is not unique if two elections have the same properties.

Solution: Added a random `uuid4` seed to enforce uniqueness.
2018-09-11 15:39:46 +02:00
Vanshdeep Singh cbfbfa8fc4 Problem: Public key format not standardized (#2508)
Solution: Add 'type' field to public key so that it can be decoded properly
2018-09-06 16:06:36 +02:00
Lev Berman 699494613f Problem: The commit mode behaves incorrectly. (#2510)
* Problem: The commit mode behaves incorrectly.

Solution: Parse the Tendermint response properly. The functionality was disabled in https://github.com/bigchaindb/bigchaindb/pull/2235/files#diff-c6511560546a7dc577e7e647b5bfdaceL68 and was not fixed since then.

* Add a test case for the sync mode.

* Do not strictly expect deliver_tx in the response.

* Fix post_mock in web/test_transactions.py.

* Check for the error field first.
2018-09-06 15:49:00 +02:00
Vanshdeep Singh 2656302c60 Problem: Bigchaindb crashes when restarted after init chain (#2519)
Solution: Return appropirate height so that Tendermint doesn't execute
init_chain more than once
2018-09-06 11:27:52 +02:00
Vanshdeep Singh cb22557771 Problem: Validation code not optimized (#2490)
Solution: memoize data
2018-09-04 15:30:52 +02:00
Lev Berman 230a5b2d69 ABCI chain migration conclusion (#2488)
* Problem: No good way to check for val set absence.

Solution: Make get_validator_set/get_validators return None/[] when there are no validators yet.

* Problem: Incompatible ABCI chain upgrades.

Solution: Record known chains and sync through InitChain. Triggering the migration and adjusting other ABCI endpoints will follow.
2018-09-03 15:41:49 +02:00
Lev Berman fe0a4c494b
Problem: No MongoDB support for election ID index. (#2507)
Solution: Election ID has to be unique but not every validator set record has it. MongoDB does not support partial indexes, does not even allow for multiple Nones. This is a temporary fix since we are introducing an `election` collection to store election IDs in #2498.
2018-09-03 15:09:16 +02:00
Muawia Khan 8e55b11da2 Problem: cyrptoconditions dependency updated because of vulnerability (#2494)
* Problem: cyrptoconditions dependency updated because of vulnerability CVE-2018-10903

* update cc to ~=0.7.2

* Fix test using b58encode

* Fixing some more tests failing because of base58 update
2018-09-03 13:25:58 +02:00
Shahbaz Nazir 2d1f670eec
Problem: BigchainDB has un-necessary code to initialize a replica set and check if MongoDB was started with replicaSet (#2491)
Solution: Remove un-necessary code. Deployment of MongoDB with or without replicaSet should be the responsibility of MongoDB admin which can and cannot be a BigchainDB node operator. As far as BigchainDB is concerned replicaset, if provided in bigchaindb configs, should be used to establish connection with MongoDB.
2018-08-31 13:54:32 +02:00
Zachary Bowen cfc2c5900b Rework `upsert-validator show status` (#2496)
* Problem: We need to store the `election_id` as part of the `validator_update` so we can efficiently check which election was resposible for the change

Solution: Added the parameter to `store_validator_set` and aligned the tests

* Problem: Logic for `upsert-validator show` is convoluted

Solution: Rewrote the function to be much simpler

* Problem: Need a uniqueness constraint for election_id wrt validator changes

Solution: Added a new key to the db schema
2018-08-31 09:47:47 +02:00
Zachary Bowen 7a0b474d11 Problem: Users want to know upsert-validator election status.
Solution: Introduce the `upsert-validator show` command. Soon to be re-implemented via storing and querying identifiers of concluded elections.
2018-08-30 10:47:37 +02:00