diff --git a/content/concepts/components.md b/content/concepts/components.md index 363880e5..70ae2d43 100644 --- a/content/concepts/components.md +++ b/content/concepts/components.md @@ -7,7 +7,7 @@ Before reading this page, you should understand some [Ocean-specific terminology ## Aquarius -Marketplaces run Aquarius to store and manage metadata about the data assets available in their marketplace. It provides an HTTP API for interacting with an off-chain database ([OceanDB](/concepts/components/#oceandb-drivers)). +Marketplaces run Aquarius to store and manage metadata about the [assets](/concepts/terminology/#asset-or-data-asset) available in their marketplace. It provides an HTTP API for interacting with an off-chain database ([OceanDB](/concepts/components/#oceandb-drivers)). @@ -15,7 +15,7 @@ Marketplaces run Aquarius to store and manage metadata about the data assets ava Publishers run Brizo to manage interactions with marketplaces and consumers. It interacts with the publisher's cloud and/or on-premise infrastructure. -The most basic scenario for a publisher is to provide access to the assets the publisher owns or manages, but [Brizo can do much more](/concepts/architecture/#brizo). +The most basic scenario for a publisher is to provide access to the [assets](/concepts/terminology/#asset-or-data-asset) the publisher owns or manages, but [Brizo can do much more](/concepts/architecture/#brizo). @@ -33,7 +33,7 @@ See also: [Run a Keeper](/setup/keeper/) ## Pleuston -An example marketplace front-end for consumers to explore, download, and publish data assets within the Ocean Protocol network. Implemented using [React](https://reactjs.org/) and [🦑 squid-js](https://github.com/oceanprotocol/squid-js). +An example marketplace front-end for consumers to explore, download, and publish [assets](/concepts/terminology/#asset-or-data-asset) within the Ocean Protocol network. Implemented using [React](https://reactjs.org/) and [🦑 squid-js](https://github.com/oceanprotocol/squid-js). @@ -49,7 +49,7 @@ Client libraries used by applications (such as Pleuston or Jupyter notebooks) to ## OceanDB Drivers -OceanDB is an off-chain database for storing metadata about data assets. +OceanDB is an off-chain database for storing metadata about [assets](/concepts/terminology/#asset-or-data-asset). @@ -61,7 +61,7 @@ These drivers are extended from a high-level module implementing OceanDB functio ## Secret Store -A [Parity Secret Store](https://wiki.parity.io/Secret-Store): software for distributed key pair generation, distributed key storage, and threshold retrieval. It's used to store asset access-control keys. +A [Parity Secret Store](https://wiki.parity.io/Secret-Store): software for distributed key pair generation, distributed key storage, and threshold retrieval. It's used to store [asset](/concepts/terminology/#asset-or-data-asset) access-control keys. There are several clients for integrating the Parity Secret Store into Ocean: diff --git a/content/concepts/terminology.md b/content/concepts/terminology.md index 44792e0a..451f365a 100644 --- a/content/concepts/terminology.md +++ b/content/concepts/terminology.md @@ -5,7 +5,7 @@ description: Terminology specific to Ocean Protocol. ## Asset or Data Asset -A data set or data service. +Anything that can be registered with and made available via the Ocean Network. Examples include data sets, trained model parameters, pipelines, and data-cleaning services. ## Data Owner or Data Service Provider @@ -17,7 +17,7 @@ Note: Initially, most data owners or data service providers will also be the pub A service which mediates access to assets on behalf of data owners or data service providers. -Note: Initially, most publishers will also be the owners of the data assets they publish. +Note: Initially, most publishers will also be the owners of the assets they publish. ## Consumer @@ -33,7 +33,7 @@ A person or a software service that checks some steps in transactions. For examp ## Service Execution Agreement (SEA) -A contract-like agreement between a provider, a consumer, and a verifier, specifying what data assets are to be delivered (from provider to consumer), the conditions that must be met, and the rewards for fulfilling the conditions. +A contract-like agreement between a provider, a consumer, and a verifier, specifying what assets are to be delivered (from provider to consumer), the conditions that must be met, and the rewards for fulfilling the conditions. We published an [Ocean Protocol blog post that explains SEAs in more detail](https://blog.oceanprotocol.com/exploring-the-sea-service-execution-agreements-65f7523d85e2). diff --git a/content/concepts/tools.md b/content/concepts/tools.md index a63b0264..31b9c348 100644 --- a/content/concepts/tools.md +++ b/content/concepts/tools.md @@ -5,11 +5,11 @@ description: Tools and examples for developing with Ocean Protocol. ## Tools -[Plecos](https://github.com/oceanprotocol/plecos) is a Python tool to check metadata (a JSON file) to see if it conforms to the [OEP8 schema](https://github.com/oceanprotocol/OEPs/tree/master/8). Plecos wraps the [jsonschema](https://github.com/Julian/jsonschema) validator. +[Plecos](https://github.com/oceanprotocol/plecos) is a Python tool to check metadata (a JSON file) to see if it conforms to the [OEP8 schema](https://github.com/oceanprotocol/OEPs/tree/master/8). Plecos wraps the [jsonschema](https://github.com/Julian/jsonschema) validator. -Users can use Plecos to check their metadata before sending it to an Aquarius instance. Plecos is manually updated with a schema file according to [OEP8](https://github.com/oceanprotocol/OEPs/tree/master/8). Find the tool in [PyPI](https://pypi.org/project/plecos/). +Users can use Plecos to check their metadata before sending it to an Aquarius instance. Plecos is manually updated with a schema file according to [OEP8](https://github.com/oceanprotocol/OEPs/tree/master/8). Find the tool in [PyPI](https://pypi.org/project/plecos/). -Plecos can be used in a microservice to facilitate data onboarding, as described in the [plecos_service](https://github.com/oceanprotocol/plecos_service) repository. +Plecos can be used in a microservice to facilitate data onboarding, as described in the [plecos_service](https://github.com/oceanprotocol/plecos_service) repository. diff --git a/content/setup/marketplace.md b/content/setup/marketplace.md index b04b4304..d7681c52 100644 --- a/content/setup/marketplace.md +++ b/content/setup/marketplace.md @@ -5,22 +5,28 @@ description: Set up and run a data marketplace in the Ocean network. ## What Does it Mean to Set Up a Marketplace? -An Ocean marketplace app is one of the primary ways that end users use the Ocean network. For example, a data scientist could use a marketplace app to see what data sets and data services (data assets) a marketplace has available. They can use the marketplace app to buy access to assets. Publishers make those data assets available. +An Ocean marketplace app is one of the primary ways that end users use the Ocean network. For example, a data scientist could use a marketplace app to see what [assets](/concepts/terminology/#asset-or-data-asset) a marketplace has available. They can use the marketplace app to buy access to assets. Publishers make those assets available. **Note: In the early days of the Ocean network, there won't be many marketplaces or publishers, so marketplaces will often also act as publishers.** If you want to set up and run a marketplace in the Ocean network, then at a technical level, you must: -1. Have [Data Assets](/concepts/terminology/#asset-or-data-asset) to offer in your marketplace. -1. Get those data assets set up to work with Ocean Protocol. +1. Have [assets](/concepts/terminology/#asset-or-data-asset) to offer in your marketplace. +1. Get those assets set up to work with Ocean Protocol. 1. Develop a marketplace application (app). 1. Run your marketplace app in production. -## Prepare Data Assets +## Prepare Assets -At the time of writing, the only kind of [data assets](/concepts/terminology/#asset-or-data-asset) supported by Ocean Protocol were datasets stored in Azure Storage. See [the tutorial about setting up Azure Storage to work with Ocean Protocol](/tutorials/azure-for-brizo/). +At the time of writing, the following kinds of [assets](/concepts/terminology/#asset-or-data-asset) were supported: -Support for other kinds of data assets (e.g. storage in AWS, computing in Azure) is coming soon. +- data sets stored in Azure Storage (i.e. with "core.windows.net" in their URL). See [the tutorial about setting up Azure Storage to work with Ocean Protocol](/tutorials/azure-for-brizo/). +- data sets stored in Amazon S3 storage (i.e. with "s3://" in their URL). See [the tutorial about setting up Amazon S3 storage to work with Ocean Protocol](/tutorials/amazon-s3-for-brizo/). +- data sets stored in on-premise storage. See [the tutorial about setting up on-premise storage to work with Ocean Protocol](/tutorials/on-premise-for-brizo/). + +Note: You can use _all_ of the above. You aren't restricted to using only one storage provider. + +Support for other kinds of assets (e.g. computing in Azure) is coming. ## Develop a Marketplace App diff --git a/content/setup/quickstart.md b/content/setup/quickstart.md index bbec90cc..20f66f1b 100644 --- a/content/setup/quickstart.md +++ b/content/setup/quickstart.md @@ -11,7 +11,7 @@ You can [try some free, online Jupyter notebooks](/tutorials/jupyter-notebooks/) You can run and try every [Ocean software component](/concepts/components/) in your local machine, all at once, using Docker Compose. -First, you must [set up some storage on Azure](/tutorials/azure-for-brizo/). (Yes, we know that's not quick. Some day there will be a local storage option and then this really will be quick.) +First, you must [set up some storage on Azure](/tutorials/azure-for-brizo/). (Yes, we know that's not quick. We're working on making a quicker option.) Then: diff --git a/content/tutorials/amazon-s3-for-brizo.md b/content/tutorials/amazon-s3-for-brizo.md new file mode 100644 index 00000000..3cb3b322 --- /dev/null +++ b/content/tutorials/amazon-s3-for-brizo.md @@ -0,0 +1,12 @@ +--- +title: Set Up Amazon S3 Storage +description: Tutorial about how to set up Amazon S3 storage for use with Ocean Protocol. +--- + +To enable Brizo to use files stored in Amazon S3 (i.e. files with an URL containing "s3://"), you must: + +1. have an Amazon AWS user account (IAM account) with permission to read those files from S3, and +1. set the AWS credentials on the machine where Brizo is running to those of the AWS user in question. Instructions are given below. +1. Note that you don't have to set any Brizo-specific configuration settings, e.g. in the `[osmosis]` section of the Brizo config file or in some special Brizo environment variables. + +Under the hood, Brizo uses [boto3](https://aws.amazon.com/sdk-for-python/) (the Python library for interacting with AWS) to interact with AWS and boto3 has a whole process for determining AWS credentials. The easiest way to set the AWS credentials on the machine where Brizo is running is to install the [AWS CLI](https://aws.amazon.com/cli/) and then use the `aws configure` command. For more details, see [the boto3 user guide about credentials](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html). diff --git a/content/tutorials/on-premise-for-brizo.md b/content/tutorials/on-premise-for-brizo.md new file mode 100644 index 00000000..1c06abb2 --- /dev/null +++ b/content/tutorials/on-premise-for-brizo.md @@ -0,0 +1,8 @@ +--- +title: Set Up On-Premise Storage +description: Tutorial about how to set up on-premise storage for use with Ocean Protocol. +--- + +To enable Brizo to use files stored in on-premise storage (i.e. files with an URL not containing "core.windows.net" or "s3://"), there is _nothing to do, other than make sure Brizo can resolve the URLs_. In particular, you don't have to set any Brizo-specific configuration settings, e.g. in the `[osmosis]` section of the Brizo config file or in some special Brizo environment variables. + +Local and private network URLs are fine so long as they can be resolved by Brizo. Potential examples include `http://localhost/helicopter_data.xls`, `http://192.168.12.34/almond_sales_2012.csv` and `http://10.12.34.56/duck_photos.zip`. diff --git a/content/tutorials/react-publish-data-set.md b/content/tutorials/react-publish-data-set.md index fa82f4b1..c4e52b70 100644 --- a/content/tutorials/react-publish-data-set.md +++ b/content/tutorials/react-publish-data-set.md @@ -11,7 +11,7 @@ Open `src/App.js` in your marketplace app from the [React App Setup](/tutorials/ ## Define Asset -First, let's add the asset that we want to publish. +First, let's add the [asset](/concepts/terminology/#asset-or-data-asset) that we want to publish. To do that, we need to add the following code after `window.ethereum.enable()` line, defining our asset based on the [OEP-08](https://github.com/oceanprotocol/OEPs/tree/master/8) metadata structure: diff --git a/data/sidebars/tutorials.yml b/data/sidebars/tutorials.yml index f801c6db..70fd5e49 100644 --- a/data/sidebars/tutorials.yml +++ b/data/sidebars/tutorials.yml @@ -8,6 +8,10 @@ link: /tutorials/get-ether-and-ocean-tokens/ - title: Set Up Azure Storage link: /tutorials/azure-for-brizo/ + - title: Set Up Amazon S3 Storage + link: /tutorials/amazon-s3-for-brizo/ + - title: Set Up On-Premise Storage + link: /tutorials/on-premise-for-brizo/ - group: React App Tutorial (squid-js) items: diff --git a/external/dev-ocean b/external/dev-ocean index fb6e9629..f7f8e10a 160000 --- a/external/dev-ocean +++ b/external/dev-ocean @@ -1 +1 @@ -Subproject commit fb6e9629ff60fc71baa9601ab449983352e8372e +Subproject commit f7f8e10a5babd85fca48d2df40f6165cfb68fdff