Merge pull request #152 from oceanprotocol/feature/now-support-many-storage-options

Revise & add docs now that we support three storage options

content/concepts/components.md

@@ -7,7 +7,7 @@ Before reading this page, you should understand some [Ocean-specific terminology
 
 ## Aquarius
 
-Marketplaces run Aquarius to store and manage metadata about the data assets available in their marketplace. It provides an HTTP API for interacting with an off-chain database ([OceanDB](/concepts/components/#oceandb-drivers)).
+Marketplaces run Aquarius to store and manage metadata about the [assets](/concepts/terminology/#asset-or-data-asset) available in their marketplace. It provides an HTTP API for interacting with an off-chain database ([OceanDB](/concepts/components/#oceandb-drivers)).
 
 <repo name="aquarius"></repo>
 
@@ -15,7 +15,7 @@ Marketplaces run Aquarius to store and manage metadata about the data assets ava
 
 Publishers run Brizo to manage interactions with marketplaces and consumers.
 It interacts with the publisher's cloud and/or on-premise infrastructure.
-The most basic scenario for a publisher is to provide access to the assets the publisher owns or manages, but [Brizo can do much more](/concepts/architecture/#brizo).
+The most basic scenario for a publisher is to provide access to the [assets](/concepts/terminology/#asset-or-data-asset) the publisher owns or manages, but [Brizo can do much more](/concepts/architecture/#brizo).
 
 <repo name="brizo"></repo>
 
@@ -33,7 +33,7 @@ See also: [Run a Keeper](/setup/keeper/)
 
 ## Pleuston
 
-An example marketplace front-end for consumers to explore, download, and publish data assets within the Ocean Protocol network. Implemented using [React](https://reactjs.org/) and [🦑 squid-js](https://github.com/oceanprotocol/squid-js).
+An example marketplace front-end for consumers to explore, download, and publish [assets](/concepts/terminology/#asset-or-data-asset) within the Ocean Protocol network. Implemented using [React](https://reactjs.org/) and [🦑 squid-js](https://github.com/oceanprotocol/squid-js).
 
 <repo name="pleuston"></repo>
 
@@ -49,7 +49,7 @@ Client libraries used by applications (such as Pleuston or Jupyter notebooks) to
 
 ## OceanDB Drivers
 
-OceanDB is an off-chain database for storing metadata about data assets.
+OceanDB is an off-chain database for storing metadata about [assets](/concepts/terminology/#asset-or-data-asset).
 
 <repo name="oceandb-mongodb-driver"></repo>
 <repo name="oceandb-bigchaindb-driver"></repo>
@@ -61,7 +61,7 @@ These drivers are extended from a high-level module implementing OceanDB functio
 
 ## Secret Store
 
-A [Parity Secret Store](https://wiki.parity.io/Secret-Store): software for distributed key pair generation, distributed key storage, and threshold retrieval. It's used to store asset access-control keys.
+A [Parity Secret Store](https://wiki.parity.io/Secret-Store): software for distributed key pair generation, distributed key storage, and threshold retrieval. It's used to store [asset](/concepts/terminology/#asset-or-data-asset) access-control keys.
 
 There are several clients for integrating the Parity Secret Store into Ocean:
 

content/concepts/terminology.md

@@ -5,7 +5,7 @@ description: Terminology specific to Ocean Protocol.
 
 ## Asset or Data Asset
 
-A data set or data service.
+Anything that can be registered with and made available via the Ocean Network. Examples include data sets, trained model parameters, pipelines, and data-cleaning services.
 
 ## Data Owner or Data Service Provider
 
@@ -17,7 +17,7 @@ Note: Initially, most data owners or data service providers will also be the pub
 
 A service which mediates access to assets on behalf of data owners or data service providers.
 
-Note: Initially, most publishers will also be the owners of the data assets they publish.
+Note: Initially, most publishers will also be the owners of the assets they publish.
 
 ## Consumer
 
@@ -33,7 +33,7 @@ A person or a software service that checks some steps in transactions. For examp
 
 ## Service Execution Agreement (SEA)
 
-A contract-like agreement between a provider, a consumer, and a verifier, specifying what data assets are to be delivered (from provider to consumer), the conditions that must be met, and the rewards for fulfilling the conditions.
+A contract-like agreement between a provider, a consumer, and a verifier, specifying what assets are to be delivered (from provider to consumer), the conditions that must be met, and the rewards for fulfilling the conditions.
 
 We published an [Ocean Protocol blog post that explains SEAs in more detail](https://blog.oceanprotocol.com/exploring-the-sea-service-execution-agreements-65f7523d85e2).
 

content/concepts/tools.md

@@ -5,11 +5,11 @@ description: Tools and examples for developing with Ocean Protocol.
 
 ## Tools
 
-[Plecos](https://github.com/oceanprotocol/plecos) is a Python tool to check metadata (a JSON file) to see if it conforms to the [OEP8 schema](https://github.com/oceanprotocol/OEPs/tree/master/8). Plecos wraps the [jsonschema](https://github.com/Julian/jsonschema) validator. 
+[Plecos](https://github.com/oceanprotocol/plecos) is a Python tool to check metadata (a JSON file) to see if it conforms to the [OEP8 schema](https://github.com/oceanprotocol/OEPs/tree/master/8). Plecos wraps the [jsonschema](https://github.com/Julian/jsonschema) validator.
 
-Users can use Plecos to check their metadata before sending it to an Aquarius instance. Plecos is manually updated with a schema file according to [OEP8](https://github.com/oceanprotocol/OEPs/tree/master/8). Find the tool in [PyPI](https://pypi.org/project/plecos/). 
+Users can use Plecos to check their metadata before sending it to an Aquarius instance. Plecos is manually updated with a schema file according to [OEP8](https://github.com/oceanprotocol/OEPs/tree/master/8). Find the tool in [PyPI](https://pypi.org/project/plecos/).
 
-Plecos can be used in a microservice to facilitate data onboarding, as described in the [plecos_service](https://github.com/oceanprotocol/plecos_service) repository. 
+Plecos can be used in a microservice to facilitate data onboarding, as described in the [plecos_service](https://github.com/oceanprotocol/plecos_service) repository.
 
 <repo name="Plecos"></repo>
 

content/setup/marketplace.md

@@ -5,22 +5,28 @@ description: Set up and run a data marketplace in the Ocean network.
 
 ## What Does it Mean to Set Up a Marketplace?
 
-An Ocean marketplace app is one of the primary ways that end users use the Ocean network. For example, a data scientist could use a marketplace app to see what data sets and data services (data assets) a marketplace has available. They can use the marketplace app to buy access to assets. Publishers make those data assets available.
+An Ocean marketplace app is one of the primary ways that end users use the Ocean network. For example, a data scientist could use a marketplace app to see what [assets](/concepts/terminology/#asset-or-data-asset) a marketplace has available. They can use the marketplace app to buy access to assets. Publishers make those assets available.
 
 **Note: In the early days of the Ocean network, there won't be many marketplaces or publishers, so marketplaces will often also act as publishers.**
 
 If you want to set up and run a marketplace in the Ocean network, then at a technical level, you must:
 
-1. Have [Data Assets](/concepts/terminology/#asset-or-data-asset) to offer in your marketplace.
+1. Have [assets](/concepts/terminology/#asset-or-data-asset) to offer in your marketplace.
-1. Get those data assets set up to work with Ocean Protocol.
+1. Get those assets set up to work with Ocean Protocol.
 1. Develop a marketplace application (app).
 1. Run your marketplace app in production.
 
-## Prepare Data Assets
+## Prepare Assets
 
-At the time of writing, the only kind of [data assets](/concepts/terminology/#asset-or-data-asset) supported by Ocean Protocol were datasets stored in Azure Storage. See [the tutorial about setting up Azure Storage to work with Ocean Protocol](/tutorials/azure-for-brizo/).
+At the time of writing, the following kinds of [assets](/concepts/terminology/#asset-or-data-asset) were supported:
 
-Support for other kinds of data assets (e.g. storage in AWS, computing in Azure) is coming soon.
+- data sets stored in Azure Storage (i.e. with "core.windows.net" in their URL). See [the tutorial about setting up Azure Storage to work with Ocean Protocol](/tutorials/azure-for-brizo/).
+- data sets stored in Amazon S3 storage (i.e. with "s3://" in their URL). See [the tutorial about setting up Amazon S3 storage to work with Ocean Protocol](/tutorials/amazon-s3-for-brizo/).
+- data sets stored in on-premise storage. See [the tutorial about setting up on-premise storage to work with Ocean Protocol](/tutorials/on-premise-for-brizo/).
+
+Note: You can use _all_ of the above. You aren't restricted to using only one storage provider.
+
+Support for other kinds of assets (e.g. computing in Azure) is coming.
 
 ## Develop a Marketplace App
 

content/setup/quickstart.md

@@ -11,7 +11,7 @@ You can [try some free, online Jupyter notebooks](/tutorials/jupyter-notebooks/)
 
 You can run and try every [Ocean software component](/concepts/components/) in your local machine, all at once, using Docker Compose.
 
-First, you must [set up some storage on Azure](/tutorials/azure-for-brizo/). (Yes, we know that's not quick. Some day there will be a local storage option and then this really will be quick.)
+First, you must [set up some storage on Azure](/tutorials/azure-for-brizo/). (Yes, we know that's not quick. We're working on making a quicker option.)
 
 Then:
 

content/tutorials/amazon-s3-for-brizo.md

@@ -0,0 +1,12 @@
+---
+title: Set Up Amazon S3 Storage
+description: Tutorial about how to set up Amazon S3 storage for use with Ocean Protocol.
+---
+
+To enable Brizo to use files stored in Amazon S3 (i.e. files with an URL containing "s3://"), you must:
+
+1. have an Amazon AWS user account (IAM account) with permission to read those files from S3, and
+1. set the AWS credentials on the machine where Brizo is running to those of the AWS user in question. Instructions are given below.
+1. Note that you don't have to set any Brizo-specific configuration settings, e.g. in the `[osmosis]` section of the Brizo config file or in some special Brizo environment variables.
+
+Under the hood, Brizo uses [boto3](https://aws.amazon.com/sdk-for-python/) (the Python library for interacting with AWS) to interact with AWS and boto3 has a whole process for determining AWS credentials. The easiest way to set the AWS credentials on the machine where Brizo is running is to install the [AWS CLI](https://aws.amazon.com/cli/) and then use the `aws configure` command. For more details, see [the boto3 user guide about credentials](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html).

content/tutorials/on-premise-for-brizo.md

@@ -0,0 +1,8 @@
+---
+title: Set Up On-Premise Storage
+description: Tutorial about how to set up on-premise storage for use with Ocean Protocol.
+---
+
+To enable Brizo to use files stored in on-premise storage (i.e. files with an URL not containing "core.windows.net" or "s3://"), there is _nothing to do, other than make sure Brizo can resolve the URLs_. In particular, you don't have to set any Brizo-specific configuration settings, e.g. in the `[osmosis]` section of the Brizo config file or in some special Brizo environment variables.
+
+Local and private network URLs are fine so long as they can be resolved by Brizo. Potential examples include `http://localhost/helicopter_data.xls`, `http://192.168.12.34/almond_sales_2012.csv` and `http://10.12.34.56/duck_photos.zip`.

content/tutorials/react-publish-data-set.md

@@ -11,7 +11,7 @@ Open `src/App.js` in your marketplace app from the [React App Setup](/tutorials/
 
 ## Define Asset
 
-First, let's add the asset that we want to publish.
+First, let's add the [asset](/concepts/terminology/#asset-or-data-asset) that we want to publish.
 
 To do that, we need to add the following code after `window.ethereum.enable()` line, defining our asset based on the [OEP-08](https://github.com/oceanprotocol/OEPs/tree/master/8) metadata structure:
 

data/sidebars/tutorials.yml

@@ -8,6 +8,10 @@
       link: /tutorials/get-ether-and-ocean-tokens/
     - title: Set Up Azure Storage
       link: /tutorials/azure-for-brizo/
+    - title: Set Up Amazon S3 Storage
+      link: /tutorials/amazon-s3-for-brizo/
+    - title: Set Up On-Premise Storage
+      link: /tutorials/on-premise-for-brizo/
 
 - group: React App Tutorial (squid-js)
   items:

external/dev-ocean

@@ -1 +1 @@
-Subproject commit fb6e9629ff60fc71baa9601ab449983352e8372e
+Subproject commit f7f8e10a5babd85fca48d2df40f6165cfb68fdff