diff --git a/content/concepts/tools.md b/content/concepts/tools.md index a63b0264..31b9c348 100644 --- a/content/concepts/tools.md +++ b/content/concepts/tools.md @@ -5,11 +5,11 @@ description: Tools and examples for developing with Ocean Protocol. ## Tools -[Plecos](https://github.com/oceanprotocol/plecos) is a Python tool to check metadata (a JSON file) to see if it conforms to the [OEP8 schema](https://github.com/oceanprotocol/OEPs/tree/master/8). Plecos wraps the [jsonschema](https://github.com/Julian/jsonschema) validator. +[Plecos](https://github.com/oceanprotocol/plecos) is a Python tool to check metadata (a JSON file) to see if it conforms to the [OEP8 schema](https://github.com/oceanprotocol/OEPs/tree/master/8). Plecos wraps the [jsonschema](https://github.com/Julian/jsonschema) validator. -Users can use Plecos to check their metadata before sending it to an Aquarius instance. Plecos is manually updated with a schema file according to [OEP8](https://github.com/oceanprotocol/OEPs/tree/master/8). Find the tool in [PyPI](https://pypi.org/project/plecos/). +Users can use Plecos to check their metadata before sending it to an Aquarius instance. Plecos is manually updated with a schema file according to [OEP8](https://github.com/oceanprotocol/OEPs/tree/master/8). Find the tool in [PyPI](https://pypi.org/project/plecos/). -Plecos can be used in a microservice to facilitate data onboarding, as described in the [plecos_service](https://github.com/oceanprotocol/plecos_service) repository. +Plecos can be used in a microservice to facilitate data onboarding, as described in the [plecos_service](https://github.com/oceanprotocol/plecos_service) repository. diff --git a/content/setup/marketplace.md b/content/setup/marketplace.md index b04b4304..62ad16bc 100644 --- a/content/setup/marketplace.md +++ b/content/setup/marketplace.md @@ -18,9 +18,15 @@ If you want to set up and run a marketplace in the Ocean network, then at a tech ## Prepare Data Assets -At the time of writing, the only kind of [data assets](/concepts/terminology/#asset-or-data-asset) supported by Ocean Protocol were datasets stored in Azure Storage. See [the tutorial about setting up Azure Storage to work with Ocean Protocol](/tutorials/azure-for-brizo/). +At the time of writing, the following kinds of [data assets](/concepts/terminology/#asset-or-data-asset) were supported: -Support for other kinds of data assets (e.g. storage in AWS, computing in Azure) is coming soon. +- data sets stored in Azure Storage (i.e. with "core.windows.net" in their URL). See [the tutorial about setting up Azure Storage to work with Ocean Protocol](/tutorials/azure-for-brizo/). +- data sets stored in Amazon S3 storage (i.e. with "s3://" in their URL). See [the tutorial about setting up Amazon S3 storage to work with Ocean Protocol](/tutorials/amazon-s3-for-brizo/). +- data sets stored in on-premise storage. See [the tutorial about setting up on-premise storage to work with Ocean Protocol](/tutorials/on-premise-for-brizo/). + +Note: You can use _all_ of the above. You aren't restricted to using only one storage provider. + +Support for other kinds of data assets (e.g. computing in Azure) is coming. ## Develop a Marketplace App diff --git a/content/setup/quickstart.md b/content/setup/quickstart.md index bbec90cc..20f66f1b 100644 --- a/content/setup/quickstart.md +++ b/content/setup/quickstart.md @@ -11,7 +11,7 @@ You can [try some free, online Jupyter notebooks](/tutorials/jupyter-notebooks/) You can run and try every [Ocean software component](/concepts/components/) in your local machine, all at once, using Docker Compose. -First, you must [set up some storage on Azure](/tutorials/azure-for-brizo/). (Yes, we know that's not quick. Some day there will be a local storage option and then this really will be quick.) +First, you must [set up some storage on Azure](/tutorials/azure-for-brizo/). (Yes, we know that's not quick. We're working on making a quicker option.) Then: diff --git a/content/tutorials/amazon-s3-for-brizo.md b/content/tutorials/amazon-s3-for-brizo.md new file mode 100644 index 00000000..3cb3b322 --- /dev/null +++ b/content/tutorials/amazon-s3-for-brizo.md @@ -0,0 +1,12 @@ +--- +title: Set Up Amazon S3 Storage +description: Tutorial about how to set up Amazon S3 storage for use with Ocean Protocol. +--- + +To enable Brizo to use files stored in Amazon S3 (i.e. files with an URL containing "s3://"), you must: + +1. have an Amazon AWS user account (IAM account) with permission to read those files from S3, and +1. set the AWS credentials on the machine where Brizo is running to those of the AWS user in question. Instructions are given below. +1. Note that you don't have to set any Brizo-specific configuration settings, e.g. in the `[osmosis]` section of the Brizo config file or in some special Brizo environment variables. + +Under the hood, Brizo uses [boto3](https://aws.amazon.com/sdk-for-python/) (the Python library for interacting with AWS) to interact with AWS and boto3 has a whole process for determining AWS credentials. The easiest way to set the AWS credentials on the machine where Brizo is running is to install the [AWS CLI](https://aws.amazon.com/cli/) and then use the `aws configure` command. For more details, see [the boto3 user guide about credentials](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html). diff --git a/content/tutorials/on-premise-for-brizo.md b/content/tutorials/on-premise-for-brizo.md new file mode 100644 index 00000000..1c06abb2 --- /dev/null +++ b/content/tutorials/on-premise-for-brizo.md @@ -0,0 +1,8 @@ +--- +title: Set Up On-Premise Storage +description: Tutorial about how to set up on-premise storage for use with Ocean Protocol. +--- + +To enable Brizo to use files stored in on-premise storage (i.e. files with an URL not containing "core.windows.net" or "s3://"), there is _nothing to do, other than make sure Brizo can resolve the URLs_. In particular, you don't have to set any Brizo-specific configuration settings, e.g. in the `[osmosis]` section of the Brizo config file or in some special Brizo environment variables. + +Local and private network URLs are fine so long as they can be resolved by Brizo. Potential examples include `http://localhost/helicopter_data.xls`, `http://192.168.12.34/almond_sales_2012.csv` and `http://10.12.34.56/duck_photos.zip`. diff --git a/data/sidebars/tutorials.yml b/data/sidebars/tutorials.yml index dfafeb9d..a945173b 100644 --- a/data/sidebars/tutorials.yml +++ b/data/sidebars/tutorials.yml @@ -8,6 +8,10 @@ link: /tutorials/get-ether-and-ocean-tokens/ - title: Set Up Azure Storage link: /tutorials/azure-for-brizo/ + - title: Set Up Amazon S3 Storage + link: /tutorials/amazon-s3-for-brizo/ + - title: Set Up On-Premise Storage + link: /tutorials/on-premise-for-brizo/ - group: React App Tutorial (squid-js) items: