9.1 KiB
| title | description |
|---|---|
| Set Up Azure Storage | Tutorial about how to set up Azure storage for use with Ocean. |
Note: This needs updating for Ocean V3.
This tutorial is for publishers who want to get started using Azure to store some of their data assets. (Some data assets could also be stored in other places.)
Publishers must run Provider to mediate consumer access to data assets stored in Azure Storage. Provider needs the following Azure credentials from the publisher:
AZURE_ACCOUNT_NAME: Azure Storage Account Name (for storing files)AZURE_ACCOUNT_KEY: Azure Storage Account keyAZURE_RESOURCE_GROUP: Azure resource groupAZURE_LOCATION: Azure RegionAZURE_CLIENT_ID: Azure Application IDAZURE_CLIENT_SECRET: Azure Application SecretAZURE_TENANT_ID: Azure Tenant IDAZURE_SUBSCRIPTION_ID: Azure Subscription ID
If you go through this tutorial, then you will get all the Azure credentials listed above.
If you already have data assets stored in Azure, then you might already have, or be able to get, the above information. You could use this tutorial to get a sense of where to look (but don't create anything new).
To give the above Azure credentials to Provider, you either put them in a Provider config file or in environment variables with the above names. Environment variables should be used if you're running Provider inside a container. If you want to use the config file option, see Provider README.
If you're using Barge to run Provider and other Ocean Protocol components, then the above Azure credentials should go in the file barge/provider.env. (That file gets used to set environment variables.)
This tutorial uses the Microsoft Azure Portal, but there are many other ways to interact with Azure.
Note: Azure is constantly changing. For that reason, we give try to give links to official Azure documentation, since it should stay up-to-date.
Sign in to Azure Portal
If you don't already have an Azure account, then you will have to create one. Go to the Microsoft Azure website and follow the links.
Once you have an Azure account, go to https://portal.azure.com/ and sign in.
Get Your Subscription ID
The Azure docs say, "A subscription is a logical grouping of Azure services that is linked to an Azure account. A single Azure account can contain multiple subscriptions."
If you see Subscriptions in the left sidebar of Azure Portal, then click that. If you don't see it, just type "Subscriptinos" into the search bar at the top, then click on Subscriptions under the SERVICES heading.
You should see a list of one or more subscriptions. Click on the one you want to use for Azure storage. Remember to use that one for the rest of this tutorial (whenever you are asked for a subscription name).
Copy the Subscription ID. That's what Provider calls AZURE_SUBSCRIPTION_ID. You now have one of the Azure credentials!
# Example AZURE_SUBSCRIPTION_ID (Azure Subscription ID)
479284be-0104-421a-8488-1aeac0caecaa
Create an Azure Active Directory (AD) Application
See the Azure docs page:
The first step there is to Create an Azure Active Directory application. Do that.
The app Name and Sign-on URL can be totally made up. The URL doesn't need to be real.
Once the app is created, copy the Application ID: that's what Provider calls the AZURE_CLIENT_ID. It should look something like this:
# Example AZURE_CLIENT_ID (Application ID)
5d25ee8a-da2c-4e6f-8fba-09b6dd091038
Get Authentication Key for Your AD Application
On the same Azure docs page, find the section titled Get application ID and authentication key or similar. You already have your application ID, but you still need generate an authentication key by following the instructions in that section.
You can make up whatever you like for the key's Description.
Once the application key is generated, copy its value: that's what Provider calls the AZURE_CLIENT_SECRET. It should look something like this:
# Example AZURE_CLIENT_SECRET (Application key)
RVJ1H5gYOmnMitikmM5ehszqmgrY5BFkoalnjfWMuDM
Get Tenant ID
On the same Azure docs page, find the section titled Get tenant ID or similar. Follow the instructions.
The tenant ID is what Provider calls AZURE_TENANT_ID.
# Example AZURE_TENANT_ID (tenant ID, Directory ID)
2a4a3887-4e2e-4a31-8006-6e2b5877640e
Create a Resource Group for Your Data Storage
See the Azure docs page:
Manage Azure resources through portal
That page says how to create a new empty resource group. Do that. You can make up whatever name you like, but it's good practice to avoid special characters and to include:
- some words to indicate what it's for, e.g.
Storage - your name
- the month and year it was created, e.g.
Nov2018
to help you and others manage it. The Resource group name is what Provider calls the AZURE_RESOURCE_GROUP and the Resource group location is what Provider calls the AZURE_LOCATION. Here are examples of both:
# Example AZURE_RESOURCE_GROUP (Resource group name)
StorageCreatedNov2018ByTroy
# Example AZURE_LOCATION (Resource group location)
West Europe
Give Your AD Application Access to Your Resource Group
Inside your new resource group:
- click Access control (IAM)
- click + Add role assignment
- In the
Rolefield, selectContributor. See the note below. - Assign access to
Azure AD user, group, or service principal - In the
Selectfield, begin entering the name of your AD application (created earlier). When it appears in the list, click on it there. It should now be listed as one of the "Selected members". - Click Save
Note: You might want to give your application fewer permissions than what a Contributor role gets. The Azure docs have a list of all the built-in roles for Azure resources.
Create a Storage Account
Follow the instructions in the Azure docs page:
except you should use the existing resource group you created earlier, i.e. don't create a new one.
The Storage account name you choose is what Provider calls the AZURE_ACCOUNT_NAME.
# Example AZURE_ACCOUNT_NAME (Storage account name)
troystorageaccount1
Use the same Location as your resource group.
The other fields can be left with their default values unless you want to change them.
Wait for it to say, "Your deployment is complete."
Get a Storage Account Access Key
See the Azure docs page:
Manage storage account settings in the Azure portal
Go to the subsection about access keys and follow the instructions to view your new storage account's credentials.
Copy the value of one of the keys (e.g. key1, not the connection string). That's what Provider calls AZURE_ACCOUNT_KEY.
# Example AZURE_ACCOUNT_KEY (Storage account access key)
93uKDkbjfnSUNPKw2tpe0LOM+3Wk+OSkNmgwhzjvzDw1d3sKVhMRTC5ikvN0r3zsx8eQrmT9Wgjz22iLPu3aGw==
You now have all the Azure credentials Provider needs. See the instructions near the top of this page about how to give those Azure credentials to Provider.
Store Some Data in Azure Storage
You now have a storage account, but you don't have any data stored under it yet. To get some data stored in Azure Storage, the easiest option is to use Azure Storage Explorer, a free desktop app that works on Windows, macOS and Linux.
Azure Storage can store blobs, files, queues and tables. To work with Ocean Network, you should store your files in Azure Blob storage (also called object storage), not Azure Files.
Besides Azure Storage Explorer, there are many other Azure Storage APIs, libraries and tools.