diff --git a/.gitbook/assets/one-drive-1.png b/.gitbook/assets/one-drive-1.png deleted file mode 100644 index 00910484..00000000 Binary files a/.gitbook/assets/one-drive-1.png and /dev/null differ diff --git a/.gitbook/assets/one-drive-2.png b/.gitbook/assets/one-drive-2.png deleted file mode 100644 index 7334561d..00000000 Binary files a/.gitbook/assets/one-drive-2.png and /dev/null differ diff --git a/.gitbook/assets/one-drive-3.png b/.gitbook/assets/one-drive-3.png deleted file mode 100644 index 73c4f043..00000000 Binary files a/.gitbook/assets/one-drive-3.png and /dev/null differ diff --git a/.gitbook/assets/one-drive-4.png b/.gitbook/assets/one-drive-4.png deleted file mode 100644 index 03d394b8..00000000 Binary files a/.gitbook/assets/one-drive-4.png and /dev/null differ diff --git a/.gitbook/assets/publish-google-drive (1).png b/.gitbook/assets/publish-google-drive (1).png deleted file mode 100644 index f472a96f..00000000 Binary files a/.gitbook/assets/publish-google-drive (1).png and /dev/null differ diff --git a/.gitbook/assets/publish-google-drive-2 (1).png b/.gitbook/assets/publish-google-drive-2 (1).png deleted file mode 100644 index 9b126c47..00000000 Binary files a/.gitbook/assets/publish-google-drive-2 (1).png and /dev/null differ diff --git a/.gitbook/assets/publish-google-drive-2.png b/.gitbook/assets/publish-google-drive-2.png deleted file mode 100644 index 9b126c47..00000000 Binary files a/.gitbook/assets/publish-google-drive-2.png and /dev/null differ diff --git a/.gitbook/assets/publish-google-drive.png b/.gitbook/assets/publish-google-drive.png deleted file mode 100644 index f472a96f..00000000 Binary files a/.gitbook/assets/publish-google-drive.png and /dev/null differ diff --git a/README.md b/README.md index 7ac0400f..b692a7cc 100644 --- a/README.md +++ b/README.md @@ -43,7 +43,7 @@ The following guides will help you get started with buying and selling data: * [Publish a data asset](using-ocean-market/marketplace-publish-data-asset.md) * [Download a data asset](using-ocean-market/marketplace-download-data-asset.md) -* [Publishing with hosting services](building-with-ocean/asset-hosting.md) +* [Publishing with hosting services](using-ocean-market/asset-hosting.md) ### Build Your Own Data Market diff --git a/SUMMARY.md b/SUMMARY.md index 603e4312..2acc2d95 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -17,7 +17,7 @@ * [Using Ocean Market](using-ocean-market/README.md) * [Publish a Data Asset](using-ocean-market/marketplace-publish-data-asset.md) * [Download a Data Asset](using-ocean-market/marketplace-download-data-asset.md) - * [Publishing with Hosting Services](building-with-ocean/asset-hosting.md) + * [Publishing with Hosting Services](using-ocean-market/asset-hosting.md) * [Liquidity Pools \[deprecated\]](using-ocean-market/remove-liquidity-using-etherscan.md) * [Building with Ocean](building-with-ocean/README.md) * [Build a Marketplace](building-with-ocean/build-a-marketplace/README.md) diff --git a/building-with-ocean/asset-hosting.md b/building-with-ocean/asset-hosting.md deleted file mode 100644 index 7a488eec..00000000 --- a/building-with-ocean/asset-hosting.md +++ /dev/null @@ -1,150 +0,0 @@ ---- -title: Publish assets using hosting services -description: Tutorial to publish assets using hosting services like Google Drive and Azure. ---- - -# Publishing with Hosting Services - -### Overview - -To publish assets on the Ocean Marketplace, publishers must provide a link(an URL) to the file. It is up to the asset publisher to decide where to host the asset. For example, a publisher can store the content on their Google Drive, AWS server, private cloud server, or other third-party hosting services. Through publishing, the URL of the asset is encrypted and stored as a part of DDO on the blockchain. Buyers don't have access directly to the URL, but they interact with the Provider, which decrypts the URL and acts as a proxy to serve the asset. The DDO only stores the location of the file, which is accessed on-demand by the Provider. Implementing a security policy that allows only the Provider to access the URL and blocks requests from other unauthorized actors is recommended. One of the possible ways to achieve this is to allow only the Provider's IP address to access the URL. But, not all hosting services provide this feature. So, the publishers must consider the security features while choosing a hosting service. - -On Ocean Marketplace, a publisher must provide the link to the asset during publish step. Once the asset is published, this link cannot be changed. So, it is essential that the publisher correctly sets this field (shown in the below image). - -![Publish - File URL field](../.gitbook/assets/marketplace-publish-file-field.png) - -### Hosting services - -Publishers can choose any hosting service of their choice. The below section explains how to use commonly used hosting services with Ocean Marketplace. - -#### Google Drive - -Google Drive allows users to share files/folders with various access policies. Publishers must set the access policy such that anyone with the link can download the file when using Ocean Marketplace with Ocean Protocol's default [Provider](https://v4.provider.mumbai.oceanprotocol.com). - -**Step 1 - Get link** - -Open https://drive.google.com and upload the file you want to publish on the Ocean Marketplace. Right-click on the uploaded file and click the `Share` option. Set the file access policy correctly and click the `Copy link` button. - -The file URL will be of the form `https://drive.google.com/file/d//view?usp=sharing`, where the `` is the unique alphanumeric string. Verify if the URL is correct by entering it in a browser and check if the file is downloaded. - -![Google Drive link](../.gitbook/assets/publish-google-drive.png) - -**Step 2 - Create a downloadable link** - -If you paste the copied URL into the browser, it will load an HTML page. Directly pasting the link on the publish page will publish the HTML page instead of a downloadable file URL. So, let's make a downloadable file URL. - -Note the `` from step 1 and create a URL as below. - -`https://drive.google.com/uc?export=download&id=` - -**Step 3 - Publish the asset using the generated link** - -After creating a downloadable file URL, fill the `File*` field with the downloadable URL created in step 2. - -![Publish - Google Drive file](../.gitbook/assets/publish-google-drive-2.png) - -_Note: Google Drive allows only shared files to be downloaded, as shown in the above steps. The above method does not work with the shared folder. As a workaround, publishers can upload a zip of a folder and upload it as a file._ - -*** - -#### Azure storage - -Azure provides various options to host data and multiple configuration possibilities. Publishers are required to do their research and decide what would be the right choice. The below steps provide one of the possible ways to host data using Azure storage and publish it on Ocean Marketplace. - -**Prerequisite** - -Create an account on [Azure](https://azure.microsoft.com/en-us/). Users might also be asked to provide payment details and billing addresses that are out of this tutorial's scope. - -**Step 1 - Create a storage account** - -**Go to Azure portal** - -Go to the Azure portal: https://portal.azure.com/#home and select `Storage accounts` as shown below. - -![Create a storage account - 1](../.gitbook/assets/azure-1.png) - -**Create a new storage account** - -![Create a storage account - 2](../.gitbook/assets/azure-2.png) - -**Fill in the details** - -![Add details](../.gitbook/assets/azure-3.png) - -**Storage account created** - -![Storage account created](../.gitbook/assets/azure-4.png) - -**Step 2 - Create a blob container** - -![Create a blob container](../.gitbook/assets/azure-5.png) - -**Step 3 - Upload a file** - -![Upload a file](../.gitbook/assets/azure-6.png) - -**Step 4 - Share the file** - -**Select the file to be published and click Generate SAS** - -![Click generate SAS](../.gitbook/assets/azure-7.png) - -**Configure the SAS details and click `Generate SAS token and URL`** - -![Generate link to file](../.gitbook/assets/azure-8.png) - -**Copy the generated link** - -![Copy the link](../.gitbook/assets/azure-9.png) - -**Step 5 - Publish the asset using the generated link** - -Now, copy and paste the link in the Publish page in the Ocean Marketplace. - -![Publish the file as an asset](../.gitbook/assets/azure-10.png) - -#### OneDrive - -Create an account on [Microsoft](https://www.microsoft.com/en-us/microsoft-365/onedrive/online-cloud-storage). - -**Step 1 - Upload a file** - -Go to [OneDrive](https://onedrive.live.com/) and upload the file to be published. - -![Upload a file](../.gitbook/assets/one-drive-1.png) - -**Step 2 - Get link** - -After the file is uploaded, right click on the file and click `Embed`, and copy the link. - -![Get an embeddable link](../.gitbook/assets/one-drive-2.png) - -Copy the highlighted content as shown in the below image: - -![Copy the iframe](../.gitbook/assets/one-drive-3.png) - -The copied content has the following format: - -```html - -``` - -**Step 3 - Generate downloadable link** - -Copy the content from `src` field from the `iframe`. The link has the following format: `https://onedrive.live.com/embed?cid=&resid=%&authkey=` - -Replace the `https://onedrive.live.com/embed` with `https://onedrive.live.com/download` from the above URL. - -The downloadable file URL has the following format: `https://onedrive.live.com/download?cid=&resid=%&authkey=` - -Enter the URL in the browser and verify if the file is downloaded correctly. - -**Step 4 - Publish the asset using the generated link** - -Copy and paste the link in the Publish page in the Ocean Marketplace. - -![Publish the file as an asset](../.gitbook/assets/one-drive-4.png) diff --git a/using-ocean-market/README.md b/using-ocean-market/README.md index 95ad46fd..b0dbc8b0 100644 --- a/using-ocean-market/README.md +++ b/using-ocean-market/README.md @@ -11,7 +11,7 @@ The following guides will help you get started with buying and selling data: * [Publish a data asset](marketplace-publish-data-asset.md) * [Download a data asset](marketplace-download-data-asset.md) -* [Publishing with hosting services](../building-with-ocean/asset-hosting.md) +* [Publishing with hosting services](asset-hosting.md) If you are new to web3 and blockchain technologies then we suggest you first read these introductory guides: diff --git a/using-ocean-market/asset-hosting.md b/using-ocean-market/asset-hosting.md new file mode 100644 index 00000000..e5e291b4 --- /dev/null +++ b/using-ocean-market/asset-hosting.md @@ -0,0 +1,214 @@ +--- +title: Publish assets using hosting services +description: Tutorial to publish assets using hosting services like AWS and Azure. +--- + +# Publishing with Hosting Services + +### Overview + +To publish assets on the Ocean Marketplace, publishers must provide a link(an URL) to the file. It is up to the asset publisher to decide where to host the asset. For example, a publisher can store the content on their AWS server, private cloud server, or other third-party hosting services. Through publishing, the URL of the asset is encrypted and stored as a part of DDO on the blockchain. Buyers don't have access directly to the URL, but they interact with the Provider, which decrypts the URL and acts as a proxy to serve the asset. The DDO only stores the location of the file, which is accessed on-demand by the Provider. Implementing a security policy that allows only the Provider to access the URL and blocks requests from other unauthorized actors is recommended. One of the possible ways to achieve this is to allow only the Provider's IP address to access the URL. But, not all hosting services provide this feature. So, the publishers must consider the security features while choosing a hosting service. + +On Ocean Marketplace, a publisher must provide the link to the asset during publish step in the field shown in the below image. + +![Publish - File URL field](../.gitbook/assets/marketplace-publish-file-field.png) + +### Hosting services + +Publishers can choose any hosting service of their choice. The below section explains how to use commonly used hosting services with Ocean Marketplace. + +Please use a proper hosting solution to keep your files. +When using other systems (for example, `Google Drive`) that are not specifically +designed for this use case, you risk not gaining traction. Systems such as `Google Drive`, +include various virus checks and rate limiters that prevent the `Provider` to download the asset once it was purchased. + +#### AWS + +AWS provides various options to host data and multiple configuration possibilities. Publishers are required to do their research and decide what would be the right choice. The below steps provide one of the possible ways to host data using AWS S3 bucket and publish it on Ocean Marketplace. + +**Prerequisite** + +Create an account on [AWS](https://aws.amazon.com/s3/). Users might also be asked to provide payment details and billing addresses that are out of this tutorial's scope. + +**Step 1 - Create a storage account** + +**Go to AWS portal** + +Go to the AWS portal for S3: https://aws.amazon.com/s3/ and select from the upper right corner `Create an AWS account` as shown below. + +![Create an account - 1](images/hosting-services/aws-1.png) + +**Fill in the details** + +![Create an account - 2](images/hosting-services/aws-2.png)) + +**Create a bucket** + +After logging into the new account, search for the available services and select `S3` type of +storage. + +![Create an account - 3](images/hosting-services/aws-3.png) + +In order to create a S3 bucket, choose `Create bucket`. + +![Create an account - 4](images/hosting-services/aws-4.png) + +Fill in the form with the necessary information. Then, the bucket is up & running. + +![Create an account - 5](images/hosting-services/aws-5.png) + +**Step 2 - Upload asset on S3 bucket** + +Now, the asset can be uploaded by selecting the bucket name and choose `Upload` +in the `Objects` tab. + +![Upload asset on S3 bucket - 1](images/hosting-services/aws-6.png) + +**Add files to the bucket** + +Get the files and add them to the bucket. + +The file is an example used in multiple Ocean repositories, and it can be +found [here](https://raw.githubusercontent.com/oceanprotocol/c2d-examples/main/branin_and_gpr/branin.arff). + +![Upload asset on S3 bucket - 3](images/hosting-services/aws-7.png) + +The permissions and properties can be set afterwards, for the moment keep them as default. + +After selecting `Upload`, make sure that the status is `Succeeded`. + +![Upload asset on S3 bucket - 4](images/hosting-services/aws-8.png) + +**Step 3 - Access the Object URL on S3 Bucket** + +By default, the permissions of accessing the file from S3 bucket are set to private. +To publish an asset on the market, the S3 URL needs to be public. +This step shows how to set up access control policies to grant permissions to others. + +**Editing permissions** + +Go to the `Permissions` tab and select `Edit` and then uncheck `Block all public access` +boxes to give everyone read access to the object and click `Save`. + +If editing the permissions is unavailable, modify the `Object Ownership` by enabling the ACLs +as shown below. + +![Access the Object URL on S3 Bucket - 1](images/hosting-services/aws-9.png) + +**Modifying bucket policy** + +In order to have the bucket granted on public access, its policy needs to be +modified likewise. + +Note that the `` must be chosen from personal buckets' dashboard. + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "Public S3 Bucket", + "Principal": "*", + "Effect": "Allow", + "Action": "s3:GetObject", + "Resource": "arn:aws:s3:::/*" + } + ] +} +``` + +After saving the changes, the bucket should appear as `Public` access. + +![Access the Object URL on S3 Bucket - 2](images/hosting-services/aws-10.png) + +**Verify the object URL on public access** + +Select the file from the bucket that needs verification and select `Open`. Now it should +be able to download the file on personal system. + +![Access the Object URL on S3 Bucket - 3](images/hosting-services/aws-11.png) + + +**Step 4 - Get the S3 Bucket Link & Publish Asset on Market** + +Now that the S3 endpoint has public access, the asset will be hosted successfully. + +Go to [Ocean Market](https://market.oceanprotocol.com/publish/1) to complete the form +for asset creation. + +Copy the `Object URL` that can be found at `Object Overview` from AWS S3 bucket +and paste it in the `File` field from the form found at [step 2](https://market.oceanprotocol.com/publish/2) as it is illustrated below. + +![Get the S3 Bucket Link & Publish Asset on Market - 1](images/hosting-services/aws-12.png) + +#### Azure storage + +Azure provides various options to host data and multiple configuration possibilities. Publishers are required to do their research and decide what would be the right choice. The below steps provide one of the possible ways to host data using Azure storage and publish it on Ocean Marketplace. + +**Prerequisite** + +Create an account on [Azure](https://azure.microsoft.com/en-us/). Users might also be asked to provide payment details and billing addresses that are out of this tutorial's scope. + +**Step 1 - Create a storage account** + +**Go to Azure portal** + +Go to the Azure portal: https://portal.azure.com/#home and select `Storage accounts` as shown below. + +![Create a storage account - 1](images/hosting-services/azure-1.png) + +**Create a new storage account** + +![Create a storage account - 2](images/hosting-services/azure-2.png) + +**Fill in the details** + +![Add details](images/hosting-services/azure-3.png) + +**Storage account created** + +![Storage account created](images/hosting-services/azure-4.png) + +**Step 2 - Create a blob container** + +![Create a blob container](images/hosting-services/azure-5.png) + +**Step 3 - Upload a file** + +![Upload a file](images/hosting-services/azure-6.png) + +**Step 4 - Share the file** + +**Select the file to be published and click Generate SAS** + +![Click generate SAS](images/hosting-services/azure-7.png) + +**Configure the SAS details and click `Generate SAS token and URL`** + +![Generate link to file](images/hosting-services/azure-8.png) + +**Copy the generated link** + +![Copy the link](images/hosting-services/azure-9.png) + +**Step 5 - Publish the asset using the generated link** + +Now, copy and paste the link in the Publish page in the Ocean Marketplace. + +![Publish the file as an asset](images/hosting-services/azure-10.png) + + + + + + + + + + + + + + + + diff --git a/using-ocean-market/images/hosting-services/aws-1.png b/using-ocean-market/images/hosting-services/aws-1.png new file mode 100644 index 00000000..7dbe1876 Binary files /dev/null and b/using-ocean-market/images/hosting-services/aws-1.png differ diff --git a/using-ocean-market/images/hosting-services/aws-10.png b/using-ocean-market/images/hosting-services/aws-10.png new file mode 100644 index 00000000..f31f8c6d Binary files /dev/null and b/using-ocean-market/images/hosting-services/aws-10.png differ diff --git a/using-ocean-market/images/hosting-services/aws-11.png b/using-ocean-market/images/hosting-services/aws-11.png new file mode 100644 index 00000000..6bfe518d Binary files /dev/null and b/using-ocean-market/images/hosting-services/aws-11.png differ diff --git a/using-ocean-market/images/hosting-services/aws-12.png b/using-ocean-market/images/hosting-services/aws-12.png new file mode 100644 index 00000000..9662a052 Binary files /dev/null and b/using-ocean-market/images/hosting-services/aws-12.png differ diff --git a/using-ocean-market/images/hosting-services/aws-2.png b/using-ocean-market/images/hosting-services/aws-2.png new file mode 100644 index 00000000..0a057d88 Binary files /dev/null and b/using-ocean-market/images/hosting-services/aws-2.png differ diff --git a/using-ocean-market/images/hosting-services/aws-3.png b/using-ocean-market/images/hosting-services/aws-3.png new file mode 100644 index 00000000..3de5511f Binary files /dev/null and b/using-ocean-market/images/hosting-services/aws-3.png differ diff --git a/using-ocean-market/images/hosting-services/aws-4.png b/using-ocean-market/images/hosting-services/aws-4.png new file mode 100644 index 00000000..27c76402 Binary files /dev/null and b/using-ocean-market/images/hosting-services/aws-4.png differ diff --git a/using-ocean-market/images/hosting-services/aws-5.png b/using-ocean-market/images/hosting-services/aws-5.png new file mode 100644 index 00000000..7d1ba64a Binary files /dev/null and b/using-ocean-market/images/hosting-services/aws-5.png differ diff --git a/using-ocean-market/images/hosting-services/aws-6.png b/using-ocean-market/images/hosting-services/aws-6.png new file mode 100644 index 00000000..845ee8ec Binary files /dev/null and b/using-ocean-market/images/hosting-services/aws-6.png differ diff --git a/using-ocean-market/images/hosting-services/aws-7.png b/using-ocean-market/images/hosting-services/aws-7.png new file mode 100644 index 00000000..c30214af Binary files /dev/null and b/using-ocean-market/images/hosting-services/aws-7.png differ diff --git a/using-ocean-market/images/hosting-services/aws-8.png b/using-ocean-market/images/hosting-services/aws-8.png new file mode 100644 index 00000000..e09238bc Binary files /dev/null and b/using-ocean-market/images/hosting-services/aws-8.png differ diff --git a/using-ocean-market/images/hosting-services/aws-9.png b/using-ocean-market/images/hosting-services/aws-9.png new file mode 100644 index 00000000..b1bf06b1 Binary files /dev/null and b/using-ocean-market/images/hosting-services/aws-9.png differ diff --git a/.gitbook/assets/azure-1.png b/using-ocean-market/images/hosting-services/azure-1.png similarity index 100% rename from .gitbook/assets/azure-1.png rename to using-ocean-market/images/hosting-services/azure-1.png diff --git a/.gitbook/assets/azure-10.png b/using-ocean-market/images/hosting-services/azure-10.png similarity index 100% rename from .gitbook/assets/azure-10.png rename to using-ocean-market/images/hosting-services/azure-10.png diff --git a/.gitbook/assets/azure-2.png b/using-ocean-market/images/hosting-services/azure-2.png similarity index 100% rename from .gitbook/assets/azure-2.png rename to using-ocean-market/images/hosting-services/azure-2.png diff --git a/.gitbook/assets/azure-3.png b/using-ocean-market/images/hosting-services/azure-3.png similarity index 100% rename from .gitbook/assets/azure-3.png rename to using-ocean-market/images/hosting-services/azure-3.png diff --git a/.gitbook/assets/azure-4.png b/using-ocean-market/images/hosting-services/azure-4.png similarity index 100% rename from .gitbook/assets/azure-4.png rename to using-ocean-market/images/hosting-services/azure-4.png diff --git a/.gitbook/assets/azure-5.png b/using-ocean-market/images/hosting-services/azure-5.png similarity index 100% rename from .gitbook/assets/azure-5.png rename to using-ocean-market/images/hosting-services/azure-5.png diff --git a/.gitbook/assets/azure-6.png b/using-ocean-market/images/hosting-services/azure-6.png similarity index 100% rename from .gitbook/assets/azure-6.png rename to using-ocean-market/images/hosting-services/azure-6.png diff --git a/.gitbook/assets/azure-7.png b/using-ocean-market/images/hosting-services/azure-7.png similarity index 100% rename from .gitbook/assets/azure-7.png rename to using-ocean-market/images/hosting-services/azure-7.png diff --git a/.gitbook/assets/azure-8.png b/using-ocean-market/images/hosting-services/azure-8.png similarity index 100% rename from .gitbook/assets/azure-8.png rename to using-ocean-market/images/hosting-services/azure-8.png diff --git a/.gitbook/assets/azure-9.png b/using-ocean-market/images/hosting-services/azure-9.png similarity index 100% rename from .gitbook/assets/azure-9.png rename to using-ocean-market/images/hosting-services/azure-9.png