Merge branch 'v4' of github.com:oceanprotocol/docs into v4

This commit is contained in:
Akshay 2022-02-16 22:25:18 +01:00
commit 89bb849417
6 changed files with 766 additions and 484 deletions

View File

@ -1,349 +0,0 @@
---
title: DDO Metadata
description: Specification of the DDO subset dedicated to asset metadata
slug: /concepts/ddo-metadata/
section: concepts
---
## Overview
This page defines the schema for asset _metadata_. Metadata is the subset of an Ocean DDO that holds information about the asset.
The schema is based on public schema.org [DataSet schema](https://schema.org/Dataset).
Standardizing labels is key to effective searching, sorting and filtering (discovery).
This page specifies metadata attributes that _must_ be included, and that _may_ be included. These attributes are organized hierarchically, from top-layer attributes like `"main"` to sub-level attributes like `"main.type"`. This page also provides DDO metadata examples.
## Rules for Metadata Storage and Control in Ocean
The publisher publishes an asset DDO (including metadata) onto the chain.
The publisher may be the asset owner, or a marketplace acting on behalf of the owner.
Most metadata fields may be modified after creation. The blockchain records the provenance of changes.
DDOs (including metadata) are found in two places:
- _Remote_ - main storage, on-chain. File URLs are always encrypted. One may actually encrypt all metadata, at a severe cost to discoverability.
- _Local_ - local cache. All fields are in plaintext.
Ocean Aquarius helps manage metadata. It can be used to write DDOs to the chain, read from the chain, and has a local cache of the DDO in plaintext with fast search.
## Fields for Metadata
An asset represents a resource in Ocean, e.g. a dataset or an algorithm.
A `metadata` object has the following attributes, all of which are objects. Some are only required for local or remote, and are specified as such.
| Attribute | Required | Description |
| --------------------------- | -------- | ---------------------------------------------------------- |
| **`main`** | **Yes** | Main attributes |
| **`encryptedFiles`** | Remote | Encrypted string of the `attributes.main.files` object. |
| **`encryptedServices`** | Remote | Encrypted string of the `attributes.main.services` object. |
| **`status`** | No | Status attributes |
| **`additionalInformation`** | No | Optional attributes |
The `main` and `additionalInformation` attributes are independent of the asset type.
## Fields for `attributes.main`
The `main` object has the following attributes.
| Attribute | Type | Required | Description |
| ------------------- | --------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **`name`** | Text |**Yes** | Descriptive name or title of the asset. |
| **`type`** | Text |**Yes** | Asset type. Includes `"dataset"` (e.g. csv file), `"algorithm"` (e.g. Python script). Each type needs a different subset of metadata attributes. |
| **`author`** | Text |**Yes** | Name of the entity generating this data (e.g. Tfl, Disney Corp, etc.). |
| **`license`** | Text |**Yes** | Short name referencing the license of the asset (e.g. Public Domain, CC-0, CC-BY, No License Specified, etc. ). If it's not specified, the following value will be added: "No License Specified". |
| **`files`** | Array of files object |**Yes** | Array of `File` objects including the encrypted file urls. |
| **`dateCreated`** | DateTime |**Yes** | The date on which the asset was created by the originator. ISO 8601 format, Coordinated Universal Time, e.g. `2019-01-31T08:38:32Z`. |
| **`datePublished`** | DateTime | Remote | The date on which the asset DDO is registered into the metadata store (Aquarius) |
## Fields for `attributes.main.files`
The `files` object has a list of `file` objects.
Each `file` object has the following attributes, with the details necessary to consume and validate the data.
| Attribute | Required | Description |
| -------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **`index`** |**Yes** | Index number starting from 0 of the file. |
| **`contentType`** |**Yes** | File format. |
| **`url`** | Local | Content URL. Omitted from the remote metadata. Supports `http(s)://` and `ipfs://` URLs. |
| **`name`** | No | File name. |
| **`checksum`** | No | Checksum of the file using your preferred format (i.e. MD5). Format specified in `checksumType`. If it's not provided can't be validated if the file was not modified after registering. |
| **`checksumType`** | No | Format of the provided checksum. Can vary according to server (i.e Amazon vs. Azure) |
| **`contentLength`** | No | Size of the file in bytes. |
| **`encoding`** | No | File encoding (e.g. UTF-8). |
| **`compression`** | No | File compression (e.g. no, gzip, bzip2, etc). |
| **`encrypted`** | No | Boolean. Is the file encrypted? If is not set is assumed the file is not encrypted |
| **`encryptionMode`** | No | Encryption mode used. Just valid if `encrypted=true` |
| **`resourceId`** | No | Remote identifier of the file in the external provider. It is typically the remote id in the cloud provider. |
| **`attributes`** | No | Key-Value hash map with additional attributes describing the asset file. It could include details like the Amazon S3 bucket, region, etc. |
## Fields for `attributes.status`
A `status` object has the following attributes.
| Attribute | Type | Required | Description |
| --------------------- | ------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **`isListed`** | Boolean | No | Use to flag unsuitable content. True by default. If it's false, the content must not be returned. |
| **`isRetired`** | Boolean | No | Flag retired content. False by default. If it's true, the content may either not be returned, or returned with a note about retirement. |
| **`isOrderDisabled`** | Boolean | No | For temporarily disabling ordering assets, e.g. when file host is in maintenance. False by default. If it's true, no ordering of assets for download or compute should be allowed. |
## Fields for `attributes.additionalInformation`
All the additional information will be stored as part of the `additionalInformation` section.
| Attribute | Type | Required |
| --------------------- | ------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **`tags`** | Array of Text | No | Array of keywords or tags used to describe this content. Empty by default. |
| **`description`** | Text | No | Details of what the resource is. For a dataset, this attribute explains what the data represents and what it can be used for. |
| **`copyrightHolder`** | Text | No | The party holding the legal copyright. Empty by default. |
| **`workExample`** | Text | No | Example of the concept of this asset. This example is part of the metadata, not an external link. |
| **`links`** | Array of Link | No | Mapping of links for data samples, or links to find out more information. Links may be to either a URL or another Asset. We expect marketplaces to converge on agreements of typical formats for linked data: The Ocean Protocol itself does not mandate any specific formats as these requirements are likely to be domain-specific. The links array can be an empty array, but if there is a link object in it, then an "url" is required in that link object. |
| **`inLanguage`** | Text | No | The language of the content. Please use one of the language codes from the [IETF BCP 47 standard](https://tools.ietf.org/html/bcp47). |
| **`categories`** | Array of Text | No | Optional array of categories associated to the asset. Note: recommended to use `"tags"` instead of this. |
## Fields - Other Suggestions
Here are example attributes to help an asset's discoverability.
| Attribute | Description |
| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **`updateFrequency`** | An indication of update latency - i.e. How often are updates expected (seldom, annually, quarterly, etc.), or is the resource static that is never expected to get updated. |
| **`structuredMarkup`** | A link to machine-readable structured markup (such as ttl/json-ld/rdf) describing the dataset. |
## DDO Metadata Example - Local
This is what the DDO metadata looks like. All fields are in plaintext. This is before it's stored on-chain or when it's retrieved and decrypted into a local cache.
```json
{
"main": {
"name": "Madrid Weather forecast",
"dateCreated": "2019-05-16T12:36:14.535Z",
"author": "Norwegian Meteorological Institute",
"type": "dataset",
"license": "Public Domain",
"price": "123000000000000000000",
"files": [
{
"index": 0,
"url": "https://example-url.net/weather/forecast/madrid/350750305731.xml",
"contentLength": "0",
"contentType": "text/xml",
"compression": "none"
}
]
},
"additionalInformation": {
"description": "Weather forecast of Europe/Madrid in XML format",
"copyrightHolder": "Norwegian Meteorological Institute",
"categories": ["Other"],
"links": [],
"tags": [],
"updateFrequency": null,
"structuredMarkup": []
},
"status": {
"isListed": true,
"isRetired": false,
"isOrderDisabled": false
}
}
```
## DDO Metadata Example - Remote
The previous example was for a local cache, with all fields in plaintext.
Here's the same example, for remote on-chain storage. That is, it's how metadata looks as a response to querying Aquarius (remote metadata).
How remote is changed, compared to local:
- `url` is removed from all objects in the `files` array
- `encryptedFiles` is added.
```json
{
"service": [
{
"index": 0,
"serviceEndpoint": "http://aquarius:5000/api/v1/aquarius/assets/ddo/{did}",
"type": "metadata",
"attributes": {
"main": {
"type": "dataset",
"name": "Madrid Weather forecast",
"dateCreated": "2019-05-16T12:36:14.535Z",
"author": "Norwegian Meteorological Institute",
"license": "Public Domain",
"files": [
{
"contentLength": "0",
"contentType": "text/xml",
"compression": "none",
"index": 0
}
],
"datePublished": "2019-05-16T12:41:01Z"
},
"encryptedFiles": "0x7a0d1c66ae861…df43aa9",
"additionalInformation": {
"description": "Weather forecast of Europe/Madrid in XML format",
"copyrightHolder": "Norwegian Meteorological Institute",
"categories": ["Other"],
"links": [],
"tags": [],
"updateFrequency": null,
"structuredMarkup": []
},
"status": {
"isListed": true,
"isRetired": false,
"isOrderDisabled": false
}
}
}
]
}
```
## Fields when `attributes.main.type = algorithm`
An asset of type `algorithm` has the following additional attributes under `main.algorithm`:
| Attribute | Type | Required | Description |
| --------------- | -------- | -------- | --------------------------------------------- |
| **`container`** | `Object` |**Yes** | Object describing the Docker container image. |
| **`language`** | `string` | No | Language used to implement the software |
| **`format`** | `string` | No | Packaging format of the software. |
| **`version`** | `string` | No | Version of the software. |
The `container` object has the following attributes:
| Attribute | Type | Required | Description |
| ---------------- | -------- | -------- | ----------------------------------------------------------------- |
| **`entrypoint`** | `string` |**Yes** | The command to execute, or script to run inside the Docker image. |
| **`image`** | `string` |**Yes** | Name of the Docker image. |
| **`tag`** | `string` |**Yes** | Tag of the Docker image. |
| **`checksum`** | `string` |**Yes** | Checksum of the Docker image. |
```json
{
"index": 0,
"serviceEndpoint": "http://localhost:5000/api/v1/aquarius/assets/ddo/{did}",
"type": "metadata",
"attributes": {
"main": {
"author": "John Doe",
"dateCreated": "2019-02-08T08:13:49Z",
"license": "CC-BY",
"name": "My super algorithm",
"type": "algorithm",
"algorithm": {
"language": "scala",
"format": "docker-image",
"version": "0.1",
"container": {
"entrypoint": "node $ALGO",
"image": "node",
"tag": "10",
"checksum": "efb2c764274b745f5fc37f97c6b0e761"
}
},
"files": [
{
"name": "build_model",
"url": "https://raw.gith ubusercontent.com/oceanprotocol/test-algorithm/master/javascript/algo.js",
"index": 0,
"checksum": "efb2c764274b745f5fc37f97c6b0e761",
"contentLength": "4535431",
"contentType": "text/plain",
"encoding": "UTF-8",
"compression": "zip"
}
]
},
"additionalInformation": {
"description": "Workflow to aggregate weather information",
"tags": ["weather", "uk", "2011", "workflow", "aggregation"],
"copyrightHolder": "John Doe"
}
}
}
```
## Fields when `attributes.main.type = compute`
An asset with a service of type `compute` has the following additional attributes under `main.privacy`:
| Attribute | Type | Required | Description |
| --------------------------------- | ------------------ | -------- | ---------------------------------------------------------- |
| **`allowRawAlgorithm`** | `boolean` |**Yes** | If True, a drag & drop algo can be runned |
| **`allowNetworkAccess`** | `boolean` |**Yes** | If True, the algo job will have network access (stil WIP) |
| **`publisherTrustedAlgorithms `** | Array of `Objects` |**Yes** | If Empty , then any published algo is allowed. (see below) |
The `publisherTrustedAlgorithms ` is an array of objects with the following structure:
| Attribute | Type | Required | Description |
| ------------------------------ | -------- | -------- | ------------------------------------------------------------------ |
| **`did`** | `string` |**Yes** | The did of the algo which is trusted by the publisher. |
| **`filesChecksum`** | `string` |**Yes** | Hash of ( algorithm's encryptedFiles + files section (as string) ) |
| **`containerSectionChecksum`** | `string` |**Yes** | Hash of the algorithm container section (as string) |
To produce `filesChecksum`:
```javascript
sha256(
algorithm_ddo.service['metadata'].attributes.encryptedFiles +
JSON.Stringify(algorithm_ddo.service['metadata'].attributes.main.files)
)
```
To produce `containerSectionChecksum`:
```javascript
sha256(
JSON.Stringify(
algorithm_ddo.service['metadata'].attributes.main.algorithm.container
)
)
```
### Example of a compute service
```json
{
"type": "compute",
"index": 1,
"serviceEndpoint": "https://provider.oceanprotocol.com",
"attributes": {
"main": {
"name": "dataAssetComputingService",
"creator": "0xA32C84D2B44C041F3a56afC07a33f8AC5BF1A071",
"datePublished": "2021-02-17T06:31:33Z",
"cost": "1",
"timeout": 3600,
"privacy": {
"allowRawAlgorithm": true,
"allowNetworkAccess": false,
"publisherTrustedAlgorithms": [
{
"did": "0xxxxx",
"filesChecksum": "1234",
"containerSectionChecksum": "7676"
},
{
"did": "0xxxxx",
"filesChecksum": "1232334",
"containerSectionChecksum": "98787"
}
]
}
}
}
}
```

View File

@ -1,173 +1,811 @@
---
title: DIDs & DDOs - Asset Identifiers & Objects
description: Specification of Ocean asset identifiers and objects using DIDs & DDOs
title: DID & DDO
description: Specification of decentralized identifiers for assets in Ocean Protocol using the DID & DDO standards.
slug: /concepts/did-ddo/
section: concepts
---
**v4.0.0**
## Overview
This document describes how Ocean assets follow the DID/DDO spec, such that Ocean assets can inherit DID/DDO benefits and enhance interoperability.
This document describes how Ocean assets follow the DID/DDO specification, such that Ocean assets can inherit DID/DDO benefits and enhance interoperability. DIDs and DDOs follow the [specification defined by the World Wide Web Consortium (W3C)](https://w3c-ccg.github.io/did-spec/).
Decentralized identifiers (DIDs) are a new type of identifier that enables verifiable, decentralized digital identity. Each DID is associated with a unique entity. DIDs may represent humans, objects, and more.
Decentralized identifiers (DIDs) are a type of identifier that enable verifiable, decentralized digital identity. Each DID is associated with a unique entity and DIDs may represent humans, objects, and more.
A DID Document (DDO) is JSON blob that holds information about the DID. Given a DID, a _resolver_ will return the DDO of that DID.
A DID Document (DDO) is a JSON blob that holds information about the DID. Given a DID, a _resolver_ will return the DDO of that DID.
If a DID is the index key in a key-value pair, then the DID Document is the value to which the index key points.
The combination of a DID and its associated DID Document forms the root record for a decentralized identifier.
## Rules for DIDs & DDOs
DIDs and DDOs follow [this specification](https://w3c-ccg.github.io/did-spec/) defined by the World Wide Web Consurtium (W3C).
An _asset_ in Ocean represents a downloadable file, compute service, or similar. Each asset is a _resource_ under control of a _publisher_. The Ocean network itself does _not_ store the actual resource (e.g. files).
## Rules for DIDs & DDOs in Ocean
An _asset_ has a DID and DDO. The DDO should include [metadata](#metadata) about the asset, and define access in at least one [service](#services). The DDO can only be modified by _owners_ or _delegated users_.
- An _asset_ in Ocean represents a downloadable file, compute service, or similar. Each asset is a _resource_ under control of a _publisher_. The Ocean network itself does _not_ store the actual resource (e.g. files).
- An asset should have a DID and DDO. The DDO should include metadata about the asset.
- The DDO can only can be modified by _owners_ or _delegated users_.
- There _must_ be at least one client library acting as _resolver_, to get a DDO from a DID.
- The DDO is stored on-chain. It's stored in in plaintext, with two exceptions: (1) the field for resource-access url is encrypted (2) the whole DDO may be encrypted, if the publisher is willing to lose 100% of discoverability.
- A metadata cache like Aquarius can help in reading and writing DDO data from the chain.
All DDOs are stored on-chain in encrypted form to be fully GDPR-compatible. A metadata cache like _Aquarius_ can help in reading, decrypting, and searching through encrypted DDO data from the chain. Because the file URLs are encrypted on top of the full DDO encryption, returning unencrypted DDOs e.g. via an API is safe to do as the file URLs will still stay encrypted.
## DID Structure
## Publishing & Retrieving DDOs
In Ocean, a DID is a string that looks like:
The DDO is stored on-chain as part of the NFT contract, and stored encrypted using the private key of the _Provider_. To resolve it, a metadata cache like _Aquarius_ must query the provider to decrypt the DDO.
Here is the complete flow:
![DDO flow](images/ddo-flow.png)
<details>
<summary>UML source</summary>
```text
title DDO flow
User(Ocean library) -> User(Ocean library): Prepare DDO
User(Ocean library) -> Provider: encrypt DDO
Provider -> User(Ocean library): encryptedDDO
User(Ocean library) -> ERC721 contract: publish encryptedDDO
Aquarius <-> ERC721 contract: monitors ERC721 contract and gets MetdadataCreated Event (contains encryptedDDO)
Aquarius -> ERC721 contract: calls getMetaData()
Aquarius -> Provider: decrypt encryptedDDO, signed request using Aquarius's private key
Provider -> ERC721 contract: checks state using getMetaData()
Provider -> Provider: depending on metadataState (expired,retired) and aquarius address, validates the request
Provider -> Aquarius: DDO
Aquarius -> Aquarius : validate DDO
Aquarius -> Aquarius : cache DDO
Aquarius -> Aquarius : enhance cached DDO in response with additional infos like events & stats
```
</details>
## DID
In Ocean, a DID is a string that looks like this:
```text
did:op:0ebed8226ada17fde24b6bf2b95d27f8f05fcce09139ff5cec31f6d81a7cd2ea
```
The part after `did:op:` is the the ERC721 contract address(in checksum format) and the chainId (expressed as decimal) the asset has been published to:
```js
const checksum = sha256(ERC721 contract address + chainId)
console.log(checksum)
// 0ebed8226ada17fde24b6bf2b95d27f8f05fcce09139ff5cec31f6d81a7cd2ea
```
It follows [the generic DID scheme](https://w3c-ccg.github.io/did-spec/#the-generic-did-scheme).
The part after `did:op:` is the asset's on-chain Ethereum address (minus the "0x"). One can be computed from the other; therefore there is a 1:1 mapping between did and Ethereum address.
## DDO
## DDO Attributes
A DDO in Ocean has these required attributes:
![DDO Content](images/ddo-content.png)
| Attribute | Type | Description |
| ----------------- | --------------------------- | -------------------------------------------------------------------------------------------------------------- |
| **`@context`** | Array of `string` | Contexts used for validation. |
| **`id`** | `string` | Computed as `sha256(address of ERC721 contract + chainId)`. |
| **`version`** | `string` | Version information in [SemVer](https://semver.org) notation referring to this DDO spec version, like `4.0.0`. |
| **`chainId`** | `number` | Stores chainId of the network the DDO was published to. |
| **`nftAddress`** | `string` | NFT contract linked to this asset |
| **`metadata`** | [Metadata](#metadata) | Stores an object describing the asset. |
| **`services`** | [Services](#services) | Stores an array of services defining access to the asset. |
| **`credentials`** | [Credentials](#credentials) | Describes the credentials needed to access a dataset in addition to the `services` definition. |
A DDO has these standard attributes:
### Metadata
- `@context`
- `id`
- `created`
- `updated`
- `publicKey`
- `authentication`
- `proof`
- `verifiableCredential`
This object holds information describing the actual asset.
In Ocean, the DDO also has:
| Attribute | Type | Required | Description |
| --------------------------- | ----------------------------------------- | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **`created`** | `ISO date/time string` | | Contains the date of the creation of the dataset content in ISO 8601 format preferably with timezone designators, e.g. `2000-10-31T01:30:00Z`. |
| **`updated`** | `ISO date/time string` | | Contains the date of last update of the dataset content in ISO 8601 format preferably with timezone designators, e.g. `2000-10-31T01:30:00Z`. |
| **`description`** | `string` | **✓** | Details of what the resource is. For a dataset, this attribute explains what the data represents and what it can be used for. |
| **`copyrightHolder`** | `string` | | The party holding the legal copyright. Empty by default. |
| **`name`** | `string` | **✓** | Descriptive name or title of the asset. |
| **`type`** | `string` | **✓** | Asset type. Includes `"dataset"` (e.g. csv file), `"algorithm"` (e.g. Python script). Each type needs a different subset of metadata attributes. |
| **`author`** | `string` | **✓** | Name of the entity generating this data (e.g. Tfl, Disney Corp, etc.). |
| **`license`** | `string` | **✓** | Short name referencing the license of the asset (e.g. Public Domain, CC-0, CC-BY, No License Specified, etc. ). If it's not specified, the following value will be added: "No License Specified". |
| **`links`** | Array of `string` | | Mapping of URL strings for data samples, or links to find out more information. Links may be to either a URL or another asset. |
| **`contentLanguage`** | `string` | | The language of the content. Use one of the language codes from the [IETF BCP 47 standard](https://tools.ietf.org/html/bcp47) |
| **`tags`** | Array of `string` | | Array of keywords or tags used to describe this content. Empty by default. |
| **`categories`** | Array of `string` | | Array of categories associated to the asset. Note: recommended to use `tags` instead of this. |
| **`additionalInformation`** | Object | | Stores additional information, this is customizable by publisher |
| **`algorithm`** | [Algorithm Metadata](#algorithm-metadata) | **✓** (for algorithm assets only) | Information about asset of `type` `algorithm` |
- `dataToken`
- `service`
- `credentials` - optional flag, which describes the credentials needed to access a dataset (see below)
Asset metadata must be included as one of the objects inside the `"service"` array, with type `"metadata"`.
## DDO Service Types
There are many possible service types for a DDO.
- `metadata` - describing the asset
- `access` - describing how the asset can be downloaded
- `compute` - describing how the asset can be computed upon
Each asset has a `metadata` service and at least one other service.
Each service is distinguished by the `DDO.service.type` attribute.
Each service has an `attributes` section holding the information related to the service. That section _must_ have a `main` sub-section, holding all the mandatory information that a service has to provide.
A part of the `attributes.main` sub-section, other optional sub-sections like `attributes.extra` can be added. These depend on the service type.
Each service has a `timeout` (in seconds) section describing how long the service can be used after consumption is initiated. A timeout of 0 represents no time limit.
The `cost` attribute is obsolete, as of Ocean V3. As of V3, to consume an asset, one sends exactly 1.0 datatokens of the asset, so a `cost` is not needed.
## DDO Service Example
Here is an example DDO service:
Example:
```json
"service": [
{
"index": 0,
"type": "metadata",
"serviceEndpoint": "https://service/api/v1/metadata/assets/ddo/did:op:0ebed8226ada17fde24b6bf2b95d27f8f05fcce09139ff5cec31f6d81a7cd2ea",
"attributes": {
"main": {},
"additionalInformation": {},
"curation": {}
}
},
{
"index": 1,
"type": "access",
"serviceEndpoint": "http://localhost:8030/api/v1/provider/services/consume",
"attributes": {
"main": {
"cost":"10",
"timeout":0
{
"metadata": {
"created": "2020-11-15T12:27:48Z",
"updated": "2021-05-17T21:58:02Z",
"description": "Sample description",
"name": "Sample asset",
"type": "dataset",
"author": "OPF",
"license": "https://market.oceanprotocol.com/terms"
}
}
```
#### Algorithm Metadata
An asset of type `algorithm` has additional attributes under `metadata.algorithm`, describing the algorithm and the Docker environment it is supposed to be run under.
| Attribute | Type | Required | Description |
| --------------- | --------------------------- | -------- | ------------------------------------------------------------------------------------------ |
| **`language`** | `string` | | Language used to implement the software. |
| **`version`** | `string` | | Version of the software preferably in [SemVer](https://semver.org) notation. E.g. `1.0.0`. |
| **`consumerParameters`** | [Consumer Parameters](#consumer-parameters) | | An object the defines required consumer input before running the algorithm |
| **`container`** | `container` | **✓** | Object describing the Docker container image. See below |
The `container` object has the following attributes defining the Docker image for running the algorithm:
| Attribute | Type | Required | Description |
| ---------------- | -------- | -------- | ----------------------------------------------------------------- |
| **`entrypoint`** | `string` | **✓** | The command to execute, or script to run inside the Docker image. |
| **`image`** | `string` | **✓** | Name of the Docker image. |
| **`tag`** | `string` | **✓** | Tag of the Docker image. |
| **`checksum`** | `string` | **✓** | Checksum of the Docker image. |
```json
{
"metadata": {
"created": "2020-11-15T12:27:48Z",
"updated": "2021-05-17T21:58:02Z",
"description": "Sample description",
"name": "Sample algorithm asset",
"type": "algorithm",
"author": "OPF",
"license": "https://market.oceanprotocol.com/terms",
"algorithm": {
"language": "Node.js",
"version": "1.0.0",
"container": {
"entrypoint": "node $ALGO",
"image": "ubuntu",
"tag": "latest",
"checksum": "44e10daa6637893f4276bb8d7301eb35306ece50f61ca34dcab550"
},
"additionalInformation": {}
"consumerParameters":{},
}
},
}
}
```
### Services
Services define the access for an asset, and each service is represented by its respective datatoken.
An asset should have at least one service to be actually accessible, and can have as many services which make sense for a specific use case.
| Attribute | Type | Required | Description |
| ---------------------- | --------------------------- | ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
| **`id`** | `string` | **✓** | Unique ID |
| **`type`** | `string` | **✓** | Type of service (`access`, `compute`, `wss`, etc. |
| **`name`** | `string` | | Service friendly name |
| **`description`** | `string` | | Service description |
| **`datatokenAddress`** | `string` | **✓** | Datatoken address |
| **`serviceEndpoint`** | `string` | **✓** | Provider URL (schema + host) |
| **`files`** | [Files](#files) | **✓** | Encrypted file URLs. |
| **`timeout`** | `number` | **✓** | Describing how long the service can be used after consumption is initiated. A timeout of `0` represents no time limit. Expressed in seconds. |
| **`compute`** | [Compute](#compute-options) | **✓** (for compute assets only) | If service is of `type` `compute`, holds information about the compute-related privacy settings & resources. |
| **`consumerParameters`** | [Consumer Parameters](#consumer-parameters) | | An object the defines required consumer input before consuming the asset|
#### Files
The `files` field is returned as a `string` which holds the encrypted file URLs.
Example:
```json
{
"files": "0x044736da6dae39889ff570c34540f24e5e084f4e5bd81eff3691b729c2dd1465ae8292fc721e9d4b1f10f56ce12036c9d149a4dab454b0795bd3ef8b7722c6001e0becdad5caeb2005859642284ef6a546c7ed76f8b350480691f0f6c6dfdda6c1e4d50ee90e83ce3cb3ca0a1a5a2544e10daa6637893f4276bb8d7301eb35306ece50f61ca34dcab550b48181ec81673953d4eaa4b5f19a45c0e9db4cd9729696f16dd05e0edb460623c843a263291ebe757c1eb3435bb529cc19023e0f49db66ef781ca692655992ea2ca7351ac2882bf340c9d9cb523b0cbcd483731dc03f6251597856afa9a68a1e0da698cfc8e81824a69d92b108023666ee35de4a229ad7e1cfa9be9946db2d909735"
}
```
During the publish process, file URLs must be encrypted with a respective _Provider_ API call before storing the DDO on-chain. For this an array of objects defining the storage access details are sent.
Type of objects supported :
<table>
<tr>
<th>Type</th>
<th>Description</th>
<th>Example</th>
</tr>
<td>'url'</td>
<td>Static URL. Contains url and HTTP method</td>
<td>
```json
[
{
"type": "url",
"url": "https://url.com/file1.csv",
"method": "GET"
}
]
```
</td>
</tr></table>
First class integrations supported in the future :
<table>
<tr>
<th>Type</th>
<th>Description</th>
<th>Example</th>
</tr>
<tr>
<td>"ipfs"</td><td>IPFS files</td>
<td>
```json
[
{
"index": 2,
"type": "compute",
"serviceEndpoint": "http://localhost:8030/api/v1/provider/services/compute",
"attributes": {
"main": {
"cost":"10",
"timeout":3600
},
"additionalInformation": {}
}
"type":"ipfs",
"hash": "XXX"
}
]
```
## DDO Credentials for Fine-Grained Permissions
</td>
By default, a consumer can access a resource if they have 1.0 datatokens. _Credentials_ allow the publisher to optionally specify finer-grained permissions.
<tr><td>"filecoin"</td><td>Filecoin storage</td><td>&nbsp;</td></tr>
<tr><td>"arwave"</td><td>Arwave</td><td>&nbsp;</td></tr>
<tr><td>"storj"</td><td>Storj</td><td>&nbsp;</td></tr>
<tr><td>"sql"</td><td>Sql connection, dataset is generated by a query</td><td>&nbsp;</td></tr>
</table>
Consider a medical data use case, where only a credentialed EU researcher can legally access a given dataset. Ocean supports this as follows: a consumer can only access the resource if they have 1.0 datatokens _and_ one of the specified `"allow"` credentials.
This is like going to an R-rated movie, where you can only get in if you show both your movie ticket (datatoken) _and_ some some id showing you're old enough (credential).
A service can contain multiple files, using multiple storage types.
Only credentials that can be proven are supported. This includes Ethereum public addresses, and (in the future) W3C Verifiable Credentials and more.
Ocean also supports `"deny"` credentials: if a consumer has any of these credentials, they cannot access the resource.
Here's an example object with both `"allow"` and `"deny"` entries.
Example:
```json
"credentials":{
"allow":[
{
"type":"address",
"values":[
"0x123",
"0x456"
]
}
]
[
{
"type": "url",
"url": "https://url.com/file1.csv",
"method": "GET"
},
"deny":[
{
"type":"address",
"values":[
"0x2222",
"0x333"
]
}
]
}
{
"type": "ipfs",
"hash": "XXXX"
}
]
```
For future usage, we can extend that with different credentials types. Example:
To get information about the files after encryption, the `/fileinfo` endpoint of _Provider_ returns based on a passed DID an array of file metadata (based on the file type):
```json
[
{
"type": "url",
"contentLength": 100,
"contentType": "application/json"
},
{
"type": "ipfs",
"contentLength": 130,
"contentType": "application/text"
}
]
```
This only concerns metadata about a file, but never the file URLs. The only way to decrypt them is to exchange at least 1 datatoken based on the respective service pricing scheme.
#### Compute Options
An asset with a service of `type` `compute` has the following additional attributes under the `compute` object. This object is required if the asset is of `type` `compute`, but can be omitted for `type` of `access`.
| Attribute | Type | Required | Description |
| ------------------------------------------ | ------------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **`allowRawAlgorithm`** | `boolean` | **✓** | If `true`, any passed raw text will be allowed to run. Useful for an algorithm drag & drop use case, but increases risk of data escape through malicious user input. Should be `false` by default in all implementations. |
| **`allowNetworkAccess`** | `boolean` | **✓** | If `true`, the algorithm job will have network access. |
| **`publisherTrustedAlgorithmPublishers `** | Array of `string` | **✓** | If empty, then any published algorithm is allowed. Otherwise, only published algorithms by some publishers are allowed. |
| **`publisherTrustedAlgorithms `** | Array of `publisherTrustedAlgorithms` | **✓** | If empty, then any published algorithm is allowed. (see below). |
The `publisherTrustedAlgorithms ` is an array of objects with the following structure:
| Attribute | Type | Required | Description |
| ------------------------------ | -------- | -------- | ------------------------------------------------------------------------- |
| **`did`** | `string` | **✓** | The DID of the algorithm which is trusted by the publisher. |
| **`filesChecksum`** | `string` | **✓** | Hash of algorithm's `files` section (as `string`). |
| **`containerSectionChecksum`** | `string` | **✓** | Hash of algorithm's `metadata.algorithm.container` section (as `string`). |
To produce `filesChecksum`:
```js
sha256(JSON.Stringify(algorithm_ddo.services[0].files))
```
To produce `containerSectionChecksum`:
```js
sha256(JSON.Stringify(algorithm_ddo.metadata.algorithm.container))
```
Example:
```json
{
"type": "credential3Box",
"values": ["profile1", "profile2"]
"services": [
{
"id": "1",
"type": "access",
"files": "0x044736da6dae39889ff570c34540f24e5e084f...",
"name": "Download service",
"description": "Download service",
"datatokenAddress": "0x123",
"serviceEndpoint": "https://myprovider.com",
"timeout": 0
},
{
"id": "2",
"type": "compute",
"files": "0x6dd05e0edb460623c843a263291ebe757c1eb3...",
"name": "Compute service",
"description": "Compute service",
"datatokenAddress": "0x124",
"serviceEndpoint": "https://myprovider.com",
"timeout": 0,
"compute": {
"allowRawAlgorithm": false,
"allowNetworkAccess": true,
"publisherTrustedAlgorithmPublishers": ["0x234", "0x235"],
"publisherTrustedAlgorithms": [
{
"did": "did:op:123",
"filesChecksum": "100",
"containerSectionChecksum": "200"
},
{
"did": "did:op:124",
"filesChecksum": "110",
"containerSectionChecksum": "210"
}
]
}
}
]
}
```
#### Consumer Parameters
Sometimes, you may need some input before downloading a dataset or running an algorithm.
Examples:
- You want to know the desired sampling interval of data in your dataset, before the user is going to download it. Your dataset URL is `https://example.com/mydata`. So you will define a field called `sampling`, ask the user to enter a value and then this parameter is going to be added to the URL of your dataset as query parameters: `https://example.com/mydata?sampling=10`
- Before running an algorithm, you need to know how many iterations should it perform. You define a field called `iterations`, ask the user to enter a value and this parameter is stored in a specific location in your Computer-to-Data pod for the algorithm to read and use that value.
It's an array of elements, each element object defines a field.
An element looks like:
```json
{
"name":"hometown",
"type": "text",
"label": "Hometown",
"required": true,
"description":"What is your hometown?",
"default": "Nowhere",
"options": []
}
```
where:
- name = defines the parameter name (this is sent as HTTP param or key towards algo)
- type = defines the form type (text, number, select, boolean)
- label = defines the label which is displayed
- required = if this field is mandatory to have a consumer input.
- default = default value
- description = description of this element
- options = for select types, a list of options
Example:
```json
[
{
"name":"hometown",
"type": "text",
"label": "Hometown",
"required": true,
"default": "Nowhere"
"description":"What is your hometown?"
},
{
"name":"age",
"type": "number",
"label": "Age",
"required": false,
"default": 0
"description":"Please fill your age"
},
{
"name":"developer",
"type": "boolean",
"label": "Developer",
"required": false,
"default": false
"description":"Are you a developer?"
},
{
"name":"preference",
"type": "select",
"label": "Date",
"required": false,
"default": "nodejs"
"options": [
{
"nodejs" : "I love NodeJs"
},
{
"python" : "I love Python"
}
],
"description": "Do you like NodeJs or Python"
},
]
```
Algorithms will have access to a json file located at /data/inputs/algoCustomData.json, which contains the keys/values for input data required. Example:
```json
{
"surname": "John",
"age": 10,
"developer": false,
"preference": "nodejs"
}
```
### Credentials
By default, a consumer can access a resource if they have 1 datatoken. _Credentials_ allow the publisher to optionally specify more fine-grained permissions.
Consider a medical data use case, where only a credentialed EU researcher can legally access a given dataset. Ocean supports this as follows: a consumer can only access the resource if they have 1 datatoken _and_ one of the specified `"allow"` credentials.
This is like going to an R-rated movie, where you can only get in if you show both your movie ticket (datatoken) _and_ some identification showing you're old enough (credential).
Only credentials that can be proven are supported. This includes Ethereum public addresses, and in the future [W3C Verifiable Credentials](https://www.w3.org/TR/vc-data-model/) and more.
Ocean also supports `"deny"` credentials: if a consumer has any of these credentials, they can not access the resource.
Here's an example object with both `"allow"` and `"deny"` entries:
```json
{
"credentials": {
"allow": [
{
"type": "address",
"values": ["0x123", "0x456"]
}
],
"deny": [
{
"type": "address",
"values": ["0x2222", "0x333"]
}
]
}
}
```
### DDO Checksum
In order to ensure the integrity of the DDO, a checksum is computed for each DDO:
```js
const checksum = sha256(JSON.stringify(ddo))
```
The checksum hash is used when publishing/updating metadata using the `setMetaData` function in the ERC721 contract, and is stored in the event generated by the ERC721 contract:
```solidity
event MetadataCreated(
address indexed createdBy,
uint8 state,
string decryptorUrl,
bytes flags,
bytes data,
bytes metaDataHash,
uint256 timestamp,
uint256 blockNumber
);
event MetadataUpdated(
address indexed updatedBy,
uint8 state,
string decryptorUrl,
bytes flags,
bytes data,
bytes metaDataHash,
uint256 timestamp,
uint256 blockNumber
);
```
_Aquarius_ should always verify the checksum after data is decrypted via a _Provider_ API call.
### State
Each asset has a state, which is held by the NFT contract. The possible states are:
| State | Description |
| ------- | ------------------------------- |
| **`0`** | Active. |
| **`1`** | End-of-life. |
| **`2`** | Deprecated (by another asset). |
| **`3`** | Revoked by publisher. |
| **`4`** | Ordering is temporary disabled. |
## Aquarius Enhanced DDO Response
The following fields are added by _Aquarius_ in its DDO response for convenience reasons, where an asset returned by _Aquarius_ inherits the DDO fields stored on-chain.
These additional fields are never stored on-chain, and are never taken into consideration when [hashing the DDO](#ddo-hash).
### NFT
The `nft` object contains information about the ERC721 NFT contract which represents the intellectual property of the publisher.
| Attribute | Type | Description |
| ------------- | ---------------------- | ------------------------------------------------------------------------- |
| **`address`** | `string` | Contract address of the deployed ERC721 NFT contract. |
| **`name`** | `string` | Name of NFT set in contract. |
| **`symbol`** | `string` | Symbol of NFT set in contract. |
| **`owner`** | `string` | ETH account address of the NFT owner. |
| **`state`** | `number` | State of the asset reflecting the NFT contract value. See [State](#state) |
| **`created`** | `ISO date/time string` | Contains the date of NFT creation. |
| **`tokenURI`** | `string` | tokenURI |
Example:
```json
{
"nft": {
"address": "0x000000",
"name": "Ocean Protocol Asset v4",
"symbol": "OCEAN-A-v4",
"owner": "0x0000000",
"state": 0,
"created": "2000-10-31T01:30:00Z"
}
}
```
### Datatokens
The `datatokens` array contains information about the ERC20 datatokens attached to [asset services](#services).
| Attribute | Type | Description |
| --------------- | -------- | ------------------------------------------------ |
| **`address`** | `string` | Contract address of the deployed ERC20 contract. |
| **`name`** | `string` | Name of NFT set in contract. |
| **`symbol`** | `string` | Symbol of NFT set in contract. |
| **`serviceId`** | `string` | ID of the service the datatoken is attached to. |
Example:
```json
{
"datatokens": [
{
"address": "0x000000",
"name": "Datatoken 1",
"symbol": "DT-1",
"serviceId": "1"
},
{
"address": "0x000001",
"name": "Datatoken 2",
"symbol": "DT-2",
"serviceId": "2"
}
]
}
```
### Event
The `event` section contains information about the last transaction that created or updated the DDO.
Example:
```json
{
"event": {
"tx": "0x8d127de58509be5dfac600792ad24cc9164921571d168bff2f123c7f1cb4b11c",
"block": 12831214,
"from": "0xAcca11dbeD4F863Bb3bC2336D3CE5BAC52aa1f83",
"contract": "0x1a4b70d8c9DcA47cD6D0Fb3c52BB8634CA1C0Fdf",
"datetime": "2000-10-31T01:30:00"
}
}
```
### Purgatory
Contains information about an asset's purgatory status defined in [`list-purgatory`](https://github.com/oceanprotocol/list-purgatory). Marketplace interfaces are encouraged to prevent certain user actions like adding liquidity on assets in purgatory.
| Attribute | Type | Description |
| ------------ | --------- | --------------------------------------------------------------------------------------------- |
| **`state`** | `boolean` | If `true`, asset is in purgatory. |
| **`reason`** | `string` | If asset is in purgatory, contains the reason for being there as defined in `list-purgatory`. |
Example:
```json
{
"purgatory": {
"state": true,
"reason": "Copyright violation"
}
}
```
```json
{
"purgatory": {
"state": false
}
}
```
### Statistics
The `stats` section contains different statistics fields.
| Attribute | Type | Description |
| -------------- | -------- | ------------------------------------------------------------------------------------------------------------- |
| **`orders`** | `number` | How often an asset was ordered, meaning how often it was either downloaded or used as part of a compute job. |
Example:
```json
{
"stats": {
"orders": 4
}
}
```
## Full Enhanced DDO Example
```json
{
"@context": ["https://w3id.org/did/v1"],
"id": "did:op:ACce67694eD2848dd683c651Dab7Af823b7dd123",
"version": "4.0.0",
"chainId": 1,
"nftAddress": "0x123",
"metadata": {
"created": "2020-11-15T12:27:48Z",
"updated": "2021-05-17T21:58:02Z",
"description": "Sample description",
"name": "Sample asset",
"type": "dataset",
"author": "OPF",
"license": "https://market.oceanprotocol.com/terms"
},
"services": [
{
"id": "1",
"type": "access",
"files": "0x044736da6dae39889ff570c34540f24e5e084f4e5bd81eff3691b729c2dd1465ae8292fc721e9d4b1f10f56ce12036c9d149a4dab454b0795bd3ef8b7722c6001e0becdad5caeb2005859642284ef6a546c7ed76f8b350480691f0f6c6dfdda6c1e4d50ee90e83ce3cb3ca0a1a5a2544e10daa6637893f4276bb8d7301eb35306ece50f61ca34dcab550b48181ec81673953d4eaa4b5f19a45c0e9db4cd9729696f16dd05e0edb460623c843a263291ebe757c1eb3435bb529cc19023e0f49db66ef781ca692655992ea2ca7351ac2882bf340c9d9cb523b0cbcd483731dc03f6251597856afa9a68a1e0da698cfc8e81824a69d92b108023666ee35de4a229ad7e1cfa9be9946db2d909735",
"name": "Download service",
"description": "Download service",
"datatokenAddress": "0x123",
"serviceEndpoint": "https://myprovider.com",
"timeout": 0,
"consumerParameters": [
{
"name":"surname",
"type": "text",
"label": "Name",
"required": true,
"default": "NoName"
"description":"Please fill your name"
},
{
"name":"age",
"type": "number",
"label": "Age",
"required": false,
"default": 0
"description":"Please fill your age"
}]
},
{
"id": "2",
"type": "compute",
"files": "0x044736da6dae39889ff570c34540f24e5e084f4e5bd81eff3691b729c2dd1465ae8292fc721e9d4b1f10f56ce12036c9d149a4dab454b0795bd3ef8b7722c6001e0becdad5caeb2005859642284ef6a546c7ed76f8b350480691f0f6c6dfdda6c1e4d50ee90e83ce3cb3ca0a1a5a2544e10daa6637893f4276bb8d7301eb35306ece50f61ca34dcab550b48181ec81673953d4eaa4b5f19a45c0e9db4cd9729696f16dd05e0edb460623c843a263291ebe757c1eb3435bb529cc19023e0f49db66ef781ca692655992ea2ca7351ac2882bf340c9d9cb523b0cbcd483731dc03f6251597856afa9a68a1e0da698cfc8e81824a69d92b108023666ee35de4a229ad7e1cfa9be9946db2d909735",
"name": "Compute service",
"description": "Compute service",
"datatokenAddress": "0x124",
"serviceEndpoint": "https://myprovider.com",
"timeout": 3600,
"compute": {
"allowRawAlgorithm": false,
"allowNetworkAccess": true,
"publisherTrustedAlgorithmPublishers": ["0x234", "0x235"],
"publisherTrustedAlgorithms": [
{
"did": "did:op:123",
"filesChecksum": "100",
"containerSectionChecksum": "200"
},
{
"did": "did:op:124",
"filesChecksum": "110",
"containerSectionChecksum": "210"
}
]
}
}
],
"credentials": {
"allow": [
{
"type": "address",
"values": ["0x123", "0x456"]
}
],
"deny": [
{
"type": "address",
"values": ["0x2222", "0x333"]
}
]
},
"nft": {
"address": "0x123",
"name": "Ocean Protocol Asset v4",
"symbol": "OCEAN-A-v4",
"owner": "0x0000000",
"state": 0,
"created": "2000-10-31T01:30:00",
"tokenURI": "xxx"
},
"datatokens": [
{
"address": "0x000000",
"name": "Datatoken 1",
"symbol": "DT-1",
"serviceId": "1"
},
{
"address": "0x000001",
"name": "Datatoken 2",
"symbol": "DT-2",
"serviceId": "2"
}
],
"event": {
"tx": "0x8d127de58509be5dfac600792ad24cc9164921571d168bff2f123c7f1cb4b11c",
"block": 12831214,
"from": "0xAcca11dbeD4F863Bb3bC2336D3CE5BAC52aa1f83",
"contract": "0x1a4b70d8c9DcA47cD6D0Fb3c52BB8634CA1C0Fdf",
"datetime": "2000-10-31T01:30:00"
},
"purgatory": {
"state": false
},
"stats": {
"orders": 4
}
}
```

Binary file not shown.

Before

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 102 KiB

View File

@ -6,27 +6,20 @@
link: /concepts/quickstart/
- title: Architecture Overview
link: /concepts/architecture/
- title: DIDs & DDOs
link: /concepts/did-ddo/
- title: Supported Networks
link: /concepts/networks/
- title: Deployments
link: /concepts/deployments/
- title: Projects using Ocean
link: /concepts/projects-using-ocean/
- group: Compute-to-Data
items:
- title: Introduction
- title: Overview
link: /concepts/compute-to-data/
- group: Specifying Assets
items:
- title: DIDs & DDOs
link: /concepts/did-ddo/
- title: DDO Metadata
link: /concepts/ddo-metadata/
- group: Contribute
items:
- title: Projects using Ocean
link: /concepts/projects-using-ocean/
- title: Ways to Contribute
link: /concepts/contributing/
- title: Get Funding

View File

@ -118,7 +118,7 @@ export const pageQuery = graphql`
query DocBySlug($slug: String!) {
markdownRemark(fields: { slug: { eq: $slug } }) {
id
tableOfContents(maxDepth: 2)
tableOfContents(maxDepth: 3)
html
htmlAst
frontmatter {