mirror of https://github.com/oceanprotocol/docs.git synced 2024-11-26 19:49:26 +01:00

GitBook: [#7 ] User better quality images

2022-08-01 09:12:56 +00:00

68 KiB

Raw Blame History

title	slug	section	description
DID & DDO	/concepts/did-ddo/	concepts	Specification of decentralized identifiers for assets in Ocean Protocol using the DID & DDO standards.

DID & DDO

v4.1.0

Overview

This document describes how Ocean assets follow the DID/DDO specification, such that Ocean assets can inherit DID/DDO benefits and enhance interoperability. DIDs and DDOs follow the specification defined by the World Wide Web Consortium (W3C).

Decentralized identifiers (DIDs) are a type of identifier that enable verifiable, decentralized digital identity. Each DID is associated with a unique entity, and DIDs may represent humans, objects, and more.

A DID Document (DDO) is a JSON blob that holds information about the DID. Given a DID, a resolver will return the DDO of that DID.

Rules for DID & DDO

An asset in Ocean represents a downloadable file, compute service, or similar. Each asset is a resource under the control of a publisher. The Ocean network itself does not store the actual resource (e.g. files).

An asset has a DID and DDO. The DDO should include metadata about the asset, and define access in at least one service. Only owners or delegated users can modify the DDO.

All DDOs are stored on-chain in encrypted form to be fully GDPR-compatible. A metadata cache like Aquarius can help in reading, decrypting, and searching through encrypted DDO data from the chain. Because the file URLs are encrypted on top of the full DDO encryption, returning unencrypted DDOs e.g. via an API is safe to do as the file URLs will still stay encrypted.

Publishing & Retrieving DDOs

The DDO is stored on-chain as part of the NFT contract and stored in encrypted form using the private key of the Provider. To resolve it, a metadata cache like Aquarius must query the provider to decrypt the DDO.

Here is the flow:

UML source

title DDO flow

User(Ocean library) -> User(Ocean library): Prepare DDO
User(Ocean library) -> Provider: encrypt DDO
Provider -> User(Ocean library): encryptedDDO
User(Ocean library) -> ERC721 contract: publish encryptedDDO
Aquarius <-> ERC721 contract: monitors ERC721 contract and gets MetdadataCreated Event (contains encryptedDDO)
Aquarius -> ERC721 contract: calls getMetaData()
Aquarius -> Provider: decrypt encryptedDDO, signed request using Aquarius's private key
Provider -> ERC721 contract: checks state using getMetaData()
Provider -> Provider: depending on metadataState (expired,retired) and aquarius address, validates the request
Provider -> Aquarius: DDO
Aquarius -> Aquarius : validate DDO
Aquarius -> Aquarius : cache DDO
Aquarius -> Aquarius : enhance cached DDO in response with additional infos like events & stats

DID

In Ocean, a DID is a string that looks like this:

did:op:0ebed8226ada17fde24b6bf2b95d27f8f05fcce09139ff5cec31f6d81a7cd2ea

The part after did:op: is the ERC721 contract address(in checksum format) and the chainId (expressed as a decimal) the asset has been published to:

const checksum = sha256(ERC721 contract address + chainId)
console.log(checksum)
// 0ebed8226ada17fde24b6bf2b95d27f8f05fcce09139ff5cec31f6d81a7cd2ea

It follows the generic DID scheme.

DDO

A DDO in Ocean has these required attributes:

Attribute	Type	Description
`@context`	Array of `string`	Contexts used for validation.
`id`	`string`	Computed as `sha256(address of ERC721 contract + chainId)`.
`version`	`string`	Version information in SemVer notation referring to this DDO spec version, like `4.1.0`.
`chainId`	`number`	Stores chainId of the network the DDO was published to.
`nftAddress`	`string`	NFT contract linked to this asset
`metadata`	Metadata	Stores an object describing the asset.
`services`	Services	Stores an array of services defining access to the asset.
`credentials`	Credentials	Describes the credentials needed to access a dataset in addition to the `services` definition.

Metadata

This object holds information describing the actual asset.

Attribute	Type	Required	Description
`created`	`ISO date/time string`		Contains the date of the creation of the dataset content in ISO 8601 format preferably with timezone designators, e.g. `2000-10-31T01:30:00Z`.
`updated`	`ISO date/time string`		Contains the date of last update of the dataset content in ISO 8601 format preferably with timezone designators, e.g. `2000-10-31T01:30:00Z`.
`description`	`string`	✓	Details of what the resource is. For a dataset, this attribute explains what the data represents and what it can be used for.
`copyrightHolder`	`string`		The party holding the legal copyright. Empty by default.
`name`	`string`	✓	Descriptive name or title of the asset.
`type`	`string`	✓	Asset type. Includes `"dataset"` (e.g. csv file), `"algorithm"` (e.g. Python script). Each type needs a different subset of metadata attributes.
`author`	`string`	✓	Name of the entity generating this data (e.g. Tfl, Disney Corp, etc.).
`license`	`string`	✓	Short name referencing the license of the asset (e.g. Public Domain, CC-0, CC-BY, No License Specified, etc. ). If it's not specified, the following value will be added: "No License Specified".
`links`	Array of `string`		Mapping of URL strings for data samples, or links to find out more information. Links may be to either a URL or another asset.
`contentLanguage`	`string`		The language of the content. Use one of the language codes from the IETF BCP 47 standard
`tags`	Array of `string`		Array of keywords or tags used to describe this content. Empty by default.
`categories`	Array of `string`		Array of categories associated to the asset. Note: recommended to use `tags` instead of this.
`additionalInformation`	Object		Stores additional information, this is customizable by publisher
`algorithm`	Algorithm Metadata	✓ (for algorithm assets only)	Information about asset of `type` `algorithm`

Example:

{
  "metadata": {
    "created": "2020-11-15T12:27:48Z",
    "updated": "2021-05-17T21:58:02Z",
    "description": "Sample description",
    "name": "Sample asset",
    "type": "dataset",
    "author": "OPF",
    "license": "https://market.oceanprotocol.com/terms"
  }
}

Algorithm Metadata

An asset of type algorithm has additional attributes under metadata.algorithm, describing the algorithm and the Docker environment it is supposed to be run under.

Attribute	Type	Required	Description
`language`	`string`		Language used to implement the software.
`version`	`string`		Version of the software preferably in SemVer notation. E.g. `1.0.0`.
`consumerParameters`	Consumer Parameters		An object that defines required consumer input before running the algorithm
`container`	`container`	✓	Object describing the Docker container image. See below

The container object has the following attributes defining the Docker image for running the algorithm:

Attribute	Type	Required	Description
`entrypoint`	`string`	✓	The command to execute, or script to run inside the Docker image.
`image`	`string`	✓	Name of the Docker image.
`tag`	`string`	✓	Tag of the Docker image.
`checksum`	`string`	✓	Digest of the Docker image. (ie: sha256:xxxxx)

{
  "metadata": {
    "created": "2020-11-15T12:27:48Z",
    "updated": "2021-05-17T21:58:02Z",
    "description": "Sample description",
    "name": "Sample algorithm asset",
    "type": "algorithm",
    "author": "OPF",
    "license": "https://market.oceanprotocol.com/terms",
    "algorithm": {
      "language": "Node.js",
      "version": "1.0.0",
      "container": {
        "entrypoint": "node $ALGO",
        "image": "ubuntu",
        "tag": "latest",
        "checksum": "sha256:44e10daa6637893f4276bb8d7301eb35306ece50f61ca34dcab550"
      },
      "consumerParameters": {}
    }
  }
}

Services

Services define the access for an asset, and each service is represented by its respective datatoken.

An asset should have at least one service to be actually accessible, and can have as many services which make sense for a specific use case.

Attribute	Type	Required	Description
`id`	`string`	✓	Unique ID
`type`	`string`	✓	Type of service (`access`, `compute`, `wss`, etc.
`name`	`string`		Service friendly name
`description`	`string`		Service description
`datatokenAddress`	`string`	✓	Datatoken address
`serviceEndpoint`	`string`	✓	Provider URL (schema + host)
`files`	Files	✓	Encrypted file URLs.
`timeout`	`number`	✓	Describing how long the service can be used after consumption is initiated. A timeout of `0` represents no time limit. Expressed in seconds.
`compute`	Compute	✓ (for compute assets only)	If service is of `type` `compute`, holds information about the compute-related privacy settings & resources.
`consumerParameters`	Consumer Parameters		An object the defines required consumer input before consuming the asset
`additionalInformation`	Object		Stores additional information, this is customizable by publisher

Files

The files field is returned as a string which holds the encrypted file URLs.

Example:

{
  "files": "0x044736da6dae39889ff570c34540f24e5e084f4e5bd81eff3691b729c2dd1465ae8292fc721e9d4b1f10f56ce12036c9d149a4dab454b0795bd3ef8b7722c6001e0becdad5caeb2005859642284ef6a546c7ed76f8b350480691f0f6c6dfdda6c1e4d50ee90e83ce3cb3ca0a1a5a2544e10daa6637893f4276bb8d7301eb35306ece50f61ca34dcab550b48181ec81673953d4eaa4b5f19a45c0e9db4cd9729696f16dd05e0edb460623c843a263291ebe757c1eb3435bb529cc19023e0f49db66ef781ca692655992ea2ca7351ac2882bf340c9d9cb523b0cbcd483731dc03f6251597856afa9a68a1e0da698cfc8e81824a69d92b108023666ee35de4a229ad7e1cfa9be9946db2d909735"
}

During the publish process, file URLs must be encrypted with a respective Provider API call before storing the DDO on-chain. For this, you need to send the following object to Provider:

{
  "datatokenAddress":"0x1",
  "nftAddress": "0x2",
  "files": [
    ...
  ]
}

where "files" contains one or more storage objects.

Type of objects supported :

Type	Description	Example
`url`	Static URL. Contains url and HTTP method	`[`
{

"type": "url",
"url": "https://url.com/file1.csv",
"method": "GET",
"headers":
  [
   {"Authorization": "Bearer 123"}, 
   {"APIKEY": "124"},
  ]

} ] |

First class integrations supported in the future :

Type	Description	Example
`ipfs`	IPFS files	`[`
{

"type": "ipfs",
"hash": "XXX"

} ] | | filecoin | Filecoin storage | | | arwave | Arwave | | | storj | Storj | | | sql | Sql connection, dataset is generated by a query | |

A service can contain multiple files, using multiple storage types.

Example:

{
  "datatokenAddress":"0x1",
  "nftAddress": "0x2",
  "files": [
    {
      "type": "url",
      "url": "https://url.com/file1.csv",
      "method": "GET"
    },
    {
      "type": "ipfs",
      "hash": "XXXX"
    }
  ]
}

To get information about the files after encryption, the /fileinfo endpoint of Provider returns based on a passed DID an array of file metadata (based on the file type):

[
  {
    "type": "url",
    "contentLength": 100,
    "contentType": "application/json"
  },
  {
    "type": "ipfs",
    "contentLength": 130,
    "contentType": "application/text"
  }
]

This only concerns metadata about a file, but never the file URLs. The only way to decrypt them is to exchange at least 1 datatoken based on the respective service pricing scheme.

Compute Options

An asset with a service of type compute has the following additional attributes under the compute object. This object is required if the asset is of type compute, but can be omitted for type of access.

allowRawAlgorithm

Type	Required	Description
`boolean`	✓	If `true`, any passed raw text will be allowed to run. Useful for an algorithm drag & drop use case, but increases risk of data escape through malicious user input. Should be `false` by default in all implementations.

Type Required Description

boolean ✓ If true, any passed raw text will be allowed to run. Useful for an algorithm drag & drop use case, but increases risk of data escape through malicious user input. Should be false by default in all implementations.

Type Required Description

allowNetworkAccess

Type	Required	Description
`boolean`	✓	If `true`, the algorithm job will have network access.

Type Required Description