The Uploader Docs (#1256)

* Adding initial page as a user guide on interacting witht the DBS

* Updating steps and instructions for the DBS user guide

* updating the navigation

* updating arweave user guide

* Adding links to the dbs upload instructions

* Adding images

* Adding images into the guide

* CHanging DBS to Uploader in user guide

* moved uploader (ex dbs) pages from gitbook

* remove all refs to dbs

* change DBSClient to UploaderClient

* removed more dbs refs

* Update README.md

- adjust some links
- fix some texts

* updated images uploader

* adding format to codeblocks

* updated images

* added uploader to asset hosting

* fix some text

* fix typo

* fix link

* replaced more images

* added text to asset hosting section related to uploader

* changed "Using Arweave with Uploader"

* removed references to "storage management"

* added missing link

* move uploader.js section from main page

* added to uploader's main readme

* rewrite uploader ui docs

* added expandable component to api calls in uploader.js

* rewrite API main paragraph in uploader.js

* fix some dbs refs

* added more to uploader ui

* moved uploader in the summary page

* added arcade to "How to store an asset on Arweave with the Uploader UI"

* added uploader links

* Update uploader.md

update arcade link

* Update uploader.md

* Updating diagrams

* Update to mention IPFS support

* Minor change in README.md

---------

Co-authored-by: Lcdo-Jeremy <enzo-vezzaro@live.it>
Co-authored-by: Ana Loznianu <ana@oceanprotocol.com>
Co-authored-by: mihaisc <mihai@oceanprotocol.com>

SUMMARY.md

@@ -23,6 +23,7 @@
     - [Publish a C2D Algorithm NFT](user-guides/compute-to-data/publish-a-c2d-algorithm-nft.md)
     - [Publish a C2D Data NFT](user-guides/compute-to-data/publish-a-c2d-data-nft.md)
   - [Host Assets](user-guides/asset-hosting/README.md)
+    - [Uploader](user-guides/asset-hosting/uploader.md)
     - [Arweave](user-guides/asset-hosting/arweave.md)
     - [AWS](user-guides/asset-hosting/aws.md)
     - [Azure Cloud](user-guides/asset-hosting/azure-cloud.md)
@@ -93,6 +94,10 @@
     - [Encryption / Decryption](developers/provider/encryption-decryption.md)
     - [Compute Endpoints](developers/provider/compute-endpoints.md)
     - [Authentication Endpoints](developers/provider/authentication-endpoints.md)
+  - [Uploader](developers/uploader/README.md)
+    - [Uploader.js](developers/uploader/uploader-js.md)
+    - [Uploader UI](developers/uploader/uploader-ui.md)
+    - [Uploader UI to Market](developers/uploader/uploader-ui-marketplace.md)
   - [Developer FAQ](developers/dev-faq.md)
 - [📊 Data Scientists](data-scientists/README.md)
   - [Ocean.py](data-scientists/ocean.py/README.md)

developers/uploader/README.md

@@ -0,0 +1,74 @@
+# Uploader
+
+### What's Uploader?
+
+The Uploader represents a cutting-edge solution designed to streamline the upload process within a decentralized network. Built with efficiency and scalability in mind, Uploader leverages advanced technologies to provide secure, reliable, and cost-effective storage solutions to users.
+
+### Architecture Overview
+
+Uploader is built on a robust architecture that seamlessly integrates various components to ensure optimal performance. The architecture consists of:
+
+- Uploader API Layer: Exposes both public and private APIs for frontend and microservices interactions, respectively.
+- 1-N Storage Microservices: Multiple microservices, each specializing in different storage types, responsible for handling storage operations.
+- IPFS Integration: Temporary storage using the InterPlanetary File System (IPFS).
+
+### Streamlined File Uploads
+
+Uploader streamlines the file uploading process, providing users with a seamless experience to effortlessly incorporate their digital assets into a decentralized network. Whether you're uploading images, documents, or other media, Uploader enhances accessibility and ease of use, fostering a more decentralized and inclusive digital landscape.
+
+### Unique Identifiers
+
+Obtain unique identifiers such as hashes or CIDs for your uploaded files. These unique identifiers play a pivotal role in enabling efficient tracking and interaction with decentralized assets. By obtaining these identifiers, users gain a crucial toolset for managing, verifying, and engaging with their digital assets on the decentralized network, ensuring a robust and secure mechanism for overseeing the lifecycle of their contributed files.
+
+### Features
+
+Uploader offers a range of powerful features tailored to meet the needs of any decentralized storage:
+
+- User Content Uploads: Users can seamlessly upload their content through the user-friendly frontend interface.
+- Payment Handling: Uploader integrates with payment systems to manage the financial aspects of storage services.
+- Decentralized Storage: Content is pushed to decentralized storage networks like Filecoin and Arweave for enhanced security and redundancy.
+- API Documentation: Comprehensive API documentation on each repo to allow users to understand and interact with the system effortlessly.
+- Uploader.js: a TypeScript library designed to simplify interaction with the Uploader API. This library provides a user-friendly and intuitive interface for calling API endpoints within the Uploader Storage system.
+
+### Components
+
+- [Uploader](https://github.com/oceanprotocol/decentralized_storage_backend)
+- [Uploader.js](https://github.com/oceanprotocol/uploader.js)
+- [Uploader UI](https://github.com/oceanprotocol/uploader-ui-lib)
+
+### Microservices:
+
+- [Filecoin](https://github.com/oceanprotocol/uploader_filecoin) (WIP)
+- [Arweave](https://github.com/oceanprotocol/uploader_arweave)
+
+### User Workflow
+
+Uploader simplifies the user workflow, allowing for easy management of storage operations:
+
+- Users fetch available storage types and payment options from the frontend.
+- Quotes for storing files on the Microservice network.
+- Files are uploaded from the frontend to Uploader, which handles temporary storage via IPFS.
+- The Microservice takes over, ensuring data is stored on the selected network securely.
+- Users can monitor upload status and retrieve links to access their stored content.
+
+#### File storage flow
+
+<figure><img src="../../.gitbook/assets/uploader/Diagrams_Upload_file_flow.png" alt=""><figcaption><p>Ocean Uploader - storage flow 1</p></figcaption></figure>
+
+#### File retrieval flow
+
+<figure><img src="../../.gitbook/assets/uploader/Diagrams_Retrieve_file_flow.png" alt=""><figcaption><p>Ocean Uploader - storage flow 1</p></figcaption></figure>
+
+### API Documentation
+
+Documentation is provided in the repos to facilitate seamless integration and interaction with the Uploader. The documentation outlines all API endpoints, payload formats, and example use cases, empowering developers to effectively harness the capabilities of the Uploader solution.
+
+### Troubleshooting
+
+Did you encounter a problem? Open an issue in Ocean Protocol's repos:
+
+- [Uploader](https://github.com/oceanprotocol/decentralized_storage_backend/issues)
+- [Uploader.js](https://github.com/oceanprotocol/uploader.js/issues)
+- [Filecoin Microservice](https://github.com/oceanprotocol/uploader_filecoin/issues)
+- [Arweave Microservice](https://github.com/oceanprotocol/uploader_arweave/issues)
+- [Uploader UI Library](https://github.com/oceanprotocol/uploader-ui-lib/issues)

developers/uploader/uploader-js.md

@@ -0,0 +1,154 @@
+# Uploader.js
+
+<a href="https://github.com/oceanprotocol/uploader.js" target="_blank">Uploader.js</a> is a robust TypeScript library that serves as a vital bridge to interact with the Ocean Uploader API. It simplifies the process of managing file storage uploads, obtaining quotes, and more within the Ocean Protocol ecosystem. This library offers developers a straightforward and efficient way to access the full range of Uploader API endpoints, facilitating seamless integration of decentralized storage capabilities into their applications. 
+
+Whether you're building a decentralized marketplace, a content management system, or any application that involves handling digital assets, Uploader.js provides a powerful toolset to streamline your development process and enhance your users' experience.
+
+### Browser Usage
+
+Ensure that the Signer object (signer in this case) you're passing to the function when you call it from the browser is properly initialized and is compatible with the browser. For instance, if you're using something like MetaMask as your Ethereum provider in the browser, you'd typically use the ethers.Web3Provider to generate a signer.
+
+### How to Safely Store Your Precious Files with Ocean Uploader Magic 🌊✨
+
+Excited to get your files safely stored? Let's breeze through the process using Ocean Uploader. First things first, install the package with npm or yarn:
+
+```bash
+npm install @oceanprotocol/uploader
+
+```bash
+yarn add @oceanprotocol/uploader
+```
+
+or
+
+```bash
+yarn add @oceanprotocol/uploader
+```
+
+Got that done? Awesome! Now, let's dive into a bit of TypeScript:
+
+```typescript
+import { ethers } from 'ethers';
+import {
+  UploaderClient,
+  GetQuoteArgs,
+  GetQuoteResult
+} from '@oceanprotocol/uploader';
+import dotenv from 'dotenv';
+
+dotenv.config();
+
+// Set up a new instance of the Uploader client
+const signer = new ethers.Wallet(process.env.PRIVATE_KEY);
+const client = new UploaderClient(process.env.UPLOADER_URL, process.env.UPLOADER_ACCOUNT, signer);
+
+async function uploadAsset() {
+  // Get storage info
+  const info = await client.getStorageInfo();
+
+  // Fetch a quote using the local file path
+  const quoteArgs: GetQuoteArgs = {
+    type: info[0].type,
+    duration: 4353545453,
+    payment: {
+      chainId: info[0].payment[0].chainId,
+      tokenAddress: info[0].payment[0].acceptedTokens[0].value
+    },
+    userAddress: process.env.USER_ADDRESS,
+    filePath: ['/home/username/ocean/test1.txt']  // example file path
+  };
+  const quoteResult: GetQuoteResult = await client.getQuote(quoteArgs);
+
+  // Upload the file using the returned quote
+  await client.upload(quoteResult.quoteId, quoteArgs.filePath);
+  console.log('Files uploaded successfully.');
+}
+
+uploadAsset().catch(console.error);
+
+```
+
+There you go! That's all it takes to upload your files using Uploader.js. Easy, right? Now go ahead and get those files stored securely. You got this! 🌟💾
+
+For additional details, please visit the [Uploader.js](https://github.com/oceanprotocol/uploader.js) repository.
+
+### API
+
+The library offers developers a versatile array of methods designed for seamless interaction with the Ocean Uploader API. These methods collectively empower developers to utilize Ocean's decentralized infrastructure for their own projects:
+
+<details>
+  <summary>
+      constructor(baseURL: string)
+  </summary>
+  
+  ```
+  Create a new instance of the UploaderClient.
+  ```
+</details>
+<details>
+  <summary>
+      getStorageInfo()
+  </summary>
+  
+  ```
+  Fetch information about supported storage types and payments.
+  ```
+</details>
+<details>
+  <summary>
+      getQuote(args: GetQuoteArgs)
+  </summary>
+
+  ```
+  Fetch a quote for storing files on a specific storage.
+  ```
+</details>
+<details>
+  <summary>
+      upload(quoteId: string, nonce: number, signature: string, files: File[])
+  </summary>
+
+  ```
+  Upload files according to the quote request.
+  ```
+</details>
+<details>
+  <summary>
+      getStatus(quoteId: string)
+  </summary>
+
+  ```
+  Fetch the status of an asset during upload.
+  ```
+</details>
+<details>
+  <summary>
+      getLink(quoteId: string, nonce: number, signature: string)
+  </summary>
+
+  ```
+  Fetch hash reference for the asset. For example: CID for Filecoin, Transaction Hash for Arweave.
+  ```
+</details>
+<details>
+  <summary>
+      registerMicroservice(args: RegisterArgs)
+  </summary>
+
+  ```
+  Register a new microservice that handles a storage type.
+  ```
+</details>
+<details>
+  <summary>
+      getHistory(page: number = 1, pageSize: number = 25)
+  </summary>
+
+  ```
+  Retrieves the quote history for the given user address, nonce, and signature.
+  ```
+</details>
+<br />
+Whether you're a developer looking to integrate Ocean Uploader into your application or a contributor interested in enhancing this TypeScript library, we welcome your involvement. By following the <a href="https://github.com/oceanprotocol/uploader.js" target="_blank">provided documentation</a>, you can harness the capabilities of Uploader.js to make the most of decentralized file storage in your projects. 
+
+Feel free to explore the API reference, contribute to the library's development, and become a part of the Ocean Protocol community's mission to democratize data access and storage.

developers/uploader/uploader-ui-marketplace.md

@@ -0,0 +1,27 @@
+---
+description: With the Uploader UI, users can effortlessly upload their files and obtain a unique hash or CID (Content Identifier) for each uploaded asset to use on the Marketplace.
+---
+
+# Uploader UI Marketplace
+
+<b>Step 1</b>: Copy the hash or CID from your upload.
+
+<img src="../../.gitbook/assets/uploader/uploader_screen_11.png" alt="" />
+
+<b>Step 2</b>: Open the Ocean Marketplace. Go to publish and fill in all the information for your dataset.
+
+<img src="../../.gitbook/assets/uploader/uploader_screen_12.png" alt="" />
+
+<b>Step 3</b>: When selecting the file to publish, open the hosting provider (e.g. "Arweave" tab)
+
+<img src="../../.gitbook/assets/uploader/uploader_screen_13.png" alt="" />
+
+<b>Step 4</b>: Paste the hash you copied earlier.
+
+<img src="../../.gitbook/assets/uploader/uploader_screen_14.png" alt="" />
+
+<b>Step 5</b>: Click on "VALIDATE" to ensure that your file gets validated correctly.
+
+<img src="../../.gitbook/assets/uploader/uploader_screen_15.png" alt="" />
+
+This feature not only simplifies the process of storing and managing files but also seamlessly integrates with the Ocean Marketplace. Once your file is uploaded via Uploader UI, you can conveniently use the generated hash or CID to interact with your assets on the Ocean Marketplace, streamlining the process of sharing, validating, and trading your digital content.

developers/uploader/uploader-ui.md

@@ -0,0 +1,178 @@
+# Uploader UI
+
+The [Uploader UI](https://github.com/oceanprotocol/uploader-ui-lib) stands as a robust UI react library dedicated to optimizing the uploading, and interaction with digital assets. 
+
+Through an intuitive platform, the tool significantly simplifies the entire process, offering users a seamless experience for uploading files, acquiring unique identifiers such as hashes or CIDs, and effectively managing their decentralized assets. Developed using React, TypeScript, and CSS modules, the library seamlessly connects to Ocean remote components by default, ensuring a cohesive and efficient integration within the ecosystem.
+
+## 🚀 Usage
+
+Integrating [Uploader UI](https://github.com/oceanprotocol/uploader-ui-lib) into your application is straightforward. The package facilitates seamless uploads but requires a wallet connector library to function optimally. Compatible wallet connection choices include [ConnectKit](https://docs.family.co/), [Web3Modal](https://web3modal.com/), [Dynamic](https://dynamic.xyz/) and [RainbowKit](https://www.rainbowkit.com/docs/installation).
+
+**Step 1:** Install the necessary packages. For instance, if you're using ConnectKit, the installation command would be:
+
+```bash
+npm install connectkit @oceanprotocol/uploader-ui-lib
+```
+
+**Step 2:** Incorporate the UploaderComponent from the uploader-ui-lib into your app. It's crucial to ensure the component is nested within both the WagmiConfig and ConnectKit providers. Here's a basic implementation:
+
+```javascript
+import React from 'react'
+import { WagmiConfig, createConfig } from 'wagmi'
+import { polygon } from 'wagmi/chains'
+import {
+  ConnectKitProvider,
+  getDefaultConfig,
+  ConnectKitButton
+} from 'connectkit'
+import UploaderComponent from 'uploader-ui-lib'
+
+export default function App () {
+  // Initialize the Wagmi client
+  const wagmiConfig = createConfig(
+    getDefaultConfig({
+      appName: 'Ocean Uploader UI',
+      infuraId: 'Your infura ID',
+      chains: [polygon],
+      walletConnectProjectId: 'Your wallet connect project ID'
+    })
+  )
+
+  return (
+    <WagmiConfig config={wagmiConfig}>
+      <ConnectKitProvider>
+        {/* Your App */}
+        <ConnectKitButton />
+        <UploaderComponent
+            uploader_url={
+                process.env.NEXT_PUBLIC_UPLOADER_URL ||'https://api.uploader.oceanprotocol.com/'
+            }
+            uploader_account={
+                process.env.NEXT_PUBLIC_UPLOADER_ACCOUNT ||
+                '0x5F8396D1BfDa5259Ee89196F892E4401BF3B596d'
+            }
+        />
+      </ConnectKitProvider>
+    </WagmiConfig>
+  )
+}
+
+```
+
+By following the steps above, you can smoothly incorporate the Uploader UI into your application while ensuring the essential providers wrap the necessary components.
+
+Alternatively, the example below shows how you could use [uploader-ui-lib](https://github.com/oceanprotocol/uploader-ui-lib) with RainbowKit:
+
+```javascript
+import React from 'react'
+import { WagmiConfig, createConfig } from 'wagmi'
+import { polygon } from 'wagmi/chains'
+import { RainbowKitProvider, ConnectButton } from '@rainbow-me/rainbowkit';
+import UploaderComponent from 'uploader-ui-lib'
+
+export default function App () {
+  // Initialize the Wagmi client
+  const wagmiConfig = createConfig(
+    getDefaultConfig({
+      appName: 'Ocean Uploader UI',
+      infuraId: 'Your infura ID',
+      chains: [polygon],
+      walletConnectProjectId: 'Your wallet connect project ID'
+    })
+  )
+
+  return (
+    <WagmiConfig config={wagmiConfig}>
+      <RainbowKitProvider>
+        {/* Your App */}
+        <ConnectButton />
+        <UploaderComponent
+            uploader_url={
+                process.env.NEXT_PUBLIC_UPLOADER_URL ||'https://api.uploader.oceanprotocol.com/'
+            }
+            uploader_account={
+                process.env.NEXT_PUBLIC_UPLOADER_ACCOUNT ||
+                '0x5F8396D1BfDa5259Ee89196F892E4401BF3B596d'
+            }
+        />
+      </RainbowKitProvider>
+    </WagmiConfig>
+  )
+}
+
+```
+
+\*\* under development
+
+## NextJS Setup for Ocean Protocol Uploader UI Library
+
+1. To use Ocean's Uploader UI library in your NextJS project modify your `next.config.js` file to include these fallbacks:
+
+```javascript
+module.exports = {
+  webpack: (config) => {
+    config.resolve.fallback = {
+      fs: false,
+      process: false,
+      net: false,
+      tls: false
+    }
+    return config
+  }
+}
+```
+
+\*\* add these fallbacks to avoid any issue related to webpack 5 Polyfills imcompatibility: https://github.com/webpack/changelog-v5#automatic-nodejs-polyfills-removed
+
+2. Install dependencies:
+
+```bash
+npm install @oceanprotocol/uploader-ui-lib
+```
+
+3. Import the library's CSS into your project:
+
+```javascript
+import '@oceanprotocol/uploader-ui-lib/dist/index.es.css';
+```
+
+4. Dynamically import the Uploader component and ensure it is not processed during server-side rendering (SSR) using the next/dynamic function:
+
+```javascript
+import dynamic from 'next/dynamic';
+...
+
+const Uploader = dynamic(() => import('@oceanprotocol/uploader-ui-lib').then((module) => module.Uploader), { ssr: false });
+```
+
+5. Import component:
+
+```javascript
+<WagmiConfig config={wagmiConfig}>
+    <ConnectKitProvider>
+    <Layout>
+        ...
+        <UploaderConnection
+            uploader_url={
+                process.env.NEXT_PUBLIC_UPLOADER_URL ||'https://api.uploader.oceanprotocol.com/'
+            }
+            uploader_account={
+                process.env.NEXT_PUBLIC_UPLOADER_ACCOUNT ||
+                '0x5F8396D1BfDa5259Ee89196F892E4401BF3B596d'
+            }
+        />
+    </Layout>
+    </ConnectKitProvider>
+</WagmiConfig>
+```
+
+When incorporating the Uploader component into your application, make sure to set 'use client' on top in your app's component. This ensures that the component operates on the client side, bypassing SSR when rendering:
+
+```javascript
+'use client'
+import dynamic from 'next/dynamic'
+```
+
+This comprehensive setup ensures the proper integration and functioning of the Ocean Protocol's Uploader UI library within a NextJS application.
+
+For more details visit the [Uploader UI](https://github.com/oceanprotocol/uploader-ui) project.

user-guides/asset-hosting/arweave.md

@@ -4,6 +4,10 @@ description: How to use decentralized hosting for your NFT assets
 
 # Arweave
 
+### Using Arweave with Uploader
+
+Enhance the efficiency of your file uploads by leveraging the simplicity of the [Ocean Uploader](./Uploader.md) storage system for Arweave. Dive into our comprehensive guide [here](./Uploader.md) to discover detailed steps and tips, ensuring a smooth and hassle-free uploading process. Your experience matters, and we're here to make it as straightforward as possible.
+
 ### Arweave
 
 [Arweave](https://www.arweave.org/) is a global, permanent, and decentralized data storage layer that allows you to store documents and applications forever. Arweave is different from other decentralized storage solutions in that there is only one up-front cost to upload each file.

user-guides/asset-hosting/uploader.md

@@ -0,0 +1,22 @@
+---
+description: How to use Ocean Uploader
+---
+
+# Ocean Uploader
+
+### What is Ocean Uploader? 
+
+Uploader is designed to simplify the process of storing your assets on decentralized networks (such as [arweave](https://www.arweave.org/) and [filecoin](https://filecoin.io/)). It provides access to multiple secure, reliable, and cost-effective storage solutions in an easy-to-use UI and JavaScript library.
+
+### What decentralized storage options are available?
+
+Currently, we support Arweave and  IPFS. We may support other storage options in the future. 
+
+### How to store an asset on Arweave with [Ocean Uploader](https://uploader.oceanprotocol.com/)? 
+
+Ready to dive into the world of decentralized storage with [Ocean Uploader](https://uploader.oceanprotocol.com/)? Let's get started:
+
+{% embed url="https://app.arcade.software/share/88CYjl3SPhTS20qKqBGU" fullWidth="false" %}
+{% endembed %}
+
+Woohoo 🎉 You did it! You now have an IPFS CID for your asset. Pop over to https://ipfs.oceanprotocol.com/ipfs/{CID} to admire your handiwork, you'll be able to access your file at that link. You can use it to publish your asset on [Ocean Market](../../developers/uploader/uploader-ui-marketplace.md).