mirror of
https://github.com/oceanprotocol/docs.git
synced 2024-11-01 15:55:34 +01:00
12 lines
1.5 KiB
Markdown
12 lines
1.5 KiB
Markdown
|
# Data Engineers
|
||
|
|
|
||
|
|
Data engineers are a key part of data value creation. Building any useful dashboards or machine-learning models requires access to curated data. Data engineers help provide this by creating robust data pipelines that ingest data from source systems, conduct transformations to clean and aggregate the data, and then make the data available for downstream use cases  
|
||
|
|
|
||
|
|
\
|
||
|
|
Some examples of useful sources of data can be found below
|
||
|
|
|
||
|
|
* **Government Open Data:** Governments serve as one of the most reliable sources of data, which, although abundant in information, often suffer from inadequate documentation or pose challenges for data scientists to work with effectively. Establishing a robust Extract, Transform, Load (ETL) pipeline to enhance accessibility to such data is crucial.
|
||
|
|
* **Public APIs:** Similarly to government open data, a wide array of freely available public APIs covering various data verticals are at the disposal of data engineers. Leveraging these APIs, data engineers can construct pipelines that enable others to efficiently access and utilize the available data.
|
||
|
|
* **On-Chain Data:** Blockchain data presents an excellent opportunity for data engineers to curate high-quality data. Whether connecting directly to the blockchain or utilizing alternative data providers, simplifying data usability holds significant value. While there is a consistent demand for well-curated decentralized finance (DeFi) data, there is also an emerging need for curated data in other domains, including social data.
|
||
|
|
|