Sign in to confirm you’re not a bot
This helps protect our community. Learn more

Armchair Architects: Is Big Data Turning into Dark Data?

Microsoft Developer
588K subscribers
<__slot-el>
<__slot-el>
1K views 2 years ago #Azure #AzureEnablementShow #ArmchairArchitects
In this episode of the #AzureEnablementShow, Uli and Eric (‪@mougue‬), our #ArmchairArchitects, talk with David about the concept of Dark Data: What is it? What patterns can lead to it? And how can you come back to good data? Uli shares a couple of examples of how business goals dictate how big data is used. Resources • Read Eric's blog post: Is Big Data Turning into Dark Data? https://techcommunity.microsoft.com/t... • Data Lake Analytics https://azure.microsoft.com/en-us/pro... • What is Azure Data Lake Analytics? https://learn.microsoft.com/en-us/azu... • Extract, Transform, and Load (ETL) https://learn.microsoft.com/en-us/azu... • What is Extract, Load, and Transform (ELT)? https://learn.microsoft.com/en-us/azu... • What does it mean to build a single source of truth? https://learn.microsoft.com/en-us/azu... • Blog: thyssenkrupp embraces digital transformation for a better future https://cloudblogs.microsoft.com/indu... Related Epi …...more
...more

Introduction

0:00

What do we mean by “dark data”?

1:07

A data lake becomes a data swamp when the data has no value

2:31

Uli uses Bing to illustrate a point about how the business objective drives how you use big data

3:35

Uli cites another example (thyssenkrupp elevators) of the business objective driving the way that big data was used

5:23

What are some of the patterns, like ETL or ELT, that can lead to dark data?

7:52

Is schema needed if there is AI pattern detection, but no human interaction?

9:52

ETL and ELT defined

11:54

How do you go from good data to bad data and back to good data again?

14:50
Armchair Architects: Is Big Data Turning into Dark Data?
36Likes
1,088Views
2023Jan 18
In this episode of the #AzureEnablementShow, Uli and Eric (‪@mougue‬), our #ArmchairArchitects, talk with David about the concept of Dark Data: What is it? What patterns can lead to it? And how can you come back to good data? Uli shares a couple of examples of how business goals dictate how big data is used. Resources • Read Eric's blog post: Is Big Data Turning into Dark Data? https://techcommunity.microsoft.com/t... • Data Lake Analytics https://azure.microsoft.com/en-us/pro... • What is Azure Data Lake Analytics? https://learn.microsoft.com/en-us/azu... • Extract, Transform, and Load (ETL) https://learn.microsoft.com/en-us/azu... • What is Extract, Load, and Transform (ELT)? https://learn.microsoft.com/en-us/azu... • What does it mean to build a single source of truth? https://learn.microsoft.com/en-us/azu... • Blog: thyssenkrupp embraces digital transformation for a better future https://cloudblogs.microsoft.com/indu... Related Episodes Armchair Architects: How architecture is changing – Machine Learning https://aka.ms/azenable/71 Chapters 0:00 Introduction 1:07 What do we mean by “dark data”? 2:31 A data lake becomes a data swamp when the data has no value 3:35 Uli uses Bing to illustrate a point about how the business objective drives how you use big data 5:23 Uli cites another example (thyssenkrupp elevators) of the business objective driving the way that big data was used 7:52 What are some of the patterns, like ETL or ELT, that can lead to dark data? 9:52 Is schema needed if there is AI pattern detection, but no human interaction? 11:54 ETL and ELT defined 14:50 How do you go from good data to bad data and back to good data again? To watch more episodes in the Well-Architected Series, check out our playlist: https://aka.ms/azenable/yt/wa-playlist To watch more episodes in the Armchair Architects series, check out this playlist! https://aka.ms/armchairarchitects #AzureEnablementShow #ArmchairArchitects #Azure

Follow along using the transcript.

Microsoft Developer

588K subscribers