Navigate Select ESC Close

Learn ETL Pipelines in Databricks in Under 1 Hour | Data Engineering in Databricks

2026-04-28 Education
1.6k
103
3
Alex The Analyst
Alex The Analyst
1.4m subscribers

Unlock all features

FREE: Get instant access to 10 AI summaries, chats, or transcripts per day.

Description

In this series we are going to dive into the Data Engineering side of Databricks! This video will orchestrating jobs to automate our data pipelines. Get the Data Here: https://github.com/AlexTheAnalyst/DatabricksSeries/tree/main/Data%20Engineering Try out Databricks Free: http://signup.databricks.com/?provider=DB_FREE_TIER&utm_source=youtube&utm_medium=video&utm_campaign=AlextheAnalystDE ____________________________________________ RESOURCES: 💻Analyst Builder - https://www.analystbuilder.com/ 📖Take my Full MySQL Course Here: https://bit.ly/3tqOipr 📖Take my Full Python Course Here: https://bit.ly/48O581R 📖Practice Technical Interview Questions: https://bit.ly/46pDqqL Coursera Courses: Google Data Analyst Certification: https://coursera.pxf.io/5bBd62 Data Analysis with Python - https://coursera.pxf.io/BXY3Wy IBM Data Analysis Specialization - https://coursera.pxf.io/AoYOdR Tableau Data Visualization - https://coursera.pxf.io/MXYqaN *Please note I may earn a small commission for any purchase through these links - Thanks for supporting the channel!* ____________________________________________ BECOME A MEMBER - Want to support the channel? Consider becoming a member! I do Monthly Livestreams and you get some awesome Emoji's to use in chat and comments! https://www.youtube.com/channel/UC7cs8q-gJRlGwj4A8OmCmXg/join ____________________________________________ Websites: 💻Website: AlexTheAnalyst.com 💾GitHub: https://github.com/AlexTheAnalyst 📱Instagram: @Alex_The_Analyst ____________________________________________ 0:00 Intro 0:48 What is ELT 2:10 Data Ingestion 8:25 Building ETL Pipelines 29:06 Orchestration and Automation 41:16 Building a Full ETL Pipeline *All opinions or statements in this video are my own and do not reflect the opinion of the company I work for or have ever worked for*

Top Comments (9)

@MMA_VID 2026-04-28

I feel like YouTube content has moved towards data engineering more and more. I think the role of data analyst is cooked.

11
@MrAlexplusplus 2026-04-28

AI: 00:00 Introduction to Databricks ETL Pipelines 00:48 Understanding ELT and Medallion Architecture 02:11 Ingesting Data via CSV Upload 04:53 Connecting Databricks to Amazon S3 08:24 Transforming Raw Data to Silver Layer 13:14 Using AI Assistant for Data Cleaning 19:18 Creating Silver to Gold Transformations 22:12 Building and Running an ETL Pipeline 29:06 Automating Pipelines with Databricks Jobs 34:40 Configuring Job Triggers and Schedules 41:17 End-to-End ETL Project Overview 44:09 Streaming Data Ingestion from S3 47:35 Developing the End-to-End Pipeline with AI 54:14 Final Job Orchestration and Table Updates 1:00:13 Verifying Pipeline Results and Data Quality

6
@The-Best-Version-Of-You 2026-04-30

im old fashion. I use sql and python in vscode. I think now im ready to start using databricks. thanks for explaining it in a very simple way.

1
@Sandroider 2026-04-28

Was thinking this 1 hour ago and you delivered!

1
@khardu 2026-05-17

Thank you, Alex. That was super helpful.

0
@fahimaftab9486 2026-04-29

Hi Alex, could you please make a video on data except csv. Like using API call, json file, or connecting with other databases

0
@ZaneFalcao 2026-05-16

can we get the notebook?

0
@adeyinkaAdedejiNaMe 2026-05-10

Hello all I got a rollback_complete when creating s3 bucket stack

0
@tauqeerahmed4810 2026-04-29

Great

0

Unlock the Data Inside
Turn Videos into Knowledge

  • Get FREE 10/day: transcripts, summaries, chats
  • Chat with videos, export text & PDF
  • $1 free API credit for RAG, chatbots & research

Free forever plan • All features unlocked

App screenshot