↑ ↓ Navigate ↡ Select ESC Close

Python AI Web Scraper Tutorial - Use AI To Scrape ANYTHING

2024-08-30 Education
350.2k
11.1k
305
Tech With Tim
Tech With Tim
2.0m subscribers

Unlock all features

FREE: Get instant access to 10 AI summaries, chats, or transcripts per day.

Description

GET MY FREE SOFTWARE DEVELOPMENT GUIDEπŸ‘‡ https://training.techwithtim.net/free-guide In this video, I'll be showing you how to build an AI web scraper using Python. The application itself is super cool as it scrapes the site based on the URL you give it, grabs the DOM content, and allows us to then pass a prompt to the AI that will be able to grab information from the website that we want. In addition to this quick demo of how everything works, I will also take a deep dive into how to build the AI web scraper utilizing tools such Selenium, BeatifulSoup, LangChain, & more! Get a free credit when you sign up for BrightData with a new account: https://brdta.com/techwithtim_ai 🎞 Video Resources 🎞 Code In This Video: https://github.com/techwithtim/AI-Web-Scraper ChromeDriver Download: https://googlechromelabs.github.io/chrome-for-testing/#stable Selenum Video Tutorial: https://www.youtube.com/watch?v=NB8OceGZGjA Ollama Download Link: https://ollama.com/ Ollama Github: https://github.com/ollama/ollama ⏳ Timestamps ⏳ 00:00 | Overview 00:15 | Install/Setup 04:30 | Starting the Streamlit UI 07:34 | Web Scraping with Selenium 15:59 | Unblocking Websites/Captchas/IP Bans 22:30 | Cleaning DOM Content 31:10 | Parsing Content with Ollama (LLM) 39:45 | Ollama Installation 42:45 | Testing the Scraper Hashtags #python #ai #webscraping

Top Comments (10)

@GokuDoku4 2024-09-16

I am creating one too using actual HTML, CSS and JavaScript and I am having a lot of fun coding this! Keep it up πŸ˜πŸ‘

35 2 replies
@norminemralino2260 2024-09-03

Tim is back with a banger tutorial! This is the kind of project/tutorial that made me subscribe to Tech With Tim in the first place. He takes a fairly complicated task and figures out how to make the task not as hard or doable. I’m really happy that he’s finally using Streamlit. It was something I commented and asked for a few projects back. Can you imagine how much worse it would be if Tim was just taking input and printing out content directly from a console? Anyway great job on this vid. I’m looking forward to the next one

22
@TehchnoGaming 2025-01-10

Building a web scraper for the first time as a Junior Engineer. Thanks Tim for the step by step instructions and clarity throughout this video.

19
@TechWithTim 2024-09-01

GET MY FREE SOFTWARE DEVELOPMENT GUIDEπŸ‘‡ https://training.techwithtim.net/free-guide

16 2 replies
@JitheshPs-r2t 2024-08-30

very well explained even a beginner could understand and great content you just earned a new subscriber :)

13
@willhelliwell 2024-12-14

This got me a long way towards what I needed. Thank you! Bit of AI help and I can now scrape iFrames inside the site too.

6
@onealoneal7047 2024-08-30

No need to see till the end you always provide great contents. Thank you . Keep working.

5
@RaghavKumarR 2024-08-30

Your content is high quality and top notch . Fantastic one brother , keep doing more stuff like this. Love to see it and really really appreciate it

2
@tengdayz2 2024-08-30

Thank you. I like that you give us alternative suggestions to your sponsor, but still invaluably represent them. Tim's gotta eat too, but you seem to get that having fun with it all comes first.

2
@jordanjackson6151 2024-09-09

So glad for this recent upload! Web Scraping is a little iffy to do since last year. Gotta stay updated.

0

Unlock the Data Inside
Turn Videos into Knowledge

  • Get FREE 10/day: transcripts, summaries, chats
  • Chat with videos, export text & PDF
  • $1 free API credit for RAG, chatbots & research

Free forever plan β€’ All features unlocked

App screenshot