Chapter #9 – Designing a Data Pipeline Solution on GCP
#datapipeline #googlecloud #gcp

What is a Data Pipeline? – https://youtu.be/PjBhIDODLM0

***Google Cloud Platform Beginner Series***

Google Cloud Platform Beginners Series (2020) – https://www.youtube.com/watch?v=OzwSBbuHY-0&list=PLcnJIHtHiTA2Jp3klIkeHiwqGrxjXBzh6

***Networking Basics***
Network and infrastructure concepts – https://www.youtube.com/playlist?list=PLcnJIHtHiTA0C9UXgU2C6eTQMVFzSlIdP

*** Other Popular Playlists ***

Crunching Data Series (2020) – https://www.youtube.com/playlist?list=PLcnJIHtHiTA2HmIleev4scARSFwrQ0bIy

Latest Technology Tutorial (2020) –
https://www.youtube.com/watch?v=l5UcUEt1IzM&list=PLcnJIHtHiTA2Hdgon51_-G3Kt43hcU6kI

Hello friends, I am Anshul Tiwari and welcome to our YouTube channel IT k Funde.

More about this video –
In this video, we'll learn something different. Instead of discussing a specific product on the Google Cloud platform, we'll learn how to design a data pipeline solution on the Google Cloud platform.

This will help us in two ways:

1. This will help us improve our design thinking
2. We get an overview of all Google Cloud products that can be used in developing a data pipeline.

We will start by understanding a typical on-premises data pipeline and enterprise data warehouse (edw) design. We will then take the same solution and attempt to implement it on the Google Cloud platform.

The on-site solution design includes the following products:
#Streaming Data – Apache Kafka
#ETL – Informatica, IBM Datastage, SAP Data Services
#EDW – Teradata
#Dashboards & Reporting – Tableau and SAP Business Objects

GCP Cloud Design includes below products –
#Streaming Data – Cloud PubSub
#ETL – Cloud Data Merger
#Storage – Google Cloud Storage
#EDW – Google Big Search
#Dashboards – Looker, Google Data Studio
#ML – AutoML
#Metadata Management – Data Catalog

WATCH ALL VIDEOS IN ONE PLAYLIST FOR BETTER UNDERSTANDING !!

Credits and Resources for Further Study –
Documentation of GCP sources for detailed investigation –
#Cloud PubSub – https://cloud.google.com/pubsub/docs/overview
#Cloud Data Fusion – https://cloud.google.com/data-fusion/docs/concepts/overview
#Google Cloud Storage – https://cloud.google.com/storage/docs/quickstart-console
#Google Big Query – https://cloud.google.com/bigquery/docs/quickstarts
#Data Studio – https://cloud.google.com/bi-engine/docs/getting-started-data-studio
#Looker – https://docs.looker.com/
#AutoML – https://cloud.google.com/automl/docs
#Datacatalog – https://cloud.google.com/data-catalog/docs

images – pixabay.com

***Social links***
Youtube – https://www.youtube.com/c/itkfunde
Facebook – https://www.facebook.com/ITkFUNDE/
Linkedin – https://www.linkedin.com/in/ansh9685
Twitter – https://twitter.com/ansh9685
Instagram -https://www.instagram.com/itkfunde/

***About this channel***
Friends ITkFUNDE channel aims to bring knowledge, information, career advice and much more about IT industry to every individual whether he or she belongs to IT or not. This channel is for everyone who wants to learn something new!

Please take the opportunity to connect and share this video with your friends and family if you found it helpful.