In this video we discuss the best way to save data as files using Python and Pandas. When you work with large data sets, there comes a time when you need to save your data. Most people choose CSV files because they are easy to share and universally used. But there are much better options! Watch Rob Mulla, Kaggle Grandmaster, discuss some alternative ways to store data files: pickle, parquet, and feather files. I run some benchmarks to show that you can save time and space while preserving the important metadata about your files!

Timeline
00:00 Introduction
00:49 Creating our data
02:08 CSVs
04:39 Setting up Dtypes for CSVs
06:15 Pickle files
07:16 Parquet️
09:07 Spring
10:31 Other options
11:02 Benchmarking
12:19 Takeaways
12:43 Off

Code overview: https://gist.github.com/RobMulla/738491f7bf7cfe79168c7e55c622efa5
Follow me on Twitch for live coding streams: https://www.twitch.tv/medallionstallion_

Other videos:
Make Pandas Faster: https://www.youtube.com/watch?vSAFmrTnEHLg
Efficient Panda Dataframes: https://www.youtube.com/watch?vu4_c2LDi4b8
Introduction to Pandas: https://www.youtube.com/watch?v_Eb0utIRdkw
Exploratory data analysis video: https://www.youtube.com/watch?vxi0vhXFPegw
Audio data in Python: https://www.youtube.com/watch?vZqpSb5p1xQo
Image data in Python: https://www.youtube.com/watch?vkSqxn6zGE0c

* Youtube: https://youtube.com/@robmulla?sub_confirmation1
* Discord: https://discord.gg/HZszek7DQc
* Twitch: https://www.twitch.tv/medallionstallion_
* Twitter: https://twitter.com/Rob_Mulla
* Kaggle: https://www.kaggle.com/robikscube

#python #code #datascience #pandas

Please take the opportunity to connect and share this video with your friends and family if you find it helpful.