你可能會喜歡
Exercise project for anyone starting in data engineering startdataengineering.com/post/data-engi… #dataengineering #bigdata #ETL #ApacheAirflow #AWS #ApacheSpark
If you are interested in using "Change Data Capture" pattern for streaming ETL, check out startdataengineering.com/post/change-da… #ETL #changedatacapture #dataengineering #debezium #BigData
Preparing for SQL interviews? Do Leetcode SQL hard, sort by freq, and do the first 40 #data #dataengineering #Software #SQL
Starting a data project is a lot of work! It can be overwhelming to define the problem, set up systems, and then code! Use this DE project as a blueprint to build your own: startdataengineering.com/post/data-engi… #data #dataengineering #Database #DataAnalytics #dataviz #Python #datapipeline
Backfilling is an inevitable part of data projects. When designing your data pipelines take some time to answer the following questions 1. Does multiple backfill runs cause duplicate data? 2. Can multiple backfills be parallelized? #data #DataEngineering #datapipeline #datasets
When data to process is larger than memory, try to stream with python generators, before jumping to distributed systems! #data #dataengineering #Python #pythonlearning #Generator E.g. Stream a file(note () and not []), get diff between date cols
![startdataeng's tweet image. When data to process is larger than memory, try to stream with python generators, before jumping to distributed systems!
#data #dataengineering #Python #pythonlearning #Generator
E.g. Stream a file(note () and not []), get diff between date cols](https://pbs.twimg.com/media/FQyi00EXoAMO4cg.jpg)
Learning data engineering? Build a pipeline locally. 1. Python to pull data from an API (e.g. Coincap) 2. Load data into a local Postgres container 3. Automate it with cron/task scheduler Start small, build, improve, & repeat. #data #dataengineering #pythonlearning #Python
Starting as a DE? 90% of what you will need is SQL (OLAP), python, & distributed system basics Don't overcomplicate! #data #dataengineering #SQL #Database #Python
Exercise project for anyone starting in data engineering startdataengineering.com/post/data-engi… #dataengineering #bigdata #ETL #ApacheAirflow #AWS #ApacheSpark
It can be overwhelming to start learning data engineering. I'd recommend starting with the basics of python, sql, UNIX commands, building a simple data project, update Github, Linkedin. Landing a DE job is 60% part learning and 40% marketing. See reply 👇🏽 for helpful links.
An orchestration tool that I've been impressed with is @dagsterio. Easy setup, powerful features and great docs. Use 👇🏽 to play around with a pipeline on dagster startdataengineering.com/post/data-engi… #data #dataengineering #Python #Database #DataAnalytics
Left anti-join is cool! Get all the data from the left table that has no matching data in the right table select t1.* from t1 left join t2 on t1.id=t2.id where https://t.co/TilIGFtVGj is null; #data #dataengineering #SQL #Database
uv by @astral_sh is truly one of the best tools you can have in your toolkit as a DE. TIL: You can quickly start a jupyter notebook with it doc: docs.astral.sh/uv/guides/inte…
Data engineers write the most complex piece of code to Upsert into tables. Here's THE command you need to know MERGE INTO/INSERT ON CONFLICT #data #dataengineering #SQL
United States 趨勢
- 1. Florida 98.9K posts
- 2. #SmallBusinessSaturday 2,102 posts
- 3. Texas 168K posts
- 4. Ohio State 26K posts
- 5. Kentucky 13.8K posts
- 6. Tyler Adams 2,040 posts
- 7. Good Saturday 34.3K posts
- 8. Buckeyes 4,401 posts
- 9. Go Bucks 2,053 posts
- 10. Go Blue 6,324 posts
- 11. #MeAndTheeSeriesEP3 1.31M posts
- 12. Raphinha 14.4K posts
- 13. Gameday 28.5K posts
- 14. #StrayKidsAtMAMA2025 12K posts
- 15. Saban 4,757 posts
- 16. #ShopSmall 1,478 posts
- 17. #SaturdayVibes 3,719 posts
- 18. TV Guide N/A
- 19. SKZ KARMA OF THE YEAR 12.2K posts
- 20. Alaves 18.1K posts
你可能會喜歡
-
DuckDB
@duckdb -
Mehdi Ouazza
@mehd_io -
dbt Labs
@dbt_labs -
Data Engineering Weekly
@data_weekly -
Darshil | Data Engineer👨🏻🔧
@parmardarshil07 -
Zach Wilson
@EcZachly -
dbt
@getdbt -
Maxime Beauchemin
@mistercrunch -
SeattleDataGuy
@SeattleDataGuy -
Mitchell van Rijkom
@MitchellvRijkom -
Neelesh Salian 💻
@neelesh_salian -
Sarah Floris
@ADutchEngineer -
Robert Yi 🐳
@imrobertyi -
Simon Späti 🏔️
@sspaeti -
Alex Merced | Open Data Lakehouse Advocate
@AMdatalakehouse
Something went wrong.
Something went wrong.