#dataengineering search results
We did a thing: three amazing data engineers and I answered the 10 most-asked questions on r/dataengineering. These genuine insights help you understand the current state of #dataengineering. Obviously, I'm biased since I pulled together the questions & answered them myself.
While men Sleep … At least when the NEPs allow me to 😂😂😂… I’m grinding. I’m doing everything within my power to turn my story… I cannot fail oh #Datafam #Dataengineering
The simpler the semantic model, the simpler everything else becomes. It must be number one, even if just for reports. This is why most are building lakehouses to drive and communicate information. #dataengineering #datamodeling youtube.com/watch?v=sC1B5s…
Maximize Snowflake performance & slash query time⚡ Even a 1-second delay = ~2.11% lower conversions 📉 ✅ Faster queries ✅ Lower compute cost ✅ Smarter data pipelines ✅ Agile analytics Full Snowflake tuning guide ⬇️ bit.ly/43QK8Xc #Snowflake #DataEngineering
Columnar DB + JSON already sounds impossible… Yet @ClickHouseDB hits ~9,000× Postgres and ~2,500× Mongo, with columnar benefits intact (vectorized exec, compression, data skipping) #DataEngineering #ClickHouse
Just dropped my Airflow + PostgreSQL ETL pipeline Modular DAGs, CLI validation, and reproducible workflows — all built from scratch. Check it out 👇 github.com/mainamuragev/a… @LuxDevHQ @HarunMbaabu #DataEngineering
Register for Subsurface NYC November 13th (Free) Talks on Agentic AI, Apache Iceberg, Apache Polaris, Apache Arrow, Data Lakehouses and much more! Details in Comments #DataLakehouse #DataEngineering #AgenticAI
I’m excited to have contributed both as an organizer and a panelist at the first ever Data Engineering Community @data_dec meetup. #DataEngineering #DECMeetUp2025 #DECLagosMeetUp2025 #BigData #CommunityBuilding #DataEngineerCommunity
Transformations are the real heart of the data stack - and dbt made us finally treat them that way. The golden age of dbt isn't about tooling - it's about trust. Link to the full blog post: luminousmen.substack.com/p/the-golden-a… #dataengineering #bigdata
Igniting Possibilities with AI As Microsoft Ignite 2025 approaches, the theme Ignite Possibilities perfectly reflects how AI is transforming how enterprises think, build, and grow. #MicrosoftIgnite #AI #DataEngineering #DigitalTransformation #MicrosoftPartner
Next-level JSON Architect in action! 🚀 Generate, validate, and test dynamic JSON formats tailored for any AI agent. Perfect for ML, data interchange & config management. Saves hours in design loops and boosts AI integration efficiency. #JSON #AI #DataEngineering ✨ Here’s your…
#DataEngineering with Google Cloud Platform #GCP — A Guide to leveling up as a #DataEngineer by building a scalable data platform with Google Cloud: amzn.to/4ecMUtM [2nd Edition] v/ @PacktDataML ————— #DataAnalytics #Analytics #CDO #CTO #AI #ML #MLOps #DataScience
![KirkDBorne's tweet image. #DataEngineering with Google Cloud Platform #GCP — A Guide to leveling up as a #DataEngineer by building a scalable data platform with Google Cloud: amzn.to/4ecMUtM [2nd Edition] v/ @PacktDataML
—————
#DataAnalytics #Analytics #CDO #CTO #AI #ML #MLOps #DataScience](https://pbs.twimg.com/media/G3RnizZXAAAh17C.jpg)
#DataEngineering with Scala and Spark — Build streaming & batch pipelines to process massive amounts of data: amzn.to/3wdmEhy v/ @PacktDataML 𝒦𝑒𝓎 𝐹𝑒𝒶𝓉𝓊𝓇𝑒𝓈: 🔴Transform data into a clean and trusted source of information for your organization using Scala…
🚀 Is your data warehouse slowing down dashboards & decisions? Before rebuilding, try targeted optimization. 👉 9 key strategies to cut costs & boost speed: schema, ETL, queries, compute & more. Read the guide ➡️ bit.ly/4qzcFdS #DataWarehouse #DataEngineering
#DataEngineering Best Practices — Architect robust and cost-effective data solutions in the cloud era: amzn.to/3BkZ6df v/ @PacktDataML 𝓚𝓮𝔂 𝓕𝓮𝓪𝓽𝓾𝓻𝓮𝓼: 🟠 Architect and engineer optimized data solutions in the cloud with best practices for performance and…
#DataEngineering with Scala and Spark — Build streaming & batch pipelines to process massive amounts of data: amzn.to/3wdmEhy v/ @PacktDataML 𝒦𝑒𝓎 𝐹𝑒𝒶𝓉𝓊𝓇𝑒𝓈: 🔴Transform data into a clean and trusted source of information for your organization using Scala…
🧵 Thread : Découvrez pourquoi Palantir Workshop révolutionne le monde des données ! Ce n'est pas juste un outil low-code, c'est la "Tour de contrôle" qui comble le fossé entre : voir un problème et le résoudre. Je suis bluffé. #Palantir: #DataEngineering 1/ Le problème…
SQL is the original data API. Master it, and everything else becomes easier. #SQL #DataEngineering
🚨 Hiring: Lead Data Platform Engineer (AI + Fintech) Build the core data foundation for an AI-first fintech platform. GCP + BigQuery + Kafka + real-time pipelines. 🌍 Remote (Europe) | 💼 High-impact, high-autonomy Apply → share.hirechain.io/apply/zgnns5pn… #Hiring #DataEngineering…
Maximize Snowflake performance & slash query time⚡ Even a 1-second delay = ~2.11% lower conversions 📉 ✅ Faster queries ✅ Lower compute cost ✅ Smarter data pipelines ✅ Agile analytics Full Snowflake tuning guide ⬇️ bit.ly/43QK8Xc #Snowflake #DataEngineering
OLAP, TSDB, VectorDB, TextDB… all coexist under one roof. #DolphinDB unifies analytics, transactions, and AI-driven queries in a single high-performance framework. #DataEngineering #AI
Avoid distributed table write conflicts in #DolphinDB! Use mr with finalFunc to combine map results first, then write to the target table—prevents parallel partition clashes. #DistributedComputing #DataEngineering
💥𝗙𝗿𝗲𝗲 𝗢𝗻𝗹𝗶𝗻𝗲 𝗗𝗲𝗺𝗼 𝗼𝗻 𝗔𝘇𝘂𝗿𝗲 𝗗𝗮𝘁𝗮 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 📅 Date: 08/11/2025 | 🕘 Time: 9:00 AM IST 👨🏫 Trainer: Mr. Sidharth (Expert in Azure & Data Solutions) 📞 𝗖𝗼𝗻𝘁𝗮𝗰𝘁 𝘂𝘀: +91 7032290546 🌐 𝗩𝗶𝘀𝗶𝘁: visualpath.in/online-azure-d… #DataEngineering
Day 42 of my #buildinginpublic journey into Data Engineering - Learned to build & share Docker images - Created Dockerfiles using FROM, RUN, COPY, CMD - Shared images via registries - Managed files inside images #Docker #DataEngineering #DevOps #Python #Automation
💡 Want faster, smarter ML pipelines? Learn how to build a Batch Feature Store using AWS EMR + DynamoDB — the 2025 way. 🚀 Read the full guide 👉 lktechacademy.com/2025/10/batch-… #MLOps #AWS #DataEngineering #MachineLearning
Data alone isn’t enough. Clean, structured, usable data = better decisions. 🔗 cal.com/vantageiqtechn… #VantageIQTechnologies #DataEngineering #AIReady
Build fast, scalable & automated data pipelines with Snowflake: ✅ Real-time ingestion ✅ Serverless processing ✅ Zero-maintenance scaling ✅ Seamless integration with DBT, Airflow, Fivetran & Kafka 🔗: snowflakemasters.in/snowflake-data… #Snowflake #SnowflakeDataPipeline #DataEngineering
🚀 Level Up Your Spark Skills! Discover the 10 must-know Apache Spark commands every Data Engineer should master 💡 👉 Read here: buff.ly/6wqw7j8 #ApacheSpark #BigData #DataEngineering #Hadoop #TechTips #Coding #MachineLearning
Field Note #49 — The SQL Scatter Problem The hardest part of migration isn’t translation; it’s discovery. SQL hides in SPs, Java strings, notebooks, even shell. ASTs + light lexical passes beat grep every time. Lesson: find all logic before you promise timelines. #DataEngineering
We used to have: Data lakes for scale Warehouses for analytics Then came the Lakehouse , open like a lake, reliable like a warehouse, ready for AI. Store once. Query everywhere. #Databricks #Lakehouse #DataEngineering
A data lake starts clean. Then comes untracked files, mixed schemas, missing lineage… and suddenly it’s a swamp. Storage is easy. Governance, reliability, and performance are hard unless you have Delta Lake. #Databricks #DeltaLake #Lakehouse
Ready to see how data powers the tech of tomorrow? 💡📊 Join Stratpoint & Jemima Villanueva this Fri, Nov 7 (9–11:30 AM) on MS Teams for WVSU’s “Unlocking the Power of Data” webinar! 🎟️ Save your slot: na2.hubs.ly/H01TYmM0 #Stratpoint #DataEngineering #DataScience
— New York City — Nov 13 — Subsurface Conference Register for free admission to the Nov 13 Dremio’s Subsurface conference in NYC: lnkd.in/ezT7n3VJ Register for free admission to Nov 11th Open Lakehouse Meetup: luma.com/89np9f3s #ApacheIceberg #DataEngineering…
Just open-sourced my ELT pipeline template! Perfect for anyone learning modern data engineering or needing a quick CSV → Snowflake solution Built with dlt + Python. Stars ⭐ appreciated! github.com/bellonbits/Sal… #OpenSource #dataengineering
The goal is to be too good to be ignored… While men sleep 🙇♂️🙇♂️ 2:00 AM check in. Who’s up?? #Datafam #DataEngineeringStudy #Dataengineering
While men Sleep … At least when the NEPs allow me to 😂😂😂… I’m grinding. I’m doing everything within my power to turn my story… I cannot fail oh #Datafam #Dataengineering
Columnar DB + JSON already sounds impossible… Yet @ClickHouseDB hits ~9,000× Postgres and ~2,500× Mongo, with columnar benefits intact (vectorized exec, compression, data skipping) #DataEngineering #ClickHouse
#DataEngineering Best Practices — Architect robust and cost-effective data solutions in the cloud era: amzn.to/3BkZ6df v/ @PacktDataML 𝓚𝓮𝔂 𝓕𝓮𝓪𝓽𝓾𝓻𝓮𝓼: 🟠 Architect and engineer optimized data solutions in the cloud with best practices for performance and…
We did a thing: three amazing data engineers and I answered the 10 most-asked questions on r/dataengineering. These genuine insights help you understand the current state of #dataengineering. Obviously, I'm biased since I pulled together the questions & answered them myself.
In 2025, data engineers need a mix of classic & emerging skills: Python & Scala, Apache Spark & Kafka, SQL & NoSQL, AI-powered tools like GitHub Copilot, and Large Language Models (LLMs) for automation. #DataEngineering #AI it.exchange/blog/5-essenti…
Curiosity led me to explore the role of LLMs in data engineering and it turns out they’re revolutionizing pipelines! Check out the full story here: betheldaniel332.medium.com/how-llms-are-r… What’s your take on LLMs in data stacks?#DataEngineering #LLMs #AI
Why is dbt So Popular? medium.com/@vutrinh274/wh… + Practical #DataEngineering guide to building a #cloud-based, pragmatic, and dependable data platform with #SQL and dbt: amzn.to/3KgVI4f —— #DataScience #Analytics #BI #CDO #CTO
Maximize Snowflake performance & slash query time⚡ Even a 1-second delay = ~2.11% lower conversions 📉 ✅ Faster queries ✅ Lower compute cost ✅ Smarter data pipelines ✅ Agile analytics Full Snowflake tuning guide ⬇️ bit.ly/43QK8Xc #Snowflake #DataEngineering
I’m excited to have contributed both as an organizer and a panelist at the first ever Data Engineering Community @data_dec meetup. #DataEngineering #DECMeetUp2025 #DECLagosMeetUp2025 #BigData #CommunityBuilding #DataEngineerCommunity
Transformations are the real heart of the data stack - and dbt made us finally treat them that way. The golden age of dbt isn't about tooling - it's about trust. Link to the full blog post: luminousmen.substack.com/p/the-golden-a… #dataengineering #bigdata
Monday's #DataEngineering CI/CD workshop #MicrosoftFabric: so much interest we ran out of credentials for unregistered attendees 🔥Watching teams go from "CI/CD seems complicated" to building working fabric-cli pipelines in one session = why hands-on workshops work.
🧵 Thread : Découvrez pourquoi Palantir Workshop révolutionne le monde des données ! Ce n'est pas juste un outil low-code, c'est la "Tour de contrôle" qui comble le fossé entre : voir un problème et le résoudre. Je suis bluffé. #Palantir: #DataEngineering 1/ Le problème…
You had me at “unlimited API calls” Try it for free now 👉 ipinfo.io/lite #IPdata #DevTools #DataEngineering
Achieve the best #pricePerformance for your #DataEngineering workloads in #microsoftFabric by enabling #NativeEngine @JustynaLucznik #bestSparkOnAzure #NativeEngine #fabconEU
Igniting Possibilities with AI As Microsoft Ignite 2025 approaches, the theme Ignite Possibilities perfectly reflects how AI is transforming how enterprises think, build, and grow. #MicrosoftIgnite #AI #DataEngineering #DigitalTransformation #MicrosoftPartner
Does Oracle get emotionally attached before splitting strings apart? Source: devhubby.com/thread/how-to-… #SQL #DataEngineering #Programming #OraclePLSQL #explore #oracle
Something went wrong.
Something went wrong.
United States Trends
- 1. Cheney 85K posts
- 2. Election Day 101K posts
- 3. Logan Wilson 6,465 posts
- 4. GO VOTE 82.7K posts
- 5. Shota 12.9K posts
- 6. Mamdani 522K posts
- 7. Iraq 51.6K posts
- 8. Cuomo 244K posts
- 9. Good Tuesday 36.5K posts
- 10. New Jersey 176K posts
- 11. #tuesdayvibe 2,104 posts
- 12. Virginia 177K posts
- 13. Halliburton 3,921 posts
- 14. Rolex 17K posts
- 15. Taco Tuesday 10.9K posts
- 16. No ID 56.7K posts
- 17. Rickey 1,800 posts
- 18. Jerry 45.6K posts
- 19. Alex Karp 7,258 posts
- 20. Chester County 8,632 posts