byte_array's profile picture. Founder @Onehousehq, Creator of @apachehudi, Built the World's first #DataLakehouse, Distributed/Data Systems, Linkedin, Uber, Confluent alum. (views are mine)

Vinoth Chandar

@byte_array

Founder @Onehousehq, Creator of @apachehudi, Built the World's first #DataLakehouse, Distributed/Data Systems, Linkedin, Uber, Confluent alum. (views are mine)

置顶

🔥 Meet Quanton — the new query execution engine from Onehouse. 👍 Same Spark & SQL. 📉 At least half the cost. 📈 1.6x-3.6x better ETL price-performance 📊 2.2x-6.5x better Ingest price-performance 👉  Read the full blog here: onehouse.ai/blog/announcin… ⬇️  Download our free…


No longer just “faster than”; Now, @apachehudi is also “faster on” #apacheiceberg . Thanks to @apachextable

[Blog] Struggling with Apache Iceberg performance when your data dimensions get too hot? 🔥🌡️ Frequent updates and deletes in Iceberg can lead to a "chilly meltdown," forcing a tough choice between fast writes and efficient reads. 🥶 But what if you didn't have to compromise? 🤔…

_xushiyan's tweet image. [Blog] Struggling with Apache Iceberg performance when your data dimensions get too hot? 🔥🌡️

Frequent updates and deletes in Iceberg can lead to a "chilly meltdown," forcing a tough choice between fast writes and efficient reads. 🥶 But what if you didn't have to compromise? 🤔…


Vinoth Chandar 已转帖

🚀 Big news for Hudi Community! We're back with the Apache Hudi Meetup | ASIA (Chinese), and this time we're hosted by the incredible team at @JD_Corporate (京东) ! Get ready to explore the "Next-Generation Lakehouse: The Intelligent Future Engine". We have a packed agenda…

apachehudi's tweet image. 🚀 Big news for Hudi Community! We're back with the Apache Hudi Meetup | ASIA (Chinese), and this time we're hosted by the incredible team at @JD_Corporate 
(京东) !

Get ready to explore the "Next-Generation Lakehouse: The Intelligent Future Engine".

We have a packed agenda…
apachehudi's tweet image. 🚀 Big news for Hudi Community! We're back with the Apache Hudi Meetup | ASIA (Chinese), and this time we're hosted by the incredible team at @JD_Corporate 
(京东) !

Get ready to explore the "Next-Generation Lakehouse: The Intelligent Future Engine".

We have a packed agenda…

💰🔥 Spark’s default autoscaler = higher latency scaling up, wasted $$ scaling down. Why? It’s based on task backlog, not actual resource usage. Costly flaw. Result: 🐢 Slow scale-ups (e.g. too few Kafka partitions during spikes) 🐢 Slow scale-downs (e.g. many tiny tasks →…

byte_array's tweet image. 💰🔥 Spark’s default autoscaler = higher latency scaling up, wasted $$ scaling down.

Why? It’s based on task backlog, not actual resource usage. Costly flaw.

Result:
🐢 Slow scale-ups (e.g. too few Kafka partitions during spikes)
🐢 Slow scale-downs (e.g. many tiny tasks →…

📊 If you’re using Apache Spark on EMR (or anywhere really), you need better visibility into where your compute spend is going. At Onehouse, we kept seeing the same pattern across Hudi and non-Hudi users alike: 👉 Jobs were under-optimized 👉 Executors were sitting idle 👉…


🗞️ OLD NEWS: but worth a shout-out. Keys are optional in Hudi ... One of Hudi’s core goals was to remove friction from building data lakes. That’s why — for a long time now — Hudi has quietly supported auto-generation of record keys. No need to think up a key field just to get…

byte_array's tweet image. 🗞️ OLD NEWS: but worth a shout-out. Keys are optional in Hudi ...

One of Hudi’s core goals was to remove friction from building data lakes. That’s why — for a long time now — Hudi has quietly supported auto-generation of record keys. No need to think up a key field just to get…

Loading...

Something went wrong.


Something went wrong.