Jay Chia’s Post

View profile for Jay Chia, graphic

Founder, Eventual

Sometimes the simplest solutions yield outsized benefits! Clark Zinzow just merged a 60 line PR (https://lnkd.in/gbPcySDe) to improve the scheduling locality of ScanTasks on the Ray runner. The fix simply made sure that we spread out the work of reading files as much as possible across the cluster. The result? Higher resource utilization, stabler Ray clusters and much better I/O throughput + performance! (Coming soon to you in the next Daft release 😁 )

  • No alternative text description for this image

To view or add a comment, sign in

Explore topics