Trelis Research’s Post

Top 10 Tricks for Google Colab Users: 10. Specify TensorFlow version 9. Use TensorBoard for visualization 8. Use TPUs when you need more processing power 7. Use Local Runtimes if you have local hardware accelerators 6. Use Colab Scratchpad for quick tests 5. Copy data to Colabs VMs for fast data loading 4. Check your RAM and resource limits to make sure you don't run out of resources 3. Close Tabs when done to end the session and save resources 2. Use GPUs only when needed to ensure you have access when you really need them. 1. What's your number 1 tip for using Google Colab?

CVE-2024-39486: Direct Rendering Manager (DRM) of video card. A race leads to use-after-free of a "struct pid" (8 Jul 2024)   Preface: The display pipeline driver responsible for interfacing with the display uses the kernel mode setting (KMS) API and the GPU responsible for drawing objects into memory uses the direct rendering manager (DRM) API.   Background: The Direct Rendering Manager (DRM) is a subsystem of the Linux kernel responsible for interfacing with GPUs of modern video cards. For plain GEM based drivers there is the DEFINE_DRM_GEM_FOPS() macro, and for DMA based drivers there is the DEFINE_DRM_GEM_DMA_FOPS() macro to make this simpler. A refcount records the number of references (i.e., pointers in the C language) to a given memory object. A positive refcount means a memory object could be accessed in the future, hence it should not be freed.   Vulnerability details: filp->pid is supposed to be a refcounted pointer; however, before this patch, drm_file_update_pid() only increments the refcount of a struct pid after storing a pointer to it in filp->pid and dropping the dev->filelist_mutex, making the race possible.   Remark: The official explanation says it may be difficult to encounter this design weakness in reality. Because process A has to pass through a synchronize_rcu() operation while process B is between mutex_unlock() and get_pid(). Vulnerability (CVE-2024-39486) has been resolved.   Official announcement: For detail, please refer to link – https://lnkd.in/gxWjvw8c

Remember when you had to press "1" 3 times, wait and then click "1" three times to write "cc" on your phone? Today i tested Groq with #Llama3 8B, which generates 800+ tokens per second, I was thoroughly impressed by the speed i get with LPUs as compared with conventional GPUs. they will scale more and more in future. With more speed and power, we can imagine the number use-cases which were impossible before. github:https://lnkd.in/gCCS5Zuj #groq #lpu #genAI #largelanguagemodel #llms #llmops #languagemodels #ollama

I put together a quick GitHub guide for anyone interested in Unraid, Plex, and AV1 transcoding: https://lnkd.in/efk8A38U This setup is designed for users with large video media libraries. If a user has 300TB of media for example, by deploying Intel ARC GPUs and transcoding videos to the AV1 format, you can compress your library down to 75TB (converting all H264/H265).

Know your GPU constants: One of the first questions teams new to GPU analytics ask is how much GPU RAM is enough to scale effectively. We put together a quick tutorial that builds your intuition for memory planning and key size ratios as data flows from Parquets on disk, to in-memory Apache Arrow, and finally, to GPUs with RAPIDS cuDF and Graphistry/GFQL. Check it out below, and stay tuned for next one on how to better diagnose your app's GPU performance Tutorial Part I: https://lnkd.in/gnGvufjP

Do you know how big of a GPU you need for an analytics task, or how big of a parquet or csv will fit into GPU memory before you need to batch? New article up on typical sizes and how to think about them. Should be helpful - one of the first we prioritized as we're revisiting common day 1 & week 1 questions!

We’ve heard it from ComfyUI users time and again: our ComfyUI integration is best-in-class! 🏆 Now we’ve made it even better. With our new "build commands" feature, you can easily run custom nodes and model checkpoints with ComfyUI on powerful GPUs. 💪🏻 🚀 Check out Het Trivedi and Rachel Rapp's post to see how: https://lnkd.in/ejDJMv7Q In case you didn't know: you can (and always could) launch ComfyUI with Truss as a callable API endpoint that you can share. Now your models spin up even faster. We’re proud to enable users with the full power of ComfyUI, while making it shareable and blazing fast. If you try it out let us know how it goes, or show us what you build! 🎉

It's been a crazy year! For our last release of 2024, we shipped: ⚒️ 𝐌𝐮𝐥𝐭𝐢-𝐆𝐏𝐔 𝐖𝐨𝐫𝐤𝐞𝐫𝐬 You can now run workloads across multiple GPUs! This lets you run workloads that might not fit on a single GPU. For example, you could run a 13B parameter LLM on 2x A10Gs, which normally would only fit on a single A100-40. ⚡️𝐈𝐧𝐬𝐭𝐚𝐧𝐭𝐥𝐲 𝐖𝐚𝐫𝐦 𝐔𝐩 𝐂𝐨𝐧𝐭𝐚𝐢𝐧𝐞𝐫𝐬 We added a "Run Now" button to the dashboard to instantly invoke an app and warm up the container. 🚢 𝐈𝐦𝐩𝐨𝐫𝐭 𝐋𝐨𝐜𝐚𝐥 𝐃𝐨𝐜𝐤𝐞𝐫𝐟𝐢𝐥𝐞𝐬 We wanted to make it easier to use existing Docker images on Beam. You can now use a Dockerfile that you have locally to create your Beam image. 🔑 𝐏𝐚𝐬𝐬 𝐒𝐞𝐜𝐫𝐞𝐭𝐬 𝐭𝐨 𝐈𝐦𝐚𝐠𝐞 𝐁𝐮𝐢𝐥𝐝𝐬 You can now pass secrets into your image builds, useful for accessing private repos or running build steps that require credentials of some kind. 𝐀𝐧𝐝 𝐰𝐞'𝐯𝐞 𝐠𝐨𝐭 𝐬𝐨𝐦𝐞 𝐚𝐦𝐚𝐳𝐢𝐧𝐠 𝐧𝐞𝐰 𝐟𝐞𝐚𝐭𝐮𝐫𝐞𝐬 𝐜𝐨𝐦𝐢𝐧𝐠 𝐢𝐧 𝐉𝐚𝐧𝐮𝐚𝐫𝐲.  It's been an excited year, and we can't wait to ship more stuff for you in 2025. Happy New Year!

ComfyUI is a modular offline stable diffusion GUI with a graph/nodes interface. It allows you to design and execute advanced stable diffusion pipelines without coding using the intuitive graph-based interface. ComfyUI supports SD1.x, SD2.x, and SDXL, and features an asynchronous queue system and smart optimizations for efficient image generation. It can be configured to work on GPUs or CPU-only, and allows you to load and save models, embeddings, and previous workflows. https://lnkd.in/gE_yNP4J

We showcased how easy it is to build and deploy GPU-powered ML workflows with any OSS package at #SnowflakeBUILD. Learn more from Dash DesAI as he demos how to speed up training for an XGBoost model with GPUs using Snowflake Notebooks on Container Runtime. Try it for yourself now: https://okt.to/sN4YAG