Seqera reposted this
Big news in Nextflow! The latest 24.04 release is packed with genuinely groundbreaking features and enhancements that transform your pipelines. From job arrays to new resource limits and Seqera Containers, new "topic channels" and a workflow output definition (adios publishDir!). This release has it all. Let's jump into the details! Workflow Output Definition - A new syntax for defining workflow outputs provides more flexibility and a cleaner DSL2-style approach. This should simplify how you publish any channel, not just process outputs, and remove the need for publishDir. Topic Channels & Eval Outputs - Collect metadata from various sources and capture standard output of shell commands easily. These features cut down boilerplate code (<<-END_VERSIONS > versions.yml in nf-core modules) and simplify complex workflows. Seqera Containers - This new flagship community service allows anyone to request and cache images built with packages from Conda and PyPI. Easily accessible directly from Nextflow, simplifying the workflow setup. Hosted on AWS, this replaces and significantly improves on biocontainers. Resource Limits - Define global resource limits for tasks to ensure your workflows run efficiently even in constrained environments. No more job max_cpus code in the config or endless queue waits! Job Arrays - Now supported in Nextflow! Submits multiple independent jobs with a single call, reducing the load on schedulers and speeding up execution times. Perfect for high-throughput workflows with 1,000+ tasks. Works on both cloud and HPC! Colored Logs & AWS Fargate Support- Enhanced log readability with colored outputs and expanded AWS Fargate support for faster run startup. Learn more about all these features and more in the detailed blog post. It even has a table of contents, so you know it is decent! https://lnkd.in/gcggh_Gy And tune into the new and improved Channels podcast, Channels Episode 41, to hear Phil and Ben give an opinioned take on these changes. https://lnkd.in/g_Wx2M7t
output made it super easy to organize result files from a single point and eval + topic is very practical. Loved the 24.04 release...
Support for job arrays is a much needed addition! Great news!
Hybrid Cloud Architect
1moNF is more cloudy now. 😁