ServiceNow (NYSE: NOW) makes the world work better for everyone. Our cloud-based platform and solutions help digitize and unify organizations so that they can find smarter, faster, better ways to make work flow. So employees and customers can be more connected, more innovative, and more agile. And we can all create the future we imagine. The world works with ServiceNow. For more information, visit www.servicenow.com. ServiceNow Research, part of the Advanced Technology Group at ServiceNow, advances the state-of-the-art in Enterprise AI. In equal measure, we innovate, research, experiment, and mature AI technologies that create compelling user experiences of the future so that every ServiceNow user benefits from AI. We believe AI should be built responsibly, without compromise to fairness, ethics, accountability, and transparency. ServiceNow Research programs drive innovation into Low Data Learning, Human Decision Support, and Human-Machine Interaction Through Language. Low Data Learning studies Machine Learning methods that enable adapting efficiently to varied and changing datasets. ServiceNow Research focuses on a wide range of downstream tasks such as language understanding, computer vision, robotic automation, and learning workflows. ServiceNow Research for Human Decision Support aims to assist decision-makers and increase customer productivity 1) Directing requests to the right sequence of decision-makers 2) Presenting appropriate information to aid decisions 3) Suggesting possible courses of action. ServiceNow Research studies in Human-Machine Interaction Through Language advance AI technology to enable the next generation of Language User Interfaces with a focus on natural conversational human-computer interactions and AI-assisted programming. The AI Trust and Governance Lab guides ServiceNow and its customers in their AI strategy and deployment via governance frameworks and applied research in trustworthiness. We’re hiring!
External link for ServiceNow Research
Montreal, Quebec, CA
Santa Clara, California, US
ServiceNow Research reposted this
Interested in time series forecasting and LLMs? We are looking for visiting researchers to work on context-aided forecasting (example below). Topics of interest: * Benchmarking * Multimodal Foundation Models * Agentic forecasting assistants Join Arjun Ashok, Andrew Robert Williams, Étienne Marcotte, Valentina Zantedeschi, Nicolas Chapados, and me in this exciting direction! When: January 2025 - 8 months Apply here: https://bit.ly/sc25q1
ServiceNow Research reposted this
Tracking the progress of LLM- and VLM-based web agents at everyday work tasks: our new WorkArena++ benchmark offers a comprehensive assessment.
🤔 Can LLM and VLM-based web agents tackle real-world enterprise tasks performed by knowledge workers in their everyday jobs? I'm thrilled to share our latest research work: "WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks". We introduce WorkArena++, a new benchmark designed to evaluate the planning, reasoning, retrieval, and memorization capabilities of large language models (LLMs) and vision-language models (VLMs) when used as autonomous web agents in enterprise settings. 🔍 Why WorkArena++: While LLMs/VLMs have demonstrated remarkable potential in mimicking human-like intelligence and following human instructions, their effectiveness in automating real-world tasks hasn't been fully explored. To address this, we developed a benchmark of 682 tasks inspired by workflows that knowledge workers frequently encounter in their everyday roles. The goal is to assess how well these models can plan, problem-solve, reason, retrieve, and understand context when deployed in real-world enterprise settings, and compare to their human counterparts. 📊 Key Findings: Our study compares state-of-the-art LLMs, VLMs, and human workers, revealing key challenges these models face in becoming effective workplace assistants. While humans achieve near-perfect performance in tackling the benchmark, even the strongest open-source and closed-source models significantly underperform in solving even the relatively easier tasks - a big gap of ~94% task success rate of humans compared to nearly 0% by AI agents. Furthermore, we also introduce a mechanism for automatically generating ground-truth traces for virtually tens of thousands of task variations in the benchmark, providing crucial resources to help the community in model fine-tuning and further development in closing the gap between open-source and closed-source language model agents. Along with WorkArena++, we would also like to present AgentLab (https://lnkd.in/ezsPT_DN), a one-stop place to develop and evaluate web agents using LLMs and VLMs on benchmarks supported by BrowserGym (https://lnkd.in/edhTp_pN), a robust gym environment for automating web tasks. We hope this benchmark and our platforms provide valuable resources for anyone working on autonomous web agents and beyond 🙂 Check out the paper here: https://lnkd.in/eTfCzDbV. Work done with my awesome collaborators: Léo Boisvert Maxime Gasse Massimo Caccia Thibault Le Sellier de Chezelles Quentin Cappart Nicolas Chapados Alexandre Lacoste Alexandre Drouin at ServiceNow Research. #AI #LLM #VLM #Agents #WebAgents #AIbenchmark #Planning #Reasoning
ServiceNow Research reposted this
We're hiring! Come help us crack hard problems related to evaluation and governance of large language models, agents and deployed AI systems. No shortage of hard and emerging challenges to sink your teeth into! ServiceNow Research's Trust & Governance Lab is looking for a Senior Applied Research Scientist. You will join a multi-disciplinary team of applied AI researchers and developers who work on test & eval challenges up and down the stack, from figuring out what capabilities and vulnerabilities foundation models have, to researching the strengths and weaknesses of using models (and juries and human-AI collaborative arrangements and...) to judge other models, to building out risk-aware test and eval methodologies and tooling for teams deploying to production, and more. Link to the job posting in comments. #artificialintelligence #trustworthyai #aisafety ServiceNow ServiceNow Research
ServiceNow Research reposted this
Are you passionate about pushing the boundaries of AI safety and security research and having an impact on enterprise AI? Join our team! **About the Role** The Reliable and Secure AI (ResAI) team at ServiceNow Research is seeking a talented Senior Research Engineer to tech lead our efforts. We're at the forefront of adapting frontier AI technologies for customers with sensitive data and strict security requirements. **What We Offer** * A chance to be part of a 0 to 1 initiative * Work on cutting-edge AI security and safety research, and help over 8000 ServiceNow customers put AI to work in a secure fashion * Collaborate with a team of top researchers and engineers **How to Apply** If you're excited about this opportunity and have relevant experience, we want to hear from you! * Apply online: https://lnkd.in/gPBKs_r3 * Reach out directly to discuss the role (Note: While we may not be able to respond to everyone individually, we seriously consider all qualified applications) Join us in shaping the future of secure AI in enterprise software! #AIResearch #JobOpportunity #ServiceNow #EnterpriseAI
ServiceNow Research reposted this
💡Today we're spotlighting Sean Hughes AI Ecosystem Director at ServiceNow! For Sean, open scientific collaboration and open-source AI provides fuel to the virtuous cycle of open innovation. Check out the post to learn more about Sean, his contributions to the open-source AI ecosystem and his work co-leading the AI Alliance Foundation Models working group and community efforts.
ServiceNow Research reposted this
Researcher in Time Series Forecasting @ ServiceNow Research, MILA-Quebec and University of Montreal
📢 Call for Papers: Time Series in the Age of Large Models We are organizing a workshop on "Time Series in the Age of Large Models" at NeurIPS 2024. We invite submissions related to the theme of time series in the age of large models. Key topics include, but are not limited to: 📌 Building Time Series Foundation Models 📌 Analysis of Pretrained Time Series Models 📌 Critiques on Time Series Foundation Models 📌 Faster and Better Inference Schemes for Autoregressive Time Series Models 📌 Leveraging Pretrained Models of Other Modalities for Time Series 📌 Multimodal Time Series Models 📌 Large-Scale Time Series Datasets and Benchmarks 📌 Time Series Evaluation 📌 Real-World Applications of Large Time Series Models Please consider submitting your latest research on these topics to our workshop. Submission deadline: Sep 15, 2024 (11:59 PM AoE) Acceptance notification: Oct 14, 2024 Workshop Date: Dec 14/15, 2024 We have an excellent lineup of keynote speakers and panelists from diverse backgrounds: 💎 Andrew Gordon Wilson 💎 Christoph Bergmeir 💎 Mihaela van der Schaar 💎 Tomas Pfister 💎 Qingsong Wen 💎 Valentina Zantedeschi Further details can be found on the workshop's website: https://lnkd.in/ePC26dUa We are also looking for reviewers for the workshop. If you would like to nominate someone (or yourself), please fill this form: https://lnkd.in/evzeNxst Organizers: Abdul Fatir Ansari, Arjun Ashok, Imry Kissos, Moshe Unger, Kashif Rasul, Pedro Mercado, Laurent Callot, Johannes Stephan. Please help us spread the word!
ServiceNow Research reposted this
The AI Alliance welcomes the U.S. Department of Commerce and the National Telecommunications and Information Administration (NTIA) for its forward-thinking report on open model weights, which recognizes the profound benefits of open foundation models for fostering innovation, economic growth, and AI safety. By concluding that "the government should not restrict the wide availability of model weights for dual-use foundation models,” the Commerce Department has taken a crucial step towards ensuring that the U.S. remains at the forefront of AI development. Read more about the AI Alliance’s input here. https://lnkd.in/g_B2-ssW
Thrilled to announce that ServiceNow has won VentureBeat's AI Innovation Award for best enterprise software implementation of GenAI! This award recognizes our work with Hugging Face and the BigCode community on StarCoder, a unique family of code LLMs built following open scientific development principles. StarCoder empowers organizations to customize LLMs for their specific needs, fostering a truly collaborative and open AI ecosystem. Huge congrats to the Hugging Face and ServiceNow Research teams for their collaboration and stewardship of BigCode. We look forward to many more exciting collaborations that push the boundaries of open innovation and open-source AI. #AI #OpenSource #GenAI #StarCoder #BigCode #ServiceNow #HuggingFace
ServiceNow Research reposted this
𝗠𝟮𝗟𝗶𝗻𝗴𝘂𝗮𝗹 🔈 🔥 𝗡𝗲𝘄 𝗗𝗮𝘁𝗮𝘀𝗲𝘁 𝗔𝗹𝗲𝗿𝘁 ❗❗❗ ⭐ 𝗠𝗨𝗟𝗧𝗜 𝗟𝗜𝗡𝗚𝗨𝗔𝗟 + 𝗠𝗨𝗟𝗧𝗜 𝗧𝗨𝗥𝗡 + 𝗦𝗬𝗡𝗧𝗛𝗘𝗧𝗜𝗖 + 𝗧𝗔𝗫𝗢𝗡𝗢𝗠𝗬 𝗚𝗨𝗜𝗗𝗘𝗗 + 𝗘𝗩𝗢𝗟𝗦 ⭐ 🥇 𝟏𝟖𝟐𝐊 𝐦𝐮𝐥𝐭𝐢𝐥𝐢𝐧𝐠𝐮𝐚𝐥 𝐢𝐧𝐬𝐭𝐫𝐮𝐜𝐭𝐢𝐨𝐧𝐬 🗣 𝟕𝟏 𝐥𝐚𝐧𝐠𝐮𝐚𝐠𝐞𝐬 🌐 𝟯𝟳 𝗹𝗼𝘄 𝗿𝗲𝘀𝗼𝘂𝗿𝗰𝗲 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲𝘀 🤹♂️ 𝟭𝟵 𝗱𝗶𝘃𝗲𝗿𝘀𝗲 𝗺𝘂𝗹𝘁𝗶𝗹𝗶𝗻𝗴𝘂𝗮𝗹 𝗡𝗟𝗣 𝘁𝗮𝘀𝗸𝘀 🏆 𝗜𝗺𝗽𝗿𝗼𝘃𝗲𝘀 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗯𝘆 𝘂𝗽𝘁𝗼 𝟯𝟬% 𝗼𝗻 𝗺𝘂𝗹𝘁𝗶-𝗹𝗶𝗻𝗴𝘂𝗮𝗹 𝘁𝗮𝘀𝗸𝘀 🔍 𝗗𝗲𝗺𝗼𝗻𝘀𝘁𝗿𝗮𝘁𝗲𝘀 𝗠𝘂𝗹𝘁𝗶-𝗹𝗶𝗻𝗴𝘂𝗮𝗹 𝗦𝗙𝗧 𝗶𝗺𝗽𝗿𝗼𝘃𝗲𝘀 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗮𝗯𝗶𝗹𝗶𝘁𝗶𝘁𝗲𝘀 𝘀𝗶𝗴𝗻𝗶𝗳𝗶𝗰𝗮𝗻𝘁𝗹𝘆 Evol mutations to generate more IFT pairs are guided by: - 𝐍𝐋𝐏 𝐭𝐚𝐬𝐤 𝐬𝐩𝐞𝐜𝐢𝐟𝐢𝐜 𝐞𝐯𝐨𝐥 𝐭𝐚𝐱𝐨𝐧𝐨𝐦𝐲. - Reasoning, slang, dialect, transliteration, etc for general instructions. - 𝗠𝐮𝐥𝐭𝐢-𝐭𝐮𝐫𝐧 𝐞𝐯𝐨𝐥 𝐭𝐚𝐱𝐨𝐧𝐨𝐦𝐲 for generating different types of - {follow-up, refinement, expansion, recollection} conversational follow ups 🔄 For more, checkout our paper below! Authors - Rishabh Maheshwary Vikas Yadav, PhD Hoang Nguyen Khyati Mahajan Sathwik Tejaswi Madhusudan 🌟 We are also actively hiring if you want to join us on our journey 🚀 Dataset: https://lnkd.in/e5E-VrN8 #genai #LLMs #research #machinelearning #deeplearning #nlp #huggingface #dataset #multilingual #sft #instructionfollowing #ai #chatgpt #gpt4 #servicenow #hiring #jobs
ServiceNow Research reposted this
🚨 It is time for foundation models for Embodied AI. Excited to Introduce GenRL! An AI agent that learns multimodal foundation world models 🌍 By connecting the multimodal knowledge of foundation models with the embodied knowledge of world models for RL, GenRL enables turning vision and language prompts into actions! We release paper, code, datasets, models, and a Gradio interactive demo! Visit the project website to know more: https://lnkd.in/e8TenzbP Pietro Mazzaglia, Aaron Courville, Tim Verbelen, ServiceNow Research, Mila - Quebec Artificial Intelligence Institute Bonus: we also tested GenRL on the Minecraft learning environment. The agent can identify biomes easily but struggles with accurate tasks, e.g. crafting objects. In the future, we will extend GenRL to more complex Minecraft tasks, e.g. using MineCLIP from Jim Fan