Concordia AI 安远AI

🚀 Issue #17 of AI Safety in China! Key Takeaways: 🌐 China announced an AI capacity building project directed at Global South countries at the UN Summit of the Future. 🇨🇳 🇺🇸 The Chinese and US governments indicated that a second round of intergovernmental dialogue on AI is likely after the US national security advisor’s trip to China. 📘 A Chinese standards body issued China’s first AI Safety Governance Framework with substantial treatment of frontier AI risks. 🔎 Recent Chinese technical AI safety papers include work on “weak-to-strong deception,” benchmarking LLM risks in science, and assessing which layers at the parameter level are most important for AI safety. 🎤 A Chinese academician and former cybersecurity official spoke on the need for further technical AI safety research. 💡 Subscribe to AI Safety in China to get biweekly updates on ✅ China's positions on international AI governance ✅ China's governance and policy initiatives to mitigate AI risks ✅ Technical safety and alignment research in China ✅ Views on AI risks by Chinese experts. Read the full issue here: https://lnkd.in/gBJx-xpW #AISafety #AI #China #InternationalCollaboration #ConcordiaAI #Newsletter

🌟Concordia AI was honoured to participate in events surrounding the UN Summit of the Future, including signing the Manhattan Declaration alongside AI luminaries such as Yoshua Bengio and Alondra Nelson—check out more in our Substack! 🌟 https://lnkd.in/g8_Uf6kY

📣 New report: China’s AI Safety Evaluations Ecosystem📣 💡 With growing interest around the world towards evaluating AI models for dangerous risks, Concordia AI is publishing the first database of Chinese AI safety evaluations and comprehensive analysis of this ecosystem to appear in English. Highlights: 🏛️ The Chinese government already requires pre-deployment testing and evaluation of certain AI systems for ideology, discrimination, commercial violations, violations of individual rights, and application in higher risk domains. There are signs that this could expand in the future to incorporate testing for frontier or catastrophic AI safety risks.  🧠 The risk areas that received the most testing by Chinese AI safety benchmarks are bias, privacy, robustness to adversarial and jailbreaking attacks, machine ethics, and misuse for cyberattacks. 💻 Chinese evaluations tested for all categories defined as frontier AI risks, with misuse for cyberattacks as the most tested frontier risk. 📏 Chinese AI safety evaluations primarily comprise static benchmarks, with a small number of open-source evaluation toolkits, agent evaluations, and domain red teaming efforts. Chinese institutions do not appear to have conducted human uplift evaluations. 🏫 Shanghai AI Lab, Tianjin University NLP Lab, and Microsoft Research Asia Societal AI team are the only research groups that have published two or more frontier AI safety evaluations in China. However, many other government-backed, academic, and private industry research groups have also published evaluations covering a broad spectrum of AI safety and social impacts concerns. Read our Substack post (https://lnkd.in/gGZh7JK7) and check out our database (https://lnkd.in/gbzQqb5G) to learn more! We welcome engagement and outreach with other organizations interested in fostering internationally interoperable AI safety evaluation practices and standards. #Evaluations #AISafety #AI #China #InternationalCollaboration #ConcordiaAI #Newsletter

📣 Announcement: Concordia AI to join Singapore’s AI Verify Foundation 📣 Concordia AI is honoured to be joining Singapore’s AI Verify Foundation together with leading global technology companies such as AWS, Google, IBM, Microsoft, and Ant Group to participate in Singapore’s efforts to advance responsible AI 🎉 https://lnkd.in/eR9ZfYTW AI Verify is an AI governance testing framework 🔍 first developed by the Infocomm Media Development Authority (IMDA) of Singapore in 2022. The AI Verify Foundation, established in 2023, brings together researchers, companies, and policymakers to support the AI Verify framework. 🌐  The Foundation will help to foster an open-source community to contribute to AI testing frameworks, code base, standards and best practices to ensure the safety and trustworthiness of AI systems. 🛡️ As a member of the AI Verify Foundation, Concordia AI looks forward to: 💡 Leveraging our expertise in AI safety and governance to to advise relevant testing and evaluation projects. 🔧 Developing and using cutting-edge tools, such as the LLM testing platform Project Moonshot, with other members of the Foundation to promote cross-language and cross-cultural AI evaluations. 🤝 Promoting exchanges and cooperation to foster internationally recognized and interoperable AI safety standards and evaluation frameworks.

📣 The Third Plenum of the Communist Party of China included the goal of “instituting oversight systems to ensure the safety of AI.” This is the strongest sign so far that top leaders in China are concerned about AI safety. But, what does this goal actually entail? 📕 Concordia AI has translated authoritative official study materials co-edited by President Xi and other top leaders expounding in greater detail the Chinese leadership’s views on AI safety. Key points: 🎯 Motivations for creating AI safety oversight systems are explained in terms of responding to rapid AI development, promoting high-quality development, and participating in global governance.  🔭 AI safety oversight should involve “forward-looking prevention and constraint-based guidance,” which suggests an active and potentially precautionary approach. ⚖️ The text argues against putting development ahead of governance. Instead, it suggests that both should go hand in hand, progressing at the same time. 🌏 The section is supportive of AI governance efforts globally, referencing China’s Global AI Governance Initiative, the UK’s Global AI Safety Summit, EU AI safety legislation, and American AI safety standards. Read our full translation on Substack (https://lnkd.in/gJVmS_wu) and subscribe for future updates! #AISafety #AI #China #InternationalCollaboration #ConcordiaAI #Newsletter

🚀 Issue #16 of AI Safety in China! Key Takeaways: 🎤 AI safety was included in China’s Third Plenum decision document laying out top domestic priorities for the next five years, the highest-level document in which this concept has been mentioned.  🌐 The 2024 World AI Conference (WAIC) included a strong safety and governance theme and featured participation of China’s Premier, the Shanghai Party Secretary, and four additional ministerial or vice-ministerial level officials.  ⚖️ A top researcher for China’s legislature cautioned against excessive focus on AI safety in a recent speech, advocating for an incremental approach to lawmaking. He also noted AI’s risks in cybersecurity and automated decision-making. 📝 Over the past two months, Chinese researchers published one of the first papers in China on mechanistic interpretability, as well as papers on unlearning, risks in superalignment, and benchmarking honesty. 💡 Subscribe to AI Safety in China to get biweekly updates on ✅ China's positions on international AI governance ✅ China's governance and policy initiatives to mitigate AI risks ✅ Technical safety and alignment research in China ✅ Views on AI risks by Chinese experts. Read the full issue here: https://lnkd.in/gVX-JJTZ #AISafety #AI #China #InternationalCollaboration #ConcordiaAI #Newsletter

🚀 Highlights from Concordia AI’s participation in the World AI Conference 🚀 Concordia AI recently hosted the WAIC Frontier AI Safety and Governance Forum in Shanghai. Highlights include: 🖥️ Assessments on the state of advanced AI capabilities and safety with Turing Award Laureate Yoshua Bengio, Former Baidu President ZHANG Ya-Qin, UC Berkeley Professor Dawn Song, and Director of Peng Cheng Lab GAO Wen. 📜 Discussions on AI safety testing and evaluation methods from the Frontier Model Forum, China Academy of Information and Communications Technology, and Shanghai AI Lab. ⚖️ Lessons-sharing on domestic AI governance from French, Singaporean, American, and Chinese policymakers or experts. 🌏 Exchanges on international AI safety cooperation from 3 members of the UN High-Level Advisory Body on AI, Carnegie Endowment for International Peace President Tino Cuellar, Tsinghua University Dean XUE Lan, and experts across industry and academia. 🎤 Plus many more insights from a total of 26 distinguished speakers from China and around the world. Concordia AI representatives also attended the WAIC opening ceremony, where Premier LI Qiang delivered opening remarks and Shanghai Party Secretary CHEN Jining announced the “Shanghai Declaration on Global AI Governance.” Read our full summary of the event on Substack (https://lnkd.in/gmXWpSUZ) and watch the recording on YouTube (https://lnkd.in/gJSHPBTC). #AI #WAIC2024 #Governance #China #Technology #Innovation #ConcordiaAI

🚀 Issue #15 of AI Safety in China! Key Takeaways: 🌐 China is diversifying its AI cooperation network through new AI dialogue efforts with Russia, Arab countries, and BRICS.  📣 Zhipu AI, a leading Chinese large model developer, signed on to the Frontier AI Safety Commitments before the AI Seoul Summit, highlighting increasing attention to AI safety issues. 💻 Technical safety work over the previous two months has covered issues such as benchmarks for frontier AI risks, protecting models from malicious fine-tuning, scalable oversight, and LLMs in cybersecurity. 📖 Four Chinese LLM startup CEOs discussed their views on AI safety at the Beijing Academy of AI Conference. 💡 Subscribe to AI Safety in China to get biweekly updates on ✅ China's positions on international AI governance ✅ China's governance and policy initiatives to mitigate AI risks ✅ Technical safety and alignment research in China ✅ Views on AI risks by Chinese experts. #AISafety #AI #China #InternationalCollaboration #ConcordiaAI #Newsletter

🚀 Announcing Concordia AI’s participation in the World AI Conference 🚀 We are thrilled to announce that Concordia AI will be hosting the Frontier AI Safety and Governance Forum at the World AI Conference (WAIC) in Shanghai on July 5th! WAIC is China’s biggest AI conference this year and has been elevated to include a High-Level Meeting on Global AI Governance attended by diplomats and experts from around the world. 🌟 Join us online or in person for an incredible lineup of 20+ Chinese and international experts who will delve into 4 key themes: AI safety research, AI safety testing, AI safety guidance, and international cooperation. 🧑🔬 Turing Award Winner Yoshua Bengio will kick off the forum with a keynote address on the state of scientific understanding on AI safety. Other keynote speakers include Academician 𝐆𝐀𝐎 𝐖𝐞𝐧, Academician 𝐙𝐇𝐀𝐍𝐆 𝐘𝐚-𝐐𝐢𝐧, and UC Berkeley Professor Dawn Song, who will share their perspectives on the risks and countermeasures for advanced AI systems.  🧪 Evaluating AI systems for dangerous risks is a nascent discipline, and we are excited to feature talks by leading experts including 𝐐𝐈𝐀𝐎 𝐘𝐮 (Shanghai AI Lab), 𝐖𝐄𝐈 𝐊𝐚𝐢 (China Academy of Information and Communications Technology), and Chris Meserole (Frontier Model Forum).  🏛️ Jurisdictions around the world are experimenting with their own AI governance frameworks. To explain the AI governance approaches in their respective countries, we have invited Gael Varoquaux from France’s Inria research center, Singapore’s Chief AI Officer Ruimin He, 𝐙𝐇𝐀𝐍𝐆 𝐋𝐢𝐧𝐠𝐡𝐚𝐧 from China University of Political Science and Law, and Mark Nitzberg from UC Berkeley’s Center for Human-Compatible AI.  🤝 International cooperation is also essential for ensuring AI safety globally. Our keynote speeches for this theme will be given by Mariano-Florentino (Tino) Cuéllar (Carnegie Endowment for International Peace), 𝐗𝐔𝐄 𝐋𝐚𝐧 (Tsinghua), Yi Zeng (Chinese Academy of Sciences), Irene Solaiman (Hugging Face), 𝐑𝐨𝐛𝐞𝐫𝐭 𝐓𝐫𝐚𝐠𝐞𝐫 (Oxford), and Duncan Cass-Beggs (Centre for International Governance Innovation). 📣 Leading AI scientist and industry veteran 𝗭𝗛𝗢𝗨 𝗕𝗼𝘄𝗲𝗻 will give closing remarks for our forum. Attendees can view the agenda online, and register to attend in-person or livestream directly for the morning (https://lnkd.in/gif4F_QG) and afternoon (https://lnkd.in/g9XdgSuc) sessions. Read more on our Substack for the schedule and full lineup of speakers: https://lnkd.in/gwj7R8wN. We are excited for you all to join us next week! #AI #WAIC2024 #Governance #China #Technology #Innovation #ConcordiaAI

🚨New Report Launch: State of AI Safety in China (Spring 2024)🚨 As the China-US intergovernmental dialogue on AI commences today, Concordia AI is excited to present an update to our State of AI Safety in China report. Since our report was first published in October 2023: ✅ There has been a big uptick in the quantity and relevance of Chinese AI safety research, with 15 Chinese technical papers on frontier AI safety per month on average in the past 6 months. The report identifies 11 key research groups who have written a substantial portion of these papers. ✅ China’s decision to sign the Bletchley Declaration, issue a joint statement on AI governance with France, and pursue an AI dialogue with the US indicates a growing convergence of views on AI safety among major powers compared to early 2023.  ✅ China's national policy and leadership show interest in developing large models while managing risks  ✅ Unofficial expert drafts of China’s forthcoming national AI law contain provisions on AI safety, such as specialized oversight for foundation models and stipulating value alignment of AGI.  ✅ Influential industry associations established projects or committees to research AI safety and security problems, but their focus is primarily on content and data security rather than frontier AI safety.   ✅ Chinese experts have discussed topics including AI "red lines," minimum funding for AI safety research, and AI's impact on biosecurity. With major events on the horizon like the AI Seoul Summit 🇰🇷, UN Summit of the Future 🇺🇳, and the Shanghai World AI Conference 🇨🇳, our report provides timely and relevant insights into the rapidly evolving landscape of AI safety in China. 📌 Sign up to our Report Launch Webinar to discuss China’s role in AI and more with experts Jeffrey Ding, Matt Sheehan, Robert Trager, and Angela Zhang on May 15 at 9 AM ET / 2 PM BST / 9 PM China Time: shorturl.at/dwMQ1. 📥 View the full report as a PowerPoint (https://lnkd.in/d-A_nfBQ) or as a PDF (https://lnkd.in/djWPvuA4). P.S. Concordia AI will also be attending the AI Seoul Summit next week––stay tuned 👀