SiFive’s Post

🗞 Electronic News! 🗞 Fujitsu is gearing up to reclaim the top spot in the supercomputer world with the development of a cutting-edge 2nm ARM-based chip for the successor to the renowned Fugaku supercomputer. The new Fujitsu-Monaka chip is built on the ARM v9-A architecture, incorporating the Scalable Vector Extension 2 extensions specifically designed for machine learning and AI workloads. #electricalengineering #electronics #embedded #embeddedsystems #electrical #computerchips Follow us on LinkedIn to get daily news: HardwareBee - Electronic News and Vendor Directory

Tachyum Prodigy new Processor ?. -------------------------------------- Market Presence: You can add a sentence mentioning that mass production of the Tachyum Prodigy CPU is expected to begin in the second half of 2024, according to press releases from Tachyum source: Tom's Hardware. It's important to note that these are still new processors, and their actual performance and efficiency will need to be evaluated through independent benchmarks once they are available. Comparison with other architectures: You can add a point about software compatibility. Since Tachyum Prodigy CPUs have a unique architecture, existing software may need to be recompiled or optimized to take full advantage of their capabilities. This could be a challenge in the initial stages of adoption. Here's the modified section for reference: Market Presence: Mass production of the Tachyum Prodigy CPU is expected to begin in the second half of 2024. As a new architecture, Prodigy CPUs may face challenges with software compatibility in the initial stages. However, their innovative design and potential performance benefits could attract attention in the future, especially in data center and HPC markets. Overall, the information you provided is excellent. Keep in mind that the tech industry evolves rapidly, so staying updated on the latest developments regarding the Prodigy CPU's performance and market adoption would be beneficial. for more information's:

"The company said the technology translates into 80 times faster bandwidth than today’s chip-to-chip communication that uses electrical technology, and it will reduce energy consumption by more than five times. It can also train a large language model (LLM) up to five times faster, reducing the time it takes to train a standard LLM from three months to three weeks, with performance gains increasing by using larger models and more GPUs. In addition to allowing GPUs and accelerators to talk to each other much faster, it could also redefine the way the computing industry transmits high-bandwidth data through circuit boards and servers"

#Chiplets are small, but they #enable big things in the #computing #world. Their adoption has seen a steep uptick in recent years, thanks in large part to the exponential growth of artificial intelligence (#AI) that’s putting increasing pressure on servers, high-performance computing (#HPC), and data centers. Rather than consolidate every part into a single #chip using a #monolithic approach, chiplets are segmented with specific segments manufactured as separate chips, which are mounted into a single, interconnected #package. A key benefit is that the different parts within a chiplet can leverage the latest fabrication methods and #shrink in size so more components can be squeezed in. https://lnkd.in/eaiazzbN

Memory-Efficient Fine-Tuning of LLMs through Sparse Adapter and Mixture of Experts Parameter-Efficient Fine-tuning (PEFT) methods on complex, knowledge-intensive tasks is constrained due to limited number of additional trainable parameters. To address this, authors of this paper introduce a novel mechanism called Memory-Efficient Fine-tuning(MEFT) that fine-tunes LLMs with adapters of larger size yet memory-efficient. 𝗞𝗲𝘆 𝗰𝗼𝗻𝘁𝗿𝗶𝗯𝘂𝘁𝗶𝗼𝗻𝘀: - propose MEFT, a novel method that utilizes sparse activations and MoE for memory-efficient fine-tuning - MEFT reduces communication overhead by artificially limiting number of activated neurons copied to GPU memory from CPU memory - propose a Key-Experts mechanism to partition a large number of parameters, using a router to allocate inputs to corresponding experts, reducing computational overhead - results show that this method achieves the best results under resource-restricted conditions 𝗠𝗘𝗙𝗧 𝗢𝘃𝗲𝗿𝘃𝗶𝗲𝘄 - During forward propagation stage, output of attention block will be transferred to the CPU to efficiently retrieve neurons highly related to the current context using a MoE-like structure, after which the activated neurons will be transferred to the GPU - During backward propagation, gradients are transferred to CPU and parameters are updated on CPU - repeated across all the transformer layers 𝗠𝗘𝗙𝗧 𝗠𝗲𝘁𝗵𝗼𝗱 i) Sparse Activation - sparse Adapter training by selectively updating only those neurons that demonstrate higher activation - during forward computation, keys with highest similarity are retrieved and activated for each FFN layer - for backward propagation, only gradients of these activated neurons are updated, as non-activated neurons do not contribute to the computation of FFN ii) Key-Experts Mechanism - proposed mechanism is based on idea of mixture-of-experts, where weights are divided into N partitions (experts) and a router R is employed to route input to some specific experts. iii) Efficiency Analysis - For training efficiency, empirical results show that MEFT method achieves at least 63% speed compared to baseline that excludes time cost of additional communication and CPU computation 𝗘𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁𝗮𝗹 𝗥𝗲𝘀𝘂𝗹𝘁𝘀 - For knowledge-intensive tasks such as NQ and SQuAD, MEFT outperforms other PEFT approaches like Parallel Adapter and LoRA, when operating under same 24G GPU memory constraints - For non knowledge-intensive tasks, such as GSM8k, increasing number of trainable parameters does not yield better performance, but does not compromise performance 𝗟𝗶𝗺𝗶𝘁𝗮𝘁𝗶𝗼𝗻𝘀 - generalization ability of LLMs has not been fully explored - lacks testing in scenario of continuous learning - amount of parameters recalled increases with length of training sequence, limiting its applicability 𝗕𝗹𝗼𝗴 𝘄𝗶𝘁𝗵 𝗮𝗻𝗮𝗹𝘆𝘀𝗶𝘀: https://lnkd.in/eybCFe8p 𝗣𝗮𝗽𝗲𝗿: https://lnkd.in/eQxzzBE9 𝗖𝗼𝗱𝗲: https://lnkd.in/eimUi9He

🗞 Electronic News! 🗞 US startup Inspire Semiconductor has collaborated with Belgian research lab imec to develop a groundbreaking chip design featuring 1,536 64-bit custom RISC-V CPU cores interconnected with low latency. The innovative 24TOPS RISC-V Thunderbird 1 chip, created by InspireSemi, is currently in the fabrication process at TSMC. The plan is to incorporate four of these chips onto a PCIe server card optimized for double precision 64-bit FP64 calculations. Targeted towards High Performance Computing , AI, graph analytics, and other compute-intensive tasks, the Thunderbird chip offers exceptional capabilities. It allows for the connection of arrays containing up to 256 Thunderbird chips through high-speed SERDES transceivers, enabling scalable and powerful computing solutions. #electricalengineering #electronics #embedded #embeddedsystems #electrical #computerchips Follow us on LinkedIn to get daily news: HardwareBee - Electronic News and Vendor Directory

🗞 Electronic News! 🗞 US startup Inspire Semiconductor has collaborated with Belgian research lab imec to develop a groundbreaking chip design featuring 1,536 64-bit custom RISC-V CPU cores interconnected with low latency. The chip, known as the 24TOPS RISC-V Thunderbird 1, is currently in the fabrication process at TSMC. It is noteworthy that four of these chips will be integrated into a PCIe server card designed to support double precision 64-bit FP64 calculations. Targeted towards High Performance Computing , AI, graph analytics, and other compute-intensive tasks, the Thunderbird 1 chip offers exceptional capabilities. It allows for the connection of arrays comprising up to 256 Thunderbird chips through high-speed SERDES transceivers, enabling scalable and powerful computing solutions. #electricalengineering #electronics #embedded #embeddedsystems #electrical #computerchips Follow us on LinkedIn to get daily news: HardwareBee - Electronic News and Vendor Directory

A closer look at Arm #chiplet game plan! Arm advances chiplet design through partnerships. Our collaboration integrates Arm Neoverse CSS into custom silicon and connectivity #chiplets, enabling seamless integration of CXL, HBMx, DDRx, and Ethernet onto Arm-based SoCs. We recently developed an advanced compute chiplet on Arm Neoverse CSS for AI, ML, HPC, and 5G/6G networking. Arm’s focus on AMBA specifications and industry standards like #UCIe and #PCIe sets benchmarks in system design. Read the full article by EDN: Voice of the Engineer to learn more: https://bit.ly/4crQ3Ey #AlphawaveSemi #ConnectivityIP #ConnectivitySolutions #Chiplets #AI #CustomSilicon #SystemDesign #Arm #Collaboration

Pitfalls and Design Considerations One challenge with RDMA and GPUDirect RDMA is the complexity of memory management. RDMA relies on pinning memory, which can be an expensive operation, potentially taking milliseconds for large memory regions. This is a critical consideration, especially when deploying RDMA in systems that need to scale to thousands of nodes For this reason, optimizations such as lazy unpinning—where memory is pinned only when absolutely necessary—are often employed to reduce overhead. In summary, NVIDIA's Blackwell architecture combined with InfiniBand RDMA creates an immensely powerful platform for AI and HPC. The reduction in latency and the ability to handle massive data flows without CPU involvement make it ideal for training, inference, and large-scale distributed computing

For AI market development, the first step is to establish high performance & secure data centers for enabling AI application services. This AI server is what current AI market focuses on. After the establishment of AI data centers, the 2nd AI wave is the secure AI endpoint devices to use AI services by data centers. The coming blooming of AI endpoint devices will be the main stream of AI era. In the AI era, lots of endpoint devices need to be changed and adopt the new AI architecture for high performance and secure data computing.