CEA-List’s Post

🧪 New Machine Learning Research: Optimizing Neural Networks with MetaMixer Researchers from the University of Seoul-서울시립대학교 have conducted a study on improving the efficiency and performance of neural networks through a new architecture called MetaMixer. - Research goal: Propose a new mixer architecture, MetaMixer, to optimize neural network performance by focusing on the query-key-value framework rather than self-attention. - Research methodology: They have developed MetaMixer by replacing inefficient sub-operations of self-attention with Feed-Forward Network (FFN) operations, and evaluated the performance across various tasks. - Key findings: MetaMixer, using simple operations like convolution and GELU activation, outperforms traditional methods. The study found that the new FFNified attention mechanism improves efficiency and performance in diverse tasks. - Practical implications: These advancements can lead to more efficient neural networks, reducing computational costs and improving the performance of AI models in applications such as image recognition, object detection, and 3D semantic segmentation. #LabelYourData #TechNews #DeepLearning #Innovation #AIResearch #MLResearch 

Approximation and interpolation of deep neural networks Vlad-Raul Constantinescu, Ionel Popescu Abstract In this paper, we prove that in the overparametrized regime, deep neural network provide universal approximations and can interpolate any data set, as long as the activation function is locally in $L^1(\RR)$ and not an affine function. Additionally, if the activation function is smooth and such an interpolation networks exists, then the set of parameters which interpolate forms a manifold. Furthermore, we give a characterization of the Hessian of the loss function evaluated at the interpolation points. In the last section, we provide a practical probabilistic method of finding such a point under general conditions on the activation function. 👉 https://lnkd.in/di43XTPc #machinelearning

🚀#mdpisignals Excited to share an article with you. "Graphical User Interface for the Development of Probabilistic Convolutional Neural Networks" 🔸https://lnkd.in/eRX3ZvFA 🔹Authors: Aníbal Chaves et al. University of Madeira Students' Union 🔸Abstract: Through the development of artificial intelligence, some capabilities of human beings have been replicated in computers. Among the developed models, convolutional neural networks stand out considerably because they make it possible for systems to have the inherent capabilities of humans, such as pattern recognition in images and signals. However, conventional methods are based on deterministic models, which cannot express the epistemic uncertainty of their predictions. The alternative consists of probabilistic models, although these are considerably more difficult to develop... #artificial_intelligence; #neural_network; 

🚀 Two weeks ago, I shared a comprehensive guide on Medium diving into the architecture of Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM) networks. Mastering these fundamental concepts is crucial for unlocking powerful capabilities in sequence models. Check out the full article for a comprehensive guide: https://lnkd.in/dSTirN6A 💡 leave a few claps if you found it insightful 👏👏 " Stay tuned for my upcoming article, where I'll dive into the Transformer architecture, focusing on the encoder structure. ⏳ " Remember, sharing is caring! 🤜🤛 #RNN #LSTM #NeuralNetworks #DeepLearning #ArtificialIntelligence #MachineLearning #DataScience #NaturalLanguageProcessing #AI #Tech #ShareIsCare

Wanna design optimal observables for CP violation searches, with better convergence properties and improving on the state of the art? Well, use our equivariant neural networks, of course: check our new paper out, https://lnkd.in/eUWVR_2G , led by Sergio Sánchez Cruz

**Unveiling the Future of Neural Networks: Liquid Neural Networks (LNNs)** Are you ready to witness the next evolution in neural networks? Liquid Neural Networks (LNNs) are revolutionizing data processing and real-time adaptation, taking inspiration from the brain of C. elegans. With just 302 neurons, these networks exhibit complex behaviors, all explained in our latest blog post. Dive into the world of LNNs and discover their potential in processing time-series data and dynamically adapting to changing conditions in real time. Learn more at GOVCRATE.org: [What are Liquid Neural Networks and What Makes Them Better]() #NeuralNetworks #Innovation #GOVCRATEBlog #govcrate #whupi #pacifictech

Training Neural Networks is NP-Hard in Fixed Dimension Vincent Froese, Christoph Hertrich Abstract We study the parameterized complexity of training two-layer neural networks with respect to the dimension of the input data and the number of hidden neurons, considering ReLU and linear threshold activation functions. Albeit the computational complexity of these problems has been studied numerous times in recent years, several questions are still open. We answer questions by Arora et al. [ICLR '18] and Khalife and Basu [IPCO '22] showing that both problems are NP-hard for two dimensions, which excludes any polynomial-time algorithm for constant dimension. We also answer a question by Froese et al. [JAIR '22] proving W[1]-hardness for four ReLUs (or two linear threshold neurons) with zero training error. Finally, in the ReLU case, we show fixed-parameter tractability for the combined parameter number of dimensions and number of ReLUs if the network is assumed to compute a convex map. Our results settle the complexity status regarding these parameters almost completely. 👉 https://lnkd.in/dTeexG3J #machinelearning

Using a dataset of transformed images, machine-learning models can simulate peripheral vision effectively, boosting their ability to detect and recognize objects located off to the side or in the corner of a scene. Deep neural networks (DNNs) have demonstrated considerable potential as models of human visual perception, allowing for the prediction of both neural response patterns and aspects of visual task performance. However, there are still significant discrepancies in how computer vision DNNs handle information compared with humans. These distinctions are clear in psychophysical experiments and adversarial instances. Read the complete story here: https://lnkd.in/g27tdZHU nasscom Ministry of Electronics and Information Technology Digital India Programme INDIAai Sushil Kumar Jangid Kavita Bhatia Massachusetts Institute of Technology Abhishek Singh #deeplearning #computervision #visualization

Convolutional neural networks (CNNs) are commonly used for instance segmentation. A model is trained on annotated data containing masks corresponding to each instance of the object. By analyzing the training data and extracting patterns and features, the model is able to identify and segment objects.

🎉 Excited to announce my first publication in the prestigious IEEE Access Q1 Journal! 📊🧠 Title: "Exploring the Impact of Conceptual Bottlenecks on Adversarial Robustness of Deep Neural Networks" As a first author alongside Bader Rasheed, I'm thrilled to share our research on the robustness of Concept Bottleneck Models (CBMs) against adversarial attacks. Our study reveals that CBMs not only enhance interpretability but also demonstrate improved defense capabilities against various adversarial attacks compared to traditional Convolutional Neural Networks (CNNs). Key findings: - CBMs maintain higher accuracy under adversarial conditions - Sequential training of CBMs shows inherent robustness against attacks - Exploration of conceptual complexity and adversarial training techniques This work contributes to the ongoing efforts to create more robust and interpretable deep learning models, with potential applications across various domains. I'd like to thank my co-authors for their invaluable contributions: Bader Rasheed, Adil Khan, Igor Menezes, and Asad Masood Khatak Looking forward to further research and collaborations in this exciting field! Read the full paper: https://lnkd.in/gU4Fwa_q #MachineLearning #AdversarialRobustness #DeepLearning #IEEE #Research