MONTREAL.AI’s Post

DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Data Xin et al.: https://lnkd.in/eRYXxgx9 #ArtificialIntelligence #DeepLearning #MachineLearning

It’s a variant of the K-Means algorithm, renowned for its simplicity and efficiency in clustering large datasets. Read more 👉 https://lttr.ai/ATgut #DynamicWorld #MachineLearning #ConstantQuest #ScalableSolution #ApproachDataSegmentation #OfferingValuableInsights #DataSegmentationStrategies #MiniBatchKMeans #MachineLearningUncover #ClusteringLargeDatasets

I had a great time talking about Synthetic Data for Computer Vision at the #Bristol  MLOps Community #Meetup. If you couldn't attend in person, check out the video below.. #AI #ML #DeepLearning #SyntheticData

Kolmogorov-Arnold Networks (KANs) are redefining neural network architectures and can outperform traditional MLPs. In our latest article, we dive into: - KANs' story and architecture - Are KANs an improved MLP? - Advantages and limitations of KANs https://lnkd.in/egY9A9Hy

Autoencoders are powerful neural network architectures for dimensionality reduction. In this article, I make a study of the fundamentals of how we can use it, and I create one example of a simple stacked autoencoder using Keras. I also published my notebook on Kaggle. The link is in the article! Feel free to comment and give your suggestions! #deeplearning #neuralnetworks #keras #datascience

In the rapidly evolving landscape of machine learning, often the focus is predominantly on model architectures and training algorithms. However, the unsung hero behind many breakthroughs, including CLIP, Stable Diffusion, and GPT-4, is the large multimodal datasets. Last October, Assistant Professor Ludwig Schmidt from the University of Washington introduced an innovative approach to enhancing the quality of these datasets through DataComp. This benchmark allows researchers to propose new training sets while keeping the training code fixed. DataComp provides a testbed for dataset experiments centered around a new candidate pool of 12.8B image-text pairs from Common Crawl. Researchers design new filtering techniques or curate new data sources and then evaluate their new dataset by running the standardized CLIP training code and testing on 38 downstream test sets. Video here: https://lnkd.in/gWKtj9UM

Using k-fold cross validation in real-world applications is flawed due to "data leakage," where future information is inadvertently used, leading to overly optimistic results. Cross-validation ignores the effects of order, space, or time, which are crucial in reality. The group k-fold method avoids leakage by considering these factors, providing accuracy scores closer to the ground truth. Cross-validation is overly optimistic, prone to overfitting, and doesn't accurately reflect estimate variability.

In the spirit of the holidays, I am posting "Twelve Days of VAST Data" this year - twelve things providing tremendous benefits to our customers and VASTly differentiating us from the other players in the market. #8 - VAST DataSpace - Our edge-to-cloud global namespace enabling access to your data anywhere, without the usual tradeoffs. The VAST DataSpace decentralizes and moves lock management to the element (file, object, table), allowing the lock to be held by the VAST cluster performing the transaction. Check out this video to learn more.   https://lnkd.in/eC3P8KmE   #AI #Artificialintelligence #data #dataplatform #VASTronauts #Storage #HPC VAST Data

Looking to understand the effects of parallelism for LLM inference on Mixture of Experts models, you have to juggle - Tensor, Pipeline, Data and Expert Parallelism. In addition you have to understand the effects of Inflight-batching and Chunking on the Prefill (attention) phase of inference. Chunking (breaking up attention) helps with processing of long sequences so it does not interrupt other generations from other requests. Plot shows 1.8T parameter model distributed across 64GPUs (H100, H200) with different levels of parallelism and what it does to the latency (TTFT) and throughput. ** Article Link in comments ** tp - tensor parallelism, pp = pipeline parallelism, dp = data parallelism, ep = expert parallelism #llm #deeplearning #machinelearning

"Overcoming barriers to innovation" is so poignant. Interviewing Chris Andrews demystified the purpose of synthetic data for me. There simply isn't enough data to train algorithms...so, what do we do? If you're new to this topic, I urge you to check out the full interview below - it's a great primer! And if you're already up on all things #GenAI, it's still worth watching to hear Chris describe how Rendered.ai fills the data void in a myriad of use cases. NVIDIA Omniverse #GenerativeAI #data #syntheticdata #MachineLearning #GTC24 #innovation