DeepLearning.AI’s Post

Managing Machine Learning Projects (Free Duke University Course). The course walks through the keys steps of a ML project from how to identify good opportunities for ML through data collection, model building, deployment, and monitoring and maintenance of production systems. Participants will learn about the data science process and how to apply the process to organize ML efforts, as well as the key considerations and decisions in designing ML system. #ML #machinelearning #freecourse

I recently completed a machine learning course and earned a certification, enhancing my skills in developing predictive models and applying advanced techniques to solve complex problems.

  Kernel tricks in SVM are wrongly understood most of the times. The simple innovation on mathematical side is often ignored by learners. For non linearly separable data, converting them to higher dimensions makes them linearly separable. But converting into higher dimension is problematic - time consuming, right functions to be obtained. At the end of the day, if you want a hyper plane separating set of points, all you need is all possible dot products between the points. So, even in higher dimensions, all we need is dot product between all possible set of points. Kernel Functions are functions which give that dot product between any two points converted into higher dimensions without actually transforming them. So kernel function is not actually transforming data into higher dimensions. It takes two points in original dimensions and returns dot product of points in higher dimension. It just gives what you want as an end outcome - the dot product / similarities between points in higher dimensions with which you can figure out the hyperplane separating the points. And this is less computationally expensive.

In the realm of science and technology, models serve as indispensable tools for understanding complex phenomena, making predictions, and guiding decision-making processes. Among the diverse array of models, computational models, mathematical models, and statistical models stand out for their distinct approaches and applications.

What is Feature Scaling and Why is it Important (analyticsvidhya.com)? This is an article about feature scaling in machine learning. It discusses why it is important and the different techniques used. Feature scaling helps improve the performance of machine learning models. It can also improve the convergence of gradient descent algorithms. There are three main techniques for feature scaling: normalization, standardization, and min-max scaling. Normalization scales the data to a range of 0 to 1, while standardization scales the data to have a mean of 0 and a standard deviation of 1. Min-max scaling scales the data to a specific range defined by the user. The choice of which technique to use can depend on the machine learning algorithm being used. PS : Applying the rightly developed math to achieve optimized results

I had the joy of reading a recent preprint, "Questionable practices in machine learning" by Leech et al. It echoes many of the sentiments I've had for the past couple of years. I wrote some thoughts about it here: https://lnkd.in/e_HkFY9n

How does synthetic data affect model training? This paper set out to look at what happened to increasing generations of ML model that had been trained only on synthetic data from the previous generation. Over the generations, the paper found that the models got increasingly worse until the performance severely degraded. The authors call this 'model collapse'. The main reason driving model collapse is that you need infinite synthetic samples to fully capture the tails of the underlying distribution. In practice though, you can only generate a finite amount of synthetic data. So, over generations, these tails get modelled less well and the distribution of the synthetic data diverges from that of the initial real-world data. This setup is hypothetical, but shows us how model collapse could happen as generations of LLMs are used to generate text, that text is published on the internet, and then is used for training the next generations of LLM. #artificialintelligence #largelanguagemodels

VisionTS: Building Superior Forecasting Models from Images Leveraging the power of images for time-series forecasting Created by author using DALLE*3 Which is the biggest challenge when building a pretrained time-series model? Answer: Finding high-quality, diverse time-series data. We’ve discussed this in previous articles. There are 2 main approaches to building a foundation forecasting model: “Bootstrap” an LLM: Repurpose a pretrained LLM like GPT-4 or Llama by applying fine-tuning or tokenization strategies tailored for time-series tasks. “From scratch“: Build a large-scale time-series dataset and pretrain a model from scratch, hoping it generalizes to new data. While the 1st approach works since Transformers are general-purpose computation engines, it doesn’t yield the best results. The 2nd approach has been more successful as seen here: MOIRAI, TimesFM, TTM, etc. However, these models seem to follow the scaling laws and their performance depends on finding extensive time-series data — which brings us back to the original challenge. But what if we could leverage a different modality, like images? This might seem counterintuitive, but some researchers explored this hypothesis and produced groundbreaking results. In this article, we’ll discuss: How do images internally encode sequential data? The concept of using a pretrained computer vision model for time series VisionTS[1], a pretrained Vision Transformer adapted for time-series data. Let’s get started: Find the hands-on project for VisionTS in the AI Projects folder, along with other cool projects! I will write a companion article for this tutorial, so stay tuned! Why use Images? Images are a 2D sequence of pixels. Hence, an image is a pixel array of numerical values — displaying known features of real-world time series, like trend, seasonality, and stationarity. (Figure 1) Figure 1: An image from the popular ImageNet dataset, exhibiting the familiar characteristics of time series (Source) As discussed earlier, pretrained text models (bootstrapped LLMs) have been used to transfer knowledge to time-series tasks but with limited success. So, what advantages do images offer? Continuous modalities: Both time series and images are continuous — unlike text which is discrete. Similar origin: Time series and images capture environmental observations — while text is a cognitive construct. Comparable information density: Text is dense with meaning, whereas images and time-series data are natural signals with more redundancy. Images encode sequential information: Unlike text, images exhibit many characteristics of time series (Figure 1). Thus, images seem like a promising modality. As Yann LeCun mentioned on Lex Fridman’s podcast, text alone is insufficient for building a powerful AGI. Images, being richer and high-dimensional, offer a deeper understanding of the world. They are also far more abundant than the other modalities — think the amount of data a LIDAR...

Equal width and equal frequency are NOT the only discretization methods. Most of us are aware of these methods to sort continuous variables into discrete intervals. And they are indeed the most commonly used. 🤔 But people have spent a lot of time trying to find better ways to discretize variables. If you don't believe me, check out this table summarizing many of the discretization methods designed so far - https://buff.ly/4fiqUOn Don't worry, many of these methods are super complex and therefore not suitable for most machine learning applications. But if you want to find the best possible way in which to create intervals for your continuous variables, now you know where to begin 😀 Want to learn more? Check out our Feature Engineering course here: https://buff.ly/3jrNOuQ #FeatureEngineering #MachineLearning #DataScience

Machine learning parallel system for integrated process-model calibration and accuracy enhancement in sewer-river system Science Direct https://lnkd.in/dn2zZFTN