Algorithms’ Post

In today's economy, the time for experimentation and science projects is over - Adaptability is the name of the game. Resources are scarce, money is hard to find and the technology landscape is shifting. How do you future-proof your applications? How do you ensure they evolve with the next big trend? We know one thing - if your data layer isn't flexible, has rigid schema constraints and requires significant remodeling when new data types are introduced, you're dead in the water. If you can't swiftly refine and scale your models as new features and requirements arise, you'll lose out on marketshare, developer talent and roadmap objectives. You NEED to be able to ingest diverse data sets directly into the database. Field additions NEED to be dynamic. And data structure changes NEED to be frictionless. Remember: 80% of data created worldwide is unstructured. If it isn't effortless to handle both structured and unstructured data, your application isn't adaptable.

"Nixtla’s TimeGPT is a generative pre-trained forecasting model for time series data. The TimeGPT model looks at windows of past data (tokens), and predicts what comes next, without training. This prediction is based on patterns the model identifies in past data and extrapolates into the future. The API provides an interface to TimeGPT. TimeGPT can also be used for other time series-related tasks, such as what-if scenarios, anomaly detection, ... " https://lnkd.in/dVNcvtFi

🧠 Understanding Regularization When you're building a machine learning model, a big challenge is avoiding overfitting. Overfitting happens when a model performs excellently on training data but poorly on new, unseen data. It's like cramming for an exam and only knowing the exact questions on the practice test but failing when new questions appear. To prevent this, we use techniques like Lasso and Ridge regression. 1. 👥 Ridge Regression - What it is: Think of Ridge regression like a coach ensuring the team doesn't rely too heavily on one star player. It spreads the importance across ALL features. - Benefit: This balance helps the model perform better with new data. 2. 🎯 Lasso Regression - What it is: Lasso regression is like a manager focusing only on the MOST IMPORTANT tasks and ignoring the rest. - Benefit: It simplifies the model by zeroing in on key features and discarding the unnecessary ones. 3. Which to Use When? - 👥 Use Ridge when you think all the features in your model have some importance, but you want to avoid relying too heavily on any single one. It's like making sure everyone on the team plays a part. - 🎯 Use Lasso when you believe that only a few features are really crucial. Lasso helps by keeping those key features and ignoring the noise, much like trimming down to just the essentials. Ridge balances importance across features, while Lasso focuses on the essentials.

Ever tried finding patterns like 𝗵𝗲𝗮𝗱 𝗮𝗻𝗱 𝘀𝗵𝗼𝘂𝗹𝗱𝗲𝗿𝘀 or 𝗱𝗼𝘂𝗯𝗹𝗲 𝗯𝗼𝘁𝘁𝗼𝗺𝘀 in millions of data points? It’s slow, tedious, and often doesn't scale well with traditional methods. That’s the exact problem I tackled with T𝗲𝗺𝗽𝗼𝗿𝗮𝗹 𝗦𝗶𝗺𝗶𝗹𝗮𝗿𝗶𝘁𝘆 𝗦𝗲𝗮𝗿𝗰𝗵 (𝗧𝗦𝗦), a direct pattern matching approach that scales to millions of time-series data points—fast. I ran a test on 10 𝗺𝗶𝗹𝗹𝗶𝗼𝗻 𝘀𝘆𝗻𝘁𝗵𝗲𝘁𝗶𝗰 𝗺𝗮𝗿𝗸𝗲𝘁 𝗱𝗮𝘁𝗮 𝗽𝗼𝗶𝗻𝘁𝘀 and identified classic patterns like 𝗰𝘂𝗽 𝗮𝗻𝗱 𝗵𝗮𝗻𝗱𝗹𝗲 in well under a second. No heavy feature engineering, no ML models—just direct comparison between time-series vectors. This method saves hours of manual work and speeds up everything from backtesting to real-time signal detection. I was able to detect any synthetic pattern I wanted, no matter how complex, simply by defining an example. Here’s what stood out: • 𝗠𝗮𝘀𝘀𝗶𝘃𝗲 𝘀𝗰𝗮𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆: TSS processes millions of data points without bottlenecks, ideal for large datasets and real-time market analysis. • 𝗖𝘂𝘀𝘁𝗼𝗺 𝗽𝗮𝘁𝘁𝗲𝗿𝗻 𝗺𝗮𝘁𝗰𝗵𝗶𝗻𝗴: You can define and search for any pattern—traditional or custom—across huge datasets. • 𝗜𝗺𝗺𝗲𝗱𝗶𝗮𝘁𝗲 𝘀𝗶𝗴𝗻𝗮𝗹 𝗱𝗲𝘁𝗲𝗰𝘁𝗶𝗼𝗻: Use it in live trading environments to spot emerging patterns instantly, without the lag of machine learning pipelines. Curious about the implementation or how it fits into your workflow? Check out the link to my article on using TSS for technical analysis in the comments!

🌳📈 Have you ever heard of Decision Tree? It is a powerful machine learning model used for classification and prediction based on historical data divided into relevant features. Its structure includes: 1. Root Node: the starting point that represents the main problem to be solved. 2. Decision Nodes: the points where the tree branches out based on specific characteristics of the data, dividing the data into smaller groups. 3. Outcome Nodes: the final points that show the final result or prediction, such as a class or a numerical value. 4. Branches: the lines that connect the nodes and represent the different options or responses for each decision. ✨ Using this tool offers greater visual clarity, facilitates decision-making, and improves the documentation of the choices made!

All in 1 : Outstanding listings of Machine Learning Error functions. Truely expensive work ! Hats off to Avi Chawla and thanks for guiding ML enthusiasts. #MLengineering #Lossfunctions #Regression #Classification #machinelearningalgorithms #optimisation

No one really knows how and why the foundation models work! But the companies building these models can at least be more transparent about what they do know. Came across this recent paper from Rishi Bommasani and Kevin Klyman on the concept of Foundation Model Transparency Reports. Authors use 100 transparency indicators from the Foundation Model Transparency Index and identify 6 design principles based on the successes and shortcomings of social media transparency reporting. These principles are: 1. Centralization: single and predictable source for finding relevant information. 2. Structure: address specific queries with clarity. 3. Contextualization: how does the model perceive its users. 4. Independent specification: don’t be selective in what information to share. 5. Standardization: specifically address all the points in the report. If certain info is omitted, explain why. 6. Methodologies: how are the statistics computed in evaluating the models. Very cool framework! Hope it is adopted like model cards and data sheets by the industry. The best part of Foundation Model Transparency Reports is that they also take account of the regulatory compliance (if any) of models. Paper attached, enjoy!

By the end of next year IDC estimates that current classification and management options will no longer be viable ways to mange your data! ⏰ Ready to learn about automated data classification? 💡🗂 Join this FREE webinar to learn... 🚨 Major issues organisations face with data classification and analysis 🚮 Reasons to ditch manual data classification method ⏰ How to classify content in weeks, as opposed to years if done manually 📑 How to remove barriers for users to find information fast Join Alyssa for the webinar using the link below and discover how AvePoint x PMD Data Solutions can get you on top of your unstructured data. https://avpt.co/3ZtgdTD 

Linearization of ratio metrics combines the trustworthiness of Bootstrapping and the Delta method with the simplicity and computational effectiveness of simpler approaches. Plus, it allows us to perform variance reduction techniques over our ratio metrics. Overall, it can make your experiments incredibly efficient. Check out my post on Medium for more: https://lnkd.in/dbZQzWKC #abtesting #dataanalysis #conversionrateoptimization

How many IT Admin hours could you save by automating Data Classification? 🤔 Join our partner hosted webinar to find out and start your journey... #datacompliance #dataprotection #datamanagement #pmddatasolutions