Quant Beckman’s Post

There are numerous protocols for evaluating a model. However, I don’t believe there is a single recipe where you plug in the model and immediately gain a certain degree of certainty. Most are dependent on multiple development processes. That said, personally, I like to use three distinctions to structure protocols that confirm whether it is worth moving a system to the incubation phase, and once in that phase, to verify whether it makes sense to deploy it for live #trading: A. Validation → Testing the model’s accuracy. -Validation is the process of checking whether the model correctly represents the inneficiency it’s supposed to describe -Validation is not a one-time process -As the model evolves and receives new data, it needs to be continuously validated -During validation, the model judges its own accuracy based on past performance -This is crucial in time-dependent models where the system itself may change B. Valuation → Assigning importance to outcomes. -Valuation is about determining the importance or worth of specific outcomes generated by the model -The model needs to prioritize and weigh different outcomes -As the model evolves, it must constantly re-evaluate the importance of its decisions -Valuation needs to be adaptive to remain effective -Valuation is not simply about maximizing returns but about weighing risks -Over time, the model assesses not just the outcomes but also the associated uncertainties -This process involves calculating expected values and incorporating probabilistic elements, especially under uncertain conditions -A utility function helps formalize valuation by assigning numerical scores to different outcomes based on preferences -Over time, as the model encounters different scenarios, the utility function adjusts its judgments to optimize the outcome according to the criteria embedded in the model C. Evaluation → Measuring performance. -The model continuously evaluates its own performance based on metrics that are tracked -Evaluation involves considering multiple dimensions, such as predictive accuracy, speed of execution, computational efficiency, and the ability to adapt to changing conditions -The model uses this evaluation process to judge its ability to generalize across different conditions -A model must not only evaluate its short-term predictions but also how it performs in the long term -Evaluation becomes more comprehensive as the time horizon increases, incorporating not only performance metrics but also robustness and stability -Over time, the model evaluates trade-offs between these dimensions to find an optimal balance These three processes occur in an ongoing, iterative cycle, creating a dynamic feedback loop that allows the model to assess its performance, adjust its structure, and adapt to evolving conditions: -Validation ensures that the model stays anchored -Valuation allows the model to assess the impact -Evaluation measures the quality of the model

For features-based models, feature selection is done during the model construction. This is the canonical procedure in data science and probably the worst option in #trading. It might sound familiar to you: -Start with all features -Apply feature selection: •Filter methods (correlation coefficient, Chi-square test, Mutual information, Variance threshold…) •Wrapper methods (Forward selection, Backward elimination, Recursive feature elimination…) •Embedded methods (LASSO regression, Decision trees and Random forests, elastic net…) … -Evaluate the model -Fine-tune This gives rise to two opposing views, leading to completely different results in terms of processes and objectives: -N features with N labels → Multi-feature, multi-target -Feature with 0 labels → Single feature, no prediction When you introduce feature selection in the research phase, you realize that the most logical approach in this environment is to go from N features to 1 feature, and from N labels to 0. So, what are you left with? Basically, rules. These rules are tied to a specific benchmark for a particular algorithm: -Instead of: Features ⟷ Labels -You use: Feature ⟷ Benchmark Using feature selection during the quantitative research phase rather than during the model development phase has several advantages: A. Reducing complexity early on. -A well-structured research phase identifies features that simplify the modeling process -Work with fewer, more relevant variables, which can lead to more interpretable models -Lower the computational complexity, making it easier to optimize models during development -Avoid trying to fit the model to an excessive number of variables that may introduce noise and irrelevant complexity B. Improving stability of models. -Models that rely on features selected during the research phase are likely to be more robust over time, as these features are selected based on their fundamental relevance, not just their performance within a specific sample of data -This enhances the stability of the model, as it is less likely to depend on transient patterns in the data -Features that consistently perform well during the research phase are more likely to generalize to different market conditions C. Focusing on long-term predictive power. -The research phase is often more exploratory and focused on the long-term validity of the relationships you're studying -Feature selection at this stage allows for a more thoughtful, data-driven process that avoids the temptation to chase short-term performance improvements -When features are selected based on a deep understanding of their role in explaining the target variable, the resulting models are more likely to have long-term predictive power, even in changing market conditions 👇👇👇Continue👇👇👇

In #trading, risk has become a buzzword, and for many, it's the focal point of obsession. Whether in strategy sessions, market analysis, or trading forums, risk management is constantly at the forefront, driving discussions that seem endless, especially when it comes to workshops and meetings dedicated to refining strategies. Why? Well, it makes a: -Difference by playing a meaningful change -Significant impact and potential measurable consequences -Key results by altering and affecting important outcomes The reasons why it has become a fetish go a bit further: -Risk is the quantifiable probability of an adverse outcome, while uncertainty refers to unknown or unpredictable elements that can’t be measured -Risk can be managed in different phases and timing models -Uncertainty, on the other hand, arises in each phase or stage from factors that are unpredictable or unknown, making it impossible to fully eliminate despite risk management efforts Let’s delve into the first timing model: A. In-Time → Reactive management at the moment. -Risk as perceived or managed in the immediate moment, focusing on present conditions and decisions made based on the current state of the market or system -Characteristics: •Short-term, reactive risk management •Emphasis on mitigating immediate threats or capitalizing on current opportunities •Real-time adjustments to protect against sudden changes B. Between-Time → Assessing transitions. -Risk evaluated between specific time points or events, considering what happens between those intervals and how decisions during one period influence outcomes in the next -Characteristics: •Intermediate focus on transitions, trends, or patterns over time •Looks at how risk evolves from one phase or cycle to another •Risk management involves planning for a certain time frame, accounting for expected events or conditions C. Through-Time → Long-term strategic risk. -Risk seen from a holistic, long-term perspective, focusing on cumulative effects and strategic decisions that span across multiple time periods or the entire lifecycle of a system -Characteristics: •Long-term, proactive risk management •Evaluates how short-term risks contribute to or are mitigated by long-term goals and strategies •Considers enduring trends, structural risks, and sustainability of decisions over time The second model is based on market stages: A. Going into-market → Entry decisions. -The risks involved when entering a market position, making decisions about when and how to initiate trades -Characteristics: •Timing risk: Determining the right moment to enter is crucial; entering too early or too late can expose you to unfavorable price movements •Information risk: Insufficient or inaccurate data at the time of entry can lead to poor decision-making •Liquidity risk: If the market is thin or illiquid, there may be difficulties executing a trade without significant slippage 👇👇👇Continue👇👇👇

Before starting to model the data, it's important to be clear about certain best practices. It’s not just about importing any library and running a .fit(), .predict() without further thought. You need to clearly understand how these elements fit together coherently: -The type of data (at a general level) -The type of inefficiency you are looking for (at a specific level) -The type of model (at a process level) Data digestion → Mapping inneficiency → Signal execution Here you will find some brief guidelines to help guide you: A. The more structure you impose in the trading model, the fewer parameters you need. -Imposing a defined structure in the model helps reduce complexity by minimizing the number of parameters required -This ensures that the model focuses on essential relationships in the market -By relying on a structured approach, the model captures key dynamics without introducing unnecessary variables that could lead to overfitting B. The less parameters, the more efficiently your model cut through noise. -Fewer parameters make a model more efficient at cutting through noise -By simplifying the model, it focuses on the most relevant features -Noise, or random price fluctuations, obscures significant signals -A well-constructed model will separate meaningful information from irrelevant noise, improving decision-making C. Models are useful to filter out noise. -Mathematical and statistical models are indispensable in filtering out the noise -A well-tuned model detects inefficiencies even in the presence of short-term volatility or external factors D. Over-simplified models can filter out signal. -Oversimplifying the model can result in filtering out the signal along with the noise -When models are too basic, they might overlook subtle but critical patterns or trends in the data -The balance lies in maintaining enough complexity to capture essential market movements without introducing excessive parameters that degrade performance E. In a data-rich and high signal-to-noise markets, you wouldn't want to use a small model. -In markets where data is abundant and the signal-to-noise ratio is high (e.g., highly liquid assets or markets with clear trends), a larger model with more parameters may be justified -More parameters allow the model to capture complex dynamics and subtle market signals, leading to potentially more accurate predictions F. With limited data and a low signal-to-noise ratio, simple models avoid overfitting. -On the other hand, in environments with limited data or when the signal-to-noise ratio is low (e.g., niche markets or during periods of high volatility), simpler models are more effective -They prevent overfitting, which occurs when a model becomes too attuned to historical data, capturing not only the signal but also noise, leading to poor out-of-sample performance 👇👇👇Continue👇👇👇

Perhaps one of the most overlooked aspects in the development of a trading system is data structures. Everyone wants a reliable data source, a professional cloud-based database, the data vendor with the lowest possible latency, to make a quick SQL query and have the table appear instantly... but defining its structure or exploring new structures? Why bother!? That's not fancy. A data structure is a scheme for organizing, storing, and retrieving data effectively. Different data structures serve different purposes: -Arrays are ideal for storing large sets of similar items with fast access -While linked lists are useful for frequent insertions or deletions The choice depends on the problem and required operations, but a good choice enhances speed, while a poor one can cause performance issues. As you see they play a key role in algorithm design. Key benefits of data structures include: -Efficiency: Optimized data access and manipulation lead to lower latency -Organization: Systematic data storage and management -Reusability: Can be reused across applications, saving development time -Clarity: Improved code readability and structured data management Why are they important? -Basic data types are insufficient for complex data processing -Data structures help manage, connect, and use data -A well-designed structure makes data easier to work with Their significance includes: -Optimized performance: Fast retrieval and manipulation of data elements -Complex relationships: Represent relationships, like trees for hierarchical data -Reusable components: Linked lists or stacks can be reused, enhancing consistency -Code readability: Clear connections make code easier to understand and maintain The wrong choice can lead to poor performance. Consider: -Type of data -Usage -Persistence -Organization for access -Memory considerations The most typical applications you will find: -Storing data: Arrays and records store data -Managing resources: Linked lists or trees manage memory and directories -Data exchange: Define data formats for communication, like TCP/IP packets -Ordering and sorting: Binary search trees and priority queues -Indexing and searching: B-trees or hash tables enhance searching -Scalability: Manage distributed data storage for large-scale applications Some of the common characteristics of data structures are: -Linear vs. non-linear: •Linear structures (arrays, lists) have sequential elements •Non-linear structures (trees, graphs) represent complex relationships. -Homogeneous vs. heterogeneous: •Homogeneous structures have similar elements •Heterogeneous ones have mixed types -Static vs. dynamic: •Static structures have fixed size •Dynamic ones adjust during runtime 👇👇👇Continue👇👇👇

Algorithms are typically designed with specific goals, time horizons, and strategies. What appears as irrelevant or random to the long-term algorithm could be the key #trading signal for the short-term one. Noise refers to the random fluctuations in asset prices. These price movements arise from unpredictable, short-term factors such as: -Trader behaviors -Market microstructure -Liquidity conditions -Order flows -Macroeconomic events The consequences? -Independent price changes -Price changes influenced by past movements -Random price variations from trading processes -Unpredictable changes in volatility All these effects occur at multiple levels of abstraction and can be classified into different types of noise, and depending on the source, they will have different characteristics: A. White noise. -It is the simplest form of noise, where each price change is completely random and independent of previous changes -It has no memory or correlation over time, meaning that today's noise provides no information about tomorrow's -Often modeled using Gaussian approach: 𝜖𝑡∼𝑁(0,𝜎^2) where 𝜖𝑡 represents the noise at time 𝑡, assumed to have a mean of zero and constant variance 𝜎^2 -Key features: Independent, identically distributed (i.i.d.), no autocorrelation B. Autocorrelated noise. -This type of noise exhibits correlations between price changes over time -Unlike white noise, where price movements are independent, autocorrelated noise means that past price movements can influence future ones -Types: 1/f noise: Power spectrum decreases with frequency, meaning that there are correlations across time, and low-frequency variations dominate Brownian noise: Similar to the previous one but with stronger temporal correlations -Often modeled using autoregressive process: 𝜖𝑡 = 𝜙𝜖_𝑡−1+𝜂𝑡 where 𝜙 is the autocorrelation coefficient and 𝜂𝑡 is white noise -Key features: Temporal dependencies, correlations over different time lags C. Market microstructure noise. -This noise arises from the trading process itself, particularly at very short time frames -Factors like bid-ask spreads, order flow, and execution delays introduce random variations in asset prices that do not reflect any fundamental value changes -Often modeled using Poisson processes for discrete price movements: 𝑑𝑃(𝑡)=𝜇𝑑𝑡+𝜎𝑑𝑊(𝑡)+𝐽(𝑡) where 𝐽(𝑡) is a jump process capturing the discrete price jumps -Key features: Discrete unpredictable price jumps due to liquidity imbalances, small trades, and transaction costs D. Volatility noise. -It refers to the unpredictable changes in the volatility (variance) of asset prices over time -Volatility isn’t constant -Clustering effect is present -GARCH model is commonly used -Key features: Time-varying volatility, clustering of high or low volatility periods

I will never understand the typical phrases that are in the mouths of some: "I already have a methodology", or "I already have a system". It seems like it sounds linear, when in reality it is a circular process, which involves recursion with value (at least to be able to improve a minimum). In algorithmic #trading, iterative refinement is a process where a model is repeatedly adjusted and improved through successive iterations to better represent a certain behaviour. The goal is to enhance the model's accuracy, efficiency, and reliability by refining its components based on feedback or new data. Here's an overview of how the process works: A. Initial model creation. -The process begins with creating a simplified model -This first model is often based on basic principles -It isn’t expected to be perfect but serves as a starting point B. Model evaluation. -After creating the initial model, it is tested to assess its performance -This step typically involves: •Quantifying the difference between the model’s predictions and actual observations •Checking whether the model captures the inefficiency of the market and functions as intended C. Feedback collection. -The evaluation step provides feedback, often in the form of error magnitudes or missing behaviors -Feedback can come from discrepancies between model outputs and actual results, or through analysis showing a lack of precision, stability, or generalizability D. Model adjustment. -Based on the feedback, adjustments are made to refine the model -This could involve: •Adding more details •Simplifying parts of the model to reduce complexity •Incorporating new data or insights E. Reiteration. -The refined model is then re-evaluated to test its new performance -This iterative process repeats, allowing incremental improvements at each cycle -Each iteration aims to improve one or more aspects of the model, such as: •Reducing the error between predictions and actual results •Making the model more efficient or easier to compute •Enhancing the model's ability to generalize across different assets •Ensuring the model behaves consistently under varied conditions F. Convergence. -The iterative refinement continues until the model reaches a point of convergence, where further refinements no longer produce significant improvements -The final model is then considered robust enough for practical application To evaluate each phase in the iterative refinement process you can structure the process as follows: A. Performance. -Assess how well the current model functions based on its initial implementation or previous iteration -In this aspect, you analyze the output of the model by comparing its predictions with expected theoretical results -The output is a clear understanding of how well the model performs and what aspects need improvement 👇👇👇Continue👇👇👇

There's one thing that makes me think: when a trader, quant, or anyone embarks on developing a model, the first thing they do is try to model price movement, direction, certain reactions... when what they should be doing is modeling the transition between states. By this, I don't mean a Markov model but the transition itself. The transaction flow gives rise to states where something: -Transits -Transfers -Transforms So here the question is: What is a transition? -A transition is the process by which a system moves from one state or condition to another -It is an essential concept in understanding dynamic systems, as it captures the idea of change over time or under certain conditions -A transition describes the evolution of a system through different phases, configurations, or behaviors -It can involve changes in structure, behavior, or properties Transitions can occur: -Deterministically, where each state leads predictably to the next -Stochastically, where the transition between states is governed by probabilities or random factors Financial markets are a mix of both, where behaviors converge into pockets of order or diverge, giving rise to noise or anomalies. In its most general sense, a transition involves: -A starting state: Where the system is at the beginning of the process -A movement or change: The process that drives the system from one state to another -An ending state: The new condition or state after the transition A transition model is a mathematical framework used to describe how a system evolves through various states over time. It defines: -The rules -The dynamics -The conditions All of them govern the movement from one state to another. So the transition model formalizes how the transition occurs, considering factors such as: -What causes the transition: These could be external inputs or internal conditions -How the system behaves in each state: Each state may have specific behaviors, properties, or characteristics -What influences the next state: This could be deterministic (specific rules or conditions) or probabilistic (random chance or external variability) The key elements can be grouped into three main categories: A. Transition process: -States: Define where the system is at any given time •Order flow states •Market states -Transition rules: Define how the system moves between states •Trigger events •Order execution rules -Time evolution: Describes whether transitions happen in discrete or continuous time •Discrete steps •Continuous monitoring 👇👇👇Continue👇👇👇

Why are there still discretionary traders? Does it have any advantage? Is trading really meant for humans, or is the problem precisely that—being human? The history of the markets includes quite a few legends who were discretionary traders, and systematic ones as well, though fewer, but they exist. Some of you might think of this as analogous to algorithmic trading, and that’s true in just one of its dimensions (decision-making), but what about the rest? They are quite different… Let’s take a deeper look at what happens inside the mind of a discretionary trader, whether they like it or not, and whether they are aware of it or not. First things first, discretionary trading is: -A trading approach where decisions to buy or sell financial instruments are made by a human trader based on their judgment, experience, and intuition rather than by automated systems or predefined algorithms -Unlike algorithmic trading, which relies on fixed rules and data-driven models, discretionary trading allows the trader to adapt to changing market conditions in real-time and incorporate subjective factors like market sentiment, economic news, and personal risk tolerance Key characteristics of discretionary trading: -Traders rely on their own analysis of the market -They can adapt their strategies on the fly -Since decisions are made by humans, emotions like fear and greed can impact the trading process -They interpret the market according to one or several sources of information -Decisions are experience-based Discretionary traders could incorporate different types of modeling to frame and interpret the market, each offering unique ways to perceive and act on market information: A. Intuitive modeling. -It is based on observing other processes that mirror the market behavior -A trader might view market movements as reflecting cycles such as trends and corrections -This approach assumes that market movements are governed by underlying principles that can be observed and interpreted -The pitfalls of this type of modeling: •Oversimplification of market behavior •Ignoring structural changes B. Narrative-Based modeling. -It relies on using archetypes, narratives, or symbolic stories to make sense of complex situations -In trading, this could involve framing market events or movements as part of a larger, symbolic narrative -Traders conceptualize the market as a battlefield between "bulls" and "bears," with each side representing distinct forces of optimism and pessimism -This model helps the trader stay mentally attuned to shifts in market sentiment, where each trade reflects a chapter in a larger market story -The pitfalls of this type of modeling: •Over-reliance on narratives •Confirmation bias 👇👇👇

From the academic world, countless models related to AI have been tested in the field of #trading. Most of them have been failures or do not perform as well as simpler models. All these models have something in common: they aim to be models of change that effectively adapt to market data. A model of change is: -A mathematical framework used to describe, analyze, and manage how a system evolves or adapts in response to internal or external changes -These changes could be triggered by shifts in the environment, data, market conditions, system performance, or other variables -The goal of a change model is to help systems, processes, or models remain effective by adjusting to new conditions Personally, if I had to choose several of these data-driven models, I would go with (all of them simultaneously): -Rule-based adaptive -Incremental updates -Multiscale models -Probabilistic change models The key lies in finding simple models through complex ones. Let’s take a look at what frameworks we’re talking about: A. Rule-based change models. -These models operate based on a predefined set of rules -Advantages: Simple and predictable; easily implementable -Challenges: Not very flexible; can’t handle unexpected changes or complex environments without expanding the rule set B. Incremental models. -In this model, changes are introduced gradually, where the model is updated incrementally as new data or insights become available -Advantages: Allows for continuous refinement of the model -Challenges: May be slow to respond to large or sudden changes (shocks); risk of overfitting C. Bayesian update models. -Bayesian methods update the model by continuously incorporating new evidence into the existing model, adjusting the probability distributions of model parameters based on observed data -Advantages: Provides a formal mechanism for updating model beliefs; handles uncertainty in data and model assumptions naturally -Challenges: Risk of overfitting to noise D. Self-tuning models. -These models can modify their structure or parameters automatically based on feedback or performance criteria -Advantages: High flexibility and responsiveness to change; minimizes the need for manual intervention -Challenges: Prone to instability if the adaptation mechanism is not well-calibrated E. Structural change models. -The model’s structure change by modifying its equations, adding/removing variables, or changing relationships between variables -Advantages: Allows models to better capture changes when incremental updates are insufficient -Challenges: Redesigning the entire model autonomously can lead to big loses F. Nonlinear transition models. -These models are used to predict or respond to phase transitions, bifurcations, or other non-continuous shifts -Advantages: More accurate in capturing sudden or unexpected changes -Challenges: Less stable and more sensitive to small changes 👇👇👇Continue👇👇👇