Reinforcement Learning for Dynamic Position Sizing in Volatile Markets

At Venym Labs, we operate on a core quantitative premise: alpha generation is only half the battle; alpha extraction is the other. In liquid, highly volatile markets—particularly digital assets and dispersed macro futures—the difference between a world-class strategy and a blown-out account rarely lies in the entry signal. It lies in position sizing. Most quantitative teams treat position sizing as an afterthought, relying on static heuristics like the Kelly Criterion or fixed fractional risk. We view this as mathematically bankrupt in non-stationary environments. When realized volatility can swing from 15% to 150% annualized in a matter of hours, a static sizing model is a liability. To solve this, we have spent the last two years building a proprietary Reinforcement Learning (RL) framework dedicated entirely to dynamic position sizing. This post details our architecture, the mathematical pitfalls of traditional approaches, and the empirical edge we have observed when handing the sizing decision over to a well-conditioned Soft Actor-Critic (SAC) agent. The Failure of Static Heuristics The Kelly Criterion, defined as , is the gold standard for theoretical optimal sizing. It maximizes the expected logarithmic growth of wealth. However, Kelly makes two catastrophic assumptions: it assumes returns are independently and identically distributed (i.i.d.), and that the estimated mean () and variance () are known constants. In volatile equity and crypto markets, these assumptions fail spectacularly. Consider a mean-reversion strategy on BTC perpetual futures. During a low-volatility regime, a 30-day rolling window might yield and . Standard Kelly might suggest a leverage of . When a sudden regime shift occurs—driven by a liquidity vacuum or macro shock—volatility expands to while the mean reversion signal temporarily inverts. The static Kelly model is still holding leverage into a blizzard, typically resulting in a drawdown exceeding 40% before the lookback window adjusts. To quantify this, we backtested a standard z-score mean reversion strategy on BTC from 2022 to 2025 using three sizing methods: 1. Fixed Fractional: Risking 1% of portfolio equity per trade. 2. Static Kelly: Recalculated at the end of every 24-hour period. 3. Volatility-Scaled Kelly: Scaling static Kelly by the ratio of short-term realized vol to long-term realized vol. While Volatility-Scaled Kelly improved the maximum drawdown from -38.2% (Static Kelly) to -24.5%, it still suffered from severe lag. The scaling factor is inherently backward-looking. By the time the short-term volatility estimate spikes, the damage to the portfolio's convexity has already been done. We need a sizing model that is forward-looking and adaptive. Formulating Position Sizing as an MDP To move beyond heuristics, we formulate dynamic position sizing as a Markov Decision Process (MDP). The RL agent does not generate the alpha signal; it sits downstream of our existing alpha engines and acts purely as a risk-adjusted capital allocator. The MDP is defined by the tuple : State Space () The state represents the current market regime and portfolio context. We encode this as a 128-dimensional continuous vector updated at 1-minute intervals. Key features include: Microstructure: Top-of-book bid/ask imbalance, rolling 5-minute order flow toxicity (VPIN). Volatility Surface: Term structure of implied volatility (e.g., 1-day vs 7-day IV), realized vs. implied volatility spread. Portfolio Context: Current net leverage, unrealized PnL, portfolio heat (sum of active position variances). Signal State: The raw alpha score from the predictive model, and its rolling Sharpe ratio over the last 4 hours. Action Space () We use a continuous action space , which is mapped to a target portfolio weight. We explicitly avoid discrete action spaces (e.g., "Buy 100%, Hold 0%, Sell 100%") because granular sizing is where edge is preserved in illiquid or highly impacted environments. Reward Function () This is where most RL implementations fail. Training an agent on raw PnL leads to catastrophic risk-seeking behavior. We utilize a risk-adjusted reward formulation based on the differential Sharpe ratio, combined with a strict drawdown penalty. Architecture: Why Soft Actor-Critic (SAC) At Venym Labs, our primary sizing agent is built on Soft Actor-Critic (SAC). While Proximal Policy Optimization (PPO) dominates the RL literature for game-playing and language model alignment, we strongly prefer SAC for financial execution due to its maximum entropy framework. In high-frequency volatile markets, deterministic policies are fragile. If a PPO agent learns that 0.5x leverage is optimal in a specific regime, it will output exactly 0.5x every time. This creates a fingerprint that can be adversarially exploited by other market participants (or simply broken by minor distribution shifts). SAC maximizes both expected return and entropy: $\mathcal{H}3x1.2x$. This ensures that even if the agent experiences a catast