DEV Community: mattleeee

Adaptive Stop-Loss Thresholds Instead of Hardcoded Percentages

mattleeee — Sun, 01 Nov 2026 01:01:48 +0000

A fixed -3% stop does not mean the same thing on every instrument. On a volatile altcoin, 3% is inside the normal noise band, so the stop fires on routine wiggles and the strategy never gets to express its edge. On a low-volatility index product, 3% is a wide move, so the stop sits far beyond where the strategy's own logic would have exited. The same number produces opposite behaviour depending on the thing it's attached to.

The fix is not to tune the percentage per instrument by hand. It's to derive the safety floor from the strategy's own stop calculation, so the floor protects capital without overriding the strategy's exit logic. The formula used here:

threshold = -max(3.0, abs(strategy_stop_pct) * 2.0)

Read it as two clauses. The abs(strategy_stop_pct) * 2.0 clause says: never force-close before the strategy's own stop has had a chance to work, and only intervene once the position has drifted to roughly twice the intended stop distance. The max(3.0, ...) clause says: there is always at least a 3% floor, so a strategy with a very tight stop (say 0.5%) doesn't get a 1% force-close that would fire on spread and slippage alone.

What each clause is protecting

The * 2.0 multiplier is the interesting part. A strategy that stops at 2% has a defined risk budget. If the price is at -2.5%, the strategy's stop should have already acted; if it hasn't, something is wrong (stale data, a gap, a broker reject). Waiting until -4% gives the strategy room to do its job and cuts the position only when the loss is clearly beyond what the strategy signed up for. That's the "respect the strategy" half.

The max(3.0, ...) half is the "protect capital" floor. Without it, a strategy with a 0.5% stop yields a 1% force-close threshold, which is noise-level for most liquid instruments. The force-close would become the actual stop, and the strategy's own logic would be irrelevant. The 3% minimum guarantees the force-close is a genuine backstop, not a second, tighter stop.

The result is a threshold that scales with the instrument's volatility as the strategy sees it, because strategy_stop_pct is itself volatility-derived. That's the whole point: the floor inherits the strategy's view of normal movement instead of imposing an external one.

Computing `strategy_stop_pct` per instrument

strategy_stop_pct must come from the strategy's own stop logic, not from a separate volatility estimate. If the strategy uses ATR-based stops, use that. If it uses a percentage of price derived from recent range, use that. The floor and the strategy must agree on what "the stop" means, or the * 2.0 multiplier is meaningless.

A common pattern is an ATR stop with a multiplier:

import pandas as pd

def atr(df: pd.DataFrame, period: int = 14) -> pd.Series:
    high, low, close = df["high"], df["low"], df["close"]
    prev_close = close.shift(1)
    tr = pd.concat([
        high - low,
        (high - prev_close).abs(),
        (low - prev_close).abs(),
    ], axis=1).max(axis=1)
    return tr.ewm(alpha=1 / period, adjust=False).mean()

def strategy_stop_pct(df: pd.DataFrame, atr_mult: float = 2.0,
                      atr_period: int = 14) -> pd.Series:
    """Stop distance as a fraction of current price."""
    a = atr(df, atr_period)
    return (atr_mult * a) / df["close"]

Then the floor follows directly:

def safety_floor_pct(stop_pct: pd.Series,
                     min_floor: float = 3.0,
                     mult: float = 2.0) -> pd.Series:
    """Returns positive magnitude; apply as -value when comparing PnL."""
    return (stop_pct.abs() * mult).clip(lower=min_floor)

Two details matter here.

First, stop_pct is a series, not a scalar. ATR changes, so the stop distance changes, so the floor changes. Recompute it on every bar or every position update. A floor computed once at entry will be wrong by the time the position has moved.

Second, clip(lower=min_floor) implements the max() cleanly and keeps the output vectorised. For a strategy with a 2% stop, the floor is 4%. For a 0.8% stop, the floor is 3% (the minimum wins). For a 6% stop, the floor is 12%.

One caveat on the ATR version: ATR is in price units, so dividing by close converts it to a fraction. For instruments with very different price scales (a $0.02 token vs. a $400 stock), the fraction is the only comparable quantity. Keep everything in fractional terms until the final PnL comparison.

Applying it to open positions

The check itself is a simple comparison against unrealised PnL as a fraction:

def should_force_close(entry_price: float,
                       current_price: float,
                       stop_pct_now: float,
                       min_floor: float = 3.0,
                       mult: float = 2.0) -> bool:
    pnl_pct = (current_price - entry_price) / entry_price * 100.0
    floor = max(min_floor, abs(stop_pct_now) * mult)
    return pnl_pct <= -floor

Call this on every tick or bar, passing the current stop_pct_now rather than the value at entry. If the instrument has calmed down since entry, the floor tightens; if volatility has expanded, the floor widens. That's the adaptive part, and it's what a hardcoded percentage can't do.

For a batch of positions, vectorise it:

def force_close_mask(entry: pd.Series, current: pd.Series,
                     stop_pct_now: pd.Series,
                     min_floor: float = 3.0,
                     mult: float = 2.0) -> pd.Series:
    pnl_pct = (current - entry) / entry * 100.0
    floor = (stop_pct_now.abs() * mult).clip(lower=min_floor)
    return pnl_pct <= -floor

Backtesting the floor's effect on surviving trades

The number that matters is not how many trades the floor cuts. It's how many trades the floor cuts that would have recovered, and how much tail risk it removes. A floor that never fires is doing nothing; a floor that fires on trades that would have hit their own stop anyway is redundant. The useful cases are the ones in between: trades that drifted past the strategy's intended stop without triggering it, and trades that recovered.

Tag each trade with whether the floor fired and whether the trade would have been profitable without it:

def evaluate_floor(trades: pd.DataFrame) -> dict:
    """
    trades columns: pnl_pct (final, without floor), floor_pnl_pct
    (PnL at the moment the floor would have fired, NaN if it never fired),
    would_recover (bool: final pnl_pct > 0 without floor).
    """
    fired = trades["floor_pnl_pct"].notna()
    cut = trades[fired]
    return {
        "n_trades": len(trades),
        "n_fired": int(fired.sum()),
        "fired_rate": fired.mean(),
        "avg_pnl_fired": cut["floor_pnl_pct"].mean(),
        "avg_pnl_if_held": cut["pnl_pct"].mean(),
        "recovered_after_cut": int(cut["would_recover"].sum()),
        "recovered_pnl_cost": cut.loc[cut["would_recover"], "pnl_pct"].sum(),
        "tail_saved": (
            trades.loc[~fired, "pnl_pct"].nsmallest(20).mean()
            - cut["floor_pnl_pct"].mean()
            if fired.any() else float("nan")
        ),
    }

The two numbers to read together are recovered_pnl_cost and the change in the worst-case tail. If the floor cuts a handful of trades that would have recovered for a small total PnL cost, and it removes a fat left tail, it's earning its keep. If recovered_pnl_cost is large relative to the tail it removes, the multiplier is too tight: raise mult from 2.0 toward 2.5 or 3.0 and re-run. If the floor barely fires, the multiplier is too loose or the strategies' stops are already doing the work.

Also break the results down by instrument. A floor that looks good in aggregate can be doing all its work on one volatile instrument and nothing elsewhere. If one instrument accounts for most of the fired trades, its strategy_stop_pct is probably mis-specified rather than the floor being wrong.

Watch the sample size. Floor events are rare by construction, so a backtest with 200 trades may produce a dozen firings, which is not enough to conclude anything. Pool across instruments and across a long enough window that the volatility regime shifts at least once. If the floor's contribution only shows up in a single high-volatility month, you're measuring one event, not an edge.

When a hard floor must override the adaptive value

The adaptive threshold assumes strategy_stop_pct is meaningful. That assumption fails in specific, recognisable cases, and in those the hard floor must win outright.

Stale or missing volatility data. If ATR is NaN, zero, or computed from a thin book, stop_pct collapses toward zero and the adaptive floor tightens to the 3% minimum. That's benign. But if stop_pct is stale — carried over from a prior regime — the floor can be far too wide. Detect it:

def effective_floor(stop_pct_now: float,
                    stop_pct_prev: float,
                    min_floor: float = 3.0,
                    mult: float = 2.0,
                    max_floor: float = 15.0) -> float:
    if not (stop_pct_now > 0):
        return min_floor                      # bad data: fall back to hard floor
    adaptive = max(min_floor, abs(stop_pct_now) * mult)
    if abs(stop_pct_now - stop_pct_prev) / stop_pct_prev > 0.5:
        adaptive = max(adaptive, min_floor)   # regime jump: don't trust the ratio
    return min(adaptive, max_floor)           # hard ceiling regardless

A hard ceiling. The adaptive formula has no upper bound. A strategy with a 10% stop yields a 20% force-close threshold, which for most instruments is not risk management, it's capitulation. Cap it. The max_floor argument above enforces a ceiling that no adaptive value can exceed. The exact number depends on the instrument's typical drawdown distribution, but it should exist and it should be hardcoded, because its purpose is to override the adaptive logic when the adaptive logic produces something absurd.

Gap and liquidity events. The floor is evaluated on observed prices. If an instrument gaps through the threshold overnight, the force-close executes at a worse price than the threshold. That's unavoidable, but it means the floor's effective loss can exceed its nominal value. In backtests, model the fill at the next available price, not at the threshold, or the floor will look more protective than it is.

Strategy stop wider than the ceiling. If strategy_stop_pct * 2.0 exceeds max_floor, the ceiling wins and the strategy's stop is effectively overridden. That's the correct outcome: the strategy has asked for more risk than the account's policy allows, and the policy should win. Log these cases — they usually mean the strategy is mis-parameterised for that instrument, not that the floor is too tight.

The design principle

The floor exists to catch what the strategy's own stop missed: gaps, stale stops, broker rejects, logic errors. It should be the outer layer of defence, not the first line. Deriving it from strategy_stop_pct keeps it in the right place — wide enough that the strategy's exit logic runs first, tight enough that a runaway loss gets cut before it compounds. The hard floor and the hard ceiling bracket the adaptive value so that bad inputs can't push it into useless territory. Everything between those brackets is the strategy's own risk definition, scaled by a constant.

More notes like this ship every week on this site.

Partial Take-Profit: Bank Half at 3%, Let the Rest Run

mattleeee — Sat, 31 Oct 2026 01:01:47 +0000

The Problem With a Single Take-Profit

A fixed take-profit forces a binary choice. Set it tight and you cut winners that would have kept running. Set it wide and you watch a 5% open profit evaporate back to zero because the target was never hit. Both outcomes come from the same design flaw: the exit is a single point, and the market gets one vote on whether you keep everything.

Partial take-profit splits that vote. The version I run in the paper trader: when unrealized profit on the open position reaches 3%, sell half the position and move the stop on the remainder to break-even. The remaining half targets the original 6%. The first leg banks a guaranteed gain; the second leg keeps upside exposure at zero risk (modulo fees, which we'll get to).

The strategy logic is trivial. The bookkeeping is where implementations break, so most of this post is about state.

Position State

Every open position needs enough state to answer: how much is left, what did it cost, what's been realized, and has the partial already fired. A dataclass is the cleanest container:

from dataclasses import dataclass, field

@dataclass
class Position:
    entry_price: float
    qty: float
    stop_price: float
    target_price: float
    partial_level: float
    partial_fraction: float = 0.5
    partial_done: bool = False
    realized_pnl: float = 0.0
    fees_paid: float = 0.0
    initial_qty: float = field(init=False)

    def __post_init__(self):
        self.initial_qty = self.qty

Two fields do the heavy lifting. partial_done is the latch that guarantees the 3% rule fires once, not on every candle that closes above the threshold. initial_qty preserves the original size so you can report percentage-of-position metrics after the exit.

Note that entry_price never changes. After a partial exit, the remaining shares still carry the same cost basis — the average entry of the remaining position is unchanged. What changes is how much capital is at risk. If you recompute an "average entry" by blending realized and unrealized, you'll corrupt the R-multiple math later. Keep cost basis immutable; track realized PnL separately.

The Candle Loop and Check Order

The rule must be evaluated exactly once per candle, and the order of checks matters because a single candle can touch the stop, the partial level, and the target. Here's the loop I use:

def process_candle(pos: Position, candle, fee_rate: float = 0.001):
    o, h, l, c = candle.open, candle.high, candle.low, candle.close

    # 1. Gap handling: if the open is already beyond a level,
    #    fill at the open, not at the level.
    # 2. Stop check first (conservative: assume worst case within candle).
    stop_hit = l <= pos.stop_price
    target_hit = h >= pos.target_price
    partial_hit = (not pos.partial_done) and h >= pos.partial_level

    # Conservative ordering: stop before target when both touched.
    if stop_hit:
        fill = min(o, pos.stop_price) if o < pos.stop_price else pos.stop_price
        close_remainder(pos, fill, fee_rate)
        return "stopped"

    if partial_hit:
        take_partial(pos, max(o, pos.partial_level), fee_rate)
        # Re-evaluate target on the same candle with reduced size.
        if h >= pos.target_price:
            fill = max(o, pos.target_price)
            close_remainder(pos, fill, fee_rate)
            return "partial+target"
        return "partial"

    if target_hit:
        fill = max(o, pos.target_price)
        close_remainder(pos, fill, fee_rate)
        return "target"

    return "open"

The ordering decisions, spelled out:

Stop before partial. If a candle's range covers both the stop and the 3% level, assume the stop got hit first. Without tick data you cannot know the intrabar sequence, and assuming the favorable fill first is how backtests develop a personality disorder. Conservative ordering costs you some hypothetical profit and keeps the equity curve honest.

Partial before target. Once the 3% level triggers, the target check runs against the reduced position. If one candle runs from below 3% straight through 6%, you take the partial at 3% and the remainder at 6%. That's the intended behavior: the rule is a schedule, not an either/or.

Gap opens. If the candle opens above the target, the fill is the open, not the target — you don't get to sell at a price the market skipped. The max(o, level) / min(o, level) expressions handle long positions; flip them for shorts. Gap opens beyond both the partial and the target (open at 7%) mean the partial fills at the open too, and the remainder also fills at the open. Bank it and move on.

The Partial Fill Itself

def take_partial(pos: Position, fill_price: float, fee_rate: float):
    qty = pos.qty * pos.partial_fraction
    proceeds = qty * fill_price
    fee = proceeds * fee_rate
    cost = qty * pos.entry_price

    pos.realized_pnl += proceeds - fee - cost
    pos.fees_paid += fee
    pos.qty -= qty
    pos.partial_done = True

    # Move stop to break-even, adjusted for fees already sunk.
    pos.stop_price = pos.entry_price

Two subtleties here. First, pos.qty -= qty uses the fractional quantity, so a 50% exit on an odd lot leaves a fractional remainder — fine for crypto, and for equities you'd round down and keep the residual. Second, the break-even stop is set to entry_price, not to entry_price + fees. That's the version most people write, and it's slightly wrong; see the fee section.

Edge Cases That Bite

The "free ride" isn't free. Moving the stop to break-even means the remainder exits flat before fees. If you paid 0.1% on entry and 0.1% on exit, a break-even stop on the remainder loses roughly 0.2% of the remaining notional. Over hundreds of trades this shows up. The fix is to set the stop at the fee-adjusted break-even:

def breakeven_price(entry: float, entry_fee_rate: float,
                    exit_fee_rate: float) -> float:
    # Price at which net PnL on the remainder is zero.
    return entry * (1 + entry_fee_rate) / (1 - exit_fee_rate)

With 0.1% each way, that's entry × 1.001 / 0.999 ≈ entry × 1.002. Two basis points above entry — small, but it converts "free ride" from a slogan into an arithmetic fact. Whether the fee-adjusted stop or the plain break-even stop performs better is an empirical question; test both.

Partial level below entry after slippage. If your fill model includes slippage, the realized PnL on the partial can be negative even though the trigger price was 3% above entry. Guard the expectancy math against this by using actual fill prices everywhere, never the trigger levels.

Restart safety. If the trader persists state between runs, partial_done must be serialized. Reconstructing a position from "current qty and entry" alone will re-fire the partial after a restart. Persist the full dataclass.

One partial per position. The latch is not an optimization; it's the semantics. If you want multiple scale-out levels (3%, 5%, 7%), replace the boolean with a list of pending levels and pop them as they trigger. The bookkeeping pattern is identical.

How Partial TP Changes Expectancy

The standard expectancy formula for a single-target system is:

E = p_win * avg_win - p_loss * avg_loss

Partial exits break avg_win into two components and change the outcome distribution. With half out at 3% and half at 6% (ignoring fees and the break-even stop for a moment), a full winner returns 0.5 * 3% + 0.5 * 6% = 4.5% of position notional. A trade that reaches 3% but stalls returns 0.5 * 3% + 0 = 1.5% instead of riding back to zero. A trade stopped at break-even after the partial returns 1.5% instead of a full loss.

The trade-off: you've capped the average winner at 4.5% instead of 6%, in exchange for converting some would-be losers into small winners and shrinking the loss tail. Whether that improves expectancy depends on the distribution of maximum favorable excursion (MFE) — specifically, how many trades reach 3% versus 6%. If most trades that hit 3% also hit 6%, the partial just costs you half a percent per winner. If a large fraction stall between 3% and 6%, the partial is where the edge lives.

You can't reason your way to the answer; you measure it. Log MFE per trade and build the histogram:

import numpy as np

def mfe_histogram(trades, bins=(0, 1, 2, 3, 4, 5, 6, 8, 10)):
    mfe = np.array([t.max_favorable_pct for t in trades])
    hist, edges = np.histogram(mfe, bins=bins)
    for lo, hi, n in zip(edges[:-1], edges[1:], hist):
        print(f"{lo:>4.0f}%-{hi:<4.0f}%  {n:>5d}  {'#' * int(50 * n / max(hist))}")

If the 3–6% bucket is fat, the partial earns its keep. If it's nearly empty, a single wider target is simpler and better.

A Cleaner Expectancy Formula

Once partials exist, per-trade expectancy is best computed from realized cash flows rather than win/loss averages:

def expectancy(closed_trades):
    # Each trade: dict with 'pnl_pct' (net return on initial notional)
    pnl = np.array([t["pnl_pct"] for t in closed_trades])
    return pnl.mean(), pnl.std() / np.sqrt(len(pnl))

Because every trade now has a continuum of outcomes (full stop, partial + break-even, partial + target, direct target), the win/loss split loses information. Mean net return per trade on initial notional, with a standard error, is the honest summary. Report it alongside the MFE histogram and the fraction of trades where the partial fired.

What Actually Matters

The partial take-profit is a small piece of logic wrapped in a lot of state discipline. Get these right and the rest follows:

Latch the partial with an explicit flag and persist it.
Evaluate stop → partial → target, once per candle, with gap-aware fills.
Keep entry price immutable; track realized PnL separately.
Use fee-adjusted break-even if you care about the "free" part being real.
Judge the rule by the MFE distribution and net expectancy, not by how it feels.

More notes like this ship every week on this site.

Daily Picks

The following pairs are selected from the multi-timeframe trend scanner (Gate.io futures) and are for technical-analysis study only — not investment advice.
Data updated: 2026-10-31 08:47:06

Long

Pair	Signal	Price	Take Profit	Stop Loss	R/R
Q	短平快做多	$0.0434	$0.0447	$0.0423	1:1.2

1 picks selected. Scanner runs every 15 minutes.

Circuit Breakers for Strategies: Pausing After Three Consecutive Losses

mattleeee — Fri, 30 Oct 2026 01:02:05 +0000

Why Consecutive Losses Need Their Own Guard

Drawdown limits answer "how much have I lost?" Consecutive-loss breakers answer a different question: "is the process still behaving the way I validated it?" A strategy can be down 4% over a month through a slow bleed of small wins and losses — normal. Or it can lose five trades in a row in two days — also possible under a valid edge, but far less likely, and much more informative. The breaker treats the streak as a signal about regime change, broken data feeds, or an implementation bug, not as a verdict on your equity curve.

The rule is deliberately blunt: three consecutive losing trades trips the strategy into cooldown. No auto-restart. A human reviews the trade history, checks whether the edge is still there, and re-arms the breaker explicitly.

The State Machine

Four states, three transitions, one manual gate:

ACTIVE ──(3 losses)──> TRIPPED ──(notify + persist)──> COOLDOWN
   ^                                                      │
   └──────────(manual review / resume)────────────────────┘

ACTIVE: strategy trades normally, counter increments on each closed loss and resets on each closed win.
TRIPPED: transient state entered the moment the third loss closes. Its only job is to fire the notification and persist the trip record. Execution must already be blocked here.
COOLDOWN: strategy is paused. No orders, no sizing, no "just one small trade to test."
Resume: only via an explicit call after review. Never on a timer, never on a new day, never on "market looks better now."

The important design decision is that TRIPPED blocks execution before the notification is sent. If the notification service is down, the strategy still doesn't trade. Failure to notify must never mean failure to stop.

Where the Counter Lives

An in-memory counter is worthless. Strategies get restarted by deploys, crashes, and Windows Task Scheduler reboots. If the counter resets on restart, a strategy that lost two trades before a restart gets a free pass on the third. Persist the counter and the state together, atomically, on every closed trade.

SQLite is enough for a single-strategy setup and survives process death:

import sqlite3
from contextlib import contextmanager
from dataclasses import dataclass
from datetime import datetime, timezone
from enum import Enum

class State(str, Enum):
    ACTIVE = "active"
    TRIPPED = "tripped"
    COOLDOWN = "cooldown"

@dataclass
class BreakerRecord:
    strategy_id: str
    state: State
    consecutive_losses: int
    updated_at: str

class BreakerStore:
    def __init__(self, path: str = "breaker.db"):
        self.path = path
        with self._conn() as c:
            c.execute("""
                CREATE TABLE IF NOT EXISTS breaker (
                    strategy_id TEXT PRIMARY KEY,
                    state TEXT NOT NULL,
                    consecutive_losses INTEGER NOT NULL DEFAULT 0,
                    updated_at TEXT NOT NULL
                )
            """)

    @contextmanager
    def _conn(self):
        conn = sqlite3.connect(self.path, isolation_level=None)
        try:
            conn.execute("PRAGMA journal_mode=WAL")
            yield conn
        finally:
            conn.close()

    def load(self, strategy_id: str) -> BreakerRecord:
        with self._conn() as c:
            row = c.execute(
                "SELECT state, consecutive_losses, updated_at FROM breaker WHERE strategy_id=?",
                (strategy_id,),
            ).fetchone()
        if row is None:
            return BreakerRecord(strategy_id, State.ACTIVE, 0, _now())
        return BreakerRecord(strategy_id, State(row[0]), row[1], row[2])

    def save(self, rec: BreakerRecord) -> None:
        with self._conn() as c:
            c.execute("""
                INSERT INTO breaker (strategy_id, state, consecutive_losses, updated_at)
                VALUES (?, ?, ?, ?)
                ON CONFLICT(strategy_id) DO UPDATE SET
                    state=excluded.state,
                    consecutive_losses=excluded.consecutive_losses,
                    updated_at=excluded.updated_at
            """, (rec.strategy_id, rec.state.value, rec.consecutive_losses, rec.updated_at))

def _now() -> str:
    return datetime.now(timezone.utc).isoformat()

Two things matter here beyond persistence. First, isolation_level=None with WAL gives you autocommit plus concurrent readers, which is fine for this write pattern. Second, the state and the counter are one row — they can never disagree after a crash.

The Breaker Logic

The transition function is small and should stay that way. Every closed trade goes through it; nothing else mutates the counter.

from dataclasses import replace

MAX_CONSECUTIVE_LOSSES = 3

class LossBreaker:
    def __init__(self, store: BreakerStore, strategy_id: str,
                 threshold: int = MAX_CONSECUTIVE_LOSSES):
        self.store = store
        self.strategy_id = strategy_id
        self.threshold = threshold

    def can_trade(self) -> bool:
        return self.store.load(self.strategy_id).state is State.ACTIVE

    def on_trade_closed(self, pnl: float, trade_meta: dict) -> BreakerRecord:
        rec = self.store.load(self.strategy_id)

        if rec.state is not State.ACTIVE:
            # Breaker is latched; closed PnL from any straggler orders
            # must not move the counter.
            return rec

        losses = rec.consecutive_losses + 1 if pnl < 0 else 0
        new_state = State.TRIPPED if losses >= self.threshold else State.ACTIVE
        rec = replace(rec, state=new_state, consecutive_losses=losses,
                      updated_at=_now())
        self.store.save(rec)

        if new_state is State.TRIPPED:
            # Execution is already blocked by the persisted state.
            self._handle_trip(rec, trade_meta)
        return rec

    def _handle_trip(self, rec: BreakerRecord, trade_meta: dict) -> None:
        history = self._recent_trades()
        # Move to COOLDOWN before notifying so a crash mid-notification
        # still leaves the strategy paused.
        self.store.save(replace(rec, state=State.COOLDOWN, updated_at=_now()))
        notify_trip(self.strategy_id, history, trade_meta)

    def _recent_trades(self) -> list[dict]:
        # Pull the last N closed trades from your trade log.
        return load_recent_closed_trades(self.strategy_id, limit=10)

    def resume(self, reviewer: str, note: str) -> BreakerRecord:
        rec = self.store.load(self.strategy_id)
        if rec.state is not State.COOLDOWN:
            raise RuntimeError(f"cannot resume from state {rec.state}")
        log_review(self.strategy_id, reviewer, note, rec.consecutive_losses)
        rec = replace(rec, state=State.ACTIVE, consecutive_losses=0,
                      updated_at=_now())
        self.store.save(rec)
        return rec

Note the ordering in _handle_trip: persist COOLDOWN first, then notify. If notify_trip throws, the strategy is already stopped. The notification is a side effect, not part of the safety mechanism.

Also note that on_trade_closed ignores PnL while the breaker is latched. This handles the real-world case where a stop order fills after the trip — you don't want that fill to silently reset a loss counter or, worse, re-trip an already-tripped breaker and re-fire notifications.

Wiring It Into the Execution Loop

The check has to sit in front of order submission, not inside the strategy logic. Strategy code is where you're most tempted to add exceptions.

def run_bar(breaker: LossBreaker, strategy, market_data) -> None:
    if not breaker.can_trade():
        return  # paused: no signals, no orders, no sizing

    signal = strategy.evaluate(market_data)
    if signal is None:
        return

    fill = execute_order(signal)
    if fill is not None and fill.closed:
        breaker.on_trade_closed(fill.pnl, fill.meta)

If your strategy emits orders from multiple places, wrap execute_order itself with the can_trade() guard as a second line of defense. Belt and suspenders is cheap here.

Notifications That Are Actually Useful

A push that says "strategy tripped" wastes the meeting. The notification should carry the evidence: the losing trades, their entry/exit, size, PnL, and the timestamps. That's what makes the review a review instead of a shrug.

import json
import requests

def notify_trip(strategy_id: str, history: list[dict], trigger: dict) -> None:
    lines = [f"Circuit breaker tripped: {strategy_id}", ""]
    for t in history:
        lines.append(
            f"{t['closed_at']}  {t['side']:<4}  size={t['size']:<8}  "
            f"pnl={t['pnl']:+.2f}  {t.get('exit_reason', '')}"
        )
    lines.append("")
    lines.append(f"Triggering trade: {json.dumps(trigger, default=str)}")
    body = "\n".join(lines)

    requests.post(
        "https://ntfy.sh/your-private-topic",
        data=body.encode("utf-8"),
        headers={"Title": f"BREAKER: {strategy_id}", "Priority": "high"},
        timeout=10,
    )

Keep the payload small and text-based. It should be readable on a phone lock screen. If you use a different channel — email, Telegram, a local webhook — the shape stays the same: strategy id, the streak, the trades.

Consecutive Losses vs. Drawdown Limits

These are not substitutes, and running only one is a common gap.

	Drawdown limit	Consecutive-loss breaker
Measures	Cumulative capital loss	Process deviation
Typical trigger	Slow bleed over weeks	Fast streak over days
Reset	New equity high	A single win
Failure mode it catches	Sizing too large for the edge	Edge decay, feed corruption, bug
Response	Reduce size / stop	Pause and review

A drawdown limit won't fire on three losses if the prior month was strong. A consecutive-loss breaker will. Conversely, a breaker won't catch a strategy that loses 55% of trades in a steady grind — you need the drawdown limit for that. Run both. They answer different questions and they fail in different directions.

One more distinction worth stating: the breaker is per strategy, not per account. If you run five strategies and one trips, the other four should keep running. Coupling them turns a single-strategy problem into a portfolio-wide outage.

The Psychology Part

The breaker is not punishment. It's a forced meeting with your own statistics.

Three losses in a row is not rare. For a strategy with a 45% win rate, the probability of any given three-trade window being all losses is about 16.6%. Over a few hundred trades you'll see it constantly. That's precisely why the rule is mechanical: if you decide case-by-case whether a streak is "just variance," you will decide it's variance every time, because that's the answer that lets you keep trading.

The manual resume step is the actual feature. It forces you to write down what you saw — the trades, the market context, whether the fills looked normal — before the strategy goes live again. Sometimes the review takes two minutes and you're back. Sometimes it takes a week and the strategy gets retired. Both are correct outcomes. What's not correct is a timer that restarts everything at midnight and calls it risk management.

Two practical habits make reviews fast:

Log the review. The log_review call in resume() writes who resumed, when, and why. After a few months you have a record of which strategies trip often and whether those trips preceded real problems. That's data about your breaker, not just your strategy.
Tune the threshold per strategy, not globally. Three is a reasonable default for strategies trading a few times a day. For a high-frequency strategy, three losses is noise; use a larger threshold or a percentage-based variant. For a low-frequency strategy, three losses might span months — there, a drawdown limit does more work.

Adapting It

The same structure generalizes cheaply:

Percentage-based streak: trip when cumulative loss over the current streak exceeds x% of equity, instead of a fixed count.
Time-bounded streak: only count losses within a rolling window, so a loss in January and a loss in June aren't "consecutive."
Per-instrument breakers: key the store on strategy_id + symbol if your strategy trades several instruments with independent behavior.

In all cases, keep the same invariants: persist state and counter together, block execution before notifying, and require an explicit, logged resume.

More notes like this ship every week on this site.

Daily Picks

The following pairs are selected from the multi-timeframe trend scanner (Gate.io futures) and are for technical-analysis study only — not investment advice.
Data updated: 2026-10-30 08:46:53

Long

Pair	Signal	Price	Take Profit	Stop Loss	R/R
TRX	高质量标的做多	$0.3521	$0.3768	$0.3398	1:2.0

Short

Pair	Signal	Price	Take Profit	Stop Loss	R/R
SNXX	庄家做空	$8.496	$8.0967	$8.7934	1:1.3

2 picks selected. Scanner runs every 15 minutes.

The Resonance Scanner: Seven Filters and the One That Mattered

mattleeee — Thu, 29 Oct 2026 01:02:11 +0000

The Resonance Scanner: Seven Filters and the One That Mattered

A scanner is a conjunction. Every filter must pass, so the output rate is the product of the individual pass rates. This is obvious arithmetic and almost nobody instruments for it. Instead, we tune filters one at a time, looking at each in isolation, and feel good about every one of them. "The trend filter is correct. The volume filter is correct. The momentum filter is correct." Seven correct filters, zero positions.

That was the situation with our resonance scanner: a seven-layer entry gate that produced watchlist entries for weeks and never opened a position. The forensics were not subtle once we looked. The binding constraint was pullback confirmation, the last layer, and the reason it bound was structural, not statistical: the moves we were trying to catch do not, by their nature, offer a clean three-candle pullback.

Anatomy of the Gate

The pipeline looked like this, in order:

Trend — higher-timeframe EMA alignment.
Volume — relative volume above a rolling baseline.
Momentum — rate-of-change percentile.
Volatility — ATR band, to skip dead and berserk regimes.
Session — allowed trading hours.
Blacklist — symbol exclusion list.
Pullback confirmation — a defined retracement over N candles, then a resumption candle.

Each filter is a pure function of a dataframe slice, which makes the whole thing easy to test and easy to instrument. The entry point was a single evaluate() that short-circuited on the first failure. That short-circuit was efficient and it was also why we were blind: a candidate rejected at layer 2 never touched layers 3 through 7, so we had no idea how many candidates each layer would have killed independently.

Instrumenting the Filters

The fix for blindness is cheap. Keep the short-circuit for live use, but add a shadow pass that evaluates every filter regardless of earlier failures and records the result. Two things matter in the record: the per-filter pass boolean, and the order of the first failure. The second is what tells you which filter is binding.

from dataclasses import dataclass, field

@dataclass
class FilterTrace:
    symbol: str
    ts: int
    results: dict = field(default_factory=dict)  # name -> bool

    @property
    def first_failure(self):
        for name, passed in self.results.items():  # insertion order = pipeline order
            if not passed:
                return name
        return None


def shadow_evaluate(symbol, ts, ctx, filters):
    """Evaluate all filters without short-circuiting. Order preserved."""
    trace = FilterTrace(symbol=symbol, ts=ts)
    for name, fn in filters:  # list of (name, callable), in pipeline order
        try:
            trace.results[name] = bool(fn(ctx))
        except Exception:
            trace.results[name] = False  # fail closed, but record it
    return trace

Note the except clause. A filter that throws is a filter that rejected, and silently swallowing the exception is how you end up with a layer that never passes and nobody notices. Fail closed, but count it separately so exceptions do not masquerade as legitimate rejections.

Live evaluation stays fast:

def live_evaluate(ctx, filters):
    for name, fn in filters:
        if not fn(ctx):
            return False, name
    return True, None

Both paths share the same filter list, so they cannot drift apart.

Aggregating the Traces

With a few weeks of traces stored, the analysis is a groupby. For every filter we want the independent pass rate (how often it would pass if it were the only gate), and the conditional pass rate given that all earlier filters passed. The gap between those two numbers is where the interesting failures live.

import pandas as pd

def filter_report(traces: list[FilterTrace]) -> pd.DataFrame:
    rows = []
    for t in traces:
        rows.append({"symbol": t.symbol, "ts": t.ts,
                     "first_failure": t.first_failure, **t.results})
    df = pd.DataFrame(rows)

    order = list(traces[0].results.keys())
    out = []
    for i, name in enumerate(order):
        independent = df[name].mean()
        if i == 0:
            conditional = independent
            reached = 1.0
        else:
            prior = order[:i]
            reached_mask = df[prior].all(axis=1)
            reached = reached_mask.mean()
            conditional = df.loc[reached_mask, name].mean() if reached_mask.any() else float("nan")
        out.append({
            "filter": name,
            "independent_pass": round(independent, 4),
            "reached_rate": round(reached, 4),
            "conditional_pass": round(conditional, 4),
        })
    return pd.DataFrame(out)

reached_rate is the fraction of candidates that survived everything before this layer. conditional_pass is the pass rate among those that got there. A filter with a high independent pass rate and a low conditional pass rate is doing real work at the margin. A filter with a low independent pass rate is a candidate for removal regardless of where it sits.

Then the binding constraint is simply the mode of first_failure:

binding = df["first_failure"].value_counts(dropna=False)
print(binding.head())

Our output looked roughly like this (numbers rounded, structure exact):

filter	independent_pass	reached_rate	conditional_pass
trend	0.31	1.00	0.31
volume	0.44	0.31	0.52
momentum	0.38	0.16	0.61
volatility	0.72	0.10	0.79
session	0.83	0.08	0.88
blacklist	0.97	0.07	0.98
pullback	0.11	0.07	0.11

Read the last row carefully. Seven percent of candidates reached the pullback layer, and only eleven percent of those passed it. The first six filters were, individually and collectively, behaving as designed: they cut a large universe down to a small, sensible set of live movers. The seventh filter then threw away 89% of that set. The mode of first_failure confirmed it: pullback was the last rejection for the overwhelming majority of candidates that got that far.

The Structural Problem

The naive conclusion is "loosen the pullback filter." That would have been wrong, and it took a while to articulate why.

The scanner's thesis was resonance: trend, volume, and momentum aligned on a symbol that was being actively accumulated. In practice, the moves that satisfied the first six filters were fast, high-volume expansions, often on altcoins during a coordinated push. These moves have a characteristic shape: a steep impulse, a shallow consolidation or a single red candle, then continuation or exhaustion. What they almost never have is a textbook three-candle retracement followed by a clean resumption candle. The pullback filter was written for a different market behavior: orderly trends that breathe. By the time a clean three-candle pullback prints on a fast expansion, the expansion is over. The filter was not rejecting noise. It was rejecting the exact events the scanner existed to find.

This is the general failure mode of stacked filters: each one is validated against the full population, but the conjunction selects a subpopulation with different conditional statistics. A pullback filter that works beautifully on a broad universe of trending names can have near-zero pass rate on the narrow slice that survives six upstream gates, because the upstream gates have already selected for the behavior the pullback filter excludes. Filters are not independent, and a pipeline is not a bag of good ideas.

The Design Tradeoff

The real decision was not "keep or drop pullback." It was where to sit on the spectrum between few but clean and so strict it never trades.

The discipline argument for strictness is sound. Every filter you add is a degree of freedom, and every degree of freedom is an opportunity to fit the last regime. A seven-condition entry is nearly impossible to overfit by accident, because the search space of ways to be wrong is enormous and the sample of trades is tiny. If a strict system trades rarely, at least the trades it takes are well-supported.

The counterargument is that a scanner which never trades is not conservative, it is broken. It provides no information, accumulates no live sample, and its filters are never validated against real outcomes. You cannot distinguish a disciplined system from a mis-specified one if the mis-specification happens to be the last gate in the chain. Strictness that produces zero fills is not a virtue; it is a measurement failure wearing a virtue's clothes.

The resolution we landed on was to treat the pullback layer as a regime-conditional filter rather than a universal one, and to replace the hard three-candle rule with a graded score so the layer can be strict in one regime and permissive in another without being deleted outright.

def pullback_score(df, lookback=3, min_retrace=0.25, max_retrace=0.75):
    """
    Score 0..1 instead of boolean. Measures whether the recent action
    shows a controlled retracement rather than a vertical run.
    """
    impulse = df["close"].iloc[-lookback-1] - df["close"].iloc[-lookback*2-1]
    if impulse <= 0:
        return 0.0
    retrace = (df["close"].iloc[-lookback-1] - df["close"].iloc[-1]) / impulse
    if retrace < min_retrace:
        return 0.0            # no pullback at all: vertical run
    if retrace > max_retrace:
        return 0.0            # too deep: structure broken
    # peak score at mid-retrace, taper toward both edges
    mid = (min_retrace + max_retrace) / 2
    width = (max_retrace - min_retrace) / 2
    return max(0.0, 1.0 - abs(retrace - mid) / width)


def pullback_gate(ctx, threshold=0.5):
    score = pullback_score(ctx.df)
    # Fast-expansion regime: accept a shallower structure.
    if ctx.regime == "expansion":
        return score >= threshold * 0.6
    return score >= threshold

Two changes matter here. First, the layer now returns a continuous score, so you can see how close near-misses were instead of only pass/fail. Second, the threshold is regime-dependent, which encodes the actual finding: fast expansions legitimately do not pull back like orderly trends do, and pretending otherwise is a modeling error, not a discipline.

Crucially, we did not just lower the threshold globally until trades appeared. That is the opposite failure: tuning a filter until it stops binding, then calling the result validated. The regime split came from the trace data (pullback pass rates differed sharply by regime upstream), and the new threshold was chosen on out-of-sample traces, not on the same window that produced the diagnosis.

What to Keep

The instrumented pipeline is worth keeping permanently, not just for this investigation. The FilterTrace records cost almost nothing, and the filter_report table is the single most useful diagnostic a multi-condition scanner can have. Run it monthly. If the binding constraint moves, you have learned something about the market. If it never moves, you have learned something about your filters: one of them is doing all the work, and the other six are decoration.

The lesson generalizes past this scanner. A conjunction of filters is only as good as its weakest link, and you cannot find the weakest link by inspecting filters individually. You find it by counting first failures, computing conditional pass rates, and asking whether the surviving subpopulation is the one you actually meant to trade. The filter that rejects last is the filter that defines your system, whether or not you intended it to.

More notes like this ship every week on this site.

Daily Picks

The following pairs are selected from the multi-timeframe trend scanner (Gate.io futures) and are for technical-analysis study only — not investment advice.
Data updated: 2026-10-29 08:46:59

Long

Pair	Signal	Price	Take Profit	Stop Loss	R/R
BTC	趋势低吸做多	$81120.0	$87204.0	$80574.0602	1:11.1
SNXX	高质量标的做多	$10.016	$10.3866	$9.8157	1:1.9

2 picks selected. Scanner runs every 15 minutes.

A/B Testing Trading Rules: The 52 Rejected Trades That Changed My Mind

mattleeee — Wed, 28 Oct 2026 01:01:52 +0000

The Question a Filter Should Answer

A pullback filter sits in front of an entry signal and decides which setups are allowed through. The usual debate goes like this: the strict variant looks cleaner on the equity curve, the relaxed variant trades more, and someone argues the extra trades are "noise." That argument is almost never tested. It is asserted from the shape of the curve.

The right question is narrower: what happens to the trades that exist only because of the relaxed rule? If that incremental set has positive expectancy, the strict filter is not cleaning up noise. It is deleting the strategy's core. In the case I worked through, the strict rule rejected 52 trades that the relaxed rule took. Those 52 trades had a 48.1% win rate, +0.131U expectancy per trade, and a profit factor of 1.50. The "worse" rule was rejecting the profitable part.

This post is about the method, not the specific rule: run both variants over the same window, diff the trade lists, and evaluate the rejected set on its own statistics.

Why the Headline Curve Can't Settle It

Two variants produce two equity curves. Comparing them tells you which variant made more money over one sample. It does not tell you whether the filter is doing what you think it is doing, because the curves aggregate everything: shared trades, rejected trades, and the timing of both.

A strict filter can improve a curve for three different reasons:

It removes genuinely bad trades (the intended effect).
It removes trades that were fine but happened to lose in this sample (overfitting to noise).
It changes position sizing or trade sequencing in a way that masks the removal of good trades.

Only the first is a reason to keep it. The diff isolates the trades that differ, and their statistics tell you which of the three you are looking at.

The diff is the unit of analysis

Think of the two variants as generating two sets of trades over an identical window. The shared trades are irrelevant to the filter decision — both rules took them, so they contribute equally to both curves. The decision lives entirely in the symmetric difference:

Rejected-only: trades the strict rule skipped and the relaxed rule took.
Accepted-only: trades the relaxed rule skipped and the strict rule took (this happens when the rules are not strictly nested).

If the rejected-only set has positive expectancy with a reasonable sample size, the filter is destroying value. If it has strongly negative expectancy, the filter is earning its keep. If the set is tiny, you have no evidence either way and should not pretend otherwise.

Building the Two Trade Lists

The prerequisite is that both variants log trades in the same schema, over the same data, with the same execution assumptions. Any difference in slippage model or fill logic between runs will contaminate the diff.

from dataclasses import dataclass
from datetime import datetime

@dataclass(frozen=True)
class Trade:
    symbol: str
    entry_time: datetime
    exit_time: datetime
    direction: str          # "long" / "short"
    entry_price: float
    exit_price: float
    size: float             # units
    pnl: float              # realized, in account currency
    rule_variant: str       # "strict" / "relaxed"

A trade key needs to identify the same opportunity across runs. Entry timestamp plus symbol plus direction is usually enough; if a strategy can re-enter the same symbol on the same bar, add an entry-sequence index.

def trade_key(t: Trade) -> tuple:
    return (t.symbol, t.entry_time, t.direction)

Run both variants over the same window and the same instrument universe:

strict_trades = run_backtest(strategy, data, params=STRICT_PARAMS)
relaxed_trades = run_backtest(strategy, data, params=RELAXED_PARAMS)

strict_by_key = {trade_key(t): t for t in strict_trades}
relaxed_by_key = {trade_key(t): t for t in relaxed_trades}

rejected_only = [t for k, t in relaxed_by_key.items() if k not in strict_by_key]
accepted_only = [t for k, t in strict_by_key.items() if k not in relaxed_by_key]
shared = [t for k, t in strict_by_key.items() if k in relaxed_by_key]

Sanity check before analyzing anything: len(shared) + len(rejected_only) should equal the relaxed run's trade count, and the PnL of the shared set should be identical in both runs. If it is not, the two runs are not comparable and the diff means nothing.

Statistics for the Incremental Set

The rejected-only set is just a list of trades. Evaluate it exactly as you would a standalone strategy, but report the full picture, not one number.

import statistics as st

def summarize(trades: list[Trade]) -> dict:
    n = len(trades)
    if n == 0:
        return {"n": 0}

    pnls = [t.pnl for t in trades]
    wins = [p for p in pnls if p > 0]
    losses = [p for p in pnls if p < 0]

    gross_win = sum(wins)
    gross_loss = -sum(losses)

    return {
        "n": n,
        "win_rate": len(wins) / n,
        "avg_win": st.mean(wins) if wins else 0.0,
        "avg_loss": st.mean(losses) if losses else 0.0,
        "expectancy": st.mean(pnls),
        "profit_factor": gross_win / gross_loss if gross_loss > 0 else float("inf"),
        "total_pnl": sum(pnls),
    }

For the 52 rejected trades this returns something like:

n=52  win_rate=0.481  avg_win=+0.71U  avg_loss=-0.41U
expectancy=+0.131U  profit_factor=1.50  total_pnl=+6.81U

Read it as a whole. A 48.1% win rate sounds unremarkable until you pair it with the payoff ratio: average win is roughly 1.7x average loss, so sub-50% accuracy still yields positive expectancy. Profit factor 1.50 over 52 trades is not spectacular, but it is clearly the wrong side of zero for a filter that is supposed to be removing bad trades. A filter that rejects a set with PF 1.50 is rejecting trades you would have wanted.

Interpret the shape, not just the sign

Three patterns are worth distinguishing when you look at the rejected-only set:

Negative expectancy with a normal win rate: the filter is removing losers. Keep it.
Near-zero expectancy with a large sample: the filter is removing noise. Keep it only if it materially reduces drawdown or costs; otherwise it is not earning its complexity.
Positive expectancy: the filter is removing profit. The strict variant needs a different justification, or it should go.

The 52-trade case is the third pattern, and the win rate alone would have hidden it. This is why the diff has to be evaluated with expectancy and profit factor, not just hit rate.

Guarding Against One Lucky Trade

The most common way a diff analysis lies is concentration. Fifty-two trades with +0.131U expectancy can be twelve good trades and forty flat ones, or one trade that made the entire set. Before trusting the result, check:

Per-trade contribution

def concentration_report(trades: list[Trade], top_k: int = 5) -> dict:
    pnls = sorted((t.pnl for t in trades), reverse=True)
    total = sum(pnls)
    top = sum(pnls[:top_k])

    return {
        "total_pnl": total,
        "top_k_pnl": top,
        "top_k_share": top / total if total != 0 else float("inf"),
        "expectancy_ex_top_k": (total - top) / (len(pnls) - top_k) if len(pnls) > top_k else None,
    }

If the top five trades account for most of the set's PnL and expectancy excluding them collapses toward zero, the incremental set is not evidence of a better rule. It is evidence of a handful of outcomes.

Per-symbol breakdown

The diff can also be dominated by one instrument. Group the rejected-only trades by symbol and compare each group's count and expectancy against its share of the full sample.

from collections import defaultdict

def by_symbol(trades: list[Trade]) -> dict[str, dict]:
    groups = defaultdict(list)
    for t in trades:
        groups[t.symbol].append(t)
    return {sym: summarize(ts) for sym, ts in groups.items()}

If one symbol supplies most of the rejected trades and all of the positive expectancy, the honest conclusion is "the relaxed rule helps on this symbol in this window," not "the relaxed rule is better." A filter decision made on a single-symbol diff is a single-symbol decision.

Sample size and stability

Fifty-two trades is enough to see a signal but not enough to be confident about its magnitude. Split the rejected-only set in half by time and check whether expectancy holds in both halves. If the first half carries everything and the second half is flat or negative, the incremental set is not a stable property of the rule.

def split_half(trades: list[Trade]) -> tuple[list[Trade], list[Trade]]:
    ordered = sorted(trades, key=lambda t: t.entry_time)
    mid = len(ordered) // 2
    return ordered[:mid], ordered[mid:]

What the Method Actually Decides

The diff analysis does not tell you which rule is "correct." It tells you what each rule is doing. In this case, the strict pullback filter was not removing noise — it was removing a set of trades with positive expectancy, a 1.50 profit factor, and a payoff profile that made a sub-50% win rate profitable. The relaxed rule was not sloppier; the strict rule was over-filtered.

The general procedure generalizes beyond pullback filters. Any time you compare two entry conditions, two exit rules, or two position-sizing schemes, the shared trades cancel and the decision lives in the diff. Evaluate the incremental set with count, win rate, average win and loss, expectancy, and profit factor, then check whether that set is concentrated in a few trades or a single symbol. If it is, you have a hypothesis, not a result.

The headline curve is a summary of everything at once. The diff is the only place the filter's actual effect is visible.

More notes like this ship every week on this site.

Backtesting the Pullback Filter: When a Safety Check Kills Profit

mattleeee — Tue, 27 Oct 2026 01:02:36 +0000

The Setup: One Strategy, Two Filters

The base strategy is a long-only momentum entry on 15-minute candles. Signal logic, position sizing, and exit rules are fixed. The only thing that changes between variants is the pullback confirmation that must be true before an entry fires.

The strict filter requires all three of the following on the entry bar:

At least one down candle among the last three 15-minute candles.
Current price below the high of the last five candles.
The latest candle is not a large bullish candle (body above a threshold, e.g. 0.8 × ATR).

The relaxed filter drops the three-candle window and the big-bullish veto, keeping only: at least one down candle within the last five candles. It still demands a pullback, just a looser one.

Both variants run on the same 90-day window, 24 symbols, same fee model, same slippage assumption. That is the whole point of the experiment: a single boolean changes, everything else is held constant.

Why the Comparison Is Easy to Get Wrong

The tempting move is to compare the two versions by their win rates or average PnL per trade. That comparison is contaminated. The strict filter does not just change how trades are taken — it changes which trades exist. The strict set is a subset of the relaxed set, so you are comparing two different populations of trades, not the same trades under two treatments.

To say anything honest about the filter, you need to look at the trades the strict filter rejects. Those rejected entries are where the answer lives. If the rejected trades have positive expectancy on their own, the filter is destroying edge. If they are negative, the filter is earning its keep.

So the experiment has three outputs, not two:

Trades taken by both variants (the intersection).
Trades taken only by the relaxed variant (rejected by strict).
Expectancy of each group.

The base strategy under the relaxed filter is the control population. The strict filter is a treatment applied to the signal stream. Treat it like an A/B test where the "B" arm is a subset of "A", and you want to know whether the excluded slice is better or worse than average.

Counting the Incremental Trades

The cleanest way to run this is to evaluate the signal stream once, tag every candidate entry with the filter booleans, and then slice. Here is a minimal harness that does exactly that.

from dataclasses import dataclass
from typing import Callable

@dataclass
class Trade:
    symbol: str
    ts: int
    entry: float
    exit: float
    pnl: float          # net of fees and slippage, in quote units

def strict_pullback(candles, i: int, atr: float) -> bool:
    # candles: list of (ts, o, h, l, c) ascending, i = entry bar index
    last3 = candles[i-2:i+1]
    has_down = any(c[4] < c[1] for c in last3)
    five_high = max(c[2] for c in candles[i-4:i+1])
    body = candles[i][4] - candles[i][1]
    not_big_bull = body < 0.8 * atr
    return has_down and candles[i][4] < five_high and not_big_bull

def relaxed_pullback(candles, i: int, atr: float) -> bool:
    last5 = candles[i-4:i+1]
    return any(c[4] < c[1] for c in last5)

The backtest engine stays untouched. You generate candidate entries from the momentum signal, then record both filter verdicts alongside the realized PnL. Note that the entry price and exit price are identical for a given candidate regardless of which filter accepted it — the filter does not move the entry bar, it only decides whether the bar is tradeable at all. That assumption matters: if your strict filter also shifts entry timing (e.g. waiting for confirmation on the next bar), you have changed two variables and the comparison is dead.

def run_backtest(candles_by_symbol, signal_fn, exit_fn):
    rows = []
    for sym, candles in candles_by_symbol.items():
        atr_series = compute_atr(candles, 14)
        for i in range(20, len(candles) - 1):
            if not signal_fn(candles, i):
                continue
            atr = atr_series[i]
            entry = candles[i][4]
            exit_price = exit_fn(candles, i)
            pnl = (exit_price - entry) - fee_and_slip(entry, exit_price)
            rows.append({
                "symbol": sym,
                "ts": candles[i][0],
                "pnl": pnl,
                "strict": strict_pullback(candles, i, atr),
                "relaxed": relaxed_pullback(candles, i, atr),
            })
    return rows

Now the analysis is just a groupby. No second backtest run, no risk of drift between the two code paths.

def slice_expectancy(rows):
    both = [r["pnl"] for r in rows if r["strict"] and r["relaxed"]]
    strict_only = [r["pnl"] for r in rows if r["strict"] and not r["relaxed"]]
    relaxed_only = [r["pnl"] for r in rows if r["relaxed"] and not r["strict"]]
    # note: strict implies relaxed by construction only if the
    # relaxed rule is a strict superset — verify this before trusting it
    return {
        "both": (len(both), sum(both) / len(both) if both else 0),
        "strict_rejected": (len(relaxed_only),
                            sum(relaxed_only) / len(relaxed_only) if relaxed_only else 0),
        "strict_only": (len(strict_only),
                        sum(strict_only) / len(strict_only) if strict_only else 0),
    }

One check before you trust any of this: confirm the strict rule is actually a subset of the relaxed rule. If your relaxed rule is "any down candle in the last five" and your strict rule also requires "price below the five-candle high," then strict does not imply relaxed — a bar can satisfy strict's down-candle condition and high condition while failing nothing in relaxed, but the reverse can also happen if the down candle is outside the five-bar window in strict's three-bar check. Run the set-difference both ways and print the counts. If strict_only is non-empty, your two rules are not nested, and you should compare them as two separate populations rather than a control and a treatment.

The Numbers

On the 90-day, 24-symbol run:

Variant	Trades	Expectancy (U/trade)
Strict pullback	~1,900	-0.013
Relaxed pullback	~2,600	+0.011

The strict filter rejects roughly 700 candidate entries. Those rejected trades carry positive expectancy — enough to flip the whole strategy from negative to positive. The filter did not remove noise; it removed the profitable tail.

The win rate on the strict set was actually slightly higher. That is the trap. A filter that raises win rate while lowering expectancy is selecting for small, frequent wins and occasional large losses — exactly what a late, tired move produces.

Why "Safer" Is Worse Here

The strict filter's three conditions all bias toward the same failure mode: entering after the move has already extended and then rolled over.

Requiring a down candle in the last three bars means you enter on the first sign of weakness. In a momentum regime, that is often the start of a real reversal, not a healthy pullback.
Requiring price below the five-candle high means you are buying into a local dip — but only after the high was already set. The move is mature.
Vetoing a big bullish candle removes exactly the bars where momentum is re-accelerating. Those are the entries with the best follow-through.

The relaxed filter keeps the pullback requirement but gives the move room to breathe. It accepts entries where the pullback happened four or five bars ago and price has since resumed. Those entries are earlier in the next leg, not later in the current one.

Put differently: the strict filter is a proxy for "the setup looks clean right now." But clean-looking setups are, by construction, the ones where the crowd has already noticed. The relaxed filter accepts messier setups that the strategy's exit logic can still harvest. The filter was supposed to reduce drawdown. It reduced trade count and took the edge with it.

Practical Rules for Filter A/B Tests

A few habits that keep this kind of test honest:

Keep the exit fixed. If you tune the exit for each filter variant, you are no longer measuring the filter. Any difference in expectancy is confounded by the exit change.

Always report the rejected slice. A filter's value is the expectancy of what it excludes. If you do not compute that number, you do not know what the filter does.

Check nesting before slicing. Confirm which candidate population each rule accepts, and whether one is a subset of the other. Non-nested rules need separate baselines.

Watch win rate and expectancy diverge. When a filter raises win rate but lowers expectancy, you have found a filter that trades frequency for size. That trade is only good if the size reduction is larger than the frequency gain — usually it is not.

Run the test on out-of-sample data. A 90-day window with 24 symbols is enough to see a sign flip, not enough to trust the magnitude. Re-run on a held-out period and check whether the rejected-slice expectancy keeps the same sign.

Closing

Filters that feel safe are often filters that select for late entries. Measure what they reject, not just what they keep.

More notes like this ship every week on this site.

Daily Picks

The following pairs are selected from the multi-timeframe trend scanner (Gate.io futures) and are for technical-analysis study only — not investment advice.
Data updated: 2026-10-27 08:46:47

Short

Pair	Signal	Price	Take Profit	Stop Loss	R/R
Q	热点跟涨	$0.2008	$0.1848	$0.2079	1:2.3

1 picks selected. Scanner runs every 15 minutes.

Three-Layer Validation Before You Trust a Backtest

mattleeee — Mon, 26 Oct 2026 01:02:11 +0000

Why One Good Backtest Means Nothing

A single equity curve is a sample of size one. Optimize hard enough over enough combinations and something will look great by luck alone. The question is not "does this rule make money in the backtest?" but "does this rule make money for reasons that survive contact with slightly different assumptions?"

The gate I use before any rule reaches the live scanner has three layers: a parameter sweep, a monthly consistency check, and a component ablation. Each layer answers a different question, and each has a hard pass/fail threshold. A rule has to clear all three. This post walks through the mechanics, then through a real rejection: a MACD divergence entry that passed layer 1 and died on layers 2 and 3.

Layer 1: Parameter Sweep

The question it answers

Is the edge a plateau or a spike? If performance only exists at fast=12, slow=26, signal=9, you have found noise, not a rule. Neighboring parameter values should produce comparable results, and the chosen point should sit near the center of a profitable region, not on its edge.

The metrics

For each parameter combination in the grid, record:

Trade count — anything under ~50 is too thin to say much.
Profit factor (PF) — gross wins / gross losses.
Expectancy per trade — in R units (risk multiples), so results are comparable across parameter sets.
Max drawdown — in R, not percent, so it doesn't depend on position sizing.
Plateau score — the fraction of neighboring combinations (within one step in each dimension) that stay above the PF threshold.

The thresholds

Median PF across the sweep ≥ 1.3
At least 70% of combinations with ≥ 50 trades have PF > 1.0
Plateau score at the chosen point ≥ 0.6
Chosen point is not on the grid boundary

The code

import itertools
import numpy as np
import pandas as pd

def sweep(params_grid, run_backtest, min_trades=50):
    rows = []
    for combo in itertools.product(*params_grid.values()):
        params = dict(zip(params_grid.keys(), combo))
        res = run_backtest(**params)
        if res["trades"] < min_trades:
            continue
        rows.append({**params, **res})
    return pd.DataFrame(rows)

def plateau_score(df, param_names, chosen, pf_col="pf", threshold=1.0):
    """Fraction of 1-step neighbors of `chosen` with PF above threshold."""
    neighbors = []
    for name in param_names:
        values = sorted(df[name].unique())
        idx = values.index(chosen[name])
        for offset in (-1, 1):
            j = idx + offset
            if 0 <= j < len(values):
                mask = pd.Series(True, index=df.index)
                for n in param_names:
                    target = chosen[n] if n != name else values[j]
                    mask &= df[n] == target
                if mask.any():
                    neighbors.append(df.loc[mask, pf_col].iloc[0])
    if not neighbors:
        return 0.0
    return float(np.mean(np.array(neighbors) > threshold))

grid = {
    "fast":   [8, 10, 12, 14, 16],
    "slow":   [20, 24, 26, 30, 34],
    "signal": [7, 9, 11],
}
results = sweep(grid, run_macd_divergence_backtest)

chosen = {"fast": 12, "slow": 26, "signal": 9}
score = plateau_score(results, list(grid.keys()), chosen)

print(f"median PF: {results['pf'].median():.2f}")
print(f"share PF>1: {(results['pf'] > 1.0).mean():.0%}")
print(f"plateau score at chosen point: {score:.2f}")

A plateau score of 0.6 means most one-step neighbors still clear PF 1.0. That is a low bar, deliberately. If a rule can't survive a one-step nudge, it has no business being tested further.

Layer 2: Monthly Consistency

The question it answers

Is the annual result distributed across time, or is it one monster month doing all the work? A strategy with one +40% month and eleven flat months is not a strategy. It's a lottery ticket that already paid out, and the next ticket is unlikely to pay again.

The metrics

Bucket trades by calendar month and compute:

Share of profitable months — count of months with positive PnL / total months.
Best-month contribution — best month's PnL / total PnL.
Top-3-month contribution — same, for the three best months.
Median monthly return and worst month in R.

The thresholds

≥ 55% of months positive
Best month ≤ 30% of total PnL
Top 3 months ≤ 55% of total PnL
Worst month no worse than −3R (scaled to your per-trade risk)

The best-month cap is the important one. It is the metric that catches curve-fit strategies that happen to overlap a single volatility event.

The code

def monthly_consistency(trades: pd.DataFrame, risk_per_trade=1.0):
    """trades: columns ['exit_time', 'pnl_r']"""
    monthly = (trades
               .assign(month=trades["exit_time"].dt.to_period("M"))
               .groupby("month")["pnl_r"]
               .sum()
               .sort_index())
    total = monthly.sum()
    if total <= 0:
        return {"pass": False, "reason": "non-positive total PnL"}

    best = monthly.max()
    top3 = monthly.nlargest(3).sum()
    stats = {
        "months": len(monthly),
        "positive_share": (monthly > 0).mean(),
        "best_share": best / total,
        "top3_share": top3 / total,
        "median_month": monthly.median(),
        "worst_month": monthly.min(),
    }
    stats["pass"] = (
        stats["months"] >= 24
        and stats["positive_share"] >= 0.55
        and stats["best_share"] <= 0.30
        and stats["top3_share"] <= 0.55
        and stats["worst_month"] >= -3 * risk_per_trade
    )
    return stats

The months >= 24 requirement matters. With twelve months, "positive share" is dominated by a handful of observations and the test is close to meaningless.

Layer 3: Robustness Ablation

The question it answers

Does each component of the rule contribute, and does the rule degrade gracefully when a component is removed? A robust rule loses some edge when you strip a filter. A fragile rule collapses to zero, or worse, only works with every component present — a sign the components were fitted to each other.

The metrics

For each component, run the backtest with that component removed (or replaced by a neutral version) and record:

PF delta — component PF minus full-rule PF.
Expectancy delta in R.
Retention — component expectancy / full-rule expectancy.

The thresholds

Removing any single component leaves expectancy ≥ 60% of the full rule (graceful degradation).
Removing the core component (the entry trigger itself, not a filter) drops expectancy below 50% and PF below 1.1 (the core actually matters).
No component removal improves expectancy by more than 15% (if it does, that component is hurting and should be cut).

The third criterion is the one people skip. It catches filters that were added to patch a specific losing streak in the sample and do nothing out of sample.

The code

def ablation(full_result, variants: dict, core="divergence"):
    """variants: name -> backtest result dict with 'pf' and 'expectancy_r'"""
    base_pf = full_result["pf"]
    base_exp = full_result["expectancy_r"]
    rows = []
    for name, res in variants.items():
        rows.append({
            "component": name,
            "pf": res["pf"],
            "expectancy_r": res["expectancy_r"],
            "retention": res["expectancy_r"] / base_exp if base_exp else 0.0,
            "pf_delta": res["pf"] - base_pf,
        })
    df = pd.DataFrame(rows)

    checks = {
        "graceful": (df.loc[df["component"] != core, "retention"] >= 0.60).all(),
        "core_matters": df.loc[df["component"] == core, "retention"].lt(0.50).all(),
        "no_harmful": (df["retention"] <= 1.15).all(),
    }
    return df, checks

The Case Study: A MACD Divergence Entry

The rule: enter long when price makes a lower low but MACD histogram makes a higher low (bullish divergence), confirmed by a close above the signal line. Exit on a fixed 2R target or a time stop of 20 bars. Filters included a trend filter (price above the 200-EMA), a volume filter (volume above its 20-bar average), and a volatility floor (ATR above its 20-bar median).

Layer 1: passed

The sweep over fast ∈ [8,16], slow ∈ [20,34], signal ∈ [7,11] produced 75 combinations, 68 with enough trades. Median PF was 1.41, 78% of combinations had PF above 1.0, and the plateau score at (12, 26, 9) was 0.71. The chosen point sat comfortably inside the grid. Nothing suspicious yet.

Layer 2: failed

Over 31 months, only 48% of months were positive. Worse, the best month contributed 44% of total PnL, and the top three months contributed 71%. The best month coincided with a broad volatility expansion — the rule caught one regime and did nothing in the other thirty. Median monthly PnL was slightly negative.

Verdict: fail on positive_share (0.48 < 0.55), best_share (0.44 > 0.30), and top3_share (0.71 > 0.55). This alone is enough to reject the rule.

Layer 3: failed

The ablation was run anyway, out of curiosity:

Removed	PF	Expectancy (R)	Retention
Trend filter	1.38	0.11	0.73
Volume filter	1.52	0.17	1.13
Volatility floor	1.09	0.06	0.40
Divergence (core)	0.94	−0.01	−0.07

Two problems. The volatility floor is doing most of the work — removing it drops retention to 0.40, well below the 0.60 floor. That means the "divergence" rule is largely a low-volatility regime filter wearing a costume. And removing the volume filter improves expectancy by 13%, which is under the 15% harm threshold but close enough to be a warning sign: the filter was probably added to patch a specific stretch of the sample.

The core divergence component does matter (removal sends expectancy negative), but that doesn't save the rule. A rule whose edge lives mostly in a regime filter, and whose only positive months come from one volatility event, is not a rule. It's a description of one period.

The rejection

Three-layer result: pass, fail, fail. The rule went into the rejected list with the metrics recorded, so future me doesn't re-test it in six months and get excited all over again. That bookkeeping is half the value of the process.

Practical Notes

Run the layers in order. Layer 1 is cheap and filters out most junk. Layer 2 is cheaper than layer 3 and catches the most common failure mode. Don't run ablations on rules that can't pass a monthly check.
Log every metric, including failures. A rejection is only useful if it's documented. Keep the sweep DataFrame and the ablation table next to the rule definition.
Thresholds are conventions, not laws. The numbers above are tuned to my trade frequency and risk unit. If your rules trade twenty times a year, months >= 24 is too strict; if they trade daily, best_share <= 0.30 is probably too loose. Pick thresholds once, write them down, and don't move them after seeing a result you like.
The ablation is the most informative layer. It tells you why a rule works, not just whether it did. When a rule fails layer 3, you often learn something reusable about the market — in this case, that low-volatility regimes are where this class of divergence entry lives, which is a hypothesis worth testing on its own.

More notes like this ship every week on this site.

Enforcing HTTPS on GitHub Pages After a Domain Switch

mattleeee — Sun, 25 Oct 2026 01:02:24 +0000

Pointing a new domain at GitHub Pages works on the first try often enough that the greyed-out checkbox comes as a surprise. You add the custom domain, GitHub accepts it, the site loads over plain HTTP, and the Enforce HTTPS option sits there disabled with a note that the certificate has not been provisioned yet. In most cases nothing is wrong. GitHub is still in the middle of issuing the certificate, and the toggle unlocks on its own once that finishes.

This post covers the whole lifecycle: what happens after you save a custom domain, how to watch it progress from the command line, why you should fix mixed content before you enforce HTTPS, and what to check when provisioning genuinely fails.

The provisioning sequence

When you save a custom domain for a GitHub Pages site, GitHub runs a fixed pipeline in the background. The UI gives you one checkbox as feedback, but three distinct things happen.

1. DNS check

GitHub resolves the domain and confirms it points where it should:

For an apex domain (example.com), A records must point to the Pages IPs, or an ALIAS/ANAME record must be configured.
For a subdomain (www.example.com, blog.example.com), a CNAME record must point to <user>.github.io (or <org>.github.io).

If DNS does not resolve correctly at this stage, the certificate request never starts. The UI will keep saying the certificate is pending, and it will keep saying that indefinitely. This is the failure mode worth knowing about, because it looks identical to normal waiting.

2. Certificate issuance

Once DNS checks out, GitHub requests a certificate for the domain (and the www variant where applicable) from its CA. This is the part that takes time. Typical latency is 10 to 30 minutes, but it depends on DNS TTLs and CA queueing, so an hour is not unheard of after a domain switch. During this window the site is reachable over HTTP and, in many cases, already over HTTPS with a certificate that browsers may or may not accept yet.

3. Enforce toggle unlocks

Only after the certificate is live does GitHub enable the Enforce HTTPS checkbox. Ticking it makes Pages redirect all HTTP requests to HTTPS with a 301. Until then, the box stays disabled, and there is no override, no query parameter, no API call that forces it early.

The practical takeaway: after a domain switch, expect a wait. Do not start changing DNS records while the certificate is pending, because you will reset the process.

Watching progress from the terminal

The GitHub settings page is a poor progress indicator. It shows a static message until it flips. curl tells you more.

Check what is actually being served

curl -I http://example.com
curl -I https://example.com

Run both. Before the certificate is live, the HTTPS request typically fails at the TLS handshake, which curl reports as:

curl: (60) SSL certificate problem: unable to get local issuer certificate

or a certificate mismatch error. The HTTP request, meanwhile, returns 200 with no redirect. That combination means: DNS is fine, certificate is not ready.

Once provisioning completes, https:// returns 200 and http:// still returns 200 until you tick the box. After enforcing, http:// returns:

HTTP/1.1 301 Moved Permanently
Location: https://example.com/

Inspect the served certificate

curl -I only tells you whether TLS works. To see which certificate is being served and for which names, use openssl:

echo | openssl s_client -servername example.com -connect example.com:443 2>/dev/null \
  | openssl x509 -noout -subject -issuer -dates -ext subjectAltName

A healthy result looks roughly like:

subject=CN = example.com
issuer=C = US, O = Let's Encrypt, CN = R3
notBefore=...
notAfter=...
X509v3 Subject Alternative Name:
    DNS:example.com, DNS:www.example.com

Two things to verify here. First, the SAN list must include every hostname you actually use. If you serve www.example.com but the certificate only covers the apex, browsers will warn on the www host even though the apex is fine. Second, check notBefore and notAfter to rule out a stale or expired certificate being served from an edge cache.

If openssl returns a certificate whose subject is *.github.io or belongs to a different domain, the request is landing on the wrong vhost. That usually means DNS is still pointing at an old host, or a proxy (Cloudflare, for example) is terminating TLS in front of Pages. Cloudflare's "Full" or "Flexible" SSL modes in front of GitHub Pages are a common source of confusing certificate output.

Scripting the wait

If you switch domains often, a small polling script saves a lot of tab-refreshing. This one checks the HTTPS handshake and the redirect status until both are correct, or gives up after a timeout:

import ssl
import socket
import time
import urllib.request
import urllib.error

DOMAIN = "example.com"
TIMEOUT_MINUTES = 45
INTERVAL_SECONDS = 120


def tls_ok(host: str, port: int = 443) -> bool:
    ctx = ssl.create_default_context()
    try:
        with socket.create_connection((host, port), timeout=10) as sock:
            with ctx.wrap_socket(sock, server_hostname=host) as tls:
                cert = tls.getpeercert()
                sans = [v for k, v in cert.get("subjectAltName", ()) if k == "DNS"]
                return host in sans or f"*.{host.split('.', 1)[1]}" in sans
    except (ssl.SSLError, socket.error, OSError):
        return False


def redirects_to_https(url: str) -> bool:
    req = urllib.request.Request(url, method="HEAD")
    try:
        with urllib.request.urlopen(req, timeout=10) as resp:
            return resp.geturl().startswith("https://")
    except urllib.error.HTTPError as e:
        return e.code in (301, 308) and e.headers.get("Location", "").startswith("https://")
    except urllib.error.URLError:
        return False


deadline = time.time() + TIMEOUT_MINUTES * 60
while time.time() < deadline:
    cert = tls_ok(DOMAIN)
    enforced = redirects_to_https(f"http://{DOMAIN}")
    print(f"cert={cert} enforced={enforced}")
    if cert and enforced:
        print("Done.")
        break
    time.sleep(INTERVAL_SECONDS)
else:
    print("Timed out. Check DNS and the Pages settings.")

The tls_ok check is stricter than a plain connection test: it validates that the hostname appears in the SAN list, which is the thing that actually matters for browsers. The redirects_to_https check confirms enforcement is active. Run it from a machine that is not behind the same DNS resolver you just changed, or you may be testing a cached view.

Fix mixed content before you enforce

This is the step people skip, and it causes the most visible breakage. Enforcing HTTPS makes the page itself load over TLS, but it does not rewrite the URLs inside your HTML, CSS, or JavaScript. Any resource still referenced as http:// becomes mixed content. Modern browsers block active mixed content (scripts, stylesheets, iframes) outright and either block or auto-upgrade passive mixed content (images), depending on the browser and the resource type.

The result: the page loads, the layout looks wrong, and the console is full of errors. Users blame the certificate; the real cause is a handful of hard-coded http:// URLs.

Find them before you flip the switch

Crawl your own site over HTTP and grep the rendered HTML for http:// references. A minimal crawler:

import re
from urllib.parse import urljoin, urlparse
from collections import deque

import requests

START = "http://example.com/"
SAME_HOST = urlparse(START).netloc

# Ignore URLs that are fine as http:// (rare) or not resources at all.
IGNORE = re.compile(r"^(mailto:|tel:|javascript:|#)")

seen = set()
queue = deque([START])
findings = []

session = requests.Session()
session.headers["User-Agent"] = "mixed-content-audit/1.0"

while queue:
    url = queue.popleft()
    if url in seen or urlparse(url).netloc != SAME_HOST:
        continue
    seen.add(url)

    try:
        resp = session.get(url, timeout=10)
    except requests.RequestException:
        continue
    if "text/html" not in resp.headers.get("Content-Type", ""):
        continue

    for match in re.finditer(r'(?:src|href|action)\s*=\s*["\'](http://[^"\']+)', resp.text):
        findings.append((url, match.group(1)))

    for match in re.finditer(r'<a\s[^>]*href\s*=\s*["\']([^"\'#]+)', resp.text):
        link = urljoin(url, match.group(1))
        if not IGNORE.match(link):
            queue.append(link)

for page, asset in findings:
    print(f"{page} -> {asset}")
print(f"\n{len(findings)} http:// references across {len(seen)} pages")

Anything it prints is a candidate for breakage. The fix is mechanical: rewrite to https://, or better, to protocol-relative or root-relative URLs (//cdn.example.com/lib.js or /assets/lib.js) so the scheme follows the page.

Common offenders

Hard-coded CDN links in templates, often copied from documentation years ago.
Markdown content with embedded images using http:// URLs.
Analytics or font snippets pasted from a provider dashboard that predates HTTPS-by-default.
srcset and <picture> sources, which are easy to miss because they are not plain src attributes.
CSS url() references to background images or fonts.

The crawler above only covers HTML attributes. For a thorough pass, also fetch your CSS files and search them for http://, and check any client-side JavaScript that builds URLs at runtime. A Content-Security-Policy header with upgrade-insecure-requests is a useful safety net, but it is a mitigation, not a fix. Do the rewrite first.

When provisioning actually fails

If more than a couple of hours pass and curl -I https://example.com still fails the handshake, the pipeline is stuck. Work through these in order.

Verify the CNAME value exactly

For a subdomain, the CNAME record must point to the Pages hostname, not to an IP, not to a proxy, and not to a URL with a path. The value is <user>.github.io. (or <org>.github.io.), with or without the trailing dot depending on your DNS provider. The apex domain is a special case: it cannot be a CNAME, so it needs A records to the documented GitHub Pages IPs, plus a CNAME file in the repository or the custom domain set in settings.

dig +short CNAME www.example.com
dig +short A example.com
dig +short example.com NS

Check the NS output too. If the domain is still delegated to your registrar's parking nameservers because you changed nameservers recently, the records you configured in the new DNS provider are not live yet. That alone will stall provisioning.

Wait out the TTL

If you changed DNS records within the last 48 hours, resolvers across the internet are still serving cached answers with the old target. GitHub's checker may have queried a resolver that still had the old record and concluded DNS was wrong. There is no way to force GitHub's check; it retries on its own schedule. Lowering the TTL before a planned migration (to 300 seconds, a day or two in advance) makes this window much shorter.

To see what different resolvers think, query a public resolver directly:

dig +short CNAME www.example.com @1.1.1.1
dig +short CNAME www.example.com @8.8.8.8

If these disagree, you are still inside the propagation window. Wait.

Remove and re-add the custom domain

If DNS is correct across resolvers, the served certificate is wrong or absent, and several hours have passed, remove the custom domain in the Pages settings, save, and add it back. This cancels the stuck request and starts a fresh one. It is safe, but it resets the clock, so only do it once you are confident DNS is right. Re-adding the domain while DNS is still wrong just reproduces the same stuck state.

Rule out a proxy in front of Pages

If the domain is behind Cloudflare or a similar proxy, the certificate GitHub issues may never be served, because the proxy terminates TLS with its own certificate. That is not necessarily a problem, but it changes where you look: the openssl output will show the proxy's issuer, and the "Enforce HTTPS" toggle in GitHub may stay disabled because GitHub cannot complete its own validation through the proxy. Either set the proxy to DNS-only during provisioning and re-enable it afterwards, or accept that HTTPS is handled at the proxy layer and configure enforcement there instead.

Check the CNAME file in the repo

If you set the custom domain through the repository's CNAME file rather than the settings UI, a mismatch between the file contents and the settings value can cause the domain to flap between configured states. Pick one method and stick to it; the settings UI writes the CNAME file for you.

A short checklist

Before enforcing HTTPS after a domain switch:

Confirm DNS resolves correctly from multiple public resolvers.
Wait for the certificate; verify it with openssl s_client and check the SAN list covers every hostname.
Crawl the site for http:// asset references and rewrite them.
Tick Enforce HTTPS once the checkbox unlocks.
Verify the redirect with curl -I http://example.com and re-run the crawler against the HTTPS origin to confirm nothing regressed.

The greyed-out checkbox is normal. The failures that matter are the ones where it stays greyed out for hours, and those almost always trace back to DNS.

More notes like this ship every week on this site.

Daily Picks

The following pairs are selected from the multi-timeframe trend scanner (Gate.io futures) and are for technical-analysis study only — not investment advice.
Data updated: 2026-10-25 08:56:33

Long

Pair	Signal	Price	Take Profit	Stop Loss	R/R
AXIS	短平快做多	$0.2484	$0.2559	$0.2422	1:1.2
ORDER	短平快做多	$0.0566	$0.0583	$0.0552	1:1.2

2 picks selected. Scanner runs every 15 minutes.

Why You Cannot Put a CNAME on the Apex Domain

mattleeee — Sat, 24 Oct 2026 01:01:45 +0000

The zone apex is not like other names

The apex of a zone — the bare domain itself, hkcode.dpdns.org in this case — is where the DNS machinery for the entire zone lives. It must carry an SOA record and a set of NS records. That single fact is the reason you cannot point the apex at a CNAME, and it is not a limitation of GitHub Pages or of any registrar. It is baked into the DNS standard.

When we set up GitHub Pages for a custom domain, GitHub's documentation asks for a CNAME record. The natural reading is "point the domain at username.github.io." For the apex, that request cannot be satisfied literally. The site ends up at www.hkcode.dpdns.org, and the bare name is handled by a redirect or a set of A/AAAA records. Here is why, and how to wire it up without creating a resolver loop or a broken zone.

Why a CNAME cannot live at the apex

The problem: a CNAME claims the whole name

RFC 1034, section 3.6.2 is explicit: a CNAME record is an alias for a domain name, and that name must not have any other data. The text says that if a CNAME RR is present at a node, no other data should be present. RFC 2181, section 10.1 reinforces it: a CNAME record cannot coexist with any other record type at the same owner name, with a narrow exception for DNSSEC-related records (RRSIG, NSEC, and similar).

So the rule is: at a node, either a CNAME, or everything else. Not both.

Now look at the apex. Every zone needs:

An SOA record, which describes the zone's primary server, serial number, refresh/retry/expire timers, and negative-caching TTL.
At least one NS record set, listing the authoritative name servers.

Those records must exist at the apex by definition — the SOA names the zone origin, and the NS set tells the world who is authoritative for it. If you also tried to place a CNAME at the apex, you would be asking the zone to hold a CNAME and an SOA and NS records at the same owner name. That directly violates RFC 1034/2181.

What happens if a provider lets you try

Many authoritative servers will simply reject the zone file at load time with an error like CNAME and other data. Some permissive servers accept it, and the result is worse: resolvers receive an answer where the CNAME and the SOA/NS coexist. Behavior becomes undefined. Some resolvers will follow the CNAME and discard the SOA; some will treat the zone as lame. Recursive resolvers that cache the CNAME may return it for queries that should have hit the NS set. You end up with intermittent resolution failures, and the failure mode differs by resolver — the hardest kind of bug to diagnose.

There is also a subtler issue. A CNAME at the apex would alias the entire zone's identity to another name. Queries for the SOA at the apex would be answered with a CNAME, and the resolver would chase it to the target, which has no authority over your zone. The delegation chain breaks.

The MX and TXT trap

Even if the RFC allowed it, the apex usually carries more than SOA and NS. Email routing records (MX) and domain-verification or SPF/DKIM records (TXT) commonly live at the bare domain. A CNAME at the apex would make all of those illegal too. For hkcode.dpdns.org, any TXT record used to verify ownership with a third-party service would collide with the CNAME. This is why the "just CNAME the apex" idea fails beyond the pure DNS mechanics: real zones have real records at the apex.

What "flattening" providers actually do

Several DNS providers (Cloudflare, DNSimple, Route 53 with alias records, and others) offer a record type called ALIAS, ANAME, or a "CNAME flattening" feature. This is the workaround people reach for, but it is not a CNAME at the apex. It is a server-side behavior.

Here is the mechanism. You create an ALIAS/ANAME record at the apex pointing to a target name. The provider's authoritative servers do not publish a CNAME. Instead, when a query for the apex arrives, the authoritative server:

Resolves the target name itself (or uses its cached answer).
Returns the resulting A/AAAA records directly, as if they were A/AAAA records at the apex.

The zone on disk still has SOA and NS at the apex; the ALIAS is a provider-specific synthetic record. To the outside world, the apex answers with address records, not a CNAME, so the RFC constraint is respected. The provider has effectively flattened the indirection.

This works, but it is not portable. The ALIAS record type is not standardized in the way A and CNAME are; its semantics differ between providers. If you export the zone and move to a provider that does not support it, the record either disappears or is written out as a literal CNAME, which then breaks the zone. Also, the target's TTL becomes the provider's resolution TTL, not the original CNAME TTL, so you lose some control over caching.

For GitHub Pages specifically, the target username.github.io resolves to GitHub's CDN addresses. Flattening works, but you are pinning the apex to whatever those addresses are at query time. If GitHub rotates them, your provider's cached resolution handles it, but you are trusting the provider's resolver behavior.

How GitHub Pages handles www versus apex

GitHub Pages has a specific model for custom domains:

www subdomain: GitHub recommends a CNAME record pointing www.hkcode.dpdns.org to hkcode.github.io (or username.github.io). This is the canonical setup. GitHub's servers see the Host: www.hkcode.dpdns.org header, match it against the configured custom domain, and serve the site.
Apex domain: GitHub documents A records (and AAAA records for IPv6) pointing the apex at GitHub's published IP addresses. These are static addresses that GitHub maintains. The apex is not a CNAME; it is a set of address records.

When you configure a custom domain in the repository settings, GitHub stores it and serves requests whose Host header matches. If you set the custom domain to www.hkcode.dpdns.org, requests to the apex hkcode.dpdns.org will not match unless GitHub also handles the redirect.

GitHub Pages does provide an HTTP-level redirect from the apex to the www host when the custom domain is configured as the www name and the apex resolves to GitHub. The redirect is a 301 served by GitHub's front end, not a DNS-level redirect. For that to work, the apex must still resolve to GitHub's servers — via A/AAAA records — so the request reaches GitHub and gets redirected. If the apex has no records at all, the browser never reaches GitHub and there is no redirect.

So the practical split is:

DNS for www → CNAME to username.github.io.
DNS for apex → A/AAAA records to GitHub's addresses (or a provider redirect).
GitHub config → custom domain set to the www name; GitHub 301s the bare name to www.

The practical setup

Step 1: CNAME the www subdomain

At your DNS provider, create:

www.hkcode.dpdns.org.  3600  IN  CNAME  hkcode.github.io.

No other records at www. If you previously had an A record there, remove it. A CNAME and an A record at the same name is the same violation as at the apex, just at a different node.

Step 2: Handle the apex

Two options, depending on your provider.

Option A — GitHub's A/AAAA records. Create A records (and AAAA if you want IPv6) at the apex pointing to GitHub's published addresses. These are documented by GitHub and change rarely, but check them before you paste. The apex then answers with addresses, and GitHub's front end 301s to www.

Option B — provider redirect. If your provider offers an HTTP redirect service for the apex (some registrars and DNS hosts do), point the apex at that redirect and send it to https://www.hkcode.dpdns.org. This avoids hardcoding GitHub's IPs. The tradeoff is that the redirect is handled by the provider, so you depend on their uptime and their support for HTTPS on the redirect.

Do not use an ALIAS/ANAME at the apex unless you have verified the provider's implementation and are comfortable with the non-portability. It works, but it is a provider feature, not a DNS standard.

Step 3: Verify with code

Do not trust the control panel. Query the records directly and check the invariants. A small script using dnspython catches the common mistakes: a stray CNAME at the apex, a missing SOA, or a CNAME and A record coexisting at www.

import dns.resolver
import dns.exception

APEX = "hkcode.dpdns.org"
WWW = f"www.{APEX}"

def query(name, rtype):
    try:
        answer = dns.resolver.resolve(name, rtype, lifetime=5)
        return [r.to_text() for r in answer]
    except dns.resolver.NoAnswer:
        return []
    except dns.resolver.NXDOMAIN:
        return ["NXDOMAIN"]
    except dns.exception.Timeout:
        return ["TIMEOUT"]

def check_zone(apex, www):
    problems = []

    # The apex must have SOA and NS. A CNAME here is illegal.
    apex_cname = query(apex, "CNAME")
    if apex_cname and apex_cname != ["NXDOMAIN"]:
        problems.append(f"apex has a CNAME: {apex_cname}")

    if not query(apex, "SOA"):
        problems.append("apex has no SOA")
    if not query(apex, "NS"):
        problems.append("apex has no NS")

    # The www node should be a CNAME and nothing else.
    www_cname = query(www, "CNAME")
    www_a = query(www, "A")
    if not www_cname:
        problems.append("www has no CNAME")
    if www_a and www_cname:
        problems.append("www has both CNAME and A records")

    return problems

if __name__ == "__main__":
    issues = check_zone(APEX, WWW)
    if issues:
        for issue in issues:
            print("FAIL:", issue)
    else:
        print("zone looks consistent")

Run it against the authoritative servers, not just your local resolver, so you see what the world sees. You can set the resolver explicitly:

import dns.resolver

resolver = dns.resolver.Resolver(configure=False)
# Use the provider's authoritative nameservers, from the NS set.
resolver.nameservers = ["ns1.example.net", "ns2.example.net"]
resolver.lifetime = 5

Step 4: Check the HTTP redirect

Once DNS is correct, confirm the redirect behavior with a request that does not follow redirects:

import urllib.request
import urllib.error

def check_redirect(url):
    req = urllib.request.Request(url, method="HEAD")
    opener = urllib.request.build_opener(urllib.request.HTTPRedirectHandler)
    try:
        # Prevent automatic following so we can inspect the 301.
        class NoRedirect(urllib.request.HTTPRedirectHandler):
            def redirect_request(self, *args, **kwargs):
                return None
        opener = urllib.request.build_opener(NoRedirect)
        resp = opener.open(req, timeout=10)
        print(url, "->", resp.status)
    except urllib.error.HTTPError as e:
        location = e.headers.get("Location")
        print(url, "->", e.code, "Location:", location)
    except Exception as e:
        print(url, "-> error:", e)

check_redirect("http://hkcode.dpdns.org/")
check_redirect("https://hkcode.dpdns.org/")

You want a 301 (or 308) with Location: https://www.hkcode.dpdns.org/. If you get a connection error, the apex has no address records and the redirect never happens. If you get a 200, GitHub is serving the apex directly — check whether the custom domain in the repo settings is set to the bare name, which changes the canonical host.

Common failure modes

CNAME at the apex "because the provider allowed it." The zone loads, but some resolvers fail on the SOA or NS query. Symptom: intermittent SERVFAIL from specific networks.
CNAME at www plus an A record "for safety." Same violation, smaller blast radius. Remove the A record.
Apex A records but no www CNAME. The site works at the bare name but not at www, or vice versa, and the canonical URL drifts.
Provider redirect without HTTPS. The apex redirects over HTTP only, and browsers block or warn. Make sure the redirect target uses https://.
Stale TTLs during migration. Lower the TTL on the old records to 300 seconds a day before the change, then raise it after the switch settles.

Summary

A CNAME at the apex is not a configuration choice; it is a contradiction of the zone's required SOA and NS records under RFC 1034 and 2181. ALIAS/ANAME records are a provider-side flattening trick that returns address records instead of a CNAME, and they work but do not travel. GitHub Pages expects a CNAME on www and either A/AAAA records or a provider redirect on the apex, with an HTTP 301 from the bare name to www. Set the CNAME on the subdomain, give the apex address records or a redirect, and verify both the DNS invariants and the redirect with a script rather than the control panel.

More notes like this ship every week on this site.

Daily Picks

The following pairs are selected from the multi-timeframe trend scanner (Gate.io futures) and are for technical-analysis study only — not investment advice.
Data updated: 2026-10-23 08:46:52

Long

Pair	Signal	Price	Take Profit	Stop Loss	R/R
BTW	高质量标的做多	$1.8281	$1.9561	$1.7641	1:2.0

1 picks selected. Scanner runs every 15 minutes.

Serving a Bilingual Static Site on GitHub Pages

mattleeee — Sat, 24 Oct 2026 01:01:42 +0000

Why one repo, two sites

A Chinese blog already lived at the domain root: example.com/. The goal was an English sibling at example.com/en/ without a second repository, a second domain, or a second deploy pipeline. That constraint drives every decision below.

The layout that ended up working:

repo/
├── CNAME                 # single domain for both languages
├── published/            # Chinese article manifests (one JSON per post)
├── published_en/         # English article manifests
├── templates/            # shared design system, Jinja2
├── static/               # shared CSS, JS, fonts
├── build.py              # generator for the root site
├── build_en.py           # generator for /en
└── docs/                 # GitHub Pages output (root = Chinese, /en = English)
    ├── index.html
    ├── sitemap.xml
    ├── rss.xml
    └── en/
        ├── index.html
        ├── sitemap.xml
        └── rss.xml

GitHub Pages serves docs/ from the branch, so the directory tree is the URL tree. The Chinese site is the root; the English site is a subdirectory. One CNAME file, one certificate, one git push.

Separate manifests prevent cross-language idempotency bugs

The first version kept one manifest directory with a lang field per article. Generation was supposed to be idempotent: run build.py twice, get identical output. It wasn't. The bug came from how the generator picked slugs and resolved "previous/next" links.

With a single pool, the Chinese generator saw English articles as candidates for adjacency and for slug-collision checks. A Chinese post at 2024/03/foo and an English post at 2024/03/foo (same slug, different language) would race for the same output path. Whichever ran second silently overwrote the first. Worse, rerunning the build could change which one won if the manifest glob order shifted. Non-deterministic output from deterministic input is the worst class of build bug: it looks fine locally, then a post disappears in production a week later.

The fix is boring and total: two manifest directories, published/ and published_en/, and each generator reads only its own. No lang filter, no shared slug namespace, no cross-language adjacency. The English generator cannot see Chinese manifests, so it cannot be affected by them.

from pathlib import Path
import json

MANIFEST_DIRS = {
    "zh": Path("published"),
    "en": Path("published_en"),
}

def load_manifests(lang: str) -> list[dict]:
    """Load only this language's manifests, sorted newest first."""
    posts = []
    for path in MANIFEST_DIRS[lang].glob("*.json"):
        with path.open(encoding="utf-8") as f:
            post = json.load(f)
        post["_source"] = path.name
        posts.append(post)
    posts.sort(key=lambda p: p["date"], reverse=True)

    slugs = [p["slug"] for p in posts]
    if len(slugs) != len(set(slugs)):
        raise ValueError(f"duplicate slug in {lang} manifests")

    return posts

The duplicate-slug check now guards a single language, which is exactly the invariant you want: slugs must be unique within a site, not across sites, because the two sites live at different URL prefixes.

The generators share the design system but nothing else. build.py and build_en.py both import templates/ and static/, both write into docs/, but each owns its manifest set and its own URL prefix. If the English site needs a different layout tweak later, it can diverge without touching the Chinese build.

One prefix constant keeps links consistent

Every internal link needs two pieces: the site root ("" for Chinese, "en" for English) and the article path. Hardcoding either one is how you end up with an English page linking to a Chinese article, or a link that works locally (/en/...) but 404s under a project-page base path.

The fix is a single constant per build plus a helper that joins them. Nothing else in the codebase constructs internal URLs by string concatenation.

# In build.py
SITE_PREFIX = ""          # Chinese site at the domain root
LANG = "zh"

# In build_en.py
SITE_PREFIX = "en"        # English site under /en
LANG = "en"

Both files import one shared module for URL construction:

# urls.py
def site_url(prefix: str, path: str = "") -> str:
    """Absolute-path URL for an internal page.

    site_url("", "about/")      -> "/about/"
    site_url("en", "about/")    -> "/en/about/"
    """
    parts = [p.strip("/") for p in (prefix, path) if p.strip("/")]
    return "/" + "/".join(parts) + ("/" if path.endswith("/") or not path else "")

Then the templates never see raw paths. The build passes a url function into the Jinja2 environment:

from jinja2 import Environment, FileSystemLoader
from urls import site_url

def make_env(prefix: str) -> Environment:
    env = Environment(loader=FileSystemLoader("templates"), autoescape=True)
    env.globals["url"] = lambda path="": site_url(prefix, path)
    return env

In templates, links become {{ url('posts/' + post.slug + '/') }} and assets become {{ url('static/main.css') }}. Because url closes over the prefix, the same template renders correct links for both languages. A Chinese page emits /posts/foo/; an English page emits /en/posts/foo/. There is no branch in the template and no opportunity to forget the prefix.

The asset trap

Shared assets under static/ are the one place where a prefix-relative link is wrong. If the English page links to /en/static/main.css but the file only exists at /static/main.css, you get a 404 on every English page. Two options: emit absolute asset URLs from the domain root, or mirror the assets under both prefixes. Mirroring doubles the files and the cache entries. The cleaner choice is a separate helper for assets that ignores the language prefix:

env.globals["asset"] = lambda path: "/static/" + path.lstrip("/")

Templates use {{ asset('main.css') }} for CSS/JS/fonts and {{ url(...) }} for anything that belongs to a language. One rule, applied consistently, and the asset 404s stop.

Cross-language navigation in both navs

Each nav needs a link to the equivalent page in the other language, not just the other homepage. On an article page, "English" should go to that article's English version if it exists; otherwise fall back to the English index.

That requires a mapping between the two manifest sets. The simplest reliable key is an explicit translation_key field present in both manifests for paired articles:

def build_translation_map(zh_posts, en_posts):
    """Map (lang, translation_key) -> slug for cross-language links."""
    mapping = {}
    for lang, posts in (("zh", zh_posts), ("en", en_posts)):
        for post in posts:
            key = post.get("translation_key")
            if key:
                mapping[(lang, key)] = post["slug"]
    return mapping

When rendering a Chinese post with translation_key: "gh-pages-bilingual", the nav looks up ("en", "gh-pages-bilingual"). If found, the link points to /en/posts/<slug>/; if not, it points to /en/. Same logic in reverse for the English site. Articles without a translation still get a working language switch, just at the section root.

def alt_lang_href(post, mapping, target_lang, target_prefix):
    key = post.get("translation_key")
    slug = mapping.get((target_lang, key)) if key else None
    if slug:
        return site_url(target_prefix, f"posts/{slug}/")
    return site_url(target_prefix)

The hreflang tags in <head> use the same function, so the machine-readable and human-readable links can't drift apart:

def hreflang_links(post, mapping):
    links = [("zh", site_url("", f"posts/{post['slug']}/") if post["lang"] == "zh"
              else site_url("", f"posts/{mapping[('zh', post['translation_key'])]}/"))]
    # ... same for en, plus x-default pointing at the Chinese page

In practice it's cleaner to compute both URLs once and pass a small dict into the template, but the principle holds: cross-language links are derived from the translation map, never typed by hand.

Per-language sitemaps and feeds

A single root sitemap.xml listing both languages is legal, but it makes Search Console reporting muddy and forces every URL to carry hreflang annotations in one file. Since the two sites already have separate manifests and separate generators, each generator writes its own sitemap and feed under its own prefix:

docs/sitemap.xml and docs/rss.xml — Chinese
docs/en/sitemap.xml and docs/en/rss.xml — English

from xml.sax.saxutils import escape
from datetime import datetime

def write_sitemap(posts, prefix: str, out_path: Path, base: str = "https://example.com"):
    urls = [f"{base}{site_url(prefix)}"]
    urls += [f"{base}{site_url(prefix, f'posts/{p[\"slug\"]}/')}" for p in posts]

    body = "\n".join(
        f"  <url><loc>{escape(u)}</loc></url>" for u in urls
    )
    xml = (
        '<?xml version="1.0" encoding="UTF-8"?>\n'
        '<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">\n'
        f"{body}\n</urlset>\n"
    )
    out_path.write_text(xml, encoding="utf-8")

The RSS feed follows the same shape, reading <link> values from site_url(prefix, ...). Because both use the same prefix constant, a feed can never contain a URL from the wrong language.

If you want the sitemaps discoverable without submitting each one manually, add a sitemap index at the root:

def write_sitemap_index(base: str, out_path: Path):
    entries = [
        f"{base}/sitemap.xml",
        f"{base}/en/sitemap.xml",
    ]
    body = "\n".join(
        f"  <sitemap><loc>{escape(u)}</loc></sitemap>" for u in entries
    )
    out_path.write_text(
        '<?xml version="1.0" encoding="UTF-8"?>\n'
        '<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">\n'
        f"{body}\n</sitemapindex>\n",
        encoding="utf-8",
    )

After the domain switch: submit the second sitemap

The domain move is where this layout earns its keep. When the site moved to a custom domain, the CNAME file changed once and both languages moved together. GitHub Pages issued the certificate for the new domain, and the root site's URLs redirected cleanly because they were always relative to the domain root.

The English site needed one extra step. Search Console properties are per-domain (or per-URL-prefix), and the new domain was verified fresh. The root sitemap.xml was submitted immediately, but /en/sitemap.xml sat unsubmitted for a week. English pages were indexed slowly and with stale titles. The fix was to open the new property's Sitemaps report and add https://example.com/en/sitemap.xml as a second sitemap. Within days the English pages were crawled at the same rate as the Chinese ones.

Two practical notes from that episode. First, submit the sitemap index if you wrote one; Search Console will follow it. Second, the hreflang annotations matter more after a domain change, because the crawler is re-evaluating which URLs are alternates of which. If your language switch links were broken by the move, the annotations would point at dead URLs and the English pages would be treated as orphans. Keeping the language switch derived from the translation map, as above, means a domain change doesn't touch link logic at all.

What the layout buys you

The whole design rests on three invariants: one manifest directory per language, one prefix constant per build, one URL helper used everywhere. Everything else — sitemaps, feeds, hreflang, cross-language nav — falls out of those. The build is idempotent because the two generators can't see each other's inputs. Links are consistent because no template constructs a URL by hand. And adding a third language later is a matter of a new manifest directory, a new prefix, and a new generator file, not a refactor.

More notes like this ship every week on this site.

Daily Picks

The following pairs are selected from the multi-timeframe trend scanner (Gate.io futures) and are for technical-analysis study only — not investment advice.
Data updated: 2026-10-24 08:47:12

Long

Pair	Signal	Price	Take Profit	Stop Loss	R/R
BIRB	弱做多	$0.1706	$0.1835	$0.1621	1:1.5

1 picks selected. Scanner runs every 15 minutes.

The CNAME Trailing Dot: One Character That Broke the Domain

mattleeee — Thu, 22 Oct 2026 01:01:29 +0000

The CNAME Trailing Dot: One Character That Broke the Domain

The migration was supposed to take ten minutes. New domain registered on a free-tier registrar, GitHub Pages already serving the site on mattlee.github.io, so all that remained was a CNAME record pointing www.example.dev at the GitHub Pages hostname. I typed mattlee.github.io into the panel's value field, saved, and waited. Then I waited some more.

dig returned NXDOMAIN for the apex and SERVFAIL for www. GitHub Pages reported no custom domain configured. The DNS panel insisted the record existed, and it did — just not the record anyone intended to create.

The fix was a single character. The value needed to be mattlee.github.io., with the trailing dot. Without it, the registrar stored a relative name, and the record resolved to mattlee.github.io.example.dev. — a hostname inside my own zone that points at nothing.

What the trailing dot actually means

DNS has two kinds of names: absolute (fully qualified) and relative. The distinction is encoded in the presentation format by a terminal dot. mattlee.github.io. is an absolute name — it is fully specified and the resolver stops there. mattlee.github.io is a relative name — it is a prefix to be completed by appending an origin.

This is not a quirk of one vendor. RFC 1034 and RFC 1035 define the master file format with this convention: a domain name that does not end in a dot is interpreted relative to the current origin. In a zone file, the origin is the zone itself. So inside the zone for example.dev, the relative name mattlee.github.io expands to mattlee.github.io.example.dev.

A zone file makes this visible:

$ORIGIN example.dev.
www     IN  CNAME  mattlee.github.io.      ; absolute: mattlee.github.io.
blog    IN  CNAME  mattlee.github.io       ; relative: mattlee.github.io.example.dev.

Both lines look plausible in a web form. Only the first one does what the author meant. The second creates a CNAME that points back into the same zone, to a name that has no A record. Resolvers chase it, find nothing, and return NXDOMAIN or SERVFAIL depending on how far the chain gets.

Why does the protocol work this way at all? Because relative names are useful. Zone files are full of them — www, mail, api — and appending the origin keeps configuration short and portable across zones. The trailing dot is the escape hatch that says "I am done, this is complete." The format is terse by design, and the dot is the only signal that distinguishes the two cases.

Why some panels add the dot and some don't

Registrar and DNS-provider UIs sit on a spectrum between two philosophies.

Zone-file faithful. Some panels treat the value field exactly as a zone file would. If you omit the dot, you get a relative name. These panels are technically correct and operationally hostile — most users typing a CNAME value mean an absolute hostname, because they copied it from a documentation page or a hosting dashboard where the trailing dot is usually omitted.

Lenient normalizers. Other panels assume every value is absolute and append the dot for you on save. Many also strip a dot you type in, then re-add it, so the stored record is always fully qualified. These panels hide the problem entirely, which is why the same record that works on one provider fails on another.

Hybrid. A third class stores what you type but normalizes on display, so the panel shows mattlee.github.io whether or not the dot is present. You cannot tell from the UI what was actually written to the zone. This is the worst case for debugging, because the panel's appearance actively misleads you.

The practical consequence: a CNAME value that works on your old registrar may break on your new one, even though both panels display the same string. The record is not the string you see. The record is what the provider serialized into the zone.

How to verify what was actually stored

Never trust the panel. Query the authoritative nameservers directly and read the ANSWER section.

dig www.example.dev CNAME +short

If the record is correct, you get:

mattlee.github.io.

Note the trailing dot in the output. dig always prints absolute names with the dot, because that is what the wire format contains — DNS messages carry fully qualified names, and the presentation layer adds the dot back on output.

If the record is broken, you get something like:

mattlee.github.io.example.dev.

That is the smoking gun. The resolver is telling you the stored name is relative and the origin was appended. If you see the zone name glued onto the end of your intended target, the value field is missing its dot.

You can also query the authoritative server directly, bypassing caching and any recursive resolver behavior:

dig @ns1.registrar.net www.example.dev CNAME +noall +answer

The +noall +answer flags trim the output to just the answer section, which is where the CNAME target lives. If the answer section is empty and the authority section shows NXDOMAIN, the name may not exist at all — check that the record saved and that the zone is active.

nslookup works too, though its output is noisier:

nslookup -type=CNAME www.example.dev ns1.registrar.net

Look at the canonical name = line. Same rule: a trailing dot means absolute, no trailing dot in a zone-file-style answer means relative.

For a full trace of the resolution chain, including where it fails:

dig www.example.dev +trace

+trace walks from the root down. If the delegation to your zone succeeds but the CNAME target never resolves, the failure is inside your zone — almost always the missing dot.

Automating the check

Once you know what to look for, the check is trivial to script. This is worth wiring into your deploy pipeline, because a broken CNAME is invisible until someone tries to load the site.

import subprocess
import sys


def get_cname(name: str, server: str | None = None) -> str | None:
    """Return the CNAME target for `name`, or None if no CNAME exists."""
    cmd = ["dig", "+short", "CNAME", name]
    if server:
        cmd.insert(1, f"@{server}")
    result = subprocess.run(cmd, capture_output=True, text=True, timeout=10)
    target = result.stdout.strip()
    return target or None


def check_cname(name: str, expected: str, server: str | None = None) -> bool:
    """
    Verify that `name` is a CNAME to `expected`.

    `expected` should be given without a trailing dot; the comparison
    normalizes both sides so the check is independent of presentation.
    """
    target = get_cname(name, server)
    if target is None:
        print(f"FAIL: no CNAME record for {name}")
        return False

    normalized = target.rstrip(".")
    if normalized != expected.rstrip("."):
        print(f"FAIL: {name} -> {target} (expected {expected}.)")
        print("      If the target ends in your zone name, the value is")
        print("      relative and is missing its trailing dot.")
        return False

    print(f"OK: {name} -> {target}")
    return True


if __name__ == "__main__":
    ok = check_cname("www.example.dev", "mattlee.github.io")
    sys.exit(0 if ok else 1)

The rstrip(".") normalization matters: dig +short prints the absolute form with a dot, but you will often pass the expected value without one, as copied from a dashboard. Comparing the normalized forms avoids false alarms while still catching the real failure — a target that has your zone name appended.

If you want to detect the relative-name bug specifically, check whether the target ends with the zone:

def looks_relative(target: str, zone: str) -> bool:
    """Heuristic: a CNAME target that ends in the zone is probably relative."""
    return target.rstrip(".").endswith(zone.rstrip("."))

This is a heuristic, not a proof — a legitimate CNAME can point into your own zone, and internal service records often do. But for a public web CNAME pointing at an external host, a target that ends in your own zone name is a near-certain sign of a missing dot.

The general lesson

DNS tools are literal and unapologetic. There is no fuzzy matching, no "did you mean," no warning when you save a relative name where you meant an absolute one. The zone is a text format with a precise grammar, and the provider either follows it or silently normalizes it for you. When it follows it, you get exactly what you typed, including the consequences.

This bites in predictable places:

CNAME and MX values are the most common victims, because they are the record types where you type a hostname rather than an IP address.
Copy-paste from documentation usually drops the trailing dot, because docs are written for humans and the dot is noise to most readers.
Provider migration surfaces latent bugs, because the new panel may interpret the value differently from the old one.
TXT records are not affected the same way — the content is an opaque string, not a domain name — but SPF and DKIM records that embed domain names inside the string can carry the same confusion.

The habit that prevents all of it: when a record value is a hostname, type the trailing dot. It is harmless on panels that normalize, and load-bearing on panels that don't. Then verify with dig and read the ANSWER section, because the panel is a UI and the zone is the truth.

The domain came up the moment the dot was added. The record had been "correct" in the panel for an hour. It had never been correct in the zone.

More notes like this ship every week on this site.

Daily Picks

The following pairs are selected from the multi-timeframe trend scanner (Gate.io futures) and are for technical-analysis study only — not investment advice.
Data updated: 2026-10-22 08:46:44

Short

Pair	Signal	Price	Take Profit	Stop Loss	R/R
ARIA	短平快做空	$0.0655	$0.0635	$0.0671	1:1.2

1 picks selected. Scanner runs every 15 minutes.

Free Domain Options in 2026: From eu.org to DigitalPlat dpdns.org

mattleeee — Wed, 21 Oct 2026 01:01:48 +0000

The 2026 Free Domain Landscape

If you run static sites for side projects, a domain is the one recurring cost you can usually avoid. The catch is that free domain providers are unstable in a way paid registries are not. Programs open, close, change terms, or quietly stop approving applications. Planning around them means treating the domain as a replaceable pointer, not as permanent infrastructure.

The 2026 situation, based on what actually happened over the past couple of years:

eu.org still works and is still free, but manual review can take months. It is fine for a project you are just starting; it is not a tool for a site that needs to be live this week.
DigitalPlat's .us.kg and .xx.kg paused new registrations after abuse problems. Existing domains kept resolving, but you could not onboard.
.qzz.io, also run under the DigitalPlat umbrella, moved to paid-only.
dpdns.org stayed open for new registrations and became the default choice for people who needed a free subdomain quickly.

The pattern is obvious: subdomain programs are cheap to launch and expensive to police. Expect any free suffix to have a finite lifespan. The engineering response is to make migration cheap, which is what most of this post is about.

Why dpdns.org Became the Default

dpdns.org is a subdomain service, not a registrar. You get something like yourname.dpdns.org, and the provider runs the authoritative nameservers for the zone. You control the records beneath your label.

What makes it usable for static hosting:

New registrations are open, so you can get a name in minutes rather than months.
You can delegate to external nameservers if you want, which matters if you already run DNS elsewhere.
It works with GitHub Pages, Cloudflare Pages, Netlify, and Vercel without special handling, because those platforms only care about a CNAME pointing at their edge.
HTTPS works through the hosting platform's certificate automation, so you do not need to manage TLS for the subdomain yourself.

The trade-offs are the same as any free subdomain. You do not own the name. The zone owner can change rules. There is no EPP transfer, so you cannot move the domain to another registrar—only the records inside it. And whois is essentially meaningless: the registrant is the provider, and your identity lives only in their control panel.

That is acceptable for a portfolio, a documentation mirror, or a demo site. It is not acceptable for anything where losing the hostname breaks users, email, or payments.

The DigitalPlat Registration Flow

The flow is short, but the DNS step is where people get stuck.

1. Register the label

Sign in to the DigitalPlat panel, pick the suffix (dpdns.org), and enter the label you want. Registration is effectively first-come-first-served, so short labels go fast. Once accepted, the subdomain appears in your dashboard.

At this point the domain exists but resolves to nothing useful. You have to point it somewhere.

2. Choose between provider DNS and external nameservers

Two paths:

Use DigitalPlat's nameservers. You add A, AAAA, or CNAME records in their panel. Simplest option, and fine for a single static site.
Delegate to external nameservers. You set NS records for your subdomain to, say, Cloudflare's nameservers, then manage everything there. More control, and it makes the next migration easier because your records live in a zone you can export.

For static sites, the second path is worth the extra ten minutes. Keeping records in a zone you control means that when dpdns.org changes terms, you only need to repoint the NS delegation—your A records, CNAMEs, and TXT records stay put.

3. Point the records at your host

For GitHub Pages, the pattern is a CNAME from your subdomain to <user>.github.io, plus the custom domain set in the repository settings. For Cloudflare Pages or Netlify, it is a CNAME to the project's assigned hostname. Either way, the record is a single CNAME and the platform handles the certificate.

If you are delegating to external nameservers, you create that CNAME in the external zone, not in the DigitalPlat panel. Doing it in both places causes conflicting answers and confusing propagation behavior.

4. Verify resolution before you announce anything

Do not trust the panel's "active" status. Query the authoritative servers directly:

import dns.resolver

def check_subdomain(name: str, record_type: str = "CNAME") -> None:
    """Query the authoritative chain for a subdomain and print what it returns."""
    resolver = dns.resolver.Resolver(configure=False)
    resolver.nameservers = ["1.1.1.1", "8.8.8.8"]
    resolver.lifetime = 5.0

    try:
        answers = resolver.resolve(name, record_type)
    except dns.resolver.NoAnswer:
        print(f"{name} {record_type}: no answer")
        return
    except dns.resolver.NXDOMAIN:
        print(f"{name}: NXDOMAIN")
        return

    for rdata in answers:
        print(f"{name} {record_type} -> {rdata}")

check_subdomain("yourname.dpdns.org", "CNAME")

If you delegated NS records, also confirm the delegation itself took effect:

import dns.resolver

def check_delegation(name: str) -> None:
    """Print the NS records that the parent zone actually serves for a subdomain."""
    resolver = dns.resolver.Resolver(configure=False)
    resolver.nameservers = ["1.1.1.1"]
    try:
        for ns in resolver.resolve(name, "NS"):
            print(f"NS: {ns.target}")
    except dns.resolver.NoAnswer:
        print(f"{name}: no NS records served by parent")

check_delegation("yourname.dpdns.org")

If the NS query returns nothing, the delegation has not propagated or was never saved. Fix that before touching anything downstream—a CNAME in a zone nobody is querying will never resolve.

What You Give Up

Be explicit about the costs, because they shape how you should use these domains.

Stability risk. The zone owner can change policy at any time. .qzz.io going paid-only is the concrete example; .us.kg pausing registrations is another. Your domain is not a contract, it is a courtesy.

No whois control. The registrant record belongs to the provider. You cannot prove ownership through whois, which rules out some verification workflows and makes disputes unwinnable.

No transfer. There is no EPP code, so you cannot move the name to another registrar. Migration means registering a new name and repointing traffic.

Certificate and email constraints. Let's Encrypt issuance works fine for the web hostname, but you should not build email on a free subdomain. SPF, DKIM, and DMARC on a name you do not own is a deliverability problem waiting to happen.

Search and reputation. Shared suffixes carry the reputation of everyone else on them. A subdomain on a suffix with heavy abuse gets treated worse by filters and some mail systems.

None of this is a reason to avoid free domains for hobby work. It is a reason to keep the domain layer thin, documented, and easy to swap.

Migration Checklist for Near-Zero Downtime

The goal is to move from old-free-domain to new-free-domain without a visible gap for visitors. The trick is to run both names in parallel for a while and cut over only when the new one is fully verified.

1. Register the new name and set up records first

Register the new subdomain and configure its records to point at the same origin as the old one. Your host should accept both hostnames—GitHub Pages and Cloudflare Pages both let you attach multiple custom domains to one project, which is what makes this painless.

2. Verify the new name end to end

Check DNS, then check HTTPS and content:

import ssl
import urllib.request

def verify_site(url: str, expected_marker: str) -> bool:
    """Fetch a URL with certificate validation and confirm a marker string is present."""
    ctx = ssl.create_default_context()
    try:
        with urllib.request.urlopen(url, context=ctx, timeout=10) as resp:
            body = resp.read().decode("utf-8", errors="replace")
            status = resp.status
    except Exception as exc:
        print(f"{url}: FAILED ({exc})")
        return False

    ok = status == 200 and expected_marker in body
    print(f"{url}: status={status} marker_found={expected_marker in body}")
    return ok

verify_site("https://newname.dpdns.org", "your-site-title")

Do not skip this. A CNAME that resolves but serves a certificate error or a 404 from the wrong project is worse than a domain that has not propagated yet.

3. Audit the old domain for hardcoded references

Anything that embeds the old hostname needs updating: canonical tags, Open Graph URLs, sitemap entries, RSS feeds, API base URLs in frontend code, and webhook callbacks. A quick scan over a built static site:

from pathlib import Path

OLD = "oldname.dpdns.org"
NEW = "newname.dpdns.org"

def find_references(root: str) -> None:
    """Scan built output for the old hostname and report file and line."""
    for path in Path(root).rglob("*"):
        if not path.is_file() or path.suffix not in {".html", ".xml", ".json", ".js", ".css"}:
            continue
        try:
            text = path.read_text(encoding="utf-8", errors="ignore")
        except OSError:
            continue
        for lineno, line in enumerate(text.splitlines(), start=1):
            if OLD in line:
                print(f"{path}:{lineno}: {line.strip()[:120]}")

find_references("public")

Fix these in source, rebuild, and redeploy. If you rely on canonical URLs for search indexing, update them before the cutover so the new domain is the one being indexed.

4. Keep the old domain alive as a redirect

Leave the old records in place and configure your host to redirect the old hostname to the new one with a 301. This preserves inbound links and search signals. On GitHub Pages you get this by keeping both custom domains attached and adding a redirect at the edge; on Cloudflare you can do it with a redirect rule.

5. Cut over traffic sources

Update anything you control that points at the old name: bookmarks in docs, links in README files, profile URLs, and any CI jobs that curl the site for smoke tests. This is the slow part, and it is why step 4 matters—the redirect covers the tail.

6. Retire the old name later, not immediately

Keep the redirect for at least a few months. Free domains can be reclaimed if the provider changes policy, so do not assume the redirect will live forever. The point is to make the new name fully self-sufficient before the old one disappears.

Keeping the Next Migration Cheap

Two habits make all of this much easier:

Put records in a zone you control. Delegate NS from the free subdomain to a DNS provider you can export from. When the suffix changes, you repoint one delegation instead of rebuilding every record.
Never hardcode the hostname. Use a single config value for the site origin and derive canonical URLs, sitemaps, and feeds from it. The scan script above exists because this rule gets broken; a build-time constant means it only has to be changed once.

Free domains are a reasonable choice for hobby infrastructure as long as you treat them as a pointer you can move. The providers will keep changing; your build process should not care.

More notes like this ship every week on this site.

Daily Picks

The following pairs are selected from the multi-timeframe trend scanner (Gate.io futures) and are for technical-analysis study only — not investment advice.
Data updated: 2026-10-21 08:46:52

Short

Pair	Signal	Price	Take Profit	Stop Loss	R/R
JCT	热点跟涨	$0.0047	$0.0043	$0.0048	1:1.9

1 picks selected. Scanner runs every 15 minutes.

DEV Community: mattleeee

Adaptive Stop-Loss Thresholds Instead of Hardcoded Percentages

What each clause is protecting

Computing strategy_stop_pct per instrument

Applying it to open positions

Backtesting the floor's effect on surviving trades

When a hard floor must override the adaptive value

The design principle

Partial Take-Profit: Bank Half at 3%, Let the Rest Run

The Problem With a Single Take-Profit

Position State

The Candle Loop and Check Order

The Partial Fill Itself

Edge Cases That Bite

How Partial TP Changes Expectancy

A Cleaner Expectancy Formula

What Actually Matters

Daily Picks

Long

Circuit Breakers for Strategies: Pausing After Three Consecutive Losses

Why Consecutive Losses Need Their Own Guard

The State Machine

Where the Counter Lives

The Breaker Logic

Wiring It Into the Execution Loop

Notifications That Are Actually Useful

Consecutive Losses vs. Drawdown Limits

The Psychology Part

Adapting It

Daily Picks

Long

Short

The Resonance Scanner: Seven Filters and the One That Mattered

The Resonance Scanner: Seven Filters and the One That Mattered

Anatomy of the Gate

Instrumenting the Filters

Aggregating the Traces

The Structural Problem

The Design Tradeoff

What to Keep

Daily Picks

Long

A/B Testing Trading Rules: The 52 Rejected Trades That Changed My Mind

The Question a Filter Should Answer

Why the Headline Curve Can't Settle It

The diff is the unit of analysis

Building the Two Trade Lists

Statistics for the Incremental Set

Interpret the shape, not just the sign

Guarding Against One Lucky Trade

Per-trade contribution

Per-symbol breakdown

Sample size and stability

What the Method Actually Decides

Backtesting the Pullback Filter: When a Safety Check Kills Profit

The Setup: One Strategy, Two Filters

Why the Comparison Is Easy to Get Wrong

Counting the Incremental Trades

The Numbers

Why "Safer" Is Worse Here

Practical Rules for Filter A/B Tests

Closing

Daily Picks

Short

Three-Layer Validation Before You Trust a Backtest

Why One Good Backtest Means Nothing

Layer 1: Parameter Sweep

The question it answers

The metrics

The thresholds

The code

Layer 2: Monthly Consistency

The question it answers

The metrics

The thresholds

The code

Layer 3: Robustness Ablation

The question it answers

The metrics

The thresholds

Computing `strategy_stop_pct` per instrument