Distributions and heavy tails

How to choose xmin for a power-law fit

Picking where a power-law tail begins (xmin) is a bias–variance trade-off. Set xmin too low and non-power-law bulk structure pulls the fitted exponent downward; set it too high and you discard most of the dataset until sampling noise dominates. A Kolmogorov–Smirnov (KS) distance scan finds the candidate that best aligns the empirical tail CDF with the fitted model, which you then audit for flat plateaus, right-edge traps and sensitivity before trusting the exponent.

Updated · 5 min read

On this page
  1. The bias–variance trade-off of a lower cutoff
  2. How the KS-distance scan picks best xmin
  3. Reading D(xmin) curves: plateaus, spikes and right-edge traps
  4. Upper cutoffs and sensitivity checks before trusting best xmin
  5. Xmin diagnostics in Autoplot
  6. Frequently asked questions

The bias–variance trade-off of a lower cutoff

Empirical heavy-tailed quantities rarely follow a power law down to their smallest values. Threshold effects, measurement resolution, or a lognormal-like core typically flatten or round the distribution below some scale, while the power law describes only the upper tail x ≥ xmin.

Cutoff mistakeEffect on αHow it appears in diagnostics
xmin set too lowBiased downward (or mixed across regimes)High KS distance D; visible curvature between data and fit at small x
xmin inside scaling plateauStable within parameter standard errorBroad shallow basin in D(xmin) with n_tail still well above ~100
xmin set too highHigh variance, sensitive to individual extreme valuesSudden drops or jumps in D alongside small n_tail

This guide focuses on cutoff selection diagnostics and failure modes. For how the chosen cutoff plugs into a binned PDF vs unbinned CCDF workflow and what other evidence a power-law claim needs, see how to fit a power law to data.

How the KS-distance scan picks best xmin

Clauset, Shalizi and Newman's standard procedure scores every distinct observation c in the sorted sample as a trial cutoff:

  1. Keep the observations x ≥ c and compute the maximum-likelihood power-law exponent α̂(c).
  2. Evaluate the fitted cumulative distribution function P_c(x) over x ≥ c alongside the empirical CDF S(x).
  3. Record the Kolmogorov–Smirnov distance D(c) = max_{x ≥ c} |S(x) − P_c(x)| — the largest vertical gap between the two cumulative curves.
  4. Select best xmin as the candidate cutoff with the smallest D(c).

Even when you expect an exponential roll-off at large values, powerlaw's standard workflow chooses best xmin using the pure power-law KS sweep first, then compares the truncated model above that fixed cutoff. Comparing D directly across different parametric families inside the sweep can reward extra parameters unfairly rather than identifying where scaling begins.

Autoplot Xmin Diagnostic card named KS sweep with its Range & analysis section open, showing the Power Law and Truncated distribution modes and the Constrain xmin scan interval checkbox checked with log10 min 0.0 and log10 max 2.0, summary metrics best xmin 11.76, alpha 2.500, D 9.455e-03 and n_tail 3177, an inset chart of KS distance D against log10(xmin), and an Export xmin/D variables button
The Xmin Diagnostic card plots KS distance D across candidate cutoffs, marks the minimum, and lets you constrain the scan interval or export the full sweep arrays for deeper inspection.

Reading D(xmin) curves: plateaus, spikes and right-edge traps

Looking at the single best xmin number without inspecting the D(xmin) curve hides whether the cutoff is well localized or accidental.

Shape of D(xmin)What it indicatesPractical response
Broad flat valley spanning ~1 decadeTail matches power law across a clear range; small D shifts barely change qualityConsider picking near the lower edge of the flat valley if α is already stable, to retain larger n_tail
Sharp dip at high xmin with tiny n_tailSmall-sample noise makes an empirical step fit closely by accidentCap the upper search boundary (log₁₀ max) so candidates with too few points cannot win
Monotonic drop across the entire rangeTrue distribution likely curves continuously (e.g. lognormal) with no pure tail regimeCompare alternatives directly via power law vs lognormal instead of pushing xmin higher
Sawtooth steps or multiple local dipsRounded values, ties, or secondary regimesPlot the raw empirical CCDF first to inspect staircases or measurement resolution

Upper cutoffs and sensitivity checks before trusting best xmin

  1. Set a hard upper cutoff (xmax) before scanning if physical finite size, sensor saturation, or collection windows cap large values. Without xmax supplied ahead of time, the pure power-law KS scan sees the upper drop-off as a bad fit and pushes xmin artificially rightward.
  2. Plot α(xmin) and n_tail beside D(xmin). Export the sweep curve or refit at half and twice best xmin: if α drifts steadily as xmin grows, the minimum of D may just be trading one curved region for another.
  3. Check whether an exponential tail explains the same range better. Run a truncated comparison above best xmin; when truncation explains the upper bend cleanly, forcing a higher xmin on a pure power law wastes data.
  4. Do not confuse D with a p-value. A small D (such as 0.02) simply identifies the closest cutoff within the assumed power-law family; it does not test whether synthetic power-law samples produce deviations as large as yours.
Autoplot observed CCDF shown as blue points on log10 axes with a green power-law tail fit
Once xmin is verified, enter log10(xmin) on the Observed CCDF card. The empirical points stay uncropped from 1 down to 1/n while the fitted model covers x ≥ xmin.
In Autoplot

Xmin diagnostics in Autoplot

Autoplot's Xmin Diagnostic card (Run Analysis → Xmin Diagnostic, Plus) requires at least 50 finite positive observations and runs powerlaw.Fit locally through Python.

  • Power Law mode: sweeps candidate cutoffs, plots D vs log₁₀(xmin), marks best xmin, and reports best xmin (in natural units), alpha, D and n_tail.
  • Truncated mode: selects best xmin via the same power-law KS sweep, then fits power_law and truncated_power_law above that cutoff and runs distribution_compare('power_law', 'truncated_power_law'), reporting alpha_tr, alpha_pl, lambda, R and p.
  • Interval and upper-cutoff controls: toggle Constrain xmin scan interval (log₁₀ min, log₁₀ max) to avoid sparse high-tail artifacts, and in Truncated mode toggle Use hard upper cutoff (log₁₀ xmax) to pass xmax before the KS scan.
  • Export xmin/D variables: saves the candidate xmin and D arrays as reusable workspace variables so you can overlay sensitivity plots or inspect plateaus at full scale.

Note that UI fields on the card (log₁₀ min, log₁₀ max, log₁₀ xmax) take base-10 logarithms, whereas best xmin in Results is printed in natural units. The card does not scan D(xmin) under a lognormal model or run Clauset, Shalizi and Newman's Monte Carlo goodness-of-fit test; run those checks via Python or Autoplot's assistant when preparing a formal publication claim. See features and pricing.

Frequently asked questions

Frequently asked questions
Why can minimizing KS distance pick an unreasonably high xmin?When very few observations remain above a large trial cutoff, a short empirical tail has fewer chances to deviate strongly from the fitted line, or ties create low-D pockets. Constraining the upper scan limit keeps at least ~50–100 observations in the tail.
Should I constrain xmin before running the scan?Only with a clear reason: excluding noisy micro-values below instrument resolution or preventing tiny-sample spikes near the maximum. Inspect the unconstrained D(xmin) curve first so your constraints stay documented and honest.
Why does Autoplot's Truncated mode still find xmin using Power Law?Following the powerlaw library workflow, selecting xmin under a single-parameter power law isolates where scaling begins, and the nested likelihood-ratio test (R and p) then compares pure vs truncated power laws over that identical tail sample.

Sources

Autoplot behaviour comes from the app's Python engine (compute_xmin_vs_ks in math_logic.py) and XminDiagnosticCardEditor, checked on 14 November 2026, plus the features page. Clauset, Shalizi and Newman, Power-law distributions in empirical data, defines the KS-minimization cutoff estimator (Section 3.3). The powerlaw package implements Fit and xmin_fitting_results.

Try it on your own data.

Autoplot's Xmin Diagnostic card scans candidate cutoffs, plots KS distance D, and exports the sweep arrays as variables. Part of Plus (free for 30 days).

Compare power law vs lognormal →
Download free for Mac

Apple silicon · macOS 15 or later · Free plan · Plus free for 30 days

Send to my Mac

Autoplot runs on a Mac. Send the link to yourself, or open it in the Mac App Store.