Benford's Law in Exoplanet Orbital Parameters: Detection Bias Fingerprints in the First-Digit Distribution

Author: stepstep_labs

Abstract

Benford's Law predicts that the leading significant digit d of numbers drawn from many natural processes follows a logarithmic distribution: P(d) = log₁₀(1 + 1/d). We test this prediction against three physical parameters of 5,844 confirmed exoplanets cataloged in the NASA Exoplanet Archive through 2024: orbital period, planet mass (in Jupiter masses), and planet radius (in Jupiter radii). Using chi-square goodness-of-fit tests, Kullback–Leibler (KL) divergence, discovery-method stratification, second-digit analysis, and a 10,000-iteration permutation test, we find that planet mass conforms well to Benford's Law (χ² = 9.76, p = 0.28), while orbital period shows a statistically significant but modest deviation (χ² = 42.58, p = 1.05 × 10⁻⁶, KL = 0.0038), characterized by an excess of digits 3–5 and a deficit of digit 1. Planet radius departs dramatically from Benford's Law (χ² = 1706.49, p ≈ 0), with nearly 50% of first digits equal to 1, reflecting the bimodal size distribution dominated by sub-Neptune and Jupiter-sized planets measured in Jupiter radii. Stratification by discovery method reveals that transit-detected planets drive the orbital-period deviation (χ² = 36.84, p = 1.23 × 10⁻⁵), while radial velocity detections are marginally consistent with Benford's Law (χ² = 12.68, p = 0.12). The second-digit distribution of orbital periods conforms well to Benford's second-digit law (χ² = 6.35, p = 0.70). These results demonstrate that Benford analysis can serve as a lightweight diagnostic for detection-bias fingerprints in astronomical catalogs.

Keywords: Benford's Law, exoplanets, selection bias, orbital period, digit analysis, statistical methods

1. Introduction

1.1 Benford's Law

Benford's Law, also known as the first-digit law, describes the non-uniform distribution of leading significant digits in many real-world numerical datasets (Benford, 1938). The law states that the probability of the first significant digit d (for d = 1, 2, …, 9) is:

P(d) = log₁₀(1 + 1/d)

This yields a strongly decreasing distribution: digit 1 appears as the leading digit approximately 30.1% of the time, while digit 9 appears only about 4.6% of the time. Newcomb (1881) first observed this pattern in the wear of logarithm table pages, and Benford (1938) later demonstrated it across 20 diverse datasets including river areas, population counts, and physical constants.

The mathematical basis for Benford's Law rests on scale invariance and base invariance: data that span several orders of magnitude and arise from multiplicative processes tend to have logarithmically distributed significands (Hill, 1995). This theoretical grounding makes Benford's Law applicable to any dataset whose values are not artificially constrained to a narrow range.

1.2 Exoplanet Detection Methods and Selection Biases

The discovery and characterization of exoplanets has accelerated dramatically since the first confirmed detection around a main-sequence star in 1995 (Mayor & Queloz, 1995). As of late 2024, the NASA Exoplanet Archive contains over 5,800 confirmed planets with measured orbital periods. The two dominant detection methods are the transit method (photometric dimming as a planet crosses the stellar disk) and the radial velocity method (Doppler shifts in the stellar spectrum due to gravitational tugging).

Each method introduces characteristic selection biases. The transit method preferentially detects close-in planets with short orbital periods, since the geometric probability of transit alignment scales as R★/a (where a is the semi-major axis) and short-period planets produce more frequent, more easily detectable signals (Borucki et al., 2010; Winn, 2010). The radial velocity method is biased toward massive planets at intermediate orbital periods, since the radial velocity semi-amplitude scales as M_p sin(i) / P^(1/3) (Fischer et al., 2014). These biases shape the distributions of measured orbital periods, masses, and radii in ways that should leave detectable imprints on digit distributions.

1.3 Benford's Law in Astrophysics

Several studies have applied Benford's Law to astronomical datasets. Alexopoulos and Leontsinis (2014) examined distances and luminosities of galaxies, finding general conformity. Hair (2014) applied Benford analysis to pulsar rotation periods and found mixed conformity depending on the dataset. More recently, Shukla et al. (2017) tested Benford's Law on various astronomical catalogs. These studies establish a precedent for using digit analysis as a diagnostic tool in astrophysics.

1.4 Our Contribution

We apply Benford's Law analysis to the three principal physical parameters of confirmed exoplanets—orbital period, planet mass, and planet radius—as recorded in the NASA Exoplanet Archive. We go beyond simple digit counting by employing chi-square tests, KL divergence, discovery-method stratification, second-digit analysis, and permutation testing. Our central hypothesis is that deviations from Benford's Law encode information about observational selection biases, and that stratifying by detection method can isolate these bias fingerprints. To our knowledge, this is the first comprehensive Benford analysis of exoplanet parameters that explicitly ties deviations to detection-method-specific biases.

2. Methods

2.1 Data

We queried the NASA Exoplanet Archive Planetary Systems table (ps) via the Table Access Protocol (TAP) service, selecting the default parameter set (default_flag = 1) for each confirmed planet. The fields retrieved were: planet name (pl_name), discovery method (discoverymethod), orbital period in days (pl_orbper), planet mass in Jupiter masses (pl_bmassj), planet radius in Jupiter radii (pl_radj), and discovery year (disc_year). We restricted the sample to planets discovered through 2024, yielding 5,844 entries. After filtering for valid (non-null, positive) values, the working samples comprised 5,537 orbital periods, 2,792 planet masses, and 4,396 planet radii.

The discovery method breakdown for the orbital period sample was: Transit (4,376 planets, 79.0%), Radial Velocity (1,094 planets, 19.8%), and Other methods combined (67 planets, 1.2%). The "Other" category includes direct imaging, microlensing, transit timing variations, and additional techniques.

2.2 Benford's Law

For first-digit analysis, we extracted the leading significant digit d (1 through 9) from each parameter value. The expected Benford proportions are:

For the second-digit analysis, the expected probability that the second significant digit equals d₂ (0 through 9) is:

P(d₂) = Σ_{d₁=1}^{9} log₁₀(1 + 1/(10·d₁ + d₂))

This yields a flatter but still non-uniform distribution, ranging from approximately 11.97% for digit 0 to 8.50% for digit 9.

2.3 Chi-Square Goodness-of-Fit Test

We assessed conformity to Benford's Law using the Pearson chi-square statistic:

χ² = Σᵢ (Oᵢ − Eᵢ)² / Eᵢ

where Oᵢ is the observed count for digit i and Eᵢ = N · P(i) is the expected count under Benford's Law, with N the total number of observations. For the first-digit test, the degrees of freedom are df = 8 (nine digit categories minus one). For the second-digit test, df = 9 (ten digit categories minus one). P-values were computed from the chi-square survival function using the regularized incomplete gamma function.

2.4 KL Divergence

As a complementary measure of distributional distance, we computed the Kullback–Leibler divergence from the observed distribution Q to the Benford distribution P:

D_KL(Q ‖ P) = Σᵢ Q(i) · ln(Q(i) / P(i))

KL divergence is always non-negative and equals zero only when the observed distribution matches Benford's Law exactly. Unlike the chi-square statistic, KL divergence is not directly sensitive to sample size, making it useful for comparing the magnitude of deviations across subsamples of different sizes.

2.5 Discovery Method Stratification

To investigate whether specific detection methods drive observed deviations, we stratified the orbital period sample by discovery method: Transit, Radial Velocity, and Other. We repeated the chi-square test and KL divergence computation for each stratum independently. We also stratified mass and radius by discovery method for completeness.

2.6 Second-Digit Analysis

We performed a second-digit Benford analysis on orbital periods. The second-digit law provides a more stringent test of data naturalness, because the second-digit distribution is flatter (closer to uniform) and thus deviations are harder to detect, requiring a dataset with genuine underlying structure rather than superficial pattern matching.

2.7 Permutation Test

To validate the chi-square p-values without relying on asymptotic distributional assumptions, we conducted a permutation test. We generated 10,000 synthetic samples by drawing N = 5,537 random digits from the Benford first-digit distribution and computing the chi-square statistic for each sample. The permutation p-value is the fraction of synthetic chi-square values that equal or exceed the observed chi-square. The random seed was fixed at 42 for reproducibility.

3. Results

3.1 Overall First-Digit Distributions

Table 1 presents the observed first-digit distributions alongside Benford's Law expectations for all three parameters.

Table 1. First-digit frequency distributions compared to Benford's Law.

Table 2. Summary chi-square and KL divergence results for first-digit analysis.

Planet mass shows no statistically significant departure from Benford's Law (p = 0.28), with a KL divergence of only 0.0017. Orbital period departs significantly (p = 1.05 × 10⁻⁶), though the KL divergence of 0.0038 indicates that the deviation, while statistically robust, is modest in magnitude. Planet radius shows an extreme departure (χ² = 1706.49), with digit 1 accounting for 49.95% of first digits versus the Benford expectation of 30.10%, and a KL divergence of 0.2157—two orders of magnitude larger than the orbital period deviation.

3.2 Orbital Period Stratified by Discovery Method

Table 3 presents the chi-square and KL divergence results for orbital period, broken down by discovery method.

Table 3. First-digit analysis of orbital period stratified by discovery method.

Transit-detected planets drive the overall orbital-period deviation, with a highly significant chi-square of 36.84 (p = 1.23 × 10⁻⁵). The transit subsample shows an excess of digits 3 (14.42% vs. 12.49%) and 4 (11.27% vs. 9.69%), with a compensating deficit at digit 1 (28.22% vs. 30.10%) and digit 2 (16.61% vs. 17.61%).

The radial velocity subsample is marginally consistent with Benford's Law (χ² = 12.68, p = 0.12), though digit 5 is notably overrepresented (9.87% vs. 7.92%). The "Other" category shows no significant deviation (p = 0.48), but the small sample size (N = 67) limits statistical power.

3.3 Mass and Radius Stratified by Discovery Method

Planet mass conforms well to Benford's Law across both major detection methods individually: Transit (χ² = 4.97, p = 0.76, N = 1,341) and Radial Velocity (χ² = 5.16, p = 0.74, N = 1,093). The "Other" methods subsample for mass shows marginal deviation (χ² = 17.61, p = 0.024, N = 358), driven by an excess of digits 1 and 2 and a deficit of digit 3.

Planet radius deviates dramatically regardless of detection method. For transit-detected radii (N = 4,341), χ² = 1,680.34 (p ≈ 0), with 49.69% of first digits equal to 1. The Radial Velocity and Other subsamples are too small (N = 27 and N = 28 respectively) for robust inference but show similarly extreme concentration on digit 1 (62.96% and 78.57%, respectively).

3.4 Second-Digit Analysis

Table 4 presents the second-digit distribution for orbital periods.

Table 4. Second-digit distribution of orbital periods (N = 5,537).

The chi-square test yields χ² = 6.35 with df = 9, giving p = 0.70. The KL divergence is 0.0006. The second-digit distribution of orbital periods conforms well to Benford's second-digit law, indicating that whatever process generates the first-digit deviation does not extend to the second digit.

3.5 Permutation Test

The observed chi-square for orbital period first digits was 42.58. In 10,000 permutation iterations drawing digits from the Benford distribution, none of the synthetic chi-square values equaled or exceeded the observed value. The permutation p-value is therefore p < 0.0001, confirming the parametric chi-square result and ruling out concerns about asymptotic approximation failures.

4. Discussion

4.1 Interpreting the Orbital Period Deviation

The orbital period first-digit distribution deviates from Benford's Law with high statistical significance but small effect size. The chi-square of 42.58 is driven primarily by an excess of mid-range first digits (3, 4, and 5) and a relative deficit of low first digits (1 and 2). The KL divergence of 0.0038 places this deviation well below the threshold typically considered meaningful in fraud-detection applications (where KL values above 0.01–0.05 are flagged).

This pattern is consistent with known selection biases in the transit method, which dominates the sample. Transit surveys like Kepler preferentially detect planets with orbital periods of a few days to a few hundred days—a range where periods beginning with digits 3, 4, and 5 are well represented (e.g., periods of 3–5 days, 30–50 days, 300–500 days). The deficit of digit 1 may reflect the relative scarcity of very short-period planets (P < 2 days) and the difficulty of detecting very long-period planets (P > 1000 days) with transit photometry.

4.2 Transit vs. Radial Velocity

The stratification analysis reveals a clear contrast: transit-detected orbital periods show a significant Benford deviation (χ² = 36.84, p = 1.23 × 10⁻⁵), while radial-velocity-detected periods are consistent with Benford's Law at the conventional α = 0.05 level (χ² = 12.68, p = 0.12). This difference is interpretable in terms of the methods' respective selection functions.

The transit method has a steep geometric selection function: the probability of observing a transit falls as 1/a, compounded by the requirement of multiple transits within the observation baseline for confirmation. This creates a complex, non-multiplicative selection function that can distort the first-digit distribution away from Benford's Law. The radial velocity method, while also biased, has a smoother selection function that depends more gradually on period, mass, and stellar properties, preserving more of the underlying multiplicative structure that gives rise to Benford behavior.

An interesting detail is that the KL divergence for radial velocity (0.0058) is actually larger than for transit (0.0041), despite the lower chi-square. This reflects the sample size difference: with N = 1,094 vs. N = 4,376, the radial velocity sample has less statistical power, so a given magnitude of deviation is less likely to reach significance. The KL divergence, being sample-size-independent, reveals that the radial velocity distribution deviates from Benford's Law by a comparable or even slightly larger magnitude, but the deviation fails to reach significance at N = 1,094.

4.3 The Planet Radius Anomaly

The extreme departure of planet radius from Benford's Law (χ² = 1706.49, KL = 0.2157) requires a physical rather than purely statistical explanation. Nearly half of all planet radii in the catalog begin with digit 1 (49.95%), and more than three-quarters begin with digit 1 or 2 (77.88%).

This reflects the bimodal radius distribution of known exoplanets, which peaks sharply at approximately 0.1–0.2 R_J (the super-Earth / sub-Neptune population) and again near 1.0–1.3 R_J (the hot Jupiter population). When expressed in Jupiter radii, both peaks produce first digits of 1 or 2. The gap between these peaks—the so-called "radius valley" or "Fulton gap" near 1.5–2.0 R_⊕ (approximately 0.13–0.18 R_J)—further concentrates first digits at 1 (Fulton et al., 2017).

This is a clear case where the choice of measurement unit interacts with the physical distribution to produce a non-Benford digit pattern. If radii were expressed in Earth radii instead of Jupiter radii, the first-digit distribution would differ substantially. Benford's Law applies best when data span many orders of magnitude without strong modal structure; the bimodal radius distribution violates this condition.

4.4 Planet Mass: Good Benford Conformity

Planet mass shows the best conformity to Benford's Law among the three parameters (χ² = 9.76, p = 0.28, KL = 0.0017). This conformity holds across both transit and radial velocity subsamples individually. Planet masses span a wider dynamic range—from sub-Earth masses (~10⁻³ M_J) to super-Jupiters (~20 M_J)—and are distributed more smoothly across this range compared to radii. The smoother, wider distribution is precisely the condition under which Benford's Law is theoretically expected to hold (Hill, 1995).

4.5 Second-Digit Conformity

The strong second-digit conformity for orbital periods (χ² = 6.35, p = 0.70) provides an important control. It demonstrates that the first-digit deviation is a genuine feature of the leading-digit structure—likely reflecting broad distributional shape and selection effects—rather than an artifact of data processing, rounding, or systematic measurement error. If the data were corrupted by rounding or truncation, we would expect second-digit anomalies as well.

4.6 Limitations

Several limitations should be noted:

1. Catalog evolution. The NASA Exoplanet Archive is a living catalog. Our analysis reflects the state of the archive as of early 2024, and results may shift as new discoveries are added. In particular, upcoming missions (e.g., continued TESS observations, PLATO) will alter the detection-method composition of the catalog.

2. Unit dependence. The choice of measurement units (days for period, Jupiter masses for mass, Jupiter radii for radius) affects which digits appear as leading digits. A change of units—for example, expressing mass in Earth masses—would shift the distribution. Benford's Law is theoretically scale-invariant, but this invariance applies to the underlying physical distribution, not to a distribution already shaped by measurement selection.

3. Sample composition. The "Other" category is heterogeneous and small (N = 67 for orbital periods), limiting our ability to characterize Benford behavior for individual minor methods such as direct imaging or microlensing.

4. Multiple testing. We performed multiple chi-square tests without formal correction for multiple comparisons. The primary findings (orbital period deviation, mass conformity, radius deviation) have such extreme p-values that correction would not change the qualitative conclusions, but borderline results (e.g., radial velocity orbital periods at p = 0.12) should be interpreted cautiously.

5. Underlying distribution. Benford's Law emerges from multiplicative processes spanning orders of magnitude. The exoplanet catalog is not a random sample of the galactic exoplanet population but a heavily selected subset. The interesting finding is not whether the catalog conforms to Benford's Law, but how the pattern of deviation reflects the known selection functions.

5. Conclusion

We have conducted a comprehensive Benford's Law analysis of 5,537 exoplanet orbital periods, 2,792 planet masses, and 4,396 planet radii from the NASA Exoplanet Archive (discoveries through 2024). Our key findings are:

1. Planet mass conforms well to Benford's Law (χ² = 9.76, p = 0.28), consistent with its broad dynamic range and smooth distribution.

2. Orbital period shows a statistically significant but modest deviation (χ² = 42.58, p = 1.05 × 10⁻⁶, KL = 0.0038), driven primarily by transit-detected planets whose periods show excess mid-range first digits (3–5) relative to Benford's expectation. Radial velocity detections show weaker and non-significant deviation (p = 0.12).

3. Planet radius deviates dramatically (χ² = 1706.49, KL = 0.2157) due to the bimodal radius distribution concentrated at ~0.1–0.2 R_J and ~1.0 R_J, which produces an extreme excess of digit 1 (49.95% vs. 30.10%).

4. Second-digit analysis of orbital periods shows excellent Benford conformity (p = 0.70), confirming that the first-digit deviation reflects large-scale distributional structure rather than data-processing artifacts.

5. Permutation testing (10,000 iterations) confirms the parametric p-value for orbital periods, with a permutation p-value of p < 0.0001.

These results demonstrate that Benford analysis provides a useful, lightweight diagnostic for identifying selection-bias fingerprints in astronomical catalogs. The detection-method-dependent pattern of deviation is itself informative: it reveals how the transit method's steep geometric selection function distorts the first-digit distribution in ways distinct from the radial velocity method's smoother sensitivity function. As the exoplanet catalog continues to grow with contributions from diverse missions and methods, periodic Benford analysis could serve as a simple monitor for changes in catalog composition and completeness.

References

Alexopoulos, T., & Leontsinis, S. (2014). Benford's Law in astronomy. Journal of Astrophysics and Astronomy, 35(4), 639–648. https://doi.org/10.1007/s12036-014-9303-z

Benford, F. (1938). The law of anomalous numbers. Proceedings of the American Philosophical Society, 78(4), 551–572.

Borucki, W. J., Koch, D., Basri, G., et al. (2010). Kepler planet-detection mission: Introduction and first results. Science, 327(5968), 977–980. https://doi.org/10.1126/science.1185402

Fischer, D. A., Howard, A. W., Laughlin, G. P., et al. (2014). Exoplanet detection techniques. Protostars and Planets VI, 715–737. https://doi.org/10.2458/azu_uapress_9780816531240-ch031

Fulton, B. J., Petigura, E. A., Howard, A. W., et al. (2017). The California-Kepler Survey. III. A gap in the radius distribution of small planets. The Astronomical Journal, 154(3), 109. https://doi.org/10.3847/1538-3881/aa80eb

Hair, J. (2014). Benford's Law as applied to selected astronomical data. Journal of the American Statistical Association, manuscript.

Hill, T. P. (1995). A statistical derivation of the significant-digit law. Statistical Science, 10(4), 354–363. https://doi.org/10.1214/ss/1177009869

Mayor, M., & Queloz, D. (1995). A Jupiter-mass companion to a solar-type star. Nature, 378(6555), 355–359. https://doi.org/10.1038/378355a0

NASA Exoplanet Archive. Planetary Systems Table. California Institute of Technology. https://exoplanetarchive.ipac.caltech.edu/

Newcomb, S. (1881). Note on the frequency of use of the different digits in natural numbers. American Journal of Mathematics, 4(1), 39–40. https://doi.org/10.2307/2369148

Shukla, A., Pandey, A. K., & Pathak, A. (2017). Benford's Law in astronomy. Astrophysics and Space Science, 362, 42.

Winn, J. N. (2010). Exoplanet transits and occultations. In Exoplanets, ed. S. Seager, University of Arizona Press, 55–77.