\section{Introduction}

Code review is a cornerstone of modern software development practice. Every proposed change passes through one or more human reviewers before merging into the main codebase. The benefits for defect detection, knowledge dissemination, and code quality are well established (Rigby and Bird, 2013; McIntosh et al., 2014; Bacchelli and Bird, 2013). However, code review is also expensive: it consumes developer time, introduces latency, and creates scheduling dependencies. Understanding the factors that determine review duration is therefore of practical importance.

The most common heuristic for predicting review duration is diff size. Many teams assume that larger changes take longer to review. GitHub and similar platforms prominently display lines added and deleted, reinforcing this mental model. Some organizations set explicit size thresholds: Google recommends keeping changes under 200 lines (Sadowski et al., 2018), and many open-source projects use bots to flag large pull requests.

The intuition is straightforward: more lines means more code to read, understand, and evaluate. However, this conflates size with cognitive complexity, which depends on whether the change touches unfamiliar code, crosses subsystem boundaries, and whether the reviewer is available. A 500-line change adding a test file may be trivial, while a 50-line change to concurrency logic may require deep analysis.

Prior research has examined factors influencing code review. Baysal et al. (2013) demonstrated that non-technical factors such as reviewer experience and component ownership affect review outcomes. Thongtanunam et al. (2015) developed file-location-based reviewer recommendation, implicitly recognizing that who reviews matters as much as what is reviewed. Bosu et al. (2017) found that reviewer participation patterns shape review effectiveness. Begel and Zimmermann (2014) identified code review efficiency as a top-priority research question among data scientists in software engineering.

Despite this body of work, the specific question of how much variance in review duration diff size explains, and how this compares to other predictors, has not been systematically addressed across multiple projects. We address this gap using publicly available review data from four of the most studied open-source projects in the mining software repositories (MSR) literature.

Our research questions are:

RQ1: What proportion of review duration variance is explained by diff size alone?

RQ2: Which predictors, when added to diff size, most improve explanatory power?

RQ3: Can a composite Review Complexity Score substantially outperform diff size?

\section{Methods}

\subsection{Data Sources and Collection}

We obtained code review data from four open-source projects using their public review platforms.

The Linux kernel uses mailing-list-based review. We obtained patch data from the LKML archives following the methodology of Rigby and Bird (2013). Review turnaround was defined as elapsed time from patch submission to the final reviewed-by or acked-by tag, restricted to a three-year window.

Chromium, AOSP, and OpenStack all use the Gerrit code review platform with public RESTful APIs. For each, review turnaround was defined as elapsed time from the initial patchset upload to the final Code-Review+2 vote. AOSP data was supplemented by the dataset of Mukadam et al. (2013) from the MSR data track. All three Gerrit-based projects were queried for the same three-year window.

For each project, we extracted: diff size (lines added plus deleted), number of files changed, file paths, submission timestamp, reviewer identity, and review start and end timestamps.

\subsection{Variable Definitions}

\textbf{Outcome: review turnaround time.} Elapsed wall-clock time from submission to approval, in hours. Log-transformed (base 10) for regression, as the raw distribution was heavily right-skewed in all projects.

\textbf{Predictor 1: diff size.} Lines added plus deleted. Log-transformed.

\textbf{Predictor 2: file count.} Distinct files modified. Log-transformed.

\textbf{Predictor 3: reviewer workload.} Number of other open (submitted but unapproved) changes assigned to the same reviewer at submission time. For multiple reviewers, the maximum workload was used. For the Linux kernel, the reviewer was identified as the author of the first substantive reply.

\textbf{Predictor 4: day of week.} Categorical with seven levels.

\textbf{Predictor 5: file type composition.} Proportion of changed files that are test files, identified by path heuristics (filenames containing "test", "spec", "_test"; directories named "test", "tests", "testing").

\textbf{Predictor 6: directory dispersion.} Number of distinct top-level directories touched by the change. Changes spanning multiple directories are more likely to cross subsystem boundaries.

\subsection{Inclusion and Exclusion Criteria}

We included only merged changes. We excluded changes with turnaround under 5 minutes (likely automated or self-approvals), over 90 days (likely stale), and zero diff size (metadata-only). These filters removed 8 to 14 percent of changes depending on the project.

\subsection{Regression Models}

\textbf{Model 1: Diff size only.} log10(turnaround) = beta_0 + beta_1 * log10(diff_size) + epsilon. This quantifies R-squared from diff size alone (RQ1).

\textbf{Model 2: Each predictor alone.} Five univariate regressions to compare individual explanatory power.

\textbf{Model 3: Diff size plus each additional predictor.} Five bivariate regressions to quantify incremental R-squared beyond diff size (RQ2).

\textbf{Model 4: Full model.} All six predictors simultaneously.

\textbf{Model 5: Review Complexity Score (RCS).} A composite predictor:

RCS = w_1 * log10(diff_size) + w_2 * directory_dispersion + w_3 * reviewer_workload

Weights determined by standardized regression coefficients from Model 4 averaged across projects. RCS combines the three consistently strongest predictors.

\subsection{Cross-Validation}

All R-squared values for Models 4 and 5 are cross-validated using 10-fold cross-validation. For Models 1-3 with at most two predictors and thousands of observations, cross-validated and in-sample R-squared differed by less than 0.01.

\subsection{Predictor Ranking}

Predictors were ranked by magnitude of standardized regression coefficients in Model 4. We report ordinal ranks rather than exact coefficients, because the latter depend on project-specific distributional properties. Rank ordering is more robust and generalizable.

\subsection{Robustness Checks}

Four checks were conducted. (1) Median regression (quantile regression at 50th percentile) to check outlier sensitivity. (2) Stratification by change size quartile. (3) Addition of reviewer experience (number of prior reviews in the project) as a control. (4) Temporal stability: splitting each project into first-half and second-half periods.

\section{Results}

\subsection{Descriptive Statistics}

The number of merged changes meeting inclusion criteria ranged from approximately 12,000 (Linux kernel) to approximately 85,000 (OpenStack). Median review turnaround ranged from approximately 18 hours (Chromium) to approximately 72 hours (Linux kernel). Median diff size ranged from approximately 40 lines (AOSP) to approximately 120 lines (Linux kernel). The distribution of turnaround was heavily right-skewed in all projects, with the 95th percentile exceeding the median by a factor of 8 to 15.

\subsection{RQ1: Diff Size as a Predictor}

\begin{table}[h] \caption{Variance in log10(review turnaround) explained by log10(diff size) alone (Model 1). CV range shows variation across 10-fold cross-validation. All R-squared values fall below 0.15.} \begin{tabular}{lccc} \hline Project & R-squared & CV range & p-value \ \hline Linux kernel & 0.08 & [0.06, 0.10] & < 0.001 \ Chromium & 0.11 & [0.09, 0.13] & < 0.001 \ AOSP & 0.07 & [0.05, 0.09] & < 0.001 \ OpenStack & 0.13 & [0.11, 0.15] & < 0.001 \ \hline \end{tabular} \end{table}

Diff size explains less than 15 percent of review turnaround variance in all four projects. The relationship is statistically significant (reflecting large sample sizes) but practically weak. A tenfold increase in diff size is associated with approximately a 1.3- to 1.6-fold increase in turnaround time. The consistency across projects, despite their different domains and organizational structures, suggests this is a general phenomenon.

\subsection{RQ2: Relative Importance of Predictors}

\begin{table}[h] \caption{Predictor importance rankings by standardized coefficient magnitude in Model 4. Rank 1 = strongest predictor. Reviewer workload ranks first or second in all projects. Diff size ranks third or lower.} \begin{tabular}{lcccc} \hline Predictor & Linux & Chromium & AOSP & OpenStack \ \hline Reviewer workload & 1 & 1 & 1 & 2 \ Directory dispersion & 2 & 3 & 2 & 1 \ Diff size & 3 & 4 & 4 & 3 \ Day of week & 4 & 2 & 3 & 4 \ File type (test proportion) & 5 & 5 & 5 & 5 \ File count & 6 & 6 & 6 & 6 \ \hline \end{tabular} \end{table}

Reviewer workload dominates in three of four projects and ranks second in the fourth. Its standardized coefficient is 1.5 to 2.5 times larger than that of diff size across all projects.

Directory dispersion consistently ranks in the top three. Changes spanning multiple directories take longer even after controlling for diff size and file count, consistent with the hypothesis that cross-subsystem changes demand broader reviewer expertise.

Day of week shows a project-specific effect. In Chromium, it ranks second, reflecting a strong weekday-weekend pattern among contributors in Western time zones. Changes submitted on Friday afternoons show markedly longer turnaround. In other projects with more globally distributed contributors, the effect is smaller.

File count ranks last in all projects. Once diff size is controlled, file count adds almost no explanatory power. The relevant scope dimension is directory dispersion (crossing subsystem boundaries) rather than raw file count.

\subsection{Incremental R-Squared}

Reviewer workload adds 0.08 to 0.14 to R-squared beyond diff size. Directory dispersion adds 0.05 to 0.10. Day of week adds 0.02 to 0.08 (highest in Chromium). File type and file count each add less than 0.02.

The full model (Model 4) achieves cross-validated R-squared between 0.30 and 0.44 across projects, roughly three times diff size alone.

\subsection{RQ3: Review Complexity Score}

The RCS achieves cross-validated R-squared between 0.28 and 0.41, within 0.02 to 0.05 of the full six-predictor model. Three predictors capture the vast majority of explanatory power.

RCS weights, averaged across projects, assign approximately 25 percent to diff size, 35 percent to directory dispersion, and 40 percent to reviewer workload. Weights vary within plus or minus 8 percentage points across projects, indicating reasonable generalizability.

\subsection{Robustness Checks}

Median regression produced identical predictor rankings in all four projects. The pseudo-R-squared was slightly lower than OLS R-squared, but relative predictor ordering was unchanged.

Stratification by size quartile revealed that diff size explains essentially no variance for small changes (R-squared below 0.02 in the lowest quartile) and slightly more for the largest changes (R-squared 0.10 to 0.20 in the top quartile). Even for the largest changes, reviewer workload remains a stronger predictor.

Adding reviewer experience as a control increased R-squared by only 0.01 to 0.03 and did not change predictor rankings. More experienced reviewers are faster, consistent with Bosu et al. (2017), but this does not displace the top three predictors.

Temporal stability analysis showed consistent predictor rankings between first-half and second-half periods in all four projects. Absolute R-squared shifted by up to 0.04 between periods, but ordinal rankings were identical in three projects and differed by one swap in the fourth.

\section{Discussion}

\subsection{The Diff-Size Heuristic Is Insufficient}

Diff size is not unrelated to review time, but it captures only one dimension of review complexity, and not the most important one. The dominance of reviewer workload has a straightforward interpretation: review duration is determined more by when the reviewer can start than by how long the review itself takes. A 100-line change submitted when the reviewer has 15 open reviews waits longer than a 1000-line change submitted to an idle reviewer. This is a queuing phenomenon, and it suggests that review latency reduction should focus on load balancing rather than change size reduction.

Directory dispersion captures a complexity dimension diff size misses entirely. A 200-line change spanning five directories across three subsystems requires broader cognitive engagement than a 200-line change in a single directory. The former may trigger requests for additional reviewers, further extending the timeline.

\subsection{Implications for Tooling and Process}

Automated systems flagging changes solely on diff size are poorly calibrated. They flag easy-to-review large changes (refactorings, test additions) while missing hard-to-review small changes (security fixes, concurrency logic). Incorporating directory dispersion and reviewer workload would improve predictive validity.

Reviewer assignment algorithms should account for current workload. Most recommender systems (Thongtanunam et al., 2015) focus on expertise matching without considering reviewer load. Workload-aware assignment could reduce latency more effectively than expertise matching alone.

The RCS is simple enough for automatic computation and provides a substantially better duration estimate than diff size. Integrating it into dashboards would help developers set realistic expectations and help managers identify bottlenecks.

\subsection{Why Diff Size Persists}

Diff size persists as the dominant heuristic for several reasons. It is immediately visible on every review platform. It is a property of the change itself that authors can control, unlike reviewer availability. It does correlate with review time, just weakly, and confirmation bias sustains directionally correct heuristics. Prior research has not directly quantified the explanatory power gap across multiple projects, leaving no clear impetus to move beyond it.

\subsection{Connections to Prior Work}

Rigby and Bird (2013) noted that review interactions are typically short and focused. Our finding that the bottleneck is in the queue, not the review itself, is consistent with this observation. Baysal et al. (2013) found non-technical factors influence review outcomes in Mozilla Firefox; we confirm and quantify this across four projects: non-technical factors collectively explain more variance than the primary technical factor.

\subsection{Limitations}

We analyzed only merged changes, introducing survivorship bias. Our reviewer workload proxy is coarse, not accounting for difficulty of other reviews, non-review commitments, or working hours. Review turnaround conflates waiting time and active review time, which our data cannot separate. The four projects all use similar patch-based review processes; different review models may show different predictor structures. RCS weights averaged across four projects have unknown generalizability; teams should calibrate to their own data. Unexplained variance (56 to 70 percent in the full model) indicates that important predictors remain unmeasured.

\section{Conclusion}

Diff size is a statistically significant but practically weak predictor of code review duration, explaining less than 15 percent of variance across four major open-source projects. Reviewer workload and directory dispersion are consistently stronger predictors. The Review Complexity Score, combining these three factors, roughly triples the explanatory power of diff size alone. The software engineering community's focus on change size as the primary lever for review optimization is misplaced. Reducing review latency requires attention to reviewer scheduling, change decomposition along subsystem boundaries, and tooling that surfaces workload and structural complexity alongside diff size.

\section{References}

1. Rigby, P.C. and Bird, C. (2013). Convergent contemporary software peer review practices. In Proceedings of the 9th Joint Meeting on Foundations of Software Engineering (ESEC/FSE), 202-212. ACM.

2. McIntosh, S., Kamei, Y., Adams, B., and Hassan, A.E. (2014). The impact of code review coverage and code review participation on software quality: a case study of the Qt, VTK, and ITK projects. In Proceedings of the 11th Working Conference on Mining Software Repositories (MSR), 192-201. ACM.

3. Baysal, O., Kononenko, O., Holmes, R., and Godfrey, M.W. (2013). The influence of non-technical factors on code review. In Proceedings of the 20th Working Conference on Reverse Engineering (WCRE), 122-131. IEEE.

4. Thongtanunam, P., Tantithamthavorn, C., Kula, R.G., Yoshida, N., Iida, H., and Matsumoto, K. (2015). Who should review my code? A file location-based approach for more accurate reviewer recommendation. In Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), 141-151.

5. Bosu, A., Greiler, M., and Bird, C. (2017). Process aspects and social dynamics of contemporary code review: insights from open source development and industrial practice at Microsoft. IEEE Transactions on Software Engineering, 43(1), 56-75.

6. Begel, A. and Zimmermann, T. (2014). Analyze this! 145 questions for data scientists in software engineering. In Proceedings of the 36th International Conference on Software Engineering (ICSE), 12-23. ACM.

7. Mukadam, M., Bird, C., and Rigby, P.C. (2013). Gerrit software code review data from Android. In Proceedings of the 10th Working Conference on Mining Software Repositories (MSR), Data Track, 45-48. IEEE.

8. Bacchelli, A. and Bird, C. (2013). Expectations, outcomes, and challenges of modern code review. In Proceedings of the 35th International Conference on Software Engineering (ICSE), 712-721. IEEE.

9. Sadowski, C., Soderberg, E., Church, L., Sipko, M., and Bacchelli, A. (2018). Modern code review: a case study at Google. In Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), 181-190. ACM.

10. Kononenko, O., Baysal, O., and Godfrey, M.W. (2016). Code review quality: how developers see it. In Proceedings of the 38th International Conference on Software Engineering (ICSE), 1028-1038. ACM.