Books, papers and documentation

Pavel Dmitriev, Somit Gupta, Dong Woo Kim, Garnet Vaz · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2017), pp. 1427-1436 · 2017

Checked 2026-10-11: Read the full text of the Microsoft Research PDF. Section 4: sample ratio mismatch detected with a chi-square test, usually implies severe selection bias. 5.2 ratio metrics can mislead when the denominator changes; report numerator and denominator counts. 5.4 underpowered metrics; 5.5 borderline p-values; 5.6 continuous monitoring and early stopping (or extending) increases Type I error; 5.8 segments affected by treatment produce Simpson's paradox, test SRM per segment, recursive segmenting is multiple testing, use corrections such as Bonferroni; 5.9 outliers, capping or log transforms; 5.10 novelty and primacy effects, segment effect by day or visit. DOI 10.1145/3097983.3098024 confirmed via Semantic Scholar.

Anastasios N. Angelopoulos, Stephen Bates · arXiv:2107.07511 · 2021

Tutorial on split conformal prediction: calibration set, nonconformity scores, finite-sample marginal coverage; Day 26 prediction intervals.

Checked 2026-10-11: Title, authors, submission and latest-version dates via arXiv API record for 2107.07511.

Ivan P. Fellegi, Alan B. Sunter · Journal of the American Statistical Association 64(328), pp. 1183-1210 · 1969

Checked 2026-10-12: Crossref metadata (2026-10-12): title, two authors, JASA vol. 64 issue 328, pp. 1183-1210, 1969. Taylor & Francis page returned 403; content statement (a decision model over comparison of records with three outcomes link / non-link / possible link, optimal procedure being a likelihood-ratio rule under conditional independence of comparison fields) read from secondary descriptions of the abstract and a university-hosted copy listing, not from the publisher page.

Isaac Gibbs, Emmanuel Candès · Advances in Neural Information Processing Systems 34 (NeurIPS 2021), 1660-1672 · 2021

ACI updates the miscoverage level online to keep long-run coverage under distribution shift; Day 26-27 online intervals.

Checked 2026-10-11: Title, authors, volume 34, pp. 1660-1672 in NeurIPS proceedings abstract page metadata; arXiv 2106.00170 matches.

Ramesh Johari, Leo Pekelis, David J. Walsh · arXiv:1512.04922 (stat.ME) · 2015

Checked 2026-10-11: arXiv abstract page: title, three authors, submission and revision dates. Abstract: standard p-values and confidence intervals are unreliable when users continuously monitor tests and choose sample size adaptively; proposes always-valid p-values and confidence intervals tied to sequential tests. A related KDD 2017 paper ('Peeking at A/B Tests', DOI 10.1145/3097983.3097992) was identified but its full text was not opened.

G. E. P. Box, D. R. Cox · Journal of the Royal Statistical Society Series B 26(2), 211-243 · 1964

Box-Cox power transformation family with likelihood-based choice of lambda; variance stabilization before modelling; Day 22.

Checked 2026-10-11: Title, authors, JRSS-B 26(2) pp. 211-243, 1964 in Crossref DOI metadata.

David Colquhoun · Royal Society Open Science 1(3), 140216 · 2014

Checked 2026-10-12: PubMed Central record via search results: title, author, R. Soc. open sci. 1(3) 140216, 2014, DOI 10.1098/rsos.140216. Abstract: a p-value of 0.05 used to claim a discovery gives a wrong conclusion at least 30% of the time, more when experiments are underpowered; a 2015 commentary notes the result depends on an assumed prior probability of a real effect. Abstract-level reading only.

Alex Deng, Ulf Knoblich, Jiannan Lu · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2018); arXiv:1803.06336 · 2018

Checked 2026-10-11: arXiv page (title, authors, v4 12 Sep 2018) and full-text PDF section 3.1: distinguishes randomization unit from analysis unit; when randomizing by user but computing page-level metrics the observations are not i.i.d., and the metric is written as a ratio of two averages of randomization-unit quantities so the delta method gives its variance. KDD 2018 venue and DOI 10.1145/3219819.3219919 from the Semantic Scholar record.

Andrew Gelman, John Carlin · Perspectives on Psychological Science 9(6), 641-651 · 2014

Checked 2026-10-11: Read the author PDF (sites.stat.columbia.edu/gelman/research/published/retropower_final.pdf): title, authors, journal 9(6) 641-651, 2014. Defines Type S (wrong sign) and Type M (exaggeration ratio) errors; in noisy small-sample settings significant results can be misleading, and when power falls well below 0.5 the exaggeration ratio grows large.

David Firth · Biometrika 80(1), pp. 27-38 · 1993

Checked 2026-10-12: Title, author, Biometrika vol. 80 pp. 27-38, 1993 in Crossref DOI metadata. That Firth's modified estimator is a penalized likelihood with Jeffreys prior penalty and is used under data separation was read in the abstract of arXiv 2304.07484 (Ogawa and Tomo); Firth's paper itself was not opened.

Roopesh Ranjan, Tilmann Gneiting · Journal of the Royal Statistical Society: Series B 72(1), pp. 71-91 · 2010

Checked 2026-10-12: Crossref DOI record: title, authors, JRSS-B vol. 72 pp. 71-91, 2010. Abstract as served by Crossref: linear pooling is the most popular way to combine probability forecasts, but any non-trivial weighted average of two or more distinct, calibrated probability forecasts is necessarily uncalibrated and lacks sharpness; linear pooling therefore requires recalibration even when the individual forecasts are calibrated; the authors propose a beta-transformed linear opinion pool. Full text not opened.

Rina Foygel Barber, Emmanuel J. Candès, Aaditya Ramdas, Ryan J. Tibshirani · The Annals of Statistics 51(2), 816-845 · 2023

Weighted conformal methods with coverage-gap bounds when exchangeability fails (drift, time series); Day 26.

Checked 2026-10-11: Title, authors, Annals of Statistics 51(2) pp. 816-845, April 2023 in Crossref DOI metadata.

Anastasios N. Angelopoulos, Stephen Bates · Foundations and Trends in Machine Learning 16(4), 494-591 · 2023

Journal version of the arXiv tutorial (2107.07511); prefer this for formal citation.

Checked 2026-10-11: Title, authors, FnT ML 16(4) pp. 494-591, March 2023 in Crossref DOI metadata.

Yaniv Romano, Evan Patterson, Emmanuel Candès · Advances in Neural Information Processing Systems 32 (NeurIPS 2019) · 2019

CQR calibrates quantile-regression intervals with conformal correction, giving adaptive-width intervals with coverage guarantees; Day 26.

Checked 2026-10-11: Title, authors, volume 32, 2019 in NeurIPS proceedings abstract page metadata; arXiv 1905.03222 matches.

Alex Deng, Jiannan Lu, Shouyuan Chen · arXiv:1602.05549 (stat.AP) · 2016

Checked 2026-10-12: arXiv abstract page: title, three authors, submitted 17 Feb 2016. Abstract: Bayesian testing with continuous monitoring is proved valid when proper stopping rules are used, supported by simulations; it identifies common improper stopping rules that invalidate results and compares with NHST corrections.

Yoav Benjamini, Yosef Hochberg · Journal of the Royal Statistical Society, Series B (Methodological) 57(1), 289-300 · 1995

Checked 2026-10-11: Oxford Academic landing page: title, authors, JRSS-B 57(1) pp. 289-300, 1995; summary introduces controlling the expected proportion of falsely rejected hypotheses (false discovery rate) and proves a sequential Bonferroni-type procedure controls it for independent test statistics.

Iavor Bojinov, David Simchi-Levi, Jinglong Zhao · arXiv:2009.00148 (stat.ME); no journal venue shown on the arXiv page · 2020

Checked 2026-10-12: arXiv abstract page: title, three authors, first submitted 31 Aug 2020. Abstract: switchback experiments repeatedly expose the same unit to randomly assigned treatments over time; carryover effects are the lingering impact of a treatment on later outcomes; derives optimal designs under assumptions about carryover length and proposes randomization-based inference.

Aleksander Fabijan, Jayant Gupchup, Somit Gupta, Jeff Omhover, Wen Qin, Lukas Vermeer, Pavel Dmitriev · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2019) · 2019

Checked 2026-10-11: Semantic Scholar record for the DOI (ACM page returned 403): title, seven authors, KDD 2019 and abstract. Abstract defines SRM as an observed sample ratio that differs from the expected one, calls it a symptom of many data-quality issues, says ignoring it can make a bad change look good, and describes a taxonomy from experiments at four companies.

Alex Deng, Victor Hu · Proceedings of the Eighth ACM International Conference on Web Search and Data Mining (WSDM 2015) · 2015

Checked 2026-10-12: Microsoft ExP page via search results: title, two authors, WSDM 2015. Content: when a feature reaches few users, analysing all users dilutes the effect and power; practitioners analyse triggered users and then dilute, and these formulas are often applied incorrectly, especially for ratio metrics; the paper merges the steps. Abstract-level reading only.

W. S. Robinson · American Sociological Review 15(3), 351-357 · 1950

Checked 2026-10-12: Citation and content from several secondary accounts and a 2009 reassessment listing (JSTOR stable 2087176; paper not opened directly). Content: ecological (area-level) and individual-level correlations can differ sharply; the 1930 census state-level foreign-born versus literacy correlation had a different sign from the individual-level one.

Leslie E. Papke, Jeffrey M. Wooldridge · Journal of Applied Econometrics 11(6), pp. 619-632 · 1996

Checked 2026-10-12: Crossref DOI record: title, authors, Journal of Applied Econometrics vol. 11 no. 6 pp. 619-632, November 1996. Metadata only; Crossref carries no abstract and the full text was not opened.

Daniël Lakens · Social Psychological and Personality Science 8(4), 355-362 · 2017

Checked 2026-10-11: PubMed Central full text (PMC5502906): title, author, journal 8(4) 355-362, 2017. TOST tests against lower and upper bounds set by the smallest effect size of interest; a non-significant result cannot show an effect is absent, equivalence testing can reject effects large enough to matter.

Joseph P. Simmons, Leif D. Nelson, Uri Simonsohn · Psychological Science 22(11), 1359-1366 · 2011

Checked 2026-10-12: SAGE and SSRN records via search results: title, authors, Psychol Sci 22(11) 1359-1366. Content: flexibility in choosing dependent variables, sample size, covariates and reported conditions substantially raises the false-positive rate above the nominal 5%, shown by simulation and two experiments. Abstract-level reading only.

Stef van Buuren

Section 1.2 for the missing-data mechanisms (Day 6, Day 9 imputation).

Checked 2026-10-11: Book title on the site; author named on the 'About the author' page (Stef van Buuren, TNO / Utrecht University); second edition inferred from the 'Preface to second edition' entry; Section 1.2 'Concepts of MCAR, MAR and MNAR' defines the three mechanisms and notes simple fixes are generally valid only under MCAR. Publisher and year not confirmed on the pages checked.

David J. Spiegelhalter · Statistics in Medicine 24(8), 1185-1202 · 2005

Funnel plots with control limits by unit size instead of league tables; Project 005.

Checked 2026-10-11: Title, author, Statistics in Medicine 24(8) pp. 1185-1202 in Crossref DOI metadata (Crossref issued date 2004, online-first; volume 24 is the 2005 volume).

statsmodels developers

Binomial/Poisson GLM APIs; Projects 003 and 007.

Checked 2026-10-11: statsmodels 0.15.0 page title; GLM families incl. Binomial and Poisson.

F. J. Anscombe · The American Statistician 27(1), pp. 17-21 · 1973

Checked 2026-10-12: Crossref metadata (2026-10-12): title, author, journal, vol. 27 issue 1, pp. 17-21, February 1973. Publisher page returned 403, so the statement that four small datasets share the same usual summary statistics yet look very different is confirmed only through secondary course pages and the Matejka and Fitzmaurice paper, which cites it as the precedent.

Gerd Gigerenzer, Ulrich Hoffrage · Psychological Review 102(4), 684-704 · 1995

Day 3 natural-frequency presentation of Bayes' theorem and base rates.

Checked 2026-10-11: Opened the authors' manuscript PDF (pages.ucsd.edu copy): first page prints Psychological Review 102(4), 1995, 684-704, title and both authors; abstract states Bayesian computations are simpler in frequency formats and that naive participants used Bayesian algorithms for up to 50% of inferences with frequency formats. DOI resolves to doi.apa.org.

Beata Strack, Jonathan P. DeShazo, Chris Gennings, Juan L. Olmo, Sebastian Ventura, Krzysztof J. Cios, John N. Clore · BioMed Research International 2014, Article 781670 · 2014

Dataset paper and attribute dictionary for Diabetes 130-US Hospitals; Project 005.

Checked 2026-10-11: Title, authors, BioMed Research International 2014 in Crossref DOI metadata; the UCI page points to its Table 1 for attribute definitions.

Alex Deng, Ya Xu, Ron Kohavi, Toby Walker · Proceedings of the Sixth ACM International Conference on Web Search and Data Mining (WSDM 2013) · 2013

CUPED variance reduction with a pre-period covariate (Day 4 power, Day 5 experiments).

Checked 2026-10-11: Microsoft ExP paper page (exp-platform.com/cuped): title, four authors, WSDM 2013 Rome; summary says CUPED uses pre-experiment data to reduce metric variability and reports roughly 50% variance reduction on Bing, equivalent to the same power with half the users or half the duration. DOI 10.1145/2433396.2433413 and authors confirmed in the Semantic Scholar record (ACM page returned 403).

Claude Nadeau, Yoshua Bengio · Machine Learning 52, pp. 239-281 · 2003

Checked 2026-10-12: Title, authors, Machine Learning vol. 52 pp. 239-281, September 2003 in Crossref DOI metadata. The corrected resampled t-test it proposes was checked through the scikit-learn example (docs-sklearn-example-grid-search-stats), which cites the 2000 NIPS version; the paper's full text was not opened.

Bradley Efron, Trevor Hastie, Iain Johnstone, Robert Tibshirani · The Annals of Statistics 32(2) · 2004

Original publication using the 442-patient diabetes data; Project 009.

Checked 2026-10-11: Title, authors, Annals of Statistics 32(2), 2004 in Crossref DOI metadata; scikit-learn's diabetes DESCR cites this paper as the data reference.

Tilmann Gneiting · Journal of the American Statistical Association 106(494), 746-762 · 2011

Checked 2026-10-12: Crossref metadata and the arXiv abstract page 0912.0902: comparing forecasts by average error scores can mislead unless the scoring function matches the task; a score is consistent for the mean iff Bregman, and quantiles require generalised piecewise linear scores; expectations and quantiles are elicitable.

Jon Vaver, Jim Koehler · Google Research technical report · 2011

Checked 2026-10-12: Google Research publication page via search results: title, authors, 2011. Content: non-overlapping geographic regions are randomly assigned to control or treatment and each receives its condition through geo-targeted ads; a later paper notes geo regression is less suitable when few geographic units are available. Abstract-level reading only.

NIST (U.S. National Institute of Standards and Technology) and SEMATECH

Day 3 common distributions (section 1.3.6); later days may use its EDA and process-modeling chapters.

Checked 2026-10-11: Handbook landing page title on itl.nist.gov; section 1.3.6.6 'Gallery of Distributions' (eda366.htm) loads and lists the normal, lognormal, exponential, gamma and other continuous distributions plus the binomial and Poisson discrete distributions. No publication date shown.

Yoshua Bengio, Yves Grandvalet · Journal of Machine Learning Research 5, pp. 1089-1105 · 2004

Checked 2026-10-12: JMLR landing page (via search result summary) 2026-10-12: title, authors, volume 5 pp. 1089-1105, 2004; abstract: no universal unbiased estimator of the variance of K-fold cross-validation exists, and naive estimators that ignore the correlations between errors caused by overlapping training sets can underestimate the true variance. Full text not opened.

G. M. Ljung, G. E. P. Box · Biometrika 65(2), 297-303 · 1978

Ljung-Box portmanteau test for residual autocorrelation (white-noise check of model residuals); Day 22-23.

Checked 2026-10-11: Title, authors, Biometrika 65(2) pp. 297-303, 1978 in Crossref DOI metadata.

A. Albert, J. A. Anderson · Biometrika 71(1), pp. 1-10 · 1984

Checked 2026-10-12: Title, authors, Biometrika vol. 71 pp. 1-10, 1984 in Crossref DOI metadata. Statement that the maximum likelihood estimate does not exist when the data are separated was read in the abstract of Ogawa and Tomo, arXiv 2304.07484 (which cites this paper for it); the 1984 paper itself was not opened.

Anastasios N. Angelopoulos, Stephen Bates, Clara Fannjiang, Michael I. Jordan, Tijana Zrnic · arXiv:2301.09633 (stat.ML); no venue shown on the arXiv page · 2023

Basis for combining a large judge-scored set with a small human-labelled set.

Checked 2026-10-12: arXiv abstract page (title, 5 authors, v1 23 Jan 2023, v4 9 Nov 2023). Abstract: a framework for valid confidence intervals for quantities such as means when a small gold-labelled dataset is supplemented with a large set of machine-learning predictions, with no assumption on the model; more accurate predictions give narrower intervals. Applying it to LLM-judge scores is this course's adaptation, not a claim of the paper.

Victor Chernozhukov, Ivan Fernandez-Val, Alfred Galichon · Econometrica 78(3), 1093-1125 · 2010

Checked 2026-10-12: Crossref metadata (title, Econometrica 78(3) pp. 1093-1125, 2010; Crossref lists no authors) and the arXiv abstract page 0704.3649 (authors Chernozhukov, Fernandez-Val, Galichon): monotone rearrangement of non-monotone estimated quantile curves, argued to be closer to the true curve in finite samples.

RAND Corporation; packaged by statsmodels · statsmodels datasets

Project 003 (price sensitivity of doctor visits under randomized coinsurance).

Checked 2026-10-11: statsmodels 0.15.0 docs page and installed package source (datasets/randhie/data.py): COPYRIGHT 'in the public domain', source RAND HIE via Cameron and Trivedi (2005) data site, subset of original data, 20,190 observations x 10 variables with definitions; bundled src/randhie.csv has 45 columns incl. plan, site, coins, year and person id zper (5,912 persons, years 1-5). Loaded locally to confirm shape.

Jennifer A. Hutcheon, Arnaud Chiolero, James A. Hanley · BMJ 340, c2289 · 2010

Checked 2026-10-12: Citation confirmed from BMJ link and citing papers (search results; BMJ page itself not opened). Content: random error in an exposure pulls estimated regression slopes toward the null, error in an outcome mainly widens intervals; a larger sample does not remove the exposure-error bias.

Jeffrey S. Vitter · ACM Transactions on Mathematical Software 11(1), pp. 37-57 · 1985

Checked 2026-10-12: Crossref metadata (2026-10-12): title, author, TOMS vol. 11 issue 1, pp. 37-57, 1985. ACM page not opened (DOI redirects to dl.acm.org). Reservoir sampling definition (k items uniformly at random from a stream of unknown length, one pass, fixed-size memory) confirmed on the NIST Dictionary of Algorithms and Data Structures entry, which cites this paper.

Robert Tibshirani · Journal of the Royal Statistical Society Series B 58(1), 267-288 · 1996

Introduces the lasso (L1 penalty) that shrinks and sets some coefficients exactly to zero; Day 11 regularization, Day 17 feature selection.

Checked 2026-10-11: Title, author, JRSS-B 58(1) pp. 267-288, 1996 in Crossref DOI metadata.

Adrian G. Barnett, Jolieke C. van der Pols, Annette J. Dobson · International Journal of Epidemiology 34(1), 215-220 · 2005

Checked 2026-10-11: Oxford Academic page: title, authors, IJE 34(1) 215-220, Feb 2005 (online Aug 2004). Abstract: natural variation in repeated data can look like real change; the effect is larger with more measurement error and when follow-up is limited to a subgroup selected for extreme baseline values; reduce it by design and suitable analysis.

statsmodels developers

Count-data regression APIs; Project 003.

Checked 2026-10-11: statsmodels 0.15.0 page title; lists count models incl. NegativeBinomial and ZeroInflatedPoisson.

Hui Zou, Trevor Hastie · Journal of the Royal Statistical Society Series B 67(2), 301-320 · 2005

Elastic net combines L1 and L2 penalties, handling groups of correlated predictors better than lasso alone; Day 11.

Checked 2026-10-11: Title, authors, JRSS-B 67(2) pp. 301-320, 2005 in Crossref DOI metadata.

Arthur E. Hoerl, Robert W. Kennard · Technometrics 12(1), 55-67 · 1970

Introduces ridge (L2-penalized) regression as a biased estimator that reduces variance under multicollinearity; Day 11 regularization.

Checked 2026-10-11: Title, authors (Hoerl, Kennard), Technometrics 12(1) pp. 55-67, Feb 1970 in Crossref DOI metadata.

Nicolai Meinshausen, Peter Buehlmann

Subsampling-based selection frequencies; Day 17.

Checked 2026-10-12: arXiv listing (0809.2932, submitted 2008, revised 2009): abstract says the method is based on subsampling in combination with high-dimensional selection algorithms and gives finite-sample control of certain false-discovery error rates; the randomised Lasso is shown to be variable-selection consistent under weaker conditions. Journal version not opened; the arXiv page was the only version read.

Jeff Gill (source); packaged by statsmodels · statsmodels datasets

Project 007 (district pass-rate analysis with a binomial GLM). The docs call the 303 cases both districts and counties; DESCRLONG says unified school districts.

Checked 2026-10-11: statsmodels 0.15.0 docs page and installed package source (datasets/star98/data.py): COPYRIGHT 'used with express permission from the original author, who retains all rights', source Gill's Generalized Linear Models: A Unified Approach, 303 district-level cases of 1998 California STAR results, NABOVE/NBELOW binomial response for 9th-grade math, variable definitions. Loaded locally: 303 rows x 22 columns.

Leo Breiman · Statistical Science 16(3), from p. 199 · 2001

Classic essay contrasting data modeling with algorithmic modeling; use it to frame prediction versus inference.

Checked 2026-10-11: Title, author, Statistical Science vol. 16 issue 3, first page 199, 2001/08 and DOI in citation metadata on the Project Euclid landing page.

Xiao-Li Meng · Annals of Applied Statistics 12(2), 685-726 · 2018

Checked 2026-10-12: Project Euclid record and search summaries (full text paywalled; a Harvard-hosted PDF exists): title, author, AoAS 12(2) 685-726, DOI 10.1214/18-AOAS1161SF. The paper decomposes the error of a sample average into data quality (correlation between outcome and the recording/response indicator), data quantity and problem difficulty, and shows that without probability sampling the error relative to the 1/sqrt(n) benchmark grows with the population size. Abstract-level reading only.

Sander Greenland, Stephen J. Senn, Kenneth J. Rothman, John B. Carlin, Charles Poole, Steven N. Goodman, Douglas G. Altman · European Journal of Epidemiology 31, 337-350 · 2016

Day 4 confidence intervals, p-values, non-significance, power; Day 5 significance vs importance.

Checked 2026-10-11: Opened the PubMed Central full text (PMC4877414): title, seven authors, Eur J Epidemiol 31:337-350 (2016). The numbered list of 25 misinterpretations includes #1 (P value as probability the test hypothesis is true), #6 (P > 0.05 means no effect was observed), #7 (significance means an important relation), #19 (a specific 95% CI has a 95% chance of containing the true effect).

statsmodels developers

APIs for power/MDE, multiple-comparison correction and binomial CIs; Projects 001, 005.

Checked 2026-10-11: statsmodels 0.15.0 page lists 'Power and Sample Size Calculations' (NormalIndPower, tt_ind_solve_power), multipletests and proportion_confint.

statsmodels developers

Checked 2026-10-12: Opened 2026-10-12. scale accepts 'X2', 'dev' or a float; the default is 1 for Binomial and Poisson; 'X2' estimates the scale as Pearson chi-square divided by df_resid and 'dev' as deviance divided by df_resid. The page does not itself call this a remedy for overdispersion; that interpretation is the standard quasi-likelihood use of an estimated scale.

statsmodels developers

Checked 2026-10-12: Opened 2026-10-12. Poisson zero-inflated model for a 1-d count response; exog_infl supplies explanatory variables for the binary inflation (mixing probability) model, a constant if omitted; inflation link logit by default or probit; exog needs an explicit constant; offset and exposure enter the Poisson mean part with coefficient fixed at 1.

statsmodels developers

Checked 2026-10-12: Opened 2026-10-12. offset is added to the linear predictor with its coefficient constrained to 1; exposure adds log(exposure) to the linear prediction with coefficient fixed at 1 and is only valid with the log link; scale is 1 for Binomial, NegativeBinomial and Poisson, and log-likelihood and deviance are not valid when variance weights make them quasi-likelihood.

statsmodels developers

Checked 2026-10-12: Opened 2026-10-12. Source defines PerfectSeparationError (docstring: error due to perfect prediction in discrete models) and a separate PerfectSeparationWarning derived from ModelWarning. Behaviour in a given release (error or warning) was not tested here.

Tilmann Gneiting, Adrian E. Raftery · Journal of the American Statistical Association 102(477), 359-378 · 2007

Theory of (strictly) proper scoring rules including CRPS, log score and interval/quantile scores; Day 26 probabilistic forecast evaluation.

Checked 2026-10-11: Title, authors, JASA 102(477) pp. 359-378, March 2007 in Crossref DOI metadata.

Andrew Gelman · Statistical Science 22(2), 153-164 · 2007

Checked 2026-10-12: arXiv abstract page (0710.5005, submitted 26 Oct 2007): title, author, Statistical Science 22(2) 153-164. Abstract: models of survey responses should account for the variables that drive inclusion, nonresponse, weighting and clustering; this can require thousands of poststratification cells and general multilevel models are hard to build. Abstract-level reading only.

Ronald L. Wasserstein, Nicole A. Lazar · The American Statistician 70(2), 129-133 · 2016

The ASA's six principles on what p-values do and do not mean; cite it for correct interpretation of significance tests.

Checked 2026-10-11: Title, authors, The American Statistician vol. 70 issue 2, pp. 129-133, published online 09 Jun 2016, free access, on the Taylor & Francis landing page (opened in a browser; plain HTTP fetch was blocked).

Yoav Benjamini, Daniel Yekutieli · Annals of Statistics 29(4), 1165-1188 · 2001

Checked 2026-10-12: Project Euclid abstract via search results: title, authors, Ann. Statist. 29(4) 1165-1188. The Benjamini-Hochberg procedure controls FDR under positive regression dependence on the true nulls; a conservative modification controls it under any dependence. Abstract-level reading only.

Hans R. Künsch · The Annals of Statistics 17(3) · 1989

Block bootstrap for dependent (time-ordered) observations; Project 004.

Checked 2026-10-11: Title, author, Annals of Statistics 17(3), 1989 in Crossref DOI metadata.

Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven, David T. Mellor · Proceedings of the National Academy of Sciences 115(11), 2600-2606 · 2018

Checked 2026-10-12: Search results (PMC6233068 and other indexes): authors, PNAS 115(11) 2600-2606, DOI. Abstract-level reading: hindsight bias makes it hard to separate postdiction from prediction; defining research questions and the analysis plan before observing outcomes (preregistration) helps, and strategies exist when data are pre-existing. Full text not read.

Galit Shmueli · Statistical Science 25(3), from p. 289 · 2010

Explains why explanatory and predictive modeling need different choices of variables, validation and evaluation; core for problem framing.

Checked 2026-10-11: Title, author, Statistical Science vol. 25 issue 3, first page 289, 2010/08 and DOI in citation metadata on the Project Euclid landing page.

John P. A. Ioannidis · PLOS Medicine 2, e124 · 2005

Shows how low prior probability, small effects, flexible analyses and bias raise the false-positive share of findings; motivates multiple-testing and pre-registration discipline.

Checked 2026-10-11: Title, author, PLOS Medicine vol. 2, article e124, Aug 30 2005 and DOI in citation metadata on the PLOS landing page.

Andrew Gelman, Jennifer Hill, Masanao Yajima · arXiv:0907.2478 (stat.AP); preprint · 2009

Checked 2026-10-12: arXiv abstract page: title, three authors, submitted 15 Jul 2009. Abstract: argues the multiple-comparisons problem largely dissolves in a hierarchical (multilevel) framework that partially pools estimates, which is more efficient, whereas classical procedures keep estimate centres fixed and widen intervals.

Peter van der Putten, Maarten van Someren · Machine Learning 57(1-2), 177-195 · 2004

Analyses why challenge entries on the COIL 2000 data varied, framed as bias and variance; Project 014.

Checked 2026-10-11: Title, authors, Machine Learning 57(1-2) pp. 177-195 in Crossref DOI metadata.

Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu · Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96), AAAI Press · 1996

DBSCAN: density-based clusters of arbitrary shape with explicit noise points, parameters eps and MinPts; Day 14.

Checked 2026-10-11: Title, authors and 'KDD-96 Proceedings, Copyright 1996 AAAI' header read from the PDF on AAAI's CDN. aaai.org HTML landing page blocked by Cloudflare; page range not confirmed, omitted.

Eibe Frank, Mark Hall · Machine Learning: ECML 2001 (Lecture Notes in Computer Science), pp. 145-156 · 2001

Checked 2026-10-12: Crossref DOI record: title, authors, ECML 2001 LNCS, pp. 145-156. Metadata only; abstract and full text not opened.

Scott M. Lundberg, Su-In Lee · Advances in Neural Information Processing Systems 30 (NeurIPS 2017) · 2017

SHAP: additive feature attributions based on Shapley values, unifying LIME/DeepLIFT-style methods; Day 17 local explanations.

Checked 2026-10-11: Title, authors, volume 30, 2017 in NeurIPS proceedings abstract page metadata; arXiv 1705.07874 matches.

Marco Saerens, Patrice Latinne, Christine Decaestecker · Neural Computation 14(1), pp. 21-41 · 2002

Checked 2026-10-11: Crossref metadata for the DOI: title, three authors, Neural Computation vol. 14 issue 1 pp. 21-41, 2002; abstract: posteriors trained under one class prior are not valid under new priors and can be adjusted with an iterative procedure.

LightGBM developers (Microsoft)

Practical rules for passing categorical features natively (integer codes, high-cardinality cautions) and how missing values are handled.

Checked 2026-10-11: Page title 'Advanced Topics' with sections Missing Value Handle and Categorical Feature Support on LightGBM docs (4.7.0.99, latest).

Aaron Fisher, Cynthia Rudin, Francesca Dominici · Journal of Machine Learning Research 20(177), 1-81 · 2019

Formalizes model reliance (permutation importance) and model class reliance across the Rashomon set of near-optimal models; Day 17.

Checked 2026-10-11: Title, authors, JMLR 20(177) pp. 1-81, 2019 in citation metadata on the JMLR landing page; arXiv 1801.01489 journal-ref agrees.

Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani · Springer · 2021

Same content as the Python edition with R labs; cite it only when the R labs are needed. Prefer book-isl-python for this academy.

Checked 2026-10-11: Title, authors, Springer New York, eBook published 29/30 July 2021 on Springer page; 2nd edition 2021 stated on statlearning.com.

scikit-learn developers

Checked 2026-10-12: Opened 2026-10-12. States the one-standard-error rule as selecting the simplest model whose performance is within one standard error of the best model's, to avoid overfitting; implements it with a custom refit callable in GridSearchCV (code uses the standard deviation of test scores across 30 splits). The example's prose has an inconsistent 0.88 figure; the printed result selects 25 PCA components.

Eleanor Dillon, Jacob LaRiviere, Scott Lundberg, Jonathan Roth, Vasilis Syrgkanis

Checked 2026-10-11: Article in the SHAP docs: SHAP shows associations the model learned, not causal effects; unobserved confounders and redundant features distort or split credit; causal questions need causal designs (double ML for observed confounding, experiments or quasi-experiments otherwise).

Database Group of Prof. Erhard Rahm, University of Leipzig

Project 032 (embedding similarity for product matching).

Checked 2026-10-11: Provider page: Abt-Buy (e-commerce; attributes name, description, manufacturer, price; 1,081 + 1,092 entities; 1,097 matches; each zip has two source files plus a perfect-mapping file); Amazon-GoogleProducts (1,363 + 3,226; 1,300 matches); released under CC BY 4.0 with a request to credit the website and the VLDB 2010 paper. HEAD on /files/datasets/Abt-Buy.zip returned 200, 134,745 bytes; not downloaded, so exact CSV column headers were not inspected.

Sudhir Varma, Richard Simon · BMC Bioinformatics 7, article 91 · 2006

Shows the minimum CV error after tuning is optimistically biased and recommends nested cross-validation; Day 15.

Checked 2026-10-11: Title, authors, BMC Bioinformatics vol. 7, Feb 2006 in Crossref DOI metadata (article number from DOI suffix).

Carolin Strobl, Anne-Laure Boulesteix, Achim Zeileis, Torsten Hothorn · BMC Bioinformatics 8, article 25 · 2007

Impurity-based RF importance is biased toward predictors with many categories / split points; supports caution about default feature_importances_; Day 17.

Checked 2026-10-11: Title, authors, BMC Bioinformatics vol. 8, Jan 2007 in Crossref DOI metadata (article number from DOI suffix).

Andrea Dal Pozzolo, Olivier Caelen, Reid A. Johnson, Gianluca Bontempi · 2015 IEEE Symposium Series on Computational Intelligence (SSCI), 159-166 · 2015

Undersampling shifts predicted probabilities and how to correct them; dataset citation for Project 010.

Checked 2026-10-11: Title, authors, SSCI 2015 proceedings pp. 159-166 in Crossref DOI metadata; OpenML data 1597 asks users to cite this paper (it names the CIDM symposium, part of SSCI 2015).

Yandex / CatBoost developers

Reference for ordered target statistics on categorical features and the training parameters; pair with the CatBoost paper.

Checked 2026-10-11: Page title 'CatBoost' and description 'gradient boosting on decision trees' on catboost.ai/docs.

Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, Andrey Gulin · Advances in Neural Information Processing Systems 31 (NeurIPS 2018) · 2018

Ordered boosting and ordered target statistics to avoid target leakage / prediction shift with categorical features; Day 9 target encoding, Day 12.

Checked 2026-10-11: Title, authors, volume 31, year 2018 in NeurIPS proceedings abstract page metadata; arXiv 1706.09516 also matches.

Christian Hennig · Computational Statistics & Data Analysis 52(1), 258-271 · 2007

Bootstrap (Jaccard) stability of individual clusters; Project 015.

Checked 2026-10-11: Title, author, CSDA 52(1) pp. 258-271 in Crossref DOI metadata.

scikit-learn developers

Checked 2026-10-11: 2.3.2 k-means assumes convex isotropic clusters, needs n_clusters, inertia not normalized, local minima; 2.3.7 DBSCAN eps is crucial (too small -> noise, too large -> merged clusters); 2.3.11.5 silhouette coefficient; 2.3.11.1 adjusted Rand index needs ground truth.

imbalanced-learn developers

Checked 2026-10-11: imbalanced-learn 0.14.2: resampling the whole dataset before splitting or CV leaks and balances the test data; the imblearn Pipeline resamples only during fit so validation folds keep the true class distribution.

scikit-learn developers

Official guidance on leakage through preprocessing, fitting transformers on the full dataset, and controlling randomness; cite it for 'fit on train only, use a Pipeline'.

Checked 2026-10-11: Page title and sections 12.1 Inconsistent preprocessing, 12.2 Data leakage (12.2.1 How to avoid data leakage) on the stable docs (version 1.9.1).

Carolin Strobl, Anne-Laure Boulesteix, Thomas Kneib, Thomas Augustin, Achim Zeileis · BMC Bioinformatics 9, article 307 · 2008

Checked 2026-10-12: Opened PubMed Central PMC2491635 2026-10-12: title, authors, BMC Bioinformatics 9:307, 11 July 2008. Abstract: random forest importance measures are biased towards correlated predictors, partly because of the unconditional permutation scheme; a conditional permutation scheme is proposed whose importance reflects each predictor's true impact more reliably.

Chris Drummond, Robert C. Holte · Machine Learning 65, pp. 95-130 · 2006

Checked 2026-10-12: Crossref DOI record (title, authors, Machine Learning vol. 65 pp. 95-130, 2006) plus the full text of the authors' November 2005 preprint (www.site.uottawa.ca/~cdrummon/pubs/ML_2006.pdf), read 2026-10-12. Abstract: cost curves visualise the error rate or expected cost of 2-class classifiers over the full range of class distributions and misclassification costs. Section 3: normalized expected cost is Norm(E[Cost]) = FN * PC(+) + FP * PC(-), a straight line in PC(+) = p(+)C(-|+) / (p(+)C(-|+) + p(-)C(+|-)); PC(+) equals p(+) when costs are equal. Section 4.2: the trivial always-negative and always-positive classifiers are the two diagonals, and the operating range of a classifier is the set of PC(+) values where it beats both.

Masashi Sugiyama, Matthias Krauledat, Klaus-Robert Mueller · Journal of Machine Learning Research 8, pp. 985-1005 · 2007

Checked 2026-10-12: JMLR page opened 2026-10-12: title, authors, vol. 8 pp. 985-1005, 2007. Abstract: under covariate shift (input distribution differs between training and test while the conditional distribution of outputs given inputs is unchanged) ordinary cross validation loses its unbiasedness; importance weighted cross validation is proved unbiased under covariate shift; demonstrated on simulations and a brain-computer interface application. Full text not read.

scikit-learn developers

Checked 2026-10-12: Opened the stable User Guide page, grouped-data section: samples from the same group are dependent, so a flexible model can look good by recognising group-specific features; GroupKFold guarantees the same group is never in both the training and the testing set of a split.

scikit-learn developers

Reference for K-fold, stratified, group-aware splits and TimeSeriesSplit (expanding-window folds for temporal data).

Checked 2026-10-11: Page title and section 3.1.2.6.1 Time Series Split (TimeSeriesSplit) on the stable docs (version 1.9.1).

scikit-learn developers

Checked 2026-10-12: Opened 2026-10-12. Predictions are piecewise constant approximations and trees are not good at extrapolation (example: a tree fitted on x in 0..5 predicts the largest training target for x = 10); min_samples_leaf=5 suggested as an initial value, a very small value usually overfits though for classification with few classes 1 is often best; min_samples_leaf guarantees a minimum leaf size; predict_proba is the fraction of training samples of the class in the leaf; min_samples_split ignores sample_weight (use min_weight_fraction_leaf).

Zachary Lipton, Yu-Xiang Wang, Alexander Smola · Proceedings of the 35th International Conference on Machine Learning (ICML), PMLR 80, pp. 3122-3130 · 2018

Checked 2026-10-11: PMLR landing page: title, authors, volume, pages; abstract defines label shift (p(y) changes, p(x|y) fixed) and black box shift estimation that works with a biased or miscalibrated predictor if its confusion matrix is invertible.

Efron, Hastie, Johnstone, Tibshirani (source); packaged by scikit-learn · scikit-learn 1.9.1 documentation, Toy datasets

Project 009 / reference r03 (tabular regression with nested CV).

Checked 2026-10-11: Toy datasets page (scikit-learn 1.9.1) has a 'Diabetes dataset' section; installed package DESCR: 442 patients, 10 baseline variables (age, sex, bmi, bp, s1-s6), target = quantitative disease progression one year after baseline, features mean-centred and scaled, source URL at NCSU and Efron et al. (2004). Files ship inside the package (sklearn/datasets/data/diabetes_*.csv.gz); loaded locally, shape 442 x 11. No license statement found for the data itself.

scikit-learn developers

Checked 2026-10-11: HistGradientBoosting sections: 1.11.1.1.1 early stopping enabled by default above 10,000 samples using validation_fraction; 1.11.1.1.2 native NaN routing learned per split; 1.11.1.1.4 native categorical support; 1.11.1.1.5 monotonic_cst per feature, constraints are marginal only, not supported for multiclass or categorical features.

Hanna Köpcke, Andreas Thor, Erhard Rahm · Proceedings of the VLDB Endowment 3(1) (VLDB 2010) · 2010

Checked 2026-10-11: Leipzig publication page: authors Köpcke, Thor, Rahm; title; PVLDB volume 3 issue 1, VLDB 2010; camera-ready PDF linked. Page numbers and DOI not shown on the page.

Alexandra Chouldechova · Big Data 5(2), 153-163 · 2017

Checked 2026-10-12: arXiv listing 1610.07524 (title, single author, submitted 24 Oct 2016, FATML 2016) and a search record giving Big Data 5(2) 153-163, DOI 10.1089/big.2016.0047. Abstract-level reading: when prevalence of the outcome differs across groups, the constraints of the predictive bias criterion force false negative and false positive rates to differ, and error-rate differences can produce disparate impact. The PDF text layer was not readable here, so theorem numbering was not checked.

Solon Barocas, Moritz Hardt, Arvind Narayanan · MIT Press · 2023

Chapter 3 (Classification) for unawareness and redundant encodings.

Checked 2026-10-12: Book home page: title, three authors, MIT Press, 2023, free online edition. Chapter 3 'Classification' (fairmlbook.org/classification.html) opened: discusses fairness through unawareness; argues that removing a sensitive attribute can be ineffective because many weakly correlated features together act as redundant encodings of it, so a model can recover an equivalent encoding.

scikit-learn developers

Hashing trick and character n-gram vectorizers.

Checked 2026-10-12: Section 8.2.2 Feature hashing: signed hash so collisions tend to cancel, alternate_sign default and use for n_features < 10000, power-of-two n_features advised because a modulo maps hash to column, no inverse_transform, MurmurHash3 32-bit. Section 8.2.3.8: character n-grams resilient against misspellings and derivations; char_wb vs char, char_wb less noisy for whitespace-separated languages.

LightGBM developers (Microsoft)

Explains histogram-based splits, leaf-wise growth and native optimal categorical splits; cite for why LightGBM is fast and how it treats categoricals.

Checked 2026-10-11: Page title and sections 'Leaf-wise (Best-first) Tree Growth' and 'Optimal Split for Categorical Features' on LightGBM docs (4.7.0.99, latest).

Jerome H. Friedman · The Annals of Statistics 29(5) · 2001

Gradient boosting as stagewise functional gradient descent on a loss; also introduces partial dependence plots; Day 12 boosting, Day 17 interpretation.

Checked 2026-10-11: Title, author, Annals of Statistics 29(5), Oct 2001 in Crossref DOI metadata (page range not present in record, omitted).

Aurélien Géron · O'Reilly Media · 2022

Keras/TensorFlow edition; the Scikit-Learn chapters (ch. 2 End-to-End Machine Learning Project, ch. 3 Classification, ch. 7 Ensemble Learning and Random Forests) remain useful. Prefer book-homl-pytorch for new references.

Checked 2026-10-11: Title '3rd Edition', author, October 2022, O'Reilly Media and chapter list on the O'Reilly landing page; homl.info still offers a Keras & TensorFlow track alongside the PyTorch book.

Milos Radovanovic, Alexandros Nanopoulos, Mirjana Ivanovic · Journal of Machine Learning Research 11, 2487-2531 · 2010

Checked 2026-10-12: JMLR page: title, three authors, vol. 11 pp. 2487-2531, 2010. Abstract: as dimensionality grows the distribution of k-occurrences becomes skewed and a few hubs appear among the nearest neighbours of many points; theory and experiments on synthetic and real data; an inherent property of high-dimensional distributions.

scikit-learn developers

Checked 2026-10-11: Section 8.4.6 Marking imputed values: MissingIndicator, add_indicator on SimpleImputer/IterativeImputer, missingness can be informative; 8.4.7 lists estimators that accept NaN natively, including HistGradientBoosting and trees.

Christoph Molnar · 2025

Model-agnostic interpretation done carefully. Most useful: ch. 17 Shapley Values, ch. 18 SHAP, ch. 19 Partial Dependence Plot, ch. 20 Accumulated Local Effects, ch. 23 Permutation Feature Importance.

Checked 2026-10-11: Title, subtitle, author, 'What's new in the 3rd edition' section and chapter list on the book site; 'Citing this Book' page gives Molnar, C. (2025), 3rd ed.

Fei Tony Liu, Kai Ming Ting, Zhi-Hua Zhou · 2008 Eighth IEEE International Conference on Data Mining (ICDM), 413-422 · 2008

Anomalies are isolated in fewer random splits, so short average path length signals an outlier; Day 14 anomaly detection.

Checked 2026-10-11: Title, authors, ICDM 2008 proceedings pp. 413-422 in Crossref DOI metadata.

Sayash Kapoor, Arvind Narayanan · Patterns 4(9), 100804 · 2023

Survey finding leakage in at least 294 studies across 17 fields, with a taxonomy of eight leakage types and 'model info sheets'; good for leakage checklists.

Checked 2026-10-11: Title, authors, Patterns vol. 4 issue 9, article 100804, September 08 2023, open access, on the Cell Press landing page (opened in a browser); matches Crossref DOI metadata.

Shachar Kaufman, Saharon Rosset, Claudia Perlich, Ori Stitelman · ACM Transactions on Knowledge Discovery from Data 6(4), Article 15, 1-21 · 2012

Defines data leakage formally and gives ways to detect and avoid it; core for validation design and feature audits.

Checked 2026-10-11: Title, four authors, TKDD vol. 6 issue 4, article 15, pp. 1-21, published 18 Dec 2012, free access, on the ACM DL landing page (opened in a browser); matches Crossref DOI metadata.

Charles Elkan, Keith Noto · Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 213-220 · 2008

Checked 2026-10-12: Opened the author's PDF (cseweb.ucsd.edu/~elkan/posonly.pdf) 2026-10-12 and read the abstract: under the assumption that labeled examples are selected randomly from the positive examples, a classifier trained on positive and unlabeled examples predicts probabilities that differ by only a constant factor from the true conditional probabilities of being positive. Crossref confirms venue, pp. 213-220, 2008.

Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, Tie-Yan Liu · Advances in Neural Information Processing Systems 30 (NeurIPS 2017) · 2017

Histogram-based boosting with gradient-based one-side sampling (GOSS) and exclusive feature bundling (EFB); Day 12.

Checked 2026-10-11: Title, all eight authors, volume 30, year 2017 in citation metadata of the NeurIPS proceedings abstract page.

scikit-learn developers

Checked 2026-10-11: 1.1.11 Logistic regression note: regularization is applied by default (common in ML, not in statistics); no regularization corresponds to a very large C; solver table marks sag/saga as not robust to unscaled data.

scikit-learn developers

Checked 2026-10-11: 3.4.6.1 R2 is dataset dependent and not meaningfully comparable across datasets, can be negative, constant mean model scores 0; 3.4.6.5 MAPE divides by max(epsilon, |y|) and returns a fraction; 3.4.8 dummy estimators as sanity baselines; 3.4.4.18 Brier score is strictly proper and mixes calibration, resolution and uncertainty; 3.4.4.4 balanced accuracy.

Olivier Chapelle · Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2014) · 2014

Checked 2026-10-12: Bibliographic data (single author, KDD 2014, pages 1097-1105, DOI 10.1145/2623330.2623634) and the described method (probabilistic delay model deciding whether a not-yet-converted user is a negative example or is excluded from training) confirmed from a web search and the Semantic Scholar entry; the ACM page returned HTTP 403, so the full text was not read.

XGBoost developers

Checked 2026-10-11: monotone_constraints with 1/-1/0 per feature or keyed by feature name; with hist/approx the constraint can remove candidate splits and give shallow trees, raising max_bin may help.

scikit-learn developers

Checked 2026-10-11: Example explains that non-nested CV tunes and evaluates on the same data and is optimistically biased; nested CV wraps GridSearchCV in cross_val_score; cites Cawley and Talbot (2010).

XGBoost developers

Checked 2026-10-11: Handle Imbalanced Dataset: if only ranking/AUC matters, balance with scale_pos_weight; if predicted probabilities must be right, do not rebalance and set max_delta_step to a finite value; overfitting control section.

scikit-learn developers

Checked 2026-10-11: Outlier vs novelty detection; contamination parameter sets the threshold applied to raw scores (for LOF, a percentile of training scores); IsolationForest path-length scoring.

Gavin C. Cawley, Nicola L. C. Talbot · Journal of Machine Learning Research 11(70), 2079-2107 · 2010

Tuning hyperparameters on the same CV estimate used for reporting yields optimistic bias; model selection itself can overfit; Day 15.

Checked 2026-10-11: Title, authors, JMLR 11 issue 70 pp. 2079-2107, 2010 in citation metadata on JMLR landing page.

E. Alpaydin, C. Kaynak · 1998

Provenance of the scikit-learn digits copy; the writer split matters for generalisation claims.

Checked 2026-10-11: UCI page: title, creators Alpaydin and Kaynak, donated 1998-06-30, 5,620 instances, 64 integer features, CC BY 4.0; 32x32 bitmaps divided into non-overlapping 4x4 blocks with on-pixel counts giving an 8x8 matrix of 0-16; 30 writers in the training set and 13 different writers in the test set.

LightGBM developers (Microsoft)

Checked 2026-10-11: is_unbalance and scale_pos_weight entries warn that they can improve the overall metric but give poor estimates of individual class probabilities; monotone_constraints and monotone_constraints_method (basic over-constrains, intermediate, advanced); early_stopping_round and first_metric_only; min_data_per_group, cat_smooth, max_cat_to_onehot.

LightGBM developers (Microsoft)

Checked 2026-10-12: Opened 2026-10-12 (added by the insights-G reviewer; the claim in insight-1141 sits on this page, not on the Parameters page). Section 'Tune Parameters for the Leaf-wise (Best-first) Tree': setting num_leaves = 2^max_depth is called not good in practice because a leaf-wise tree is typically much deeper than a depth-wise tree for a fixed number of leaves; num_leaves should be smaller than 2^max_depth; min_data_in_leaf is 'a very important parameter to prevent over-fitting in a leaf-wise tree', a large value avoids growing too deep a tree but may underfit, and hundreds or thousands is enough for a large dataset; with max_depth set, also set num_leaves <= 2^max_depth.

scikit-learn developers

Checked 2026-10-11: PDP and ICE assume the features of interest are independent of the complement features and can create unrealistic points when correlated; ICE shows heterogeneity the PDP average hides; kind='both' and centered=True options.

Alex Goldstein, Adam Kapelner, Justin Bleich, Emil Pitkin · Journal of Computational and Graphical Statistics 24(1), 44-65 · 2015

ICE plots show per-observation prediction curves, revealing heterogeneity/interactions that an averaged partial dependence plot hides; Day 17.

Checked 2026-10-11: Title, authors, JCGS 24(1) pp. 44-65, 2015 in Crossref DOI metadata.

scikit-learn developers

Shared record: also cited by project batches after their duplicate was removed.

Checked 2026-10-11: Warning that importances of a poorly scoring model are not meaningful; 5.2.2 impurity-based (MDI) importance is computed on training data and biased toward high-cardinality features; 5.2.3 correlated features lower each other's permutation importance, suggested fix is clustering correlated features; held-out vs training set computation.

Curtis G. Northcutt, Anish Athalye, Jonas Mueller · NeurIPS 2021 Datasets and Benchmarks Track (per arXiv journal reference) · 2021

Label noise in evaluation data (days 18, 41).

Checked 2026-10-11: arXiv 2103.14749 confirms title and authors; journal_ref NeurIPS 2021 Track on Datasets and Benchmarks. Abstract: an estimated average of at least 3.3% test-label errors across 10 common datasets; lower-capacity models can be preferable when a test set has many label errors.

scikit-learn developers

Pipeline and ColumnTransformer usage, so that preprocessing is fit inside cross-validation folds and applied per column type.

Checked 2026-10-11: Page title and section 8.1.4 ColumnTransformer for heterogeneous data on the stable docs (version 1.9.1).

Alexandru Niculescu-Mizil, Rich Caruana · Proceedings of the 22nd International Conference on Machine Learning (ICML 2005), 625-632 · 2005

Compares calibration of common learners (e.g., boosted trees, SVMs distort probabilities) and Platt scaling vs isotonic regression; Day 13.

Checked 2026-10-11: Title, authors, ICML '05 proceedings pp. 625-632, 2005 in Crossref DOI metadata.

scikit-learn developers

Scaling, sparse scaling, one-hot options, infrequent categories, polynomial and spline features.

Checked 2026-10-11: scikit-learn 1.9.1 page 8.3: 8.3.1 standardization note that RBF-kernel SVMs and l1/l2 regularizers of linear models assume centered, similarly scaled features; 8.3.1.2 centering sparse data destroys sparsity, MaxAbsScaler recommended, StandardScaler needs with_mean=False; 8.3.1.3 RobustScaler for data with outliers; 8.3.4 OneHotEncoder drop and collinearity in non-regularized regression, handle_unknown; 8.3.4.1 min_frequency / max_categories infrequent categories; 8.3.7 PolynomialFeatures interaction_only, SplineTransformer has no interactions.

scikit-learn developers

Reliability diagrams, sigmoid (Platt) versus isotonic calibration and CalibratedClassifierCV; cite for when predicted probabilities need to be trusted.

Checked 2026-10-11: Page title, section 1.16.1 Calibration curves and CalibratedClassifierCV on the stable docs (version 1.9.1).

George Forman · Data Mining and Knowledge Discovery 17, pp. 164-206 · 2008

Checked 2026-10-12: Crossref DOI record: title, author, DMKD vol. 17 pp. 164-206, published June 2008. Metadata only; abstract and full text not opened (publisher page blocked automated access).

Leo Breiman · Machine Learning 45(1), 5-32 · 2001

Introduces random forests (bagged trees with random feature subsets), out-of-bag error and permutation-style variable importance; Day 12.

Checked 2026-10-11: Title, author, Machine Learning 45(1) pp. 5-32, Oct 2001 in Crossref DOI metadata.

James Bergstra, Yoshua Bengio · Journal of Machine Learning Research 13, pp. 281-305 · 2012

Checked 2026-10-12: Opened JMLR page 2026-10-12: title, authors, vol. 13 no. 10 pp. 281-305, 2012. Abstract: randomly chosen trials are more efficient than trials on a grid; random search matched or beat grid-configured neural networks in a fraction of the computation time; experiments are neural networks and deep belief networks, and random search is proposed as a natural baseline for adaptive methods.

C. Okan Sakar, S. Olcay Polat, Mete Katircioglu, Yomi Kastro · Neural Computing and Applications 31(10), 6893-6908 · 2019

Dataset paper for Online Shoppers Purchasing Intention; Project 006.

Checked 2026-10-11: Title, authors, Neural Computing and Applications 31(10) pp. 6893-6908 in Crossref DOI metadata (issued online 2018).

scikit-learn developers

Projects 034 (training dynamics, reference r07) and 035 (debugging transfer check).

Checked 2026-10-11: API page (scikit-learn 1.9.1): 1,797 samples, 10 classes, 64 features (8x8 images), integer values 0-16; page states it is a copy of the test set of the UCI handwritten digits data. Executed locally: load_digits() returns (1797, 64) with values 0-16 and about 180 per class; the file digits.csv.gz ships inside sklearn/datasets/data, so no download is needed.

scikit-learn developers

Checked 2026-10-11: scikit-learn 1.9.1 docs section 8.3.4.2: fit_transform uses cross fitting (cv folds, default 5); fit(X, y).transform(X) on the same data does not and can leak; shrinkage toward the global mean controlled by 'smooth'; unseen categories get the global target mean.

Yonatan Geifman, Ran El-Yaniv · arXiv:1705.08500 (cs.LG) · 2017

The coverage versus risk trade-off behind abstention thresholds; supports day 51 and 13.

Checked 2026-10-12: arXiv abstract page: title, two authors, May 2017. Abstract: a trained network can be wrapped in a reject option so that it meets a user-set risk level by rejecting inputs, trading coverage for lower error; one result is 2% top-5 ImageNet error with 99.9% probability at nearly 60% coverage. Image classification; the page does not use the phrase risk-coverage curve.

SHAP developers

Checked 2026-10-11: model_output defaults to 'raw' (log-odds for XGBoost binary classification); 'probability' requires feature_perturbation='interventional' with background data; tree_path_dependent uses leaf counts as the background.

Jesse Dodge, Suchin Gururangan, Dallas Card, Roy Schwartz, Noah A. Smith · arXiv:1909.03004 (cs.LG); EMNLP 2019 per the paper's own venue, not shown on the arXiv page · 2019

Supports reporting how many variants or trials were tried alongside the best score.

Checked 2026-10-12: arXiv abstract page (title, 5 authors, v1 6 Sep 2019). Abstract: test scores alone cannot reliably show which model is best; the authors propose reporting expected validation performance of the best model as a function of the budget (hyperparameter trials or training time) and show conclusions that change with more or less compute.

Peter J. Rousseeuw · Journal of Computational and Applied Mathematics 20, 53-65 · 1987

Defines the silhouette width for judging cluster cohesion vs separation and choosing k; Day 14.

Checked 2026-10-11: Title, author, J. Comput. Appl. Math. vol. 20 pp. 53-65, Nov 1987 in Crossref DOI metadata.

scikit-learn developers

Checked 2026-10-12: Opened 2026-10-12. score_samples is documented as the opposite of the anomaly score defined in the original paper, so lower means more abnormal; decision_function = score_samples - offset_, negative values are outliers; with contamination='auto' offset_ is -0.5 as in the paper, with a float contamination offset_ is set so that the expected share of training samples with a negative decision function equals it.

scikit-learn developers

Checked 2026-10-12: discrete_features default 'auto' (False for dense X, True for sparse X); warning that treating continuous as discrete or vice versa will usually give incorrect results; nonparametric k-nearest-neighbour entropy estimation (Kraskov, Ross); n_neighbors default 3 (higher reduces variance, may add bias); random_state controls noise added to continuous variables to remove repeated values.

scikit-learn developers

Checked 2026-10-12: features parameter: 'missing-only' (default) covers only features with missing values at fit, 'all' covers every feature; error_on_new applies only with 'missing-only' and, if True, raises at transform for features with missing values that had none in fit.

scikit-learn developers

Checked 2026-10-12: Opened 2026-10-12. lasso_path computes the Lasso regularization path with coordinate descent and returns the alphas and the coefficients for every alpha (coefs of shape n_features x n_alphas); alphas are set automatically when None, with eps = alpha_min / alpha_max.

scikit-learn developers

Checked 2026-10-12: Opened 2026-10-12. AP summarises a precision-recall curve as the weighted mean of precisions at each threshold, weighted by the increase in recall from the previous threshold (sum over n of (R_n - R_{n-1}) P_n). The description says the implementation is not interpolated and differs from computing the area under the precision-recall curve with the trapezoidal rule, which uses linear interpolation and can be too optimistic.

scikit-learn developers

Checked 2026-10-12: Opened the page: Cohen's kappa measures agreement between two annotators on a classification problem, as (p_o - p_e) / (1 - p_e) with p_e the agreement expected by chance; range -1 to 1, 0 means no better than chance; symmetric in its two arguments.

scikit-learn developers

Checked 2026-10-12: Opened 2026-10-12. k restricts the ranking to the highest k scores; ties in y_score are averaged by default (ignore_ties=False); ignore_ties=True assumes no ties and gives wrong results if ties exist; docs example: ties averaged gives 0.75 against 0.5 with ignore_ties=True.

scikit-learn developers

Checked 2026-10-12: Opened 2026-10-12. Inputs are (latitude, longitude) pairs in radians, two columns; the result is the angular great-circle distance, and the page example converts degrees with math.radians for two airports and multiplies by an Earth radius of 6371 km to get kilometres (about 11,100 km between Buenos Aires and Paris in that example).

scikit-learn developers

Checked 2026-10-12: Opened 2026-10-12. groups gives group labels used while splitting and is only used with a Group cv instance such as GroupKFold; train_sizes are fractions of the maximum training-set size or absolute counts.

scikit-learn developers

Checked 2026-10-12: n_bins_ attribute: bins whose width is too small (<= 1e-8) are removed with a warning; strategy quantile, quantile_method, encode onehot / onehot-dense / ordinal, subsample default 200000.

scikit-learn developers

Checked 2026-10-12: clip parameter (default False, added 0.24) clips transformed held-out values to feature_range, inverse_transform may not restore data; note that clip=True does not prevent feature drift; note that MinMaxScaler does not reduce the effect of outliers but linearly scales them into a fixed range.

scikit-learn developers

Checked 2026-10-12: cv (default 5): for int/None, KFold if target_type continuous else StratifiedKFold; custom splitters or iterables allowed since 1.9 but each sample index must appear in a validation fold exactly once, otherwise ValueError; shuffle (default True, deprecated since 1.9); smooth default auto (empirical Bayes); fit_transform uses cross fitting and fit(X, y).transform(X) does not equal it; unseen categories get the target mean.

scikit-learn developers

Checked 2026-10-12: Opened 2026-10-12. class_weight: 'Set the parameter C of class i to class_weight[i]*C'; 'balanced' uses n_samples / (n_classes * np.bincount(y)); fit(sample_weight) rescales C per sample; regularization strength is inversely proportional to C.

N. V. Chawla, K. W. Bowyer, L. O. Hall, W. P. Kegelmeyer · Journal of Artificial Intelligence Research 16, 321-357 · 2002

Introduces SMOTE (interpolated synthetic minority examples); cite alongside van den Goorbergh 2022 for its effect on calibration; Day 13.

Checked 2026-10-11: Title, authors, JAIR vol. 16 pp. 321-357, 2002 in Crossref DOI metadata.

Alexandros Kalousis, Julien Prados, Melanie Hilario · Knowledge and Information Systems 12(1), pp. 95-116 · 2007

Selection stability measures; Day 17.

Checked 2026-10-12: Title, authors, journal, volume, pages and DOI confirmed from search-result records of the open-access repository copy (doc.rero.ch) and a later JMLR paper citing it; abstract summary (stability as sensitivity of feature preferences to training-data changes, comparison of stability measures, stability profiles to choose an algorithm, proteomics experiments). Full text not read.

scikit-learn developers

Checked 2026-10-12: Opened 2026-10-12. Fold-wise scores of models evaluated on the same 10x10-fold partitions are correlated; treating them as independent underestimates the variance of the test statistic and inflates false positives (Dietterich 1998); corrected resampled t-test of Nadeau and Bengio multiplies the naive variance term 1/(k*r) by (1/(k*r) + n_test/n_train); the worked example gives an uncorrected t of 2.611 (p = 0.005) against a corrected t of 0.750 (p = 0.227).

statsmodels developers

Checked 2026-10-12: Opened 2026-10-12. The Binomial family accepts a 1d array of proportions as endog, with the number of trials supplied through var_weights; the default link is the logit link. The page does not itself discuss fractional-response inference.

statsmodels developers

Checked 2026-10-12: Opened 2026-10-12. Ordinal model based on a logistic or normal distribution; the logistic case corresponds to the proportional odds model; the endogenous variable is assumed ordered but its labels have no numeric interpretation besides the ordering; a latent linear variable is observed only through a discretisation at cut points; K levels need K-1 thresholds besides the slope coefficients.

Ravid Shwartz-Ziv, Amitai Armon · Information Fusion 81, 84-90 · 2022

Tabular deep learning vs XGBoost (days 12, 35).

Checked 2026-10-11: Crossref record for DOI 10.1016/j.inffus.2021.11.011 confirms title, authors, Information Fusion vol 81 pp 84-90, 2022. Abstract (arXiv 2106.03253): XGBoost outperformed the proposed deep tabular models across datasets, needed much less tuning, and an ensemble of both did best.

Stefan Axelsson · ACM Transactions on Information and System Security 3(3), pp. 186-205 · 2000

Checked 2026-10-11: Crossref metadata for the DOI: title, author, journal, vol. 3 issue 3, pp. 186-205, August 2000; abstract: because of the base-rate fallacy the false alarm rate limits detector usefulness, P(intrusion | alarm) needs a very low false alarm rate.

I-Cheng Yeh, Che-hui Lien · Expert Systems with Applications 36(2), 2473-2480 · 2009

Dataset paper; argues estimated default probability matters more than a binary label; Project 011.

Checked 2026-10-11: Title, authors, ESWA 36(2) pp. 2473-2480 in Crossref DOI metadata; cited as the intro paper on the UCI page.

Trevor Hastie, Robert Tibshirani, Jerome Friedman · Springer · 2009

Deeper theory behind ISL. Most useful: Model Assessment and Selection (bias-variance, cross-validation, bootstrap), Linear Methods for Regression, Boosting and Additive Trees, Random Forests.

Checked 2026-10-11: Title, subtitle, 'Second Edition', three authors, Springer New York, 2009 and chapter list on the Springer landing page. The authors' free-PDF site (hastie.su.domains/ElemStatLearn) refused connections at check time, so it is not linked.

Charles Elkan · Proceedings of the 17th International Joint Conference on Artificial Intelligence (IJCAI 2001), pp. 973-978 · 2001

Checked 2026-10-11: ML Anthology entry: title, author, IJCAI 2001, pages 973-978; abstract recommends training on the data as given and making decisions by applying the cost matrix to probability estimates; rebalancing has little effect on Bayesian and tree learners.

Ruben van den Goorbergh, Maarten van Smeden, Dirk Timmerman, Ben Van Calster · Journal of the American Medical Informatics Association 29(9), 1525-1534 · 2022

Random over/undersampling and SMOTE led to strongly miscalibrated (overestimated) risks without improving discrimination in their study; Day 13.

Checked 2026-10-11: Title, authors, JAMIA 29(9) pp. 1525-1534, 2022 in Crossref DOI metadata.

Takaya Saito, Marc Rehmsmeier · PLOS ONE 10(3), e0118432 · 2015

PR curves reflect performance on the minority class under imbalance where ROC can look optimistic; Day 13 and Day 16 metric choice.

Checked 2026-10-11: Title, authors, PLOS ONE 10(3) e0118432, March 2015 in Crossref DOI metadata.

Jesse Davis, Mark Goadrich · Proceedings of the 23rd International Conference on Machine Learning (ICML 2006), pp. 233-240; University of Wisconsin-Madison CS Technical Report TR1551 · 2006

Checked 2026-10-12: Technical-report record page opened 2026-10-12 (TR1551, University of Wisconsin-Madison). Abstract: a curve dominates in ROC space if and only if it dominates in precision-recall space; the authors give an efficient algorithm for the achievable PR curve; PR space differs from ROC space in that points must not be linearly interpolated; optimising area under the ROC curve does not guarantee optimising area under the PR curve. ICML pages and DOI taken from search metadata only; full text not read.

Himabindu Lakkaraju, Jon Kleinberg, Jure Leskovec, Jens Ludwig, Sendhil Mullainathan · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2017), 275-284 · 2017

Checked 2026-10-12: Full text opened (author copy at Cornell, text extracted). Abstract and introduction: outcomes are observed only for instances a decision-maker selected, so labelled instances are not a random sample; the contraction technique evaluates a model against human decision-makers by using the heterogeneity of decision-makers, under the assumption that cases are as if randomly assigned across them; demonstrated on criminal justice, health care and insurance data. Venue, pages and DOI 10.1145/3097983.3098066 from a bibliographic record.

scikit-learn developers

Post-hoc threshold choice from a business cost; Projects 008, 010, 012.

Checked 2026-10-11: Page title in scikit-learn 1.9.1 docs; documents TunedThresholdClassifierCV and FixedThresholdClassifier and links a cost-sensitive learning example.

scikit-learn developers

Checked 2026-10-12: Opened 2026-10-12. 3.2.2: randomized search budget is chosen independent of the number of parameters, adding parameters that do not influence performance does not decrease efficiency; loguniform is the continuous version of a log-spaced parameter; 3.2.3: successive halving (HalvingGridSearchCV, HalvingRandomSearchCV) is experimental, candidates start with few resources and only the best survive, min_resources should be high if many samples are needed to tell good from bad candidates.

Thorsten Joachims, Adith Swaminathan, Tobias Schnabel · arXiv:1608.04468 (cs.IR) · 2016

Checked 2026-10-12: arXiv abstract page opened 2026-10-12: title, three authors, submitted 16 Aug 2016. Abstract: implicit feedback such as clicks is plentiful but biased by position (higher-ranked results attract more clicks), so training on click data directly is suboptimal; the authors give a counterfactual framework with propensity weighting and a Propensity-Weighted Ranking SVM. Publication venue other than arXiv was not checked and is not recorded.

Shai Shalev-Shwartz, Shai Ben-David · Cambridge University Press · 2014

Learning-theory reference for why generalization works. Most useful: ch. 2 A Gentle Start, ch. 5 The Bias-Complexity Trade-off, ch. 6 The VC-Dimension, ch. 11 Model Selection and Validation, ch. 14 Stochastic Gradient Descent.

Checked 2026-10-11: Title, two authors, Cambridge University Press, publication date 2014/05, DOI and chapter list on Cambridge Core. The authors' huji.ac.il page did not respond at check time.

scikit-learn developers

Checked 2026-10-11: 3.5.1 validation curve under/overfitting patterns and the warning that a validation score used for tuning is biased; 3.5.2 learning curve: more data helps while a train-validation gap remains open, not when both converge to a low score.

Daniel W. Apley, Jingyu Zhu · Journal of the Royal Statistical Society Series B 82(4), 1059-1086 · 2020

Accumulated local effects (ALE) plots avoid PDP extrapolation into unlikely regions when features are correlated; Day 17.

Checked 2026-10-11: Title, authors, JRSS-B 82(4) pp. 1059-1086, 2020 in Crossref DOI metadata; arXiv 1612.08468 title matches.

Leo Grinsztajn, Edouard Oyallon, Gael Varoquaux · 2022

GBM baseline before neural nets on tabular data (days 12, 35).

Checked 2026-10-11: arXiv 2207.08815 confirms title, authors and July 2022 date. Abstract: on 45 tabular datasets, tree-based models remain state of the art on medium-sized data (about 10K samples) even before accounting for speed. Venue not recorded here.

XGBoost developers (DMLC)

Reference for parameters, early stopping, categorical support and the Python API.

Checked 2026-10-11: Page title and description 'optimized distributed gradient boosting library' on XGBoost 3.4.2 stable docs.

XGBoost contributors

Checked 2026-10-12: Opened 2026-10-12. min_child_weight (default 1) is the minimum sum of instance weight (hessian) needed in a child; the page notes that in linear regression this corresponds to a minimum number of instances per node; larger values make the algorithm more conservative. max_cat_to_onehot: below the threshold one-hot-based splits, otherwise categories are partitioned.

Tianqi Chen, Carlos Guestrin · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2016), 785-794 · 2016

Regularized second-order tree boosting with sparsity-aware split finding and systems optimizations; Day 12.

Checked 2026-10-11: Authors, KDD 2016 proceedings, pp. 785-794, Aug 2016 in Crossref DOI metadata (Crossref title field shows 'XGBoost'; full title is the published paper title).

Larry R. Weatherford, Sheryl E. Kimes · International Journal of Forecasting 19(3), 401-415 · 2003

Hotel arrivals forecasting with Choice and Marriott data: pickup and regression had lowest error on Choice data; exponential smoothing, pickup and moving average were most robust on Marriott data (abstract); Day 26/28 revenue-management case.

Checked 2026-10-11: Title, author surnames (Weatherford, Kimes), IJF 19(3) pp. 401-415, July 2003 in Crossref bibliographic search result for this DOI.

Ivo Welch, Amit Goyal · Review of Financial Studies 21(4), 1455-1508 · 2008

Limits of financial return forecastability; Day 21/25 insights.

Checked 2026-10-11: Title, authors, RFS 21(4) pp. 1455-1508 in Crossref DOI metadata; RePEc abstract: proposed equity-premium predictors performed poorly both in-sample and out-of-sample over about 30 years, appeared unstable, and would not have helped a real-time investor time the market.

Christoph Bergmeir, Rob J. Hyndman, Bonsoo Koo · Computational Statistics & Data Analysis 120, 70-83 · 2018

Standard K-fold CV is valid for purely autoregressive models when residuals are uncorrelated; nuance to 'never shuffle time series'; Day 25.

Checked 2026-10-11: Title, authors, CSDA vol. 120 pp. 70-83, Apr 2018 in Crossref DOI metadata.

Souhaib Ben Taieb, Gianluca Bontempi, Amir F. Atiya, Antti Sorjamaa · Expert Systems with Applications 39(8), 7067-7083 · 2012

Direct vs recursive vs multi-output strategies; Day 24. The finding is for NN5 daily cash-withdrawal series and the models studied.

Checked 2026-10-11: Title, authors, ESWA 39(8) pp. 7067-7083 in Crossref DOI metadata; arXiv 1108.3259 abstract: 111 NN5 series, multiple-output strategies performed best, deseasonalization uniformly improved accuracy.

Rob J. Hyndman, Anne B. Koehler, Ralph D. Snyder, Simone Grose · International Journal of Forecasting 18(3), 439-454 · 2002

Exponential smoothing as innovations state space models (ETS), enabling likelihood, AIC model selection and prediction intervals; Day 23.

Checked 2026-10-11: Title, authors, IJF 18(3) pp. 439-454, July 2002 in Crossref DOI metadata.

Rob J. Hyndman, Anne B. Koehler · International Journal of Forecasting 22(4), 679-688 · 2006

Critiques MAPE/sMAPE/relative measures and proposes MASE, scaling errors by the in-sample (seasonal) naive MAE; Day 16 and Day 25.

Checked 2026-10-11: Title, authors, IJF 22(4) pp. 679-688, Oct 2006 in Crossref DOI metadata.

Rob J. Hyndman, Yeasmin Khandakar · Journal of Statistical Software 27(3) · 2008

Describes automatic ETS and the auto.arima stepwise order-selection algorithm; Day 23.

Checked 2026-10-11: Title, authors, JSS vol. 27 issue 3, 2008 in Crossref DOI metadata.

Christoph Bergmeir, Rob J. Hyndman, Jose M. Benitez · International Journal of Forecasting 32(2), 303-312 · 2016

Bagged ETS via STL-remainder bootstrap; cite the FPP3 section for the procedure and this paper for the reported M3 result.

Checked 2026-10-12: Title, venue, volume, pages and DOI via IDEAS/RePEc listing returned by search (v32y2016i2p303-312); abstract summary: Box-Cox plus STL, moving-block bootstrap of the remainder, ensemble of exponential smoothing forecasts, consistent gains over the original ETS on the M3 data. Publisher full text not read. FPP3 section 'Bootstrapping and bagging' (otexts.com/fpp3/bootstrap.html) was opened and states the procedure and that bagging gave better forecasts than ETS alone on average, at much higher compute cost.

L. R. Weatherford, S. Pölt · Journal of Revenue and Pricing Management 1(3), 234-254 · 2002

Compares methods for unconstraining capacity-limited airline booking data; supports the claim that observed bookings understate demand when flights close.

Checked 2026-10-11: Title, authors, JRPM 1(3) pp. 234-254, Oct 2002 in Crossref DOI metadata.

Hadi Fanaee-T · UCI Machine Learning Repository · 2013

Hourly and daily rental counts (2011-2012) with calendar and realised-weather columns; used for the exogenous-availability project (weather is observed, not forecast) and the casual + registered = cnt leakage trap.

Checked 2026-10-11: UCI page: title, creator, donation date 12/19/2013, DOI 10.24432/C5W894, files hour.csv (1.1 MB) and day.csv (56.2 KB), 17,389 instances, variable list (dteday, hr, holiday, workingday, weathersit, temp, atemp, hum, windspeed, casual, registered, cnt), license CC BY 4.0.

Francis X. Diebold, Roberto S. Mariano · Journal of Business & Economic Statistics 13(3), 253-263 · 1995

Checked 2026-10-12: Crossref DOI metadata (title, authors, JBES 13(3) pp. 253-263, July 1995) and the RePEc abstract: tests of the null of no difference in accuracy of two forecasts, asymptotic and exact finite-sample, valid for non-quadratic and asymmetric loss and for non-Gaussian, nonzero-mean, serially and contemporaneously correlated forecast errors. Test statistic formula not read (full text paywalled).

Kamile Stankeviciute, Ahmed M. Alaa, Mihaela van der Schaar · Advances in Neural Information Processing Systems 34 (NeurIPS 2021), 6216-6228 · 2021

Conformal intervals for multi-horizon RNN forecasts with Bonferroni-style correction across horizons, assuming independent series; Day 26-27.

Checked 2026-10-11: Title, authors, volume 34, pp. 6216-6228 in NeurIPS proceedings abstract page metadata.

Artur Trindade · UCI Machine Learning Repository · 2015

Multiple-seasonality project (96 and 672 intervals per day/week plus annual cycle); the documented DST artefacts and pre-start zeros are planted-looking but real quality rules.

Checked 2026-10-11: UCI page: title, creator, donation date 3/12/2015, DOI 10.24432/C58C86, 370 clients as columns, 15-minute kW values, Portuguese local time, daylight-saving handling (March: 1:00-2:00 zeros; October: two hours aggregated), clients created after 2011 recorded as zero before start, LD2011_2014.txt 678.1 MB, zip 249.2 MB, license CC BY 4.0.

Johannes Bracher, Evan L. Ray, Tilmann Gneiting, Nicholas G. Reich · PLOS Computational Biology 17(2), e1008618 · 2021

Checked 2026-10-12: Crossref metadata and the PLOS article page: the weighted interval score is a proper score for forecasts given as a median and central intervals, combines sharpness (width) and penalties for observations outside intervals, approximates CRPS with many equally spaced levels and equals absolute error for a median-only forecast; PIT histograms recommended for calibration checks.

Peter F. Christoffersen · International Economic Review 39(4), 841-862 · 1998

Checked 2026-10-12: Crossref DOI metadata (title, author, IER 39(4), Nov 1998) and the RePEc abstract: prior work mostly tested unconditional coverage only; the paper develops a framework for conditional interval forecast evaluation. Specific test names and the full text were not read.

Stephan Kolassa · International Journal of Forecasting 32(3), 788-803 · 2016

Checked 2026-10-12: Crossref metadata and the RePEc abstract: low counts at fine granularity call for full discrete predictive distributions; MAD, MASE and wMAPE are argued to be inherently unsuitable for count data; evaluation uses the randomized PIT and proper scoring rules on two daily retail datasets.

Vitor Cerqueira, Luis Torgo, Igor Mozetič · Machine Learning 109(11), 1997-2028 · 2020

174 real series + synthetic: blocked CV works for stationary series, but for non-stationary series out-of-sample methods (repeated holdout over multiple test periods) gave the most accurate estimates (abstract); Day 25.

Checked 2026-10-11: Title, authors, Machine Learning 109(11) pp. 1997-2028, 2020 in Crossref DOI metadata.

SAS Institute

Checked 2026-10-11: Landing page title and description only: FVA analysis is presented as a way to find waste (steps that do not improve accuracy) in the forecasting process, with guidance on data gathering, analysis and reporting. Full text not read; vendor white paper.

J. D. Croston · Operational Research Quarterly 23(3), 289-303 (now Journal of the Operational Research Society) · 1972

Croston's method: separately smooth nonzero demand sizes and inter-demand intervals for intermittent series; Day 26.

Checked 2026-10-11: Title, author, vol. 23(3) pp. 289-303, Sept 1972 in Crossref DOI metadata (Crossref lists the container as Journal of the Operational Research Society).

Alysha M. De Livera, Rob J. Hyndman, Ralph D. Snyder · Journal of the American Statistical Association 106(496), 1513-1527 · 2011

BATS/TBATS: exponential smoothing with multiple (including non-integer) seasonal periods via trigonometric terms; multiple-seasonality project.

Checked 2026-10-11: Title, authors, JASA 106(496) pp. 1513-1527, Dec 2011 in Crossref DOI metadata.

George Athanasopoulos, Rob J. Hyndman, Nikolaos Kourentzes, Fotios Petropoulos · European Journal of Operational Research 262(1), 60-74 · 2017

Checked 2026-10-12: Bibliographic data (EJOR 262(1) 60-74, DOI 10.1016/j.ejor.2017.02.046) from a library-portal record. Read the 2015 working-paper PDF (MPRA 66362, text extracted): abstract (temporal hierarchy from non-overlapping temporal aggregation; predictions at all levels combined into temporally reconciled forecasts; mitigates modelling uncertainty; accuracy gain over conventional forecasting, particularly under increased modelling uncertainty) and Section 3 (monthly data aggregated at k in {12, 6, 4, 3, 2, 1}; evaluation on 1,428 monthly and 756 quarterly M3 series). The journal text was not opened, so claims are limited to the abstract.

Rob J Hyndman, George Athanasopoulos · OTexts, Melbourne · 2021

Standard forecasting text (R/fable). Most useful: ch. 3 Time series decomposition, ch. 5 The forecaster's toolbox (benchmarks, residuals, accuracy, time-series CV), ch. 8 Exponential smoothing, ch. 9 ARIMA models.

Checked 2026-10-11: Title, authors, 3rd edition, citation 'Hyndman & Athanasopoulos (2021) ... OTexts: Melbourne' and chapter list on the otexts.com landing page.

Fotios Petropoulos, Daniele Apiletti, Vassilios Assimakopoulos, et al. (80 authors) · International Journal of Forecasting 38(3), 705-871 · 2022

Encyclopedic review of forecasting methods and applications; reference map for Days 21-28.

Checked 2026-10-11: Title, full author list (first author Petropoulos), IJF 38(3) pp. 705-871, July 2022 in Crossref DOI metadata.

Taha Aksu, Gerald Woo, Juncheng Liu, Xu Liu, Chenghao Liu, Silvio Savarese, Caiming Xiong, Doyen Sahoo · arXiv:2410.10393 · 2024

Benchmark spanning many datasets, frequencies and horizons with a non-leaking pretraining split; evaluates statistical, DL and foundation models; Day 27.

Checked 2026-10-11: Title, authors and dates via arXiv API record 2410.10393; linked from the official SalesforceAIResearch/gift-eval README. Peer-reviewed venue not confirmed.

Nixtla

Python reconciliation library used in the hierarchical-forecasting project.

Checked 2026-10-11: Docs landing page: lists BottomUp, TopDown, MiddleOut, MinTrace, ERM, probabilistic methods (Normality, Bootstrap, PERMBU, Conformal), temporal hierarchies; Apache 2.0 license section.

Georges Hebrail, Alice Berard

Project 030 (1-D CNN / GRU vs classical day-ahead baselines).

Checked 2026-10-11: UCI page: title, creators Hebrail and Berard, donated 2012-08-29, 2,075,259 instances, 9 variables (Date, Time, Global_active_power kW, Global_reactive_power kW, Voltage V, Global_intensity A, Sub_metering_1/2/3 Wh with kitchen / laundry / water-heater-and-AC descriptions); one-minute sampling, December 2006 to November 2010, one house in Sceaux near Paris; missing values in about 1.25% of rows shown as empty fields between separators; CC BY 4.0; zip 19.7 MB, extracted text file 126.8 MB. Static download URL answered HEAD with 200; not downloaded.

Ruud H. Teunter, Aris A. Syntetos, M. Zied Babai · European Journal of Operational Research 214(3), 606-615 · 2011

TSB method: updates the demand probability every period (not only after a demand), so forecasts decay for items that stop selling; intermittent-demand project.

Checked 2026-10-11: Title, authors, EJOR 214(3) pp. 606-615, Nov 2011 in Crossref DOI metadata.

Spyros Makridakis, Evangelos Spiliotis, Vassilios Assimakopoulos · International Journal of Forecasting 38(4), 1346-1364 · 2022

Official M5 Accuracy results: 42,840 hierarchical Walmart unit-sales series; best-performing methods and findings; Day 24 ML forecasting (check the paper before attributing wins to specific packages).

Checked 2026-10-11: Title, authors, IJF 38(4) pp. 1346-1364, Oct 2022 in Crossref DOI metadata.

Kaggle competition page · Kaggle · 2020

Hierarchical Walmart unit sales (store/state/category/department/item levels; 42,840 series across levels per the M5 results paper). Do not redistribute raw files in a portfolio.

Checked 2026-10-11: Competition page loads (HTTP 200) with title 'M5 Forecasting - Accuracy' and description 'Estimate the unit sales of Walmart retail goods'. Data tab content and rules are not readable without login, so file names, row counts and the data license were NOT confirmed on the page; series counts come from the registered M5 results paper (paper-makridakis-2022-m5-accuracy).

Andrey Davydenko, Robert Fildes · International Journal of Forecasting 29(3), 510-522 · 2013

Checked 2026-10-12: Crossref metadata and the RePEc abstract: recommends aggregating performance ratios across series by the weighted geometric mean, described as treating over- and under-forecasting evenly, symmetric in distribution and robust. The exact ratio definition (relative MAE) was not confirmed from the abstract.

Nixtla

Global ML forecasting with leakage-safe lag construction per series; reference for entity-safe lags.

Checked 2026-10-11: Docs landing page: lag features and lag transforms (rolling/expanding), date features, target transforms (e.g. differences) reversed at prediction, static and exogenous features, scikit-learn-compatible regressors, Dask/Ray/Spark scaling, conformal prediction intervals.

Kasun Bandara, Rob J Hyndman, Christoph Bergmeir · arXiv preprint · 2021

Extends STL to several seasonal periods by iterative decomposition; the MSTL model in statsforecast follows this idea.

Checked 2026-10-11: arXiv export API: title, authors, first version 2021-07-28. Later journal publication not checked.

A. A. Syntetos, J. E. Boylan, J. D. Croston · Journal of the Operational Research Society 56(5), 495-503 · 2005

Source for routing intermittent series between Croston-type methods by interval and size variability; do not state numeric cut-offs from this entry.

Checked 2026-10-12: Title, three authors, JORS 56(5) pp. 495-503, May 2005 in Crossref DOI metadata. Abstract content (classification rules stated in terms of the average inter-demand interval and the squared coefficient of variation of demand sizes, derived by comparing forecasting methods on theoretical error measures and tested on 3,000 real intermittent automotive series) taken from the Salford repository record as returned by a search summary; the publisher and repository pages returned HTTP 403 to direct fetches. The numeric cut-off values often quoted for the scheme were NOT confirmed from a primary source.

Christoph Bergmeir, José M. Benítez · Information Sciences 191, 192-213 · 2012

Empirical comparison of CV, blocked CV and last-block holdout for time series predictor evaluation; Day 25.

Checked 2026-10-11: Title, authors, Information Sciences vol. 191 pp. 192-213, May 2012 in Crossref DOI metadata.

Daqing Chen · UCI Machine Learning Repository · 2019

Transaction log that aggregates to sparse SKU x day/week demand; used for the intermittent-demand project.

Checked 2026-10-11: UCI page: title, creator, donation date 2019-09-20, DOI 10.24432/C5CG6D, 1,067,371 instances, period 01/12/2009-09/12/2011, UK non-store gift-ware retailer with many wholesale customers, variables (InvoiceNo, StockCode, Description, Quantity, InvoiceDate, UnitPrice, CustomerID, Country), invoice numbers starting with 'c' are cancellations, missing values present, license CC BY 4.0.

Rob J. Hyndman, Roman A. Ahmed, George Athanasopoulos, Han Lin Shang · Computational Statistics & Data Analysis 55(9), 2579-2589 · 2011

Forecast all levels independently and reconcile via regression (optimal combination), versus bottom-up/top-down; Day 26.

Checked 2026-10-11: Title, authors, CSDA 55(9) pp. 2579-2589, Sept 2011 in Crossref DOI metadata.

Shanika L. Wickramasuriya, George Athanasopoulos, Rob J. Hyndman · Journal of the American Statistical Association 114(526), 804-819 · 2019

MinT reconciliation: minimizes trace of reconciled forecast error covariance, generalizing OLS/WLS reconciliation; Day 26 hierarchical forecasting.

Checked 2026-10-11: Title, authors, JASA 114(526) pp. 804-819 in Crossref DOI metadata (Crossref issued date 2018-10-26 = online publication).

Shanika L. Wickramasuriya, Berwin A. Turlach, Rob J. Hyndman · Statistics and Computing 30(5), 1167-1182 · 2020

Checked 2026-10-12: Crossref metadata and the author's publication page (robjhyndman.com): MinT and variants do not guarantee non-negative coherent forecasts; non-negativity constraints are added and a quadratic programme solved with three algorithms; tourism study showed slight gains at the most disaggregated level and slight losses at aggregate levels.

Leonard J. Tashman · International Journal of Forecasting 16(4), 437-450 · 2000

Reviews holdout design for forecast evaluation: rolling origins, fixed vs rolling windows, multiple horizons; Day 25.

Checked 2026-10-11: Title, author, IJF 16(4) pp. 437-450, Oct 2000 in Crossref DOI metadata.

Pablo Montero-Manso, Rob J. Hyndman · International Journal of Forecasting 37(4), 1632-1653 · 2021

Global vs local forecasting models; Day 24.

Checked 2026-10-11: Title, authors, IJF 37(4) pp. 1632-1653 in Crossref DOI metadata; arXiv 2008.00444 abstract: global methods are not more restrictive than local ones, keep constant complexity as the number of series grows, and performed well even on heterogeneous datasets.

Tilmann Gneiting, Fadoua Balabdaoui, Adrian E. Raftery · Journal of the Royal Statistical Society Series B 69(2), 243-268 · 2007

Paradigm of maximizing sharpness subject to calibration; PIT histograms as calibration diagnostics; Day 26.

Checked 2026-10-11: Title, authors, JRSS-B 69(2) pp. 243-268, 2007 in Crossref DOI metadata.

Tilmann Gneiting · International Journal of Forecasting 27(2), 197-207 · 2011

Choosing a forecast quantile from asymmetric costs (newsvendor-style decisions); Day 26 decision economics.

Checked 2026-10-11: Title, author, IJF 27(2) pp. 197-207 (2011) in Crossref DOI metadata; abstract on RePEc (ideas.repec.org) states quantiles are optimal point forecasts under generalized piecewise linear losses, which nest asymmetric piecewise linear loss, with the quantile level set by the possibly distinct costs of under- and overprediction.

Salesforce AI Research

Use for leakage-aware comparison of foundation models (MASE/CRPS ranks, test-data-leakage flag); leaderboard positions change, so cite with date; Day 27.

Checked 2026-10-11: GitHub API repo record (Apache-2.0, last push 2026-10-09); README update log (2025-08-05 added TestData Leakage and MASE_Rank columns; 2025-08-25 zero-shot tag; 2025-10-17 replication-code column); leaderboard HF Space huggingface.co/spaces/Salesforce/GIFT-Eval loads (title checked only).

scikit-learn developers

Gap/embargo between training and test windows; Day 25.

Checked 2026-10-11: API page documents gap (samples excluded from the end of each training set before the test set, added in 0.24), test_size and max_train_size, with a worked example.

Nixtla

Day 23 forecasting API used in the lesson code (forecast and cross_validation).

Checked 2026-10-11: Opened the core API page: StatsForecast(models=[...], freq=..., n_jobs=..., verbose=...); forecast(h, df, X_df, level, fitted, prediction_intervals, id_col, time_col, target_col); cross_validation(h, df, n_windows, step_size, test_size, input_size, ...); input needs unique_id, ds and y columns; output has one column per model.

Nixtla

Fast statistical baselines (AutoARIMA, ETS, Theta, seasonal naive) and built-in time-series cross-validation; the library used in FPP Pythonic Way.

Checked 2026-10-11: Page title 'Statistical Forecast - Nixtla' with StatsForecast getting-started, cross-validation and AutoARIMA/ETS tutorial navigation on nixtlaverse.nixtla.io.

Nixtla

Day 23 model names and arguments (season_length) used in the lesson code.

Checked 2026-10-11: Opened the models page (first 100,000 characters): AutoETS(season_length=1, model='ZZZ', damped=None, ...), AutoARIMA(..., seasonal=True, season_length=1, ...), SeasonalNaive(season_length), Naive, RandomWalkWithDrift, HistoricAverage, WindowAverage(window_size), SeasonalWindowAverage; 'ZZZ' lets AutoETS search the error, trend and seasonal components; both AutoETS and AutoARIMA select by AICc by default (AutoARIMA exposes ic='aicc').

statsmodels developers

Day 23 likelihood-based ETS with AIC/AICc, for the AIC-is-not-accuracy demonstration.

Checked 2026-10-11: Opened the ETSModel page (signature ETSModel(endog, error='add', trend=None, damped_trend=False, seasonal=None, seasonal_periods=None, initialization_method='estimated', ...), fit by maximising the log-likelihood) and the linked ETSResults page (attributes aic, aicc, bic, llf, resid; methods forecast, simulate, get_prediction).

statsmodels developers

Day 22 Ljung-Box test of white noise and Day 23 residual checks; remember model_df on fitted-model residuals.

Checked 2026-10-11: Opened the stable-docs page: signature acorr_ljungbox(x, lags=None, boxpierce=False, model_df=0, period=None, return_df=True, auto_lag=False); default lags min(10, nobs // 5); model_df subtracts estimated ARMA parameters from the degrees of freedom; period sets max lag to min(2 * period, nobs // 5); returns a DataFrame with lb_stat and lb_pvalue.

statsmodels developers

Day 22 STL decomposition and seasonal adjustment in Python.

Checked 2026-10-11: Opened the stable-docs page: signature STL(endog, period=None, seasonal=7, trend=None, low_pass=None, seasonal_deg=1, trend_deg=1, low_pass_deg=1, robust=False, seasonal_jump=1, trend_jump=1, low_pass_jump=1); a NumPy array needs an explicit period; robust=True uses a weighted fit less sensitive to outliers; fit() returns a DecomposeResult.

statsmodels developers

Day 22 and Day 23: seasonal ARIMA fit and residual extraction in Python.

Checked 2026-10-11: Opened the SARIMAX class page (signature SARIMAX(endog, exog=None, order=(1, 0, 0), seasonal_order=(0, 0, 0, 0), trend=None, ...); seasonal_order = (P, D, Q, s); simple_differencing discards the first s*D + d rows), the SARIMAX.fit page (documented disp, maxiter, method arguments) and the SARIMAXResults page (attributes resid, aic, aicc, bic, llf; methods forecast, get_forecast, predict).

statsmodels developers

Day 22 sample ACF; note that the confidence intervals use Bartlett's formula, not the flat 1.96/sqrt(n) white-noise band.

Checked 2026-10-11: Opened the stable-docs page: signature acf(x, adjusted=False, nlags=None, qstat=False, fft=True, alpha=None, bartlett_confint=True, missing='none', *, result_object=None); output includes lag 0; confidence intervals use Bartlett's formula; default nlags formula min(10 * log10(nobs), nobs - 1).

statsmodels developers

Day 21 and Day 22: unit-root test paired with KPSS.

Checked 2026-10-11: Opened the stable-docs page: signature adfuller(x, maxlag=None, regression='c', autolag='AIC', store=False, regresults=False, *, result_object=None); null hypothesis of a unit root; regression options c, ct, ctt, n; autolag options AIC, BIC, t-stat, None; ADFullerResult with statistic, pvalue, lags, nobs, critical_values; tuple default scheduled to change.

statsmodels developers

Day 21 stationarity checks; use result_object=True to avoid the deprecated tuple return.

Checked 2026-10-11: Opened the stable-docs page: signature kpss(x, regression='c', nlags='auto', store=False, *, result_object=None); null hypothesis that the series is level or trend stationary; regression 'c' or 'ct'; p-values interpolated from the Kwiatkowski et al. table and a boundary value returned when the statistic is outside it; the default tuple return is deprecated in favour of KPSSResult.

Robert B. Cleveland, William S. Cleveland, Jean E. McRae, Irma Terpenning · Journal of Official Statistics 6(1), 3-73 · 1990

STL decomposes a series into trend, seasonal and remainder via iterated loess, with robustness weights; Day 21-22.

Checked 2026-10-11: Title, all four authors, 'Journal of Official Statistics Vol. 6, No. 1, 1990, pp. 3-73, Statistics Sweden' read from page 1 of the scanned PDF hosted by Statistics Sweden (scb.se). No DOI.

Aris A. Syntetos, John E. Boylan · International Journal of Forecasting 21(2), 303-314 · 2005

Shows Croston's estimator is biased and evaluates the bias-corrected Syntetos-Boylan Approximation (SBA); Day 26 intermittent demand.

Checked 2026-10-11: Title, authors, IJF 21(2) pp. 303-314, April 2005 in Crossref DOI metadata.

J. M. Bates, C. W. J. Granger · Operational Research Quarterly 20(4), 451-468 (now Journal of the Operational Research Society) · 1969

Founding paper showing combined forecasts can beat individual ones; basis for ensembles/combination in forecasting; Day 26.

Checked 2026-10-11: Title, authors, vol. 20(4) pp. 451-468, Dec 1969 in Crossref DOI metadata (container listed as Journal of the Operational Research Society).

Spyros Makridakis, Evangelos Spiliotis, Vassilios Assimakopoulos · International Journal of Forecasting 36(1), 54-74 · 2020

Official M4 results paper: 100,000 series, 61 methods, point forecasts and prediction intervals; top methods and findings by category; benchmark context for Days 23-24 (read the findings section before quoting specific rankings).

Checked 2026-10-11: Title, authors, IJF 36(1) pp. 54-74, Jan 2020 in Crossref DOI metadata.

M Open Forecasting Center (organizers of the M5 competition)

Primary description of the M5 accuracy metric and its dollar-sales weights.

Checked 2026-10-12: Downloaded the 16-page PDF from the Mcompetitions/M5-methods repository and read the Evaluation and Weighting sections: 42,840 series, 28-day horizon, ranking by WRMSSE = sum of weight x RMSSE; weights computed from the cumulative actual dollar sales (units times price) over the last 28 observations of the training sample; 12 aggregation levels, all levels equally weighted. Publication date not printed in the sections read.

Kalyan T. Talluri, Garrett J. van Ryzin · Springer, International Series in Operations Research & Management Science · 2004

Standard RM reference: demand forecasting, unconstraining, capacity control and overbooking; supports aviation/hotel forecasting cases (Days 26-29). Chapter-level mapping not checked.

Checked 2026-10-11: Title, authors, series and publisher (Springer US), 2004 in Crossref DOI metadata.

Robert H. Shumway, David S. Stoffer · Springer Cham (Springer Texts in Statistics) · 2025

Graduate-level time-series theory. Most useful: Characteristics of Time Series, Time Series Regression and Exploratory Data Analysis, ARIMA Models, State-Space Models.

Checked 2026-10-11: Title, authors, '5th edition' in the description, Springer Cham, eBook published 27/28 Jan 2025 and chapter list on the Springer landing page.

statsmodels developers

Reference for ARIMA/SARIMAX, autocorrelation and stationarity tests, decomposition and exponential smoothing in Python.

Checked 2026-10-11: Page title and API listing (AutoReg, ARIMA, stattools, X-13 helpers) on statsmodels 0.15.0 stable docs.

Eurostat · Eurostat database

A recent, regularly updated public panel for evaluating time-series foundation models on months after their release; values are revised between releases (vintage caveat).

Checked 2026-10-11: Eurostat dissemination API (statistics/1.0/data/une_rt_m, JSON): label 'Unemployment by sex and age - monthly data', last update 2026-10-08, dataset DOI 10.2908/UNE_RT_M, monthly coverage 1983-01 to 2026-09 for Germany (total, NSA, % of labour force), 38 geographic entities in the latest period, dimensions freq/s_adj/age/unit/sex/geo/time. No login needed for the API.

Miguel A. Hernan, Sonia Hernandez-Diaz · Clinical Trials 9(1), 48-55 · 2012

Checked 2026-10-12: SAGE record and PMC3731071 via search results: title, authors, Clin Trials 9(1) 48-55 (online Sep 2011). Content: the intention-to-treat comparison is not guaranteed to estimate effectiveness and can understate an effect when adherence differs; recommends reporting as-treated or per-protocol analyses with weighting adjustments that rely on untestable assumptions. Abstract-level reading only.

Scott Cunningham

Updated 2nd edition with a stronger panel-data section: ch. 9 Difference-in-Differences Fundamentals, ch. 10 Complex Diff-in-Diff Designs, ch. 11 Synthetic Control. It is still being corrected, so check passages before citing.

Checked 2026-10-11: mixtape.scunning.com now hosts 'Causal Inference: The Remix', labelled the online second edition and a work in progress; author and chapter list on the page. Publication year not stated, so omitted.

Judea Pearl · Cambridge University Press · 2009

Primary reference for structural causal models, DAGs, d-separation and the do-operator; cite it for formal definitions, not as first reading.

Checked 2026-10-11: Title 'Causality', '2nd edition', author, Cambridge University Press, publication date 2009/09 and DOI on Cambridge Core. Subtitle not shown on Cambridge Core, so omitted.

Felix Elwert, Christopher Winship · Annual Review of Sociology · 2014

Checked 2026-10-11: Semantic Scholar record for the DOI: title, authors, venue Annual Review of Sociology, 2014, abstract. Abstract: endogenous selection bias arises from conditioning (controlling, stratifying or selecting) on a collider caused by two variables, one associated with the treatment and one with the outcome; uses DAGs to show it. Volume and pages not checked.

Stefan Wager, Susan Athey · arXiv:1510.04342 (stat.ME); listed as to appear in the Journal of the American Statistical Association · 2018

Checked 2026-10-12: arXiv abstract page: title, two authors, first submitted Oct 2015, v4 July 2017. Abstract: causal forests estimate heterogeneous treatment effects non-parametrically under unconfoundedness, with asymptotically Gaussian estimates and a practical confidence-interval construction. Final journal details were not shown on the page.

Kay H. Brodersen, Fabian Gallusser, Jim Koehler, Nicolas Remy, Steven L. Scott · The Annals of Applied Statistics 9(1), from p. 247 · 2015

The method behind the CausalImpact package: a counterfactual forecast from control series via a Bayesian structural time-series model; use for marketing/intervention impact lessons.

Checked 2026-10-11: Title, five authors, Annals of Applied Statistics vol. 9 issue 1, first page 247, 2015/03 and DOI in citation metadata on the Project Euclid landing page.

Douglas Staiger, James H. Stock · Econometrica 65(3), 557-586 · 1997

Checked 2026-10-12: Econometric Society listing and search summaries: title, authors, Econometrica 65(3) May 1997. Abstract: asymptotic theory for IV regression when instruments are only weakly correlated with the endogenous regressor; two-stage least squares with many weak instruments is pulled toward OLS in the Angrist-Krueger illustration. Abstract-level reading only.

James Lopez Bernal, Steven Cummins, Antonio Gasparrini · International Journal of Epidemiology 46(1), 348-355 · 2017

Checked 2026-10-11: Oxford Academic article page: title, three authors, IJE 46(1) 348-355 (Feb 2017, online June 2016). Content: seasonality can bias estimates when pre and post periods contain uneven seasonal months (adjust with month terms, Fourier terms or splines); ITS is protected against slowly changing confounders but events coinciding with the intervention can bias it, addressed by controlled ITS or other design extensions; check residual autocorrelation; no fixed minimum number of time points.

C. W. J. Granger · Econometrica 37(3), 424-438 · 1969

Defines Granger causality as predictive precedence; cite it when explaining why 'Granger-causes' does not mean causal effect.

Checked 2026-10-11: Title, author, Econometrica vol. 37 no. 3 (Aug 1969), pp. 424-438, The Econometric Society, on the JSTOR landing page (opened in a browser); matches Crossref DOI metadata.

Justin McCrary · Journal of Econometrics 142(2), 698-714 · 2008

Checked 2026-10-12: NBER working-paper version (t0334) abstract via search results: identification relies on continuity of counterfactual outcomes in the running variable, which may fail if agents can manipulate it; the paper tests continuity of the running-variable density at the cutoff. Journal citation (J. Econometrics 142(2) 698-714) from the same results; journal page not opened.

Peter C. Austin, Elizabeth A. Stuart · Statistics in Medicine 34(28), 3661-3679 · 2015

Checked 2026-10-12: PubMed Central record (via search results): title, authors, Stat Med 34(28) 3661-3679, 2015, DOI 10.1002/sim.6607. Abstract: IPTW creates a pseudo-sample in which treatment assignment no longer depends on measured covariates; the paper proposes qualitative and quantitative balance diagnostics for the weighted sample. Abstract-level reading only.

Ramesh Johari, Orrie B. Page, Gabriel Y. Weintraub · arXiv:2310.17165 (stat.ME); no journal venue shown on the arXiv page · 2023

Checked 2026-10-12: arXiv abstract page: title, three authors, first submitted 26 Oct 2023 (latest revision Jan 2026). Abstract: price A/B tests are biased when market participants interfere with each other; applied to a standard revenue-management pricing model, the standard estimator of the profit change can have the wrong sign in expectation; proposes a debiasing technique needing an equal treatment/control split.

Tyler J. VanderWeele, Peng Ding · Annals of Internal Medicine 167(4), 268-274 · 2017

Checked 2026-10-12: Harvard DASH manuscript record and search summaries: title, authors, Ann Intern Med 167(4) 268-274. Abstract: the E-value is the minimum strength of association (risk-ratio scale) an unmeasured confounder would need with both treatment and outcome to fully explain away an association, given measured covariates; it should be reported for the estimate and the confidence limit closest to the null.

Alberto Abadie, Alexis Diamond, Jens Hainmueller · Journal of the American Statistical Association 105(490), 493-505 · 2010

Introduces the synthetic-control estimator with placebo-based inference (California Proposition 99 case); core for single-unit policy evaluation.

Checked 2026-10-11: Title, three authors, JASA vol. 105 issue 490 (2010), pp. 493-505, on the Taylor & Francis landing page (opened in a browser; plain HTTP fetch was blocked).

Susan Athey, Raj Chetty, Guido Imbens, Hyunseung Kang · NBER Working Paper 26463 · 2019

Checked 2026-10-12: NBER abstract and PDF listing via search results: title, four authors, Nov 2019 (revised Apr 2024). Content: the treatment effect on the surrogate index equals the effect on the long-term outcome if the long-term outcome is independent of treatment conditional on the surrogates; bias characterised when this fails. Abstract-level reading only.

Alberto Abadie · Journal of Economic Literature 59(2), 391-425 · 2021

Checked 2026-10-11: AEA landing page (title, author, JEL 59(2) 391-425, June 2021) and the full text (NBER-hosted PDF): when pre-intervention fit is poor the method is not recommended because of potential bias; a long pre-period cannot remove bias if the fit is bad; over-fitting risk grows with donor pool size when the pre-period is short; donors must be chosen judiciously; backdating (in-time placebo) checks credibility.

Jonathan Roth, Pedro H. C. Sant'Anna, Alyssa Bilinski, John Poe · Journal of Econometrics (2023); arXiv:2201.01194 · 2023

Checked 2026-10-11: arXiv page and v3 PDF (9 Jan 2023): no significant pre-trend does not imply parallel trends, pre-tests often have low power, conditioning on passing a pre-test causes pre-test bias; with staggered timing and heterogeneous effects TWFE can make 'forbidden' comparisons with already-treated units and even get the sign wrong. Journal of Econometrics venue from the Semantic Scholar record for DOI 10.1016/j.jeconom.2023.03.008 (journal page returned 403).

Jonathan Roth, Pedro H. C. Sant'Anna · arXiv:2010.04814 (econ.EM); no journal venue shown on the arXiv page · 2020

Checked 2026-10-12: arXiv abstract page: title, two authors, first submitted Oct 2020, v5 Sep 2022. Abstract: parallel trends holds under every strictly monotonic transformation of the outcome only under a stronger condition on the untreated potential-outcome distributions, which in practice holds essentially only when treatment is effectively random in a subgroup and the rest is stable; introduces falsification tests.

Andrew Gelman, Guido Imbens · Journal of Business & Economic Statistics 37(3), 447-456 (NBER Working Paper 20405, 2014) · 2019

Checked 2026-10-12: NBER working-paper page and search summaries: title, authors, JBES 37(3) 447-456 per IDEAS/RePEc (the NBER page cites DOI 10.1080/07350015.2017.1366909). Argument: estimators controlling for high-order (third, fourth or higher) polynomials of the forcing variable can be misleading, and local linear or quadratic estimators are recommended. Abstract-level reading only.

Dan Hendrycks, Kevin Gimpel · ICLR 2017 (per arXiv journal reference) · 2017

Max-softmax confidence as a routing baseline (days 32, 41).

Checked 2026-10-11: arXiv 1610.02136 confirms title and authors; journal_ref ICLR 2017. Abstract: correctly classified examples tend to have higher maximum softmax probability than misclassified or out-of-distribution ones; the baseline can be surpassed.

Abhimanyu Das, Weihao Kong, Rajat Sen, Yichen Zhou · Proceedings of the 41st International Conference on Machine Learning (ICML 2024), PMLR 235, 10148-10167 · 2024

TimesFM: patched decoder-only Transformer pretrained on a large time-series corpus for zero-shot point forecasting; Day 27.

Checked 2026-10-11: Title, authors, pp. 10148-10167, ICML, PMLR v235 (publication date 2024-07-08) in PMLR landing page metadata; arXiv 2310.10688 matches.

Vincent Dumoulin, Francesco Visin · 2016

CNN shape arithmetic (day 36).

Checked 2026-10-11: arXiv 1603.07285 confirms title and authors (preprint). Abstract: relationships between input shape, kernel shape, zero padding, strides and output shape for convolution, pooling and transposed convolution.

Andrej Karpathy · 2019

Practitioner debugging checklist; cite as practitioner experience, not as an empirical study (days 33-35).

Checked 2026-10-11: Page loads with title "A Recipe for Training Neural Networks", dated Apr 25, 2019. Contains "verify loss @ init" (expect -log(1/n_classes) for softmax), "overfit one batch", and "use backprop to chart dependencies" (detect mixing across the batch dimension).

Ting Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey Hinton · Proceedings of the 37th International Conference on Machine Learning (ICML), PMLR 119 · 2020

Contrastive learning with augmentations, a projection head and large negative sets; supports day 38 (contrastive representation learning).

Checked 2026-10-11: PMLR v119 landing page confirms title and ICML; arXiv 2002.05709 confirms four authors and ICML 2020 comment.

Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, Jayant Kalagnanam · International Conference on Learning Representations (ICLR 2023) · 2023

PatchTST: patching plus channel independence for Transformer long-horizon forecasting; Day 27.

Checked 2026-10-11: Title, authors via arXiv API (comment: 'Accepted by ICLR 2023'); listed in ICLR 2023 virtual papers page as poster 10876. OpenReview forum blocked by browser-verification challenge.

Xavier Bouthillier, Pierre Delaunay, Mirko Bronzi, Assya Trofimov, Brennan Nichyporuk, Justin Szeto, Naz Sepah, Edward Raff, Kanika Madan, Vikram Voleti, Samira Ebrahimi Kahou, Vincent Michalski, Dmitriy Serdyuk, Tal Arbel, Chris Pal, Gael Varoquaux, Pascal Vincent · 2021

Seeds and variance in model comparisons (days 35, 40).

Checked 2026-10-11: arXiv 2103.03098 confirms title and authors; comment "Submitted to MLSys2021" (final venue not recorded here). Abstract: variance from data sampling, parameter initialization and hyperparameter choice markedly affects benchmark conclusions.

Priya Goyal, Piotr Dollar, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, Kaiming He · Technical report (arXiv) · 2017

Coupling of batch size and learning rate; warmup (days 33, 34).

Checked 2026-10-11: arXiv 1706.02677 confirms title and authors; comment "Tech report". Abstract describes the linear scaling rule for learning rate vs minibatch size and a warmup scheme.

PyTorch

L2 vs decoupled weight decay (days 33, 34).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page "Adam". weight_decay is documented as weight decay (L2 penalty), default 0; decoupled_weight_decay=True makes it equivalent to AdamW.

PyTorch

AdamW defaults and parameter groups (days 33, 34, 40).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page "AdamW". weight_decay coefficient default 1e-2; refers to Decoupled Weight Decay Regularization; params may be an iterable of dicts defining parameter groups.

Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Nicolas Papernot, Ross Anderson, Yarin Gal · Nature 631(8022), 755-759 · 2024

Indiscriminately training on model-generated content causes irreversible defects in which tails of the original distribution disappear (shown for LLMs, VAEs and Gaussian mixtures). Supports insight 0545.

Checked 2026-10-11: Crossref record for DOI 10.1038/s41586-024-07566-y (title, authors, journal, volume/issue/pages, online 2024-07-24, abstract). A correction was published 2025-03-21 (DOI 10.1038/s41586-025-08905-3).

Amazon Science · 2024

Official code/checkpoints for Chronos, Chronos-Bolt (patch-based, direct multi-step quantile decoder) and Chronos-2; Day 27 lab.

Checked 2026-10-11: GitHub API repo record (Apache-2.0, not archived, last push 2026-10-05), releases list (v2.3.2 2026-09-08), README news: Chronos (13 Mar 2024), Chronos-Bolt (26 Nov 2024), Chronos-2 (20 Oct 2025); model table lists amazon/chronos-2 (120M), chronos-bolt tiny/mini/small/base, chronos-t5 sizes.

Sam McCandlish, Jared Kaplan, Dario Amodei, OpenAI Dota Team · arXiv:1812.06162 · 2018

Checked 2026-10-12: arXiv abstract page: title, authors, submitted 14 Dec 2018, no journal reference. Abstract: the gradient noise scale, a simple measurable quantity, predicts the largest useful batch size across supervised, reinforcement-learning and generative tasks, and tends to grow as the loss falls.

Ailing Zeng, Muxi Chen, Lei Zhang, Qiang Xu · Proceedings of the AAAI Conference on Artificial Intelligence 37(9), 11121-11128 · 2023

Simple linear models (DLinear/NLinear) outperformed several Transformer LTSF models on standard benchmarks: a baseline lesson; Day 27.

Checked 2026-10-11: Title, authors, AAAI 37(9) pp. 11121-11128, 2023 in Crossref DOI metadata; arXiv 2205.13504 lists code at github.com/cure-lab/LTSF-Linear.

Diederik P. Kingma, Max Welling · ICLR 2014 · 2014

Variational autoencoder, ELBO and the reparameterization trick; supports day 46 (Generative Modeling).

Checked 2026-10-11: arXiv abs 1312.6114 confirms title and authors (first submitted 2013-12-20). The ICLR 2014 conference-proceedings page on iclr.cc lists "Auto-Encoding Variational Bayes".

PyTorch

How the dynamic computation graph records operations and saves tensors for backward; supports day 33.

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 notes page titled "Autograd mechanics".

PyTorch

API reference for backward(), grad and gradient tracking; supports days 33 and 35 (autograd in the NN lab).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page titled "Automatic differentiation package - torch.autograd".

PyTorch

Unscale before gradient clipping under AMP (day 34).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 notes page. Gradient clipping section states that clipping without unscaling uses scaled gradients, which invalidates the threshold, and shows scaler.unscale_(optimizer) before clipping.

PyTorch

Mixed precision caveats and gradient scaling (days 34, 40, 54).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page. Gradient Scaling section states small float16 gradients can underflow to zero and that gradient scaling multiplies the loss by a scale factor to prevent this.

Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, Andrew Gordon Wilson · UAI 2018 (per arXiv comment) · 2018

Checked 2026-10-12: arXiv abstract page: title, five authors, submitted 14 Mar 2018, comment 'Appears at UAI 2018'. Abstract: averaging several points along an SGD trajectory with a cyclical or constant learning rate generalizes better than conventional training, finds flatter solutions, and costs almost nothing extra.

Sergey Ioffe, Christian Szegedy · Proceedings of the 32nd International Conference on Machine Learning (ICML), PMLR 37 · 2015

Batch normalization and its train/inference behaviour difference; supports day 34. The paper's own "internal covariate shift" explanation should be presented as the authors' motivation, not settled fact.

Checked 2026-10-11: PMLR v37 landing page confirms title and ICML 2015 (publication date 2015/06/01); arXiv 1502.03167 confirms authors.

PyTorch

Batch statistics vs running statistics (days 34, 36, 40).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page "BatchNorm2d". By default the layer keeps running estimates of mean and variance during training (momentum 0.1) and uses them for normalization during evaluation.

PyTorch

Numerically stable binary and multi-label losses (day 32).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page "BCEWithLogitsLoss". Page states the class combines Sigmoid and BCELoss and is more numerically stable than a plain Sigmoid followed by BCELoss because it uses the log-sum-exp trick; documents pos_weight.

PyTorch

Shape bugs that do not raise errors (day 31).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 notes page. States PyTorch follows NumPy broadcasting; tensors are broadcastable when, from the trailing dimension, sizes are equal, one of them is 1, or one does not exist; expansion happens without copying data.

Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip H.S. Torr, Puneet K. Dokania · NeurIPS 2020 (per arXiv comment) · 2020

Checked 2026-10-12: arXiv abstract page: title, six authors, submitted 21 Feb 2020, comment 'accepted at NeurIPS 2020'. Abstract: models trained with focal loss are better calibrated than with cross-entropy, and combining focal loss with temperature scaling gives top calibration while preserving accuracy.

Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D Sculley, Sebastian Nowozin, Joshua V. Dillon, Balaji Lakshminarayanan, Jasper Snoek · NeurIPS 2019 (per arXiv comment) · 2019

Checked 2026-10-12: arXiv abstract page: title, nine authors, submitted 6 Jun 2019, comment 'Advances in Neural Information Processing Systems, 2019'. Abstract: benchmarks uncertainty methods on classification under dataset shift; predictive uncertainty quality degrades with shift; post-hoc recalibration often falls short while methods that average over several models perform surprisingly well.

Abdul Fatir Ansari, Oleksandr Shchur, Jaris Küken, Andreas Auer, Boran Han, Pedro Mercado, et al. · arXiv:2510.15821 (technical report) · 2025

Chronos-2 adds zero-shot multivariate and covariate-informed forecasting; vendor benchmark claims (fev-bench, GIFT-Eval) are self-reported; Day 27.

Checked 2026-10-11: Title, leading authors, date via arXiv API record 2510.15821; linked as 'Chronos-2 Report' from the official amazon-science/chronos-forecasting README.

Abdul Fatir Ansari, Lorenzo Stella, Ali Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, Jasper Zschiegner, Danielle C. Maddix, Hao Wang, Michael W. Mahoney, Kari Torkkola, Andrew Gordon Wilson, Michael Bohlke-Schneider, Bernie Wang · Transactions on Machine Learning Research (TMLR) · 2024

Tokenizes scaled, quantized series and trains T5-style language models for zero-shot probabilistic forecasting; Day 27.

Checked 2026-10-11: Title and authors on the TMLR accepted-papers list at jmlr.org/tmlr/papers (dated November 2024, OpenReview id gerNCVqqtR); arXiv API record 2403.07815 matches. OpenReview page itself blocked by browser-verification challenge.

Jonathan Ho, Tim Salimans · NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications (short version) · 2022

Guidance trades sample diversity for fidelity, in the spirit of low-temperature or truncated sampling, by mixing conditional and unconditional estimates. Supports insight 0573.

Checked 2026-10-11: arXiv abs page (submitted 2022-07-26; Comments: short version in a NeurIPS 2021 workshop; no main-track venue) and abstract.

Ajay Jaiswal, Zhe Gan, Xianzhi Du, Bowen Zhang, Zhangyang Wang, Yinfei Yang · ICLR 2024 · 2024

Perplexity is an inadequate measure for compressed LLMs; on knowledge-intensive tasks (LLM-KICK) pruning degraded performance even at modest sparsity while quantization held up better. Supports insight 0551.

Checked 2026-10-11: arXiv abs page (title, authors, abstract); venue from the arXiv Comments field (Accepted to ICLR 2024).

Curtis G. Northcutt, Lu Jiang, Isaac L. Chuang · Journal of Artificial Intelligence Research (JAIR), 2021 (per arXiv journal reference) · 2021

Checked 2026-10-12: arXiv abstract page: title, three authors, submitted 31 Oct 2019, journal reference JAIR 2021. Abstract: confident learning finds label errors by pruning suspect examples, estimating noise with thresholded counting and ranking examples, under a class-conditional noise assumption; reported to improve accuracy after cleaning.

PyTorch

CNN shape and parameter arithmetic (day 36).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page "Conv2d". Gives H_out = floor((H_in + 2*padding - dilation*(kernel_size - 1) - 1)/stride + 1) and weight shape (out_channels, in_channels/groups, kernel_size[0], kernel_size[1]).

Yoon Kim · EMNLP 2014; arXiv:1408.5882 · 2014

Checked 2026-10-12: arXiv abstract page: title, single author, to appear at EMNLP 2014. Abstract: a simple CNN on top of pretrained word vectors with little tuning does well on sentence-level classification benchmarks; task-specific vectors add gains; beats prior results on four of seven tasks.

Gael Varoquaux · arXiv:1706.07581 (the paper appeared in NeuroImage; journal details not shown on the arXiv page) · 2017

Checked 2026-10-12: arXiv abstract page: title, single author, submitted 23 Jun 2017. Abstract: cross-validation error bars are often underestimated; for typical neuroimaging sample sizes (about 100 samples) the uncertainty is on the order of +-10% and is not captured by the spread across folds. Journal venue not confirmed from the page.

PyTorch

Logits vs probabilities at the loss boundary; class weights for imbalance (days 32, 41).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page "CrossEntropyLoss". Page states the input is expected to contain unnormalized logits per class, that the optional class weight tensor is useful for an unbalanced training set, that ignore_index targets do not contribute to the gradient, and that label_smoothing mixes targets with a uniform distribution.

Stanford CS231n course staff

Checked 2026-10-11: Course notes page: centred-difference numerical gradient; relative error with rough thresholds (above 1e-2 likely wrong, below 1e-7 good, up to about 1e-4 tolerable with kinks such as ReLU); use double precision; turn off dropout/augmentation during checks; check data loss and regularisation separately; loss at chance (for 10-class softmax about 2.302); overfit a tiny subset of about 20 examples; update-to-weight ratio around 1e-3; random search with learning rate and regularisation on a log scale.

PyTorch

GPU memory measurement and budgeting (days 35, 40, 54).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 notes page. Memory management section states PyTorch uses a caching allocator whose unused memory still appears as used in nvidia-smi, and names memory_allocated, max_memory_allocated, memory_reserved and max_memory_reserved.

Leslie N. Smith · WACV 2017 (per author comment) · 2017

Learning-rate range test (days 33, 34).

Checked 2026-10-11: arXiv 1506.01186 confirms title and author; comment says presented at WACV 2017. Abstract describes estimating reasonable learning-rate bounds by linearly increasing the learning rate for a few epochs (the LR range test).

Ilya Loshchilov, Frank Hutter · ICLR 2019 · 2019

Shows L2 regularization and weight decay differ for adaptive optimizers and proposes AdamW; supports days 33 and 40.

Checked 2026-10-11: arXiv abs 1711.05101 confirms title and authors; author comment states "Published as a conference paper at ICLR 2019". OpenReview page was behind a browser challenge, so venue rests on the arXiv comment.

Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, Ilya Sutskever · arXiv:1912.02292 (ICLR 2020 per common citation; not shown on the arXiv page) · 2019

Checked 2026-10-12: arXiv abstract page: title, six authors, submitted 4 Dec 2019, no journal reference shown. Abstract: test performance can get worse and then better as model size grows, the same pattern appears as training time grows (epoch-wise), and more data can in some regimes reduce test accuracy.

Ian Goodfellow, Yoshua Bengio, Aaron Courville · MIT Press · 2016

Foundational theory (pre-transformer). Most useful: ch. 5 Machine Learning Basics, ch. 6 Deep Feedforward Networks, ch. 7 Regularization for Deep Learning, ch. 8 Optimization for Training Deep Models, ch. 11 Practical Methodology.

Checked 2026-10-11: Title, authors, MIT Press, 2016 bibtex and table of contents on deeplearningbook.org.

Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md. Mostofa Ali Patwary, Yang Yang, Yanqi Zhou · arXiv:1712.00409 · 2017

Checked 2026-10-12: arXiv abstract page: title, nine authors, submitted 1 Dec 2017, no journal reference. Abstract: across machine translation, language modelling, image classification and speech recognition, generalization error falls as a power law in training-set size; architecture changes shift the curve without changing its exponent.

David Salinas, Valentin Flunkert, Jan Gasthaus, Tim Januschowski · International Journal of Forecasting 36(3), 1181-1191 · 2020

Global autoregressive RNN trained across many related series producing probabilistic (likelihood-based) forecasts; Day 27.

Checked 2026-10-11: Title, authors, IJF 36(3) pp. 1181-1191, July 2020 in Crossref DOI metadata; arXiv 1704.04110 is the preprint.

Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun · 2015 IEEE International Conference on Computer Vision (ICCV), 1026-1034 · 2015

Introduces He (Kaiming) initialization for ReLU networks and PReLU; supports day 34 on initialization choice for ReLU-family activations.

Checked 2026-10-11: Crossref record for DOI 10.1109/ICCV.2015.123 confirms title, authors and ICCV 2015 pp 1026-1034; arXiv 1502.01852 matches title and authors.

Jonathan Ho, Ajay Jain, Pieter Abbeel · Advances in Neural Information Processing Systems 33 (NeurIPS 2020) · 2020

Forward noising and learned reverse denoising process (noise-prediction objective); supports day 46.

Checked 2026-10-11: NeurIPS proceedings abstract page metadata confirms title, three authors, 2020; arXiv 2006.11239 matches.

Geoffrey Hinton, Oriol Vinyals, Jeff Dean · arXiv:1503.02531 (stat.ML) · 2015

Origin of the teacher-student (distillation) recipe; says nothing about LLM outputs, so use it for the mechanism only.

Checked 2026-10-12: arXiv abstract page: title, three authors, submitted 9 Mar 2015. Abstract: running an ensemble is costly at deployment; the paper compresses an ensemble into a single model with a different technique than Caruana et al., reports MNIST results and a commercial acoustic-model improvement, and introduces specialist models for fine-grained classes.

Aston Zhang, Zachary C. Lipton, Mu Li, Alexander J. Smola · Cambridge University Press · 2023

Code-first deep learning with PyTorch. Most useful: ch. 5 Multilayer Perceptrons, ch. 11 Attention Mechanisms and Transformers, ch. 12 Optimization Algorithms, ch. 19 Hyperparameter Optimization.

Checked 2026-10-11: Title, four authors, Cambridge University Press 2023 bibtex, online version 1.0.3 and table of contents on d2l.ai.

PyTorch

Dropout at inference (day 34).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page "Dropout". During training elements are zeroed with probability p and outputs scaled by 1/(1-p); during evaluation the module computes an identity function.

Yarin Gal, Zoubin Ghahramani · ICML 2016 (PMLR 48) · 2016

Monte Carlo dropout at inference (days 34, 26).

Checked 2026-10-11: PMLR v48 landing page shows the title; arXiv 1506.02142 confirms authors. Casts dropout training as approximate Bayesian inference, giving uncertainty estimates from dropout networks.

Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov · Journal of Machine Learning Research 15, 1929-1958 · 2014

Dropout as regularization and its train vs eval behaviour; supports day 34 (Training Deep Networks).

Checked 2026-10-11: JMLR landing page citation metadata confirms title, five authors, JMLR vol 15 pp 1929-1958, 2014.

Patrick Helber, Benjamin Bischke, Andreas Dengel, Damian Borth · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12(7), 2217-2226 · 2019

Checked 2026-10-11: Crossref metadata for the DOI: title, four authors, journal, volume 12 issue 7 pages 2217-2226, July 2019. arXiv 1709.00029 abstract: 13 bands, 10 classes, 27,000 images, CNN benchmarks, overall accuracy 98.57% reported by the authors.

Patrick Helber, Benjamin Bischke, Andreas Dengel, Damian Borth · 2018

Project 031 (frozen-backbone transfer learning, docs-only).

Checked 2026-10-11: GitHub README: 27,000 labelled, geo-referenced Sentinel-2 images in 10 classes, 13-band and RGB versions, MIT license plus a pointer to the Copernicus Sentinel data terms, hosted on Zenodo. Zenodo record 7711810: EuroSAT_RGB.zip 94.7 MB (HEAD: 94,658,721 bytes), EuroSAT_MS.zip 2.1 GB, MIT. TensorFlow Datasets catalog: rgb config images are 64x64x3 uint8, a single 'train' split of 27,000 examples (no official test split). Class names were not confirmed on the pages read; read them from the folder names after unzipping.

Han Xiao, Kashif Rasul, Roland Vollgraf (Zalando Research) · 2017

Project 029 (small CNN vs pixel baselines).

Checked 2026-10-11: Repository README: 60,000 training and 10,000 test images, 28x28 grayscale, 10 labelled classes (T-shirt/top, Trouser, Pullover, Dress, Coat, Sandal, Shirt, Sneaker, Bag, Ankle boot); MIT license, copyright Zalando SE; MNIST-format gz files on an S3 bucket with MD5 checksums (train images 26 MB); loaders listed for PyTorch, Keras, TensorFlow Datasets, Hugging Face Datasets. HEAD request on the train-images file returned 200 with 26,421,880 bytes; not downloaded. Citation arXiv 1708.07747.

Han Xiao, Kashif Rasul, Roland Vollgraf · arXiv:1708.07747 · 2017

Checked 2026-10-11: arXiv API record: title, three authors, first version 2017-08-25; cited by the dataset README.

Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, Percy Liang · ICLR 2022 (per author comment) · 2022

Linear probe then fine-tune (day 40).

Checked 2026-10-11: arXiv 2202.10054 confirms title and authors; comment "ICLR (Oral) 2022". Abstract: across 10 shift datasets full fine-tuning averaged 2% higher in-distribution but 7% lower out-of-distribution accuracy than linear probing; a random head distorts features.

Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, Christopher Re · 2022

Attention cost and memory (days 39, 42, 54).

Checked 2026-10-11: arXiv 2205.14135 confirms title and authors. Abstract: self-attention time and memory are quadratic in sequence length; FlashAttention is exact attention that uses tiling to reduce GPU high-bandwidth-memory reads and writes. Venue not recorded here.

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollar · 2017

Loss reweighting under class imbalance (days 13, 41).

Checked 2026-10-11: arXiv 1708.02002 confirms title and authors. Abstract: focal loss reshapes cross-entropy to down-weight well-classified examples so that many easy negatives do not dominate training under extreme imbalance. Venue not recorded here.

Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, Sepp Hochreiter · Advances in Neural Information Processing Systems 30 (NIPS 2017) · 2017

Introduces the Fréchet Inception Distance (FID), comparing statistics of real and generated images in Inception feature space. Supports insight 0574.

Checked 2026-10-11: arXiv abs page (journal-ref: Advances in Neural Information Processing Systems 30 (NIPS 2017)) and abstract.

Dan Hendrycks, Kevin Gimpel · 2016

Defines GELU, the activation used in BERT/GPT-style Transformers; supports day 32 (Activations and Losses).

Checked 2026-10-11: arXiv abs 1606.08415 metadata confirms title, authors and first submission 2016-06-27. No peer-reviewed venue confirmed.

Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio · Advances in Neural Information Processing Systems 27 (NIPS 2014) · 2014

Generator-discriminator minimax game; supports day 46 comparison of generative families (GAN vs VAE vs diffusion).

Checked 2026-10-11: NeurIPS proceedings abstract page metadata confirms title "Generative Adversarial Nets", 8 authors (first Goodfellow), 2014. arXiv 1406.2661 uses the title "Generative Adversarial Networks".

Google Research · 2024

Official TimesFM code and version history; check weight licence before production use; Day 27.

Checked 2026-10-11: GitHub API repo record (Apache-2.0 code, last push 2026-09-29), releases (v3.0.0 2026-08-28, v2.0.2 2026-07-02), README: latest model TimesFM 3.0 (Aug 2026), TimesFM 2.5 released Sept 15 2025, 1.0/2.0 archived; README license notice says TimesFM 3.0 weights use a separate non-commercial license while weights up to 2.5 remain Apache-2.0.

Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, Sumit Sanghai · Proceedings of EMNLP 2023, 4895-4901 · 2023

Multi-query attention (one key-value head) speeds decoder inference but can lose quality; grouped-query attention (an intermediate number of key-value heads) gets close to multi-head quality at close to multi-query speed. Supports insight 0506.

Checked 2026-10-11: ACL Anthology page (title, authors, booktitle, pages, DOI 10.18653/v1/2023.emnlp-main.298); arXiv 2305.13245 Comments (Accepted at EMNLP 2023) and abstract.

Y. LeCun, L. Bottou, Y. Bengio, P. Haffner · Proceedings of the IEEE 86(11), 2278-2324 · 1998

Convolutional networks (LeNet) trained end-to-end with gradient descent; supports day 36 (CNNs) on weight sharing and local receptive fields.

Checked 2026-10-11: Crossref record for DOI 10.1109/5.726791 confirms title, four authors, Proceedings of the IEEE vol 86 issue 11 pp 2278-2324, 1998.

Yuxin Wu, Kaiming He · 2018

Small-batch normalization choice (days 34, 36).

Checked 2026-10-11: arXiv 1803.08494 confirms title and authors. Abstract states BN error increases rapidly as batch size shrinks and that GN is batch-size independent (10.6% lower error than BN at batch size 2 on ResNet-50/ImageNet). Venue not recorded here.

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, Björn Ommer · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 10674-10685 · 2022

Diffusion in an autoencoder's latent space to cut compute, with cross-attention conditioning; supports day 46.

Checked 2026-10-11: Crossref record for DOI 10.1109/CVPR52688.2022.01042 confirms title, five authors, CVPR 2022 pp 10674-10685; arXiv 2112.10752 matches (comment "CVPR 2022").

Jason Yosinski, Jeff Clune, Yoshua Bengio, Hod Lipson · Advances in Neural Information Processing Systems 27 (NIPS 2014), 3320-3328 · 2014

Early layers are general, later layers task-specific, and transfer degrades with task distance; supports day 40 (transfer learning / fine-tuning).

Checked 2026-10-11: NeurIPS proceedings abstract page confirms title, authors, 2014; arXiv 1411.1792 journal_ref gives NIPS 27 pages 3320-3328.

Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, Ameet Talwalkar · Journal of Machine Learning Research 18 (2018) 1-52 · 2018

Checked 2026-10-12: arXiv abstract page: title, five authors, submitted 21 Mar 2016, journal reference JMLR 18 (2018) 1-52. Abstract: speeds up random search by adaptively allocating resources and early-stopping poor configurations; framed as a pure-exploration bandit problem; reported to beat Bayesian optimization methods on deep learning and kernel tasks.

Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton · Advances in Neural Information Processing Systems 25 (NIPS 2012) · 2012

Deep CNN with ReLU, dropout and GPU training on ImageNet; historical anchor for day 36 (CNNs).

Checked 2026-10-11: NeurIPS proceedings abstract page metadata confirms title, three authors, 2012. A later CACM version exists (Crossref DOI 10.1145/3065386, CACM 60(6) 2017).

Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, Wieland Brendel · ICLR 2019; arXiv:1811.12231 · 2019

Checked 2026-10-12: arXiv abstract page: title, six authors, ICLR 2019. Abstract: on images with conflicting texture and shape cues ImageNet-trained CNNs rely strongly on texture, unlike humans; training ResNet-50 on a texture-altered Stylized-ImageNet gives shape-based representations and improves detection and robustness to distortions.

Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, Timo Aila · Advances in Neural Information Processing Systems 32 (NeurIPS 2019) · 2019

Measures sample quality (precision) and coverage of the real distribution (recall) separately, which single-number metrics such as FID conflate. Supports insight 0574.

Checked 2026-10-11: NeurIPS proceedings page and arXiv 1904.06991 (Comments: NeurIPS 2019 final version); Introduction notes FID-style metrics fold quality and coverage into one value.

Joelle Pineau, Philippe Vincent-Lamarre, Koustuv Sinha, Vincent Lariviere, Alina Beygelzimer, Florence d'Alche-Buc, Emily Fox, Hugo Larochelle · JMLR (arXiv comment: 'To appear at JMLR') · 2021

Checked 2026-10-12: arXiv abstract page: title, eight authors, submitted 27 Mar 2020, comment 'To appear at JMLR'. Abstract: reports the NeurIPS 2019 reproducibility programme of code submission, a community reproducibility challenge and a reproducibility checklist for papers. Volume/pages not confirmed.

Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton · 2016

Per-example normalization independent of batch size, used in Transformers; supports days 34 and 39.

Checked 2026-10-11: arXiv abs 1607.06450 metadata confirms title, three authors and submission date 2016-07-21. No peer-reviewed venue recorded (none confirmed).

Y. Bengio, P. Simard, P. Frasconi · IEEE Transactions on Neural Networks 5(2), 157-166 · 1994

Formal account of vanishing/exploding gradients over long sequences; supports days 34 and 37 (why plain RNNs struggle).

Checked 2026-10-11: Crossref record for DOI 10.1109/72.279181 confirms title, authors, IEEE Trans. Neural Networks vol 5 issue 2 pp 157-166, March 1994.

David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams · Nature 323(6088), 533-536 · 1986

Classic statement of backpropagation as gradient computation through hidden layers; supports day 33 (Backpropagation and Optimization).

Checked 2026-10-11: Crossref record for DOI 10.1038/323533a0 confirms title, authors Rumelhart/Hinton/Williams, Nature vol 323 issue 6088 pp 533-536, Oct 1986.

PyTorch

Parameter counting (day 31).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page "Linear". Weight has shape (out_features, in_features); optional bias of shape (out_features).

Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation 9(8), 1735-1780 · 1997

Gated memory cell for long-range dependencies; supports day 37 (Sequence Models).

Checked 2026-10-11: Crossref record for DOI 10.1162/neco.1997.9.8.1735 confirms title, authors, Neural Computation vol 9 issue 8 pp 1735-1780, Nov 1997.

Dan Biderman et al. (12 authors) · Transactions on Machine Learning Research (TMLR), 2024 · 2024

In code and math (instruction fine-tuning and continued pretraining), LoRA substantially underperformed full fine-tuning on the target domain but better preserved base-model performance outside it. Supports insight 0547.

Checked 2026-10-11: arXiv abs page (title, 12 authors, abstract); venue from the arXiv Comments field (accepted to TMLR, Featured Certification); no formal journal-ref.

Aditya Kusupati et al. · arXiv:2205.13147 (cs.LG) · 2022

Why some embedding models can be shortened by dropping trailing dimensions; supports day 38 and 48.

Checked 2026-10-12: arXiv abstract page: title, eleven authors, submitted May 2022 (v4 Feb 2024). Abstract: representations encode information at multiple granularities so one embedding can be truncated to a task's budget; reports up to 14x smaller embeddings at the same ImageNet-1K accuracy and up to 14x faster large-scale retrieval. Vision-centred results.

Jeremy Nixon, Mike Dusenberry, Ghassen Jerfel, Timothy Nguyen, Jeremiah Liu, Linchuan Zhang, Dustin Tran · arXiv:1904.01685 · 2019

Checked 2026-10-12: arXiv abstract page: title, seven authors, submitted 2 Apr 2019, no journal reference shown. Abstract: expected calibration error has known flaws; the paper empirically studies measurement design choices on several image datasets and shows they can change which recalibration method ranks best.

Christopher J. Shallue, Jaehoon Lee, Joseph Antognini, Jascha Sohl-Dickstein, Roy Frostig, George E. Dahl · Journal of Machine Learning Research 20 (2019) 1-49 · 2019

Checked 2026-10-12: arXiv abstract page: title, six authors, submitted 8 Nov 2018, journal reference JMLR 20 (2019) 1-49. Abstract: empirically measures how larger batches change the number of steps needed to reach a target out-of-sample error across workloads; the effect varies greatly between workloads; conflicting prior findings often trace to tuning and compute budgets; no evidence that larger batches hurt generalization was found.

Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, Hao Wu · ICLR 2018 (per author comment) · 2018

Mixed precision caveats (days 34, 54).

Checked 2026-10-11: arXiv 1710.03740 confirms title and authors; comment "Published as a conference paper at ICLR 2018". Abstract proposes an fp32 master copy of weights and loss scaling to handle half-precision range limits.

Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, David Lopez-Paz · ICLR 2018 (per arXiv comment) · 2018

Checked 2026-10-12: arXiv abstract page: title, four authors, submitted 25 Oct 2017, comment 'ICLR camera ready version'. Abstract: mixup trains on convex combinations of pairs of examples and their labels; reported benefits include better generalization, less memorization of corrupted labels and greater robustness to adversarial examples.

PyTorch

Train vs eval mode (day 34).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page "Module". eval() sets evaluation mode, affects only certain modules such as Dropout and BatchNorm, and is equivalent to train(False).

PyTorch

Checked 2026-10-12: Stable URL redirects to the 2.14 page. Page states that, unlike a plain Python list, a ModuleList registers the submodules it holds so every Module method can see them.

Chenghao Liu, Taha Aksu, Juncheng Liu, Xu Liu, Hanshu Yan, Quang Pham, Silvio Savarese, Doyen Sahoo, Caiming Xiong, Junnan Li · arXiv:2511.11698 · 2025

Successor in the Moirai family (preprint); Day 27 optional reading.

Checked 2026-10-11: Title, authors and version dates via arXiv API record 2511.11698. No peer-reviewed venue confirmed.

Alex Kendall, Yarin Gal, Roberto Cipolla · CVPR 2018 (per arXiv comment) · 2018

Checked 2026-10-12: arXiv abstract page: title, three authors, submitted 19 May 2017, comment 'CVPR 2018'. Abstract: multi-task performance depends strongly on the relative weighting of task losses and manual tuning is expensive; the paper weights losses by homoscedastic task uncertainty so regression and classification tasks of different scales can be learned together.

PyTorch

Padding masks in attention (days 39, 41).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page. key_padding_mask of shape (N, S) marks key elements to ignore for attention (padding).

Boris N. Oreshkin, Dmitri Carpov, Nicolas Chapados, Yoshua Bengio · International Conference on Learning Representations (ICLR 2020) · 2020

Deep stacks of fully connected blocks with backward/forward residual links and optional interpretable trend/seasonality bases; evaluated on M3/M4/TOURISM; Day 27.

Checked 2026-10-11: Title and authors via arXiv API record 1905.10437; ICLR 2020 venue confirmed on iclr.cc/virtual_2020/poster_r1ecqn4YwB.html (page title). OpenReview forum r1ecqn4YwB blocked by browser-verification challenge.

Cristian Challu, Kin G. Olivares, Boris N. Oreshkin, Federico Garza Ramirez, Max Mergenthaler Canseco, Artur Dubrawski · Proceedings of the AAAI Conference on Artificial Intelligence 37(6), 6989-6997 · 2023

Extends N-BEATS with multi-rate input pooling and hierarchical interpolation for efficient long-horizon forecasting; Day 27.

Checked 2026-10-11: Title, authors, AAAI 37(6) pp. 6989-6997, 2023 in Crossref DOI metadata; arXiv 2201.12886 (titled N-HiTS) notes AAAI-23 acceptance.

Nixtla · Latest release v0.9.0 on 2026-09-21 (GitHub releases)

Official client SDK for TimeGPT; the repo description (vendor text) names TimeGPT-2.1 as the current model; Day 27.

Checked 2026-10-11: GitHub API repo record (description names 'TimeGPT-2.1'; license reported as NOASSERTION; last push 2026-10-08) and releases list (v0.9.0 2026-09-21).

Chuan Guo, Geoff Pleiss, Yu Sun, Kilian Q. Weinberger · Proceedings of the 34th International Conference on Machine Learning (ICML 2017), PMLR 70, 1321-1330 · 2017

Modern deep nets are often overconfident; temperature scaling is a simple effective post-hoc fix; ECE and reliability diagrams; Day 13, Day 34.

Checked 2026-10-11: Title, authors, pp. 1321-1330, conference ICML, PMLR v70 in citation metadata on PMLR landing page.

Yury Gorishniy, Ivan Rubachev, Artem Babenko · NeurIPS 2022 (per arXiv comment) · 2022

Checked 2026-10-12: arXiv abstract page: title, three authors, submitted 10 Mar 2022, comment 'NeurIPS 2022 camera-ready'. Abstract: embedding scalar numerical features (piecewise linear encodings, periodic activations) before the main network substantially improves tabular deep models, and simple MLP-type models can then match attention-based ones, sometimes competing with gradient-boosted trees.

Dami Choi, Christopher J. Shallue, Zachary Nado, Jaehoon Lee, Chris J. Maddison, George E. Dahl · arXiv:1910.05446 · 2019

Checked 2026-10-12: arXiv abstract page: title, six authors, submitted 11 Oct 2019, no journal reference shown. Abstract: optimizer rankings change with the hyperparameter tuning protocol; the tuning search space may matter more than anything else in explaining conflicting past conclusions; a more general optimizer should not lose to one it can emulate when tuning is unconstrained; tuning advice is given.

Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, Tie-Yan Liu · ICML 2020 (per arXiv journal reference) · 2020

Normalization placement and warmup (days 34, 39).

Checked 2026-10-11: arXiv 2002.04745 confirms title and authors; journal_ref "Published on ICML 2020". Abstract shows Post-LN has large gradients near the output at initialization (hence warmup), while Pre-LN gradients are well-behaved and warmup can be removed.

Razvan Pascanu, Tomas Mikolov, Yoshua Bengio · ICML 2013 (PMLR 28) · 2013

Gradient clipping; exploding and vanishing gradients (days 34, 37).

Checked 2026-10-11: PMLR v28 landing page shows the title; arXiv 1211.5063 confirms authors. Abstract proposes gradient norm clipping for exploding gradients and a soft constraint for vanishing gradients.

James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, et al. · Proceedings of the National Academy of Sciences 114(13), 3521-3526 · 2017

Catastrophic forgetting (days 40, 44).

Checked 2026-10-11: Crossref record for DOI 10.1073/pnas.1611835114 confirms title, PNAS vol 114 issue 13 pp 3521-3526, March 2017; arXiv 1612.00796 confirms authors. Abstract: networks generally forget earlier tasks when trained sequentially; EWC slows learning on weights important to old tasks.

PyTorch

Checked 2026-10-12: Opened the file. repackage_hidden wraps hidden states in new detached tensors to cut their connection to past computation (recursing into LSTM tuples); the training loop calls it before each batch for non-Transformer models, with a comment that otherwise backpropagation would reach back to the start of the dataset.

Aaron van den Oord, Yazhe Li, Oriol Vinyals · 2018

Introduces the InfoNCE contrastive loss used by many embedding models; supports day 38.

Checked 2026-10-11: arXiv abs 1807.03748 metadata confirms title, three authors, submitted 2018-07-10. No peer-reviewed venue confirmed.

Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo-Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, Ludwig Schmidt · CVPR 2022 (per arXiv comment) · 2022

Checked 2026-10-12: arXiv abstract page: title, eleven authors, submitted 4 Sep 2021, comment 'CVPR 2022'. Abstract: fine-tuning a zero-shot model often lowers robustness under distribution shift; WiSE-FT interpolates the weights of the zero-shot and fine-tuned models, improving shifted-data accuracy while keeping target accuracy, with no extra cost at fine-tuning or inference. Evaluated on CLIP/ALIGN-style models.

Salesforce AI Research · 2024

Official Moirai / Moirai-MoE / Moirai 2.0 code; Day 27 lab.

Checked 2026-10-11: GitHub API repo record (Apache-2.0, last push 2026-06-02), releases (2.0.0 2025-11-04, 1.2.0 2024-11-28, 1.1.0 2024-06-14), README news: Moirai-2.0-R-small released Aug 2025, Moirai-MoE Oct 2024, Moirai-1.1-R Jun 2024, ICML 2024 oral acceptance May 2024.

PyTorch

Checked 2026-10-12: Page text: a general checkpoint for resuming training stores epoch, model state_dict, optimizer state_dict and loss; the optimizer state_dict is saved because it holds buffers and parameters updated during training; call model.train() to resume training and model.eval() before inference.

Samy Bengio, Oriol Vinyals, Navdeep Jaitly, Noam Shazeer · arXiv:1506.03099 (cs.LG); no venue shown on the arXiv page · 2015

Checked 2026-10-12: arXiv abstract page: title, four authors, v3 23 Sep 2015. Abstract: RNNs are usually trained to predict each token from the true previous token but at inference use their own outputs, so errors can compound; a curriculum that gradually shifts to the model's own tokens improved several sequence prediction tasks.

Ilya Loshchilov, Frank Hutter · ICLR 2017 (per arXiv comment) · 2017

Checked 2026-10-12: arXiv abstract page: title, two authors, submitted 13 Aug 2016, comment 'ICLR 2017 conference paper'. Abstract: periodically restarting the learning rate in SGD gives faster early progress; tested on CIFAR-10, CIFAR-100, an EEG dataset and downsampled ImageNet.

Balaji Lakshminarayanan, Alexander Pritzel, Charles Blundell · NIPS 2017 (per arXiv comment) · 2017

Checked 2026-10-12: arXiv abstract page: title, three authors, submitted 5 Dec 2016, comment 'NIPS 2017'. Abstract: a non-Bayesian ensemble recipe that is easy to implement, parallelizable and needs little tuning, giving well-calibrated uncertainty and higher uncertainty on out-of-distribution examples; scales to ImageNet.

William Fedus, Barret Zoph, Noam Shazeer · Journal of Machine Learning Research 23(120), 1-39 · 2022

Mixture-of-experts selects different parameters for each input, giving a sparsely activated model with a very large parameter count at constant computational cost. Supports insight 0505.

Checked 2026-10-11: JMLR paper page (title, authors, volume 23, number 120, pages 1-39, 2022) and abstract.

PyTorch

Checked 2026-10-12: Stable URL redirects to the 2.14 page. Page states that mean and standard deviation are computed over all mini-batches of the same process group, supports DistributedDataParallel with one GPU per process, uses convert_sync_batchnorm before wrapping in DDP, and synchronizes only in training mode.

Bryan Lim, Sercan Ö. Arık, Nicolas Loeff, Tomas Pfister · International Journal of Forecasting 37(4), 1748-1764 · 2021

Attention model handling static covariates, known-future and observed inputs with variable selection and quantile outputs; Day 27.

Checked 2026-10-11: Title, authors, IJF 37(4) pp. 1748-1764, Oct 2021 in Crossref DOI metadata; arXiv 1912.09363 is the preprint.

Ashia C. Wilson, Rebecca Roelofs, Mitchell Stern, Nathan Srebro, Benjamin Recht · 2017

Adam vs SGD generalization trade-off (day 33).

Checked 2026-10-11: arXiv 1705.08292 confirms title, authors and May 2017 date. Abstract reports solutions found by adaptive methods generalized worse than SGD on several deep-learning tasks even with better training performance. Venue not recorded here.

Azul Garza, Cristian Challu, Max Mergenthaler-Canseco · arXiv:2310.03589 · 2023

Commercial API time-series foundation model from Nixtla; preprint only, closed weights; Day 27 build-vs-buy discussion.

Checked 2026-10-11: Title, authors and version dates via arXiv API record 2310.03589. Not peer reviewed (preprint).

Ayush Jain, Rajat Sen (Google Research) · Google Research Blog · 2026

Vendor announcement of TimesFM-3 (native multivariate + covariates); benchmark rankings are self-reported, not peer reviewed; Day 27.

Checked 2026-10-11: Title, authors, date August 31, 2026 on the Google Research blog page; page states models up to TimesFM-2.5 (Sept 2025) were univariate-only.

PyTorch

Gradient checking of custom operations (days 33, 35).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page. Notes that default tolerances are designed for double-precision inputs, that the check will likely fail for lower precision, and that it may fail at non-differentiable points.

PyTorch

Reference for layers, activations, losses and normalization modules; supports days 31-35.

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page titled "torch.nn".

PyTorch

Checked 2026-10-12: PyTorch 2.14 page: padding_idx entries receive no gradient, so the vector is not updated during training; it starts as zeros and can be set to other values.

PyTorch

Checked 2026-10-12: PyTorch 2.14 page: a boolean attn_mask value of True means the element takes part in attention, the inverse of MultiheadAttention's key_padding_mask where True marks padding to ignore (the page says to invert with ~mask when migrating); the reference code fills positions where a boolean mask is False with negative infinity before the softmax. The page does not discuss fully masked rows or NaN. Marked beta.

PyTorch

Exploding gradients (days 34, 37).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page. Clips the gradient norm of an iterable of parameters, computed as if all gradients were concatenated into one vector; modifies gradients in place.

PyTorch

Checked 2026-10-12: PyTorch 2.14 page: takes the true lengths of the sequences in the batch and returns a PackedSequence without the padding; enforce_sorted=True requires longest-to-shortest order, False sorts internally (True only needed for ONNX export). The page says nothing about the final hidden state.

PyTorch

Checked 2026-10-12: Stable URL redirects to the 2.14 page. Learning-rate scheduler section: scheduler.step() is expected after optimizer.step(); calling it first skips the first value of the schedule (behaviour changed in 1.1.0). swa_utils section: AveragedModel (SWA and EMA), SWALR, and update_bn, which recomputes BatchNorm activation statistics with one pass over a loader at the end of SWA/EMA training. Optimizer state_dict/load_state_dict exist.

PyTorch

Activation memory reduction (days 34, 40, 44).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page. Describes activation checkpointing as trading compute for memory by not keeping forward tensors alive until backward and recomputing them.

PyTorch

Data loader seeding and throughput (day 35).

Checked 2026-10-11: Stable URL redirects to the PyTorch 2.14 page. "Randomness in multi-process data loading" states each worker's PyTorch seed is base_seed + worker_id but seeds for other libraries may be duplicated across workers; worker_init_fn can seed them. Memory Pinning section describes pin_memory=True.

PyTorch / torchvision maintainers

Checked 2026-10-11: Docs page: ResNet18_Weights.IMAGENET1K_V1 (alias DEFAULT); ImageNet-1K acc@1 69.758 / acc@5 89.078; 11,689,512 parameters; 1.81 GFLOPS; weight file 44.7 MB; inference transforms resize 256, centre crop 224, rescale to [0,1], normalise with ImageNet mean/std. No licence terms for the weights are stated on the page.

Yantao Shen, Yuanjun Xiong, Wei Xia, Stefano Soatto · CVPR 2020 (arXiv:2003.11942) · 2020

Why a model update means re-embedding the whole gallery unless the new model was trained to be compatible; supports day 38 and 48.

Checked 2026-10-12: arXiv abstract page: title, four authors, 2020, accepted to CVPR 2020. Abstract: updating an embedding model normally forces recomputing ('backfilling') embeddings of all previously seen items because new and old features are not directly comparable; backward-compatible training is proposed so old and new features can be compared without backfill. Image-retrieval setting, not text; abstract-level reading.

Tianqi Chen, Bing Xu, Chiyuan Zhang, Carlos Guestrin · 2016

Gradient (activation) checkpointing (days 34, 40).

Checked 2026-10-11: arXiv 1604.06174 confirms title and authors (preprint). Abstract: O(sqrt(n)) memory for an n-layer network at the cost of roughly one extra forward pass per minibatch.

Keras team

Checked 2026-10-12: Guide text: BatchNormalization has non-trainable running mean and variance; setting trainable to False puts the layer in inference mode so statistics stop updating; when unfreezing a base model for fine-tuning, call it with training=False so BatchNorm statistics are not updated and the learned features are not disturbed.

Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, Oriol Vinyals · ICLR 2017 (per author comment) · 2017

Memorization capacity; label noise; overfitting a batch (days 34, 35, 41).

Checked 2026-10-11: arXiv 1611.03530 confirms title and authors; comment "Published in ICLR 2017". Abstract reports image CNNs trained with SGD easily fit random labels, qualitatively unaffected by explicit regularization.

Xavier Glorot, Yoshua Bengio · Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR 9, 249-256 · 2010

Origin of variance-preserving (Xavier/Glorot) initialization and analysis of saturating activations; supports days 32 and 34 (activations, training deep networks).

Checked 2026-10-11: PMLR landing page citation metadata confirms title, authors Glorot and Bengio, AISTATS 2010 proceedings, pages 249-256.

Xiang Li, Shuo Chen, Xiaolin Hu, Jian Yang · arXiv:1801.05134 · 2018

Checked 2026-10-12: arXiv abstract page: title, four authors, submitted 16 Jan 2018, no journal reference shown. Abstract: dropout shifts a unit's variance between training and test while batch normalization keeps the variance accumulated in training; this mismatch can cause unstable inference and more errors; modified dropout strategies are tested.

Wenjie Luo, Yujia Li, Raquel Urtasun, Richard Zemel · arXiv:1701.04128 (cs.CV); preprint, no venue shown · 2017

Checked 2026-10-12: arXiv abstract page: title, four authors, v1 15 Jan 2017. Abstract: the effective receptive field follows a Gaussian distribution and covers only a fraction of the theoretical receptive field; analyses the effect of activations, dropout, subsampling and skip connections.

Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, Doyen Sahoo · Proceedings of the 41st International Conference on Machine Learning (ICML 2024), PMLR 235, 53140-53164 · 2024

Moirai: masked-encoder universal forecaster with multi-patch-size projections and any-variate attention, trained on LOTSA; Day 27.

Checked 2026-10-11: Title, authors, pp. 53140-53164, ICML, PMLR v235 in PMLR landing page metadata; arXiv 2402.02592 matches.

Rafael Muller, Simon Kornblith, Geoffrey Hinton · NeurIPS 2019 (per author comment) · 2019

Label smoothing trade-offs (days 32, 34).

Checked 2026-10-11: arXiv 1906.02629 confirms title and authors; comment "Accepted at NeurIPS 2019". Abstract reports label smoothing improves calibration but makes distillation from a smoothed teacher much less effective.

Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, Yuxiong He · 2019

GPU memory budgeting for training (days 40, 44, 54).

Checked 2026-10-11: arXiv 1910.02054 confirms title and authors. Section 3.1 (read via ar5iv HTML) states mixed-precision Adam needs 2+2 bytes per parameter for fp16 weights and gradients plus 12 bytes for fp32 weights, momentum and variance, i.e. 16 bytes per parameter before activations. Venue not recorded here.

Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, Matt Gardner · Proceedings of NAACL-HLT 2021, 4599-4610 · 2021

Describes how QASPER questions, answers and evidence were collected; supports project-038 dataset description.

Checked 2026-10-11: ACL Anthology .bib for 2021.naacl-main.365 confirms title, first author Dasigi, NAACL 2021 proceedings, pages 4599-4610; arXiv 2105.03011 lists the six authors.

Prasann Singhal, Tanya Goyal, Jiacheng Xu, Greg Durrett · COLM 2024 · 2023

Reward improvements in RLHF came largely from longer outputs; a length-only reward reproduced most downstream gains over SFT; reward models were the main source of the length bias. Supports insight 0570.

Checked 2026-10-11: arXiv abs page (v2 2024-07-10; Comments: accepted to COLM 2024) and abstract. arXiv year 2023, venue year 2024.

Karen Sparck Jones · Journal of Documentation 28(1), 11-21 · 1972

Origin of inverse document frequency weighting; supports day 41 (TF-IDF baseline).

Checked 2026-10-11: Crossref record for DOI 10.1108/eb026526 confirms title, author Sparck Jones, Journal of Documentation vol 28 issue 1 pp 11-21, Jan 1972.

Evan Miller · 2024

Recommends reporting standard errors, clustering them when questions are grouped, comparing two models with paired question-level differences, and using power analysis to choose the number of questions. Supports insights 0555 and 0556.

Checked 2026-10-11: arXiv abs page (title, sole author, abstract; no venue, preprint); recommendations checked in the arXiv HTML full text (Sections 2.1, 2.2, 4.2, 5).

Jiaqi Mu, Suma Bhat, Pramod Viswanath · arXiv:1702.01417 (cs.CL, stat.ML); no venue shown on the arXiv page · 2017

Checked 2026-10-12: arXiv abstract page: title, three authors, v1 5 Feb 2017, v2 19 Mar 2018. Abstract: removing the shared mean vector and a few dominant directions from word2vec/GloVe vectors improves lexical tasks and sentence-level tasks (semantic textual similarity, text classification) across datasets and languages. Preprint; no peer-reviewed venue confirmed.

Stefan Larson et al. · Proceedings of EMNLP-IJCNLP 2019, 1311-1316 · 2019

Introduces CLINC150 and the out-of-scope evaluation problem; supports project-049.

Checked 2026-10-11: ACL Anthology .bib for D19-1131 confirms title, first author Larson, EMNLP-IJCNLP 2019, pages 1311-1316.

Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel R. Bowman, Noah A. Smith · NAACL 2018; arXiv:1803.02324 · 2018

Checked 2026-10-12: arXiv abstract page: title, six authors, NAACL 2018. Abstract: in much of the data the hypothesis alone reveals the label; negation and vague wording correlate with specific labels; a hypothesis-only text classifier reaches about 67% on SNLI and 53% on MultiNLI.

Mor Geva, Yoav Goldberg, Jonathan Berant · EMNLP-IJCNLP 2019; arXiv:1908.07898 · 2019

Checked 2026-10-12: arXiv abstract page: title, three authors, EMNLP-IJCNLP 2019. Abstract: on three datasets, adding annotator identifiers as features improves performance, models can identify prolific annotators and often fail to generalise to examples from annotators absent from training; recommends keeping test-set annotators separate from training-set annotators.

Tao Lei et al. (repository taolei87/askubuntu) · 2016

Real support-forum questions with human relevance labels; used for support-ticket semantic search (project-047).

Checked 2026-10-11: README describes about 167K AskUbuntu questions from the 2014 dump (bodies truncated to 100 words), user-marked duplicates for training and dev/test files with 20 BM25 candidates per query and human similar/non-similar labels. dev.txt and test.txt were downloaded (download-verified) - 200 queries each with 20 candidates; 11 dev and 14 test queries have no similar candidate. GitHub API reports no repository licence. Stack Exchange help page states that public user contributions are CC BY-SA 2.5 (before 2011-04-08), 3.0 (to 2018-05-02) or 4.0 (after).

Sarthak Jain, Byron C. Wallace · Proceedings of NAACL-HLT 2019 · 2019

Attention weights often do not track feature importance; supports day 39 caveat against reading attention maps as explanations.

Checked 2026-10-11: ACL Anthology page N19-1357 confirms title; arXiv 1902.10186 confirms authors and "Accepted as NAACL 2019 Long Paper".

Sarah Wiegreffe, Yuval Pinter · EMNLP-IJCNLP 2019 · 2019

Nuance to Jain and Wallace (day 39).

Checked 2026-10-11: ACL Anthology D19-1002 shows the title; arXiv 1908.04626 confirms authors and "Accepted to EMNLP 2019". Abstract: whether attention explains depends on the definition of explanation; proposes tests including a uniform-weights baseline and seed-variance calibration.

Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, Song Han · arXiv:2306.00978 (cs.CL) · 2023

Venue not read from the page; cite as arXiv. The calibration-set robustness is the authors' claim.

Checked 2026-10-12: arXiv abstract page: title, ten authors, v1 1 Jun 2023. Abstract: weights are not equally important; protecting about 1% salient weights greatly reduces quantisation error; salient channels are found from the activation distribution, not the weights; the method uses activation statistics collected offline, needs no backpropagation or reconstruction and is said not to overfit the calibration set. GPTQ comparison not read.

Armand Joulin, Edouard Grave, Piotr Bojanowski, Tomas Mikolov · EACL 2017 · 2017

Linear text-classification baselines (day 41).

Checked 2026-10-11: ACL Anthology E17-2068 shows the title; arXiv 1607.01759 confirms authors. Abstract: fastText is often on par with deep learning classifiers in accuracy and orders of magnitude faster.

PolyAI (Casanueva et al.) · 2020

Fine-grained single-domain intents; used for LLM-vs-TF-IDF classification (project-045) and the PEFT plan (project-048).

Checked 2026-10-11: GitHub API reports license CC-BY-4.0 for PolyAI-LDN/task-specific-datasets; banking_data/train.csv and test.csv were downloaded (download-verified) - columns text, category; 10,003 train and 3,080 test rows; 77 categories in each. Hugging Face card PolyAI/banking77 also states cc-by-4.0 but the Hub entry is a loading-script dataset that the dataset viewer cannot load.

Nandan Thakur, Nils Reimers, Andreas Ruckle, Abhishek Srivastava, Iryna Gurevych · NeurIPS 2021 Datasets and Benchmarks Track (per author comment) · 2021

Embedding domain shift (days 38, 48, 51).

Checked 2026-10-11: arXiv 2104.08663 confirms title and authors; comment "Accepted at NeurIPS 2021 Dataset and Benchmark Track". Abstract: across 18 datasets BM25 is a robust zero-shot baseline, while dense retrievers often underperform out of domain.

Fanjia Yan, Huanzhi Mao, Charlie Cheng-Jie Ji, Ion Stoica, Joseph E. Gonzalez, Tianjun Zhang, Shishir G. Patil · Gorilla project blog, UC Berkeley · 2024

Supports insight 1513. Methodology of one benchmark, not a universal standard.

Checked 2026-10-12: Blog page opened 2026-10-12. AST evaluation parses each model output into a syntax tree and checks function name, required parameters present, no undocumented parameters, type and value checks (strings compared case-insensitively after normalisation, lists in order); executable evaluation runs the calls and compares outputs (exact match, 20% tolerance for live numeric data, structural checks); AST is used because not all functions are easily executable (for example Java); relevance detection gives functions that do not fit the request and expects no call. The title field here is descriptive (the exact post title was not recorded); the URL, authors and last-updated date were read on the page.

Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, Sameer Singh · Proceedings of the 58th Annual Meeting of the ACL, 4902-4912 · 2020

Capability-by-test-type matrix (minimum functionality, invariance, directional expectation tests); supports golden-set design in project-043.

Checked 2026-10-11: ACL Anthology .bib for 2020.acl-main.442 confirms title, first author Ribeiro, ACL 2020, pages 4902-4912.

FiscalNote (Kornilova and Eidelman) · 2019

Public-sector source documents with human reference summaries; used for summary faithfulness evaluation (project-046).

Checked 2026-10-11: Hub API and card show license cc0-1.0, not gated, parquet files; splits train 18,949, test 3,269, ca_test 1,237; fields text, summary, title. The card states US bills come from GovInfo (GPO) under CC0-1.0, while California bills come from the legislature website with no licence stated in the card.

Anastassia Kornilova, Vladimir Eidelman · Proceedings of the 2nd Workshop on New Frontiers in Summarization, 48-56 · 2019

Documents how BillSum was assembled; supports project-046.

Checked 2026-10-11: ACL Anthology .bib for D19-5406 confirms title, authors Kornilova and Eidelman, workshop proceedings, pages 48-56; arXiv 1910.00523 matches.

Sebastian Raschka · Manning Publications · 2024

Builds a GPT-style model in PyTorch step by step. Most useful: ch. 2 Working with Text Data, ch. 3 Coding Attention Mechanisms, ch. 4 Implementing a GPT Model from Scratch, ch. 7 Finetuning to Follow Instructions.

Checked 2026-10-11: Title, author, September 2024, ISBN 9781633437166 on the Manning landing page; chapter titles from the author's official GitHub repo (rasbt/LLMs-from-scratch) README.

Erik S., Barry Zhang (Anthropic) · 2024

Vendor engineering guidance (practitioner experience, not a controlled study); supports day 49 workflow-vs-agent choice.

Checked 2026-10-11: Page title, date 2024-12-19, authors as bylined. Content: workflows use predefined code paths, agents direct their own tool use; start with the simplest solution and add agentic complexity only when needed; agents trade latency and cost for performance and risk compounding errors; recommends sandboxed testing, guardrails and stopping conditions such as a maximum number of iterations; tool definitions deserve the same care as a human interface.

Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, Sameer Singh · Proceedings of the 38th International Conference on Machine Learning (ICML 2021), PMLR 139, 12697-12706 · 2021

Few-shot accuracy swings from near chance to near state of the art depending on prompt format, chosen examples and their order; names majority-label, recency and common-token biases. Supports insight 0523.

Checked 2026-10-11: PMLR page (title, authors, volume, pages, abstract); majority-label and recency bias named in the paper body (Sections 1 and 4).

Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, Bryan Hooi · ICLR 2024 · 2024

LLMs tend to be overconfident when verbalising confidence; sampling consistency and aggregation help but no single method wins consistently. Supports insight 0565.

Checked 2026-10-11: arXiv abs page (Comments: Accepted by ICLR 2024) and abstract; also listed in the proceedings.iclr.cc 2024 index.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou · Advances in Neural Information Processing Systems 35 (NeurIPS 2022), 24824-24837 · 2022

Few-shot exemplars with intermediate reasoning steps improve multi-step tasks in large models; supports day 47 (prompt engineering).

Checked 2026-10-11: arXiv abs 2201.11903 confirms title and 9 authors. Crossref record for DOI 10.52202/068431-1800 confirms NeurIPS 35 proceedings pp 24824-24837.

Hugging Face

Models, even those fine-tuned from the same base, expect different chat formats and control tokens; wrong control tokens can make performance drastically worse. Supports insight 0508.

Checked 2026-10-11: Live docs page (links to v5.19.0 docs): text on per-model chat formats and control tokens, apply_chat_template, add_generation_prompt and the warning about duplicated special tokens. No publication date shown.

Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios N. Angelopoulos, Tianle Li, Dacheng Li, Hao Zhang, Banghua Zhu, Michael Jordan, Joseph E. Gonzalez, Ion Stoica · arXiv:2403.04132 (cs.AI); no venue shown on the arXiv page · 2024

Supports the claim that many-variant pairwise comparisons are summarised with a Bradley-Terry model and reported with confidence intervals.

Checked 2026-10-12: arXiv abstract page (title, 11 authors, submitted 7 Mar 2024) and PDF text. Full text: ranking uses the Bradley-Terry model on pairwise votes with an inverse-probability-weighted estimator, confidence intervals on the BT coefficients by pivot bootstrap or sandwich robust standard errors (sandwich deployed), users can vote tie, and model-pair sampling is chosen actively to speed ranking convergence.

Clinc (Larson et al.) · 2019

Labelled domains and an explicit out-of-scope class make it possible to inject known drift for monitoring experiments (project-049).

Checked 2026-10-11: Repository LICENSE file is Creative Commons Attribution 3.0 Unported; README describes 150 in-scope intents over 10 domains. data/data_full.json and data/domains.json were downloaded (download-verified) - 15,000 train, 3,000 val, 4,500 test in-scope queries plus 100/100/1,000 out-of-scope train/val/test; domains.json maps 15 intents to each of 10 domains. Hugging Face clinc/clinc_oos (cc-by-3.0) mirrors it as parquet.

European Commission (published via EUR-Lex) · 2024

Longer interpretative text that paraphrases and extends the regulation; creates realistic near-duplicate evidence and version conflicts for retrieval evaluation (project-036).

Checked 2026-10-11: EUR-Lex page for document 52024XC05687 loads with the title above and the reference OJ C, C/2024/5687, 25.9.2024; the earlier 2016 version (CELEX 52016XC0615(01)) also loads. Reuse terms as in the EUR-Lex legal notice (Decision 2011/833/EU).

Yufeng Du, Minyang Tian, Srikanth Ronanki, Subendhu Rongali, Sravan Bodapati, Aram Galstyan, Azton Wells, Roy Schwartz, Eliu A Huerta, Hao Peng · Findings of EMNLP 2025; arXiv 2510.05381 · 2025

Supports insight 1506.

Checked 2026-10-12: arXiv abstract page opened 2026-10-12: title, ten authors, submitted 2025-10-06, accepted at Findings of EMNLP 2025. Abstract: with perfect retrieval, performance still drops by 13.9% to 85% as input length grows within claimed limits; the drop persists when irrelevant tokens are replaced by whitespace or masked, and when evidence is placed right before the question; five open and closed LLMs on math, QA and coding; mitigation of reciting retrieved evidence before solving improved GPT-4o by up to 4% on RULER. Full text not read.

Anthropic

At check time the temperature description says results are not fully deterministic even at 0.0; top_p and top_k are marked for advanced use; the page also marks these sampling parameters as deprecated (restricted) for newer models. Vendor behaviour changes: re-check before citing specifics. Supports insight 0510.

Checked 2026-10-11: Messages API reference page: temperature, top_p and top_k parameter descriptions and the stop_reason enum. No publication date shown.

Kalervo Järvelin, Jaana Kekäläinen · ACM Transactions on Information Systems 20(4), 422-446 · 2002

Defines (normalized) discounted cumulated gain for graded relevance; supports day 51 (retrieval metrics such as nDCG@k).

Checked 2026-10-11: Crossref record for DOI 10.1145/582415.582418 confirms title, two authors, ACM TOIS vol 20 issue 4 pp 422-446, Oct 2002.

scikit-learn developers

TruncatedSVD on a tf-idf matrix is LSA; used in the Day 41 application code.

Checked 2026-10-11: Opened the User Guide page: the section is numbered 2.5.3 under 2.5 'Decomposing signals in components'; it states that TruncatedSVD differs from PCA in not requiring a centred matrix, that applying it to tf-idf matrices (rather than raw counts) is recommended for LSA, and suggests sublinear_tf and use_idf. No publication date shown.

Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, Nicholas Carlini · ACL 2022 · 2022

Near-duplicate detection and evaluation validity (days 41, 43).

Checked 2026-10-11: ACL Anthology 2022.acl-long.577 shows the title; arXiv 2107.06499 confirms authors. Abstract: LM datasets contain many near-duplicates; train-test overlap affects over 4% of the validation set of standard datasets; deduplication tools released.

Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, Dario Amodei · Advances in Neural Information Processing Systems 30 (NIPS 2017) · 2017

Learning a reward model from pairwise human comparisons; conceptual origin of RLHF for day 45.

Checked 2026-10-11: NeurIPS proceedings abstract page metadata confirms title, six authors and 2017; arXiv 1706.03741 matches.

Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, Wen-tau Yih · Proceedings of EMNLP 2020 · 2020

Dual-encoder dense retrieval trained with in-batch negatives, compared with BM25; supports days 48 and 51.

Checked 2026-10-11: ACL Anthology page 2020.emnlp-main.550 confirms title; arXiv 2004.04906 confirms 8 authors and comment "EMNLP 2020".

Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, Chelsea Finn · Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 53728-53741 · 2023

Optimizes preferences directly with a classification-style loss, no explicit reward model or RL loop; supports day 45.

Checked 2026-10-11: arXiv abs 2305.18290 confirms title and six authors. Crossref record for DOI 10.52202/075280-2338 confirms NeurIPS 36 proceedings pp 53728-53741.

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, Jeffrey Dean · Advances in Neural Information Processing Systems 26 (NIPS 2013) · 2013

Negative sampling and subsampling for skip-gram; supports day 38 and the contrastive-objective thread.

Checked 2026-10-11: NeurIPS proceedings abstract page metadata confirms title, five authors and 2013; arXiv 1310.4546 matches.

Zorik Gekhman, Gal Yona, Roee Aharoni, Matan Eyal, Amir Feder, Roi Reichart, Jonathan Herzig · Proceedings of EMNLP 2024, 7765-7784 · 2024

Fine-tuning examples that introduce new knowledge are learned much more slowly than examples consistent with the model's knowledge, and as they are learned the model's tendency to hallucinate increases linearly. Supports insight 0540.

Checked 2026-10-11: arXiv abs page (v3 2024-10-01; Comments: accepted as a long paper at EMNLP 2024) and abstract; Crossref record for DOI 10.18653/v1/2024.emnlp-main.444 confirms proceedings and pages.

Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, Noah A. Smith · ACL 2020 · 2020

Domain adaptation of pretrained encoders (days 38, 40, 43).

Checked 2026-10-11: ACL Anthology 2020.acl-main.740 shows the title; arXiv 2004.10964 confirms authors. Abstract: a second phase of in-domain pretraining improves performance across four domains and eight classification tasks.

Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, Noah A. Smith · Proceedings of the 58th Annual Meeting of the ACL, 8342-8360 · 2020

A second phase of pretraining on in-domain text (DAPT) and on unlabelled task data (TAPT) improved downstream performance across four domains. Supports insight 0544.

Checked 2026-10-11: ACL Anthology page (title, authors, venue, pages, DOI 10.18653/v1/2020.acl-main.740) and abstract.

Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, Zhiyi Ma, Tristan Thrush, Sebastian Riedel, Zeerak Waseem, Pontus Stenetorp, Robin Jia, Mohit Bansal, Christopher Potts, Adina Williams · NAACL 2021 (arXiv:2104.14337) · 2021

Checked 2026-10-12: arXiv abstract page (title, 19 authors, v1 7 Apr 2021, NAACL 2021 noted). Abstract: models reach strong benchmark scores quickly yet still fail simple challenge examples; Dynabench collects new examples with annotators writing items that a target model gets wrong but a person answers correctly.

Office of the Federal Register / U.S. Government Publishing Office (eCFR)

Second jurisdiction for the flagship corpus (project-036); EU-vs-US rule conflicts are a deliberate retrieval failure mode.

Checked 2026-10-11: eCFR versioner API (titles.json and structure for title 14, up to date as of 2026-10-07) lists Part 250 Oversales, Part 259 Enhanced Protections for Airline Passengers and Part 260 Refunds for Airline Fare and Ancillary Service Fees. GovInfo policies page restates 17 U.S.C. 105 (no copyright in works of the U.S. Government) and notes that government publications may contain third-party copyrighted material.

Prithvi Rajasekaran, Ethan Dixon, Carly Ryan, Jeremy Hadfield (Anthropic) · 2025

Vendor engineering guidance (practitioner experience); supports context and memory management in long agent runs.

Checked 2026-10-12: Opened 2026-10-12. Title, bylined authors, date 2025-09-29. Content: compaction summarizes a conversation near the context limit and continues in a fresh window, keeping key decisions and open issues and dropping redundant output; tool result clearing removes raw tool outputs deep in history; structured note-taking persists notes outside the context window; just-in-time retrieval keeps lightweight references and loads data on demand; sub-agents return condensed summaries.

Justin Young (Anthropic) · 2025

Vendor engineering report (one team's experience, not a controlled study); supports long-running task design and completion verification.

Checked 2026-10-12: Opened 2026-10-12. Title, byline and date 2025-11-26 as shown. Content: agents that tried to do everything in one pass ran out of context mid-feature, and later sessions sometimes saw partial progress and declared the project finished; an initializer session writes a JSON feature list with every feature failing plus a progress file and git history; later coding agents may only change a feature's pass/fail status; one feature per session; agents often marked features done after only unit tests or curl checks, and prompting for end-to-end browser testing improved results.

Inigo Casanueva, Tadas Temcinas, Daniela Gerz, Matthew Henderson, Ivan Vulic · Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI, 38-45 · 2020

Introduces BANKING77 and reports few-shot intent detection with sentence encoders; supports projects 045 and 048.

Checked 2026-10-11: ACL Anthology .bib for 2020.nlp4convai-1.5 confirms title, first author Casanueva, NLP4ConvAI 2020 workshop, pages 38-45; arXiv 2003.04807 lists the five authors.

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, Ion Stoica · Proceedings of the 29th Symposium on Operating Systems Principles (SOSP 2023), ACM, 611-626 · 2023

Each request's key-value cache is large and grows and shrinks dynamically; wasted cache memory limits batch size and throughput. Supports insight 0506.

Checked 2026-10-11: arXiv abs page (title, authors, Comments SOSP 2023, abstract); Crossref record for DOI 10.1145/3600006.3613165 confirms SOSP '23 proceedings pp 611-626.

Tianyu Gao, Howard Yen, Jiatong Yu, Danqi Chen · Proceedings of EMNLP 2023, 6465-6488 · 2023

Citation-quality evaluation for retrieval-augmented answers; supports days 48, 51 and 57.

Checked 2026-10-11: ACL Anthology 2023.emnlp-main.398 (via search listing) and arXiv 2305.14627 comment 'Accepted by EMNLP 2023'. Abstract: ALCE benchmark scores fluency, correctness and citation quality; on ELI5 even the best models lack complete citation support about half the time.

Piotr Bojanowski, Edouard Grave, Armand Joulin, Tomas Mikolov · arXiv:1607.04606 (cs.CL); accepted to TACL per the arXiv page · 2017

Checked 2026-10-12: arXiv abstract page: title, four authors, v1 15 Jul 2016, v2 19 Jun 2017. Abstract: each word is represented as a bag of character n-grams and the word vector is the sum of its n-gram vectors, which handles rare and unseen words and is stated to matter especially for morphologically rich languages; evaluated on word similarity and analogy tasks in nine languages.

Mark Chen, Jerry Tworek and 56 co-authors · arXiv:2107.03374 (cs.LG); no venue shown on the arXiv page · 2021

Checked 2026-10-12: arXiv abstract page (title, first authors, v1 7 Jul 2021) and PDF text. Full text: pass@k = fraction of problems solved if any of k samples passes unit tests; the naive computation has high variance, so the paper generates n >= k samples per task (n=200, k<=100), counts c correct, and uses the unbiased estimator 1 - C(n-c,k)/C(n,k); the shortcut 1-(1-p)^k with p the pass@1 estimate is shown biased (Appendix A).

Nelson F. Liu, Tianyi Zhang, Percy Liang · Findings of the Association for Computational Linguistics - EMNLP 2023, 7001-7025 · 2023

Citation recall vs citation precision; supports days 51 and 57.

Checked 2026-10-11: arXiv 2304.09848 abstract and comment (Findings of EMNLP 2023); Anthology id 2023.findings-emnlp.467 with pages 7001-7025 from the Anthology BibTeX listing. Human audit of four generative search engines: about 51.5% of generated sentences fully supported by citations, about 74.5% of citations support their sentence; defines citation recall and citation precision.

Anthropic

Registered here during verification for the reasoning-token claims; also cited by another batch (insight 1639).

Checked 2026-10-12: Page opened 2026-10-12 (plus the Thinking overview page and the Messages API reference). Thinking tokens are billed as output tokens even when thinking text is not returned and count toward max_tokens alongside the response text; budget_tokens must be below max_tokens (except interleaved thinking); the budget is a target not a strict cap; usage.output_tokens_details.thinking_tokens reports the reasoning tokens; adaptive thinking may skip thinking on easy inputs at lower effort. Models 4.7 and later reject type enabled with a 400.

Sewon Min et al. (9 authors) · Proceedings of EMNLP 2023, 12076-12100 · 2023

Binary judgments are inadequate for long-form text that mixes supported and unsupported claims; FActScore is the share of atomic facts supported by a knowledge source. Supports insight 0514.

Checked 2026-10-11: ACL Anthology page (title, nine authors, booktitle, pages, DOI 10.18653/v1/2023.emnlp-main.741) and abstract.

Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, Pontus Stenetorp · Proceedings of the 60th Annual Meeting of the ACL (Volume 1, Long Papers), 8086-8098 · 2022

The order of few-shot samples can make the difference between near state-of-the-art and random-guess performance; good orders do not transfer between models. Supports insight 0523.

Checked 2026-10-11: ACL Anthology page (title, authors, venue, pages, DOI 10.18653/v1/2022.acl-long.556) and abstract.

Oded Ovadia, Menachem Brief, Moshik Mishaeli, Oren Elisha · Proceedings of EMNLP 2024, 237-250 · 2024

Unsupervised fine-tuning gave some improvement, but RAG consistently outperformed it for both knowledge seen in training and entirely new knowledge. Supports insight 0540.

Checked 2026-10-11: ACL Anthology page (title, authors, venue, pages, DOI 10.18653/v1/2024.emnlp-main.15); arXiv 2312.05934 abstract.

Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, Noah Smith · 2020

Fine-tuning variance across seeds (days 40, 43).

Checked 2026-10-11: arXiv 2002.06305 confirms title and authors (preprint). Abstract: distinct seeds give substantially different BERT fine-tuning results; weight initialization and data order contribute comparably; many trials diverge on small datasets.

Lingjiao Chen, Matei Zaharia, James Zou · 2023

Model cascades and routing; results are task-specific and from 2023 API prices; supports day 54.

Checked 2026-10-11: arXiv abs 2305.05176: title, three authors, submitted 2023-05-09, no venue listed (preprint). Abstract: API prices differ by up to two orders of magnitude; strategies are prompt adaptation, LLM approximation and LLM cascades; the learned cascade matched GPT-4 at up to 98% lower cost on the authors' tasks.

Hugging Face

Checked 2026-10-12: Opened the page (v5.19.0 rendering). Attention mask section: for the BERT tokenizer 1 marks tokens that should be attended to and 0 marks padded values; position IDs section: transformers are unaware of token position without them, and absolute positional embeddings are selected in [0, config.max_position_embeddings - 1].

Jeffrey Pennington, Richard Socher, Christopher D. Manning · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · 2014

Embeddings from global co-occurrence statistics; supports day 38 comparison of count-based vs predictive embeddings.

Checked 2026-10-11: ACL Anthology page D14-1162 citation metadata confirms title, three authors, EMNLP 2014 proceedings (2014/10).

OpenAI (280 listed authors) · 2023

Figure 8 shows calibration of answer probabilities worsening after post-training. Vendor technical report about one model; supports insight 0565.

Checked 2026-10-11: arXiv abs page (v6 2024-03-04; no journal-ref, not peer reviewed). Section 5 (Limitations) and Figure 8 read in the PDF: on an MMLU subset the pre-trained model's answer log-probabilities are well calibrated and post-training (PPO) significantly reduces calibration.

OpenAI (Cobbe et al.) · 2021

Automatically checkable answers make it a convenient stand-in workload for a cost/quality routing study (project-044).

Checked 2026-10-11: Hub API and card show license mit, not gated; main config train 7,473 and test 1,319. The main/test parquet was downloaded (download-verified) - 1,319 rows, columns question and answer, with the final numeric answer after a "####" marker.

Anthropic

Vendor API rule as read on 2026-10-12; other providers use different message formats and rules.

Checked 2026-10-12: Page opened 2026-10-12 (as read). Each tool_use block has an id used to match the later tool_result via tool_use_id; tool_result blocks must immediately follow their tool_use blocks in the history with no messages between, and must come first in the user message's content array; a history that breaks this fails with an error such as 'tool_use ids were found without tool_result blocks immediately after'.

Angela Fan, Mike Lewis, Yann Dauphin · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 889-898 · 2018

Early, widely cited use of top-k sampling for text generation (Day 42).

Checked 2026-10-11: ACL Anthology page confirms title, three authors, ACL 2018 long papers, pages 889-898, DOI 10.18653/v1/P18-1082. The arXiv PDF (1805.04833), Section 5.4 'Generation', describes sampling from the k = 10 most likely candidates at each step (top-k random sampling) and tuning a softmax temperature at generation time; the abstract itself does not mention top-k.

Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras et al. (50 authors, Stanford CRFM) · Transactions on Machine Learning Research (TMLR), 2023 · 2023

Multi-metric, multi-scenario evaluation (accuracy, calibration, robustness, fairness, efficiency); supports day 45 argument against single-number leaderboards.

Checked 2026-10-11: arXiv abs 2211.09110 confirms title, first author Liang, 50 authors; journal_ref field reads "Published in Transactions on Machine Learning Research (TMLR), 2023". arXiv first version 2022-11-16.

Kawin Ethayarajh · EMNLP-IJCNLP 2019 · 2019

Anisotropy makes raw cosine values hard to interpret (day 38).

Checked 2026-10-11: ACL Anthology D19-1006 shows the title; arXiv 1909.00512 confirms author and "Accepted to EMNLP 2019". Abstract: contextualized representations of all words are not isotropic in any layer.

Phillip Rust, Jonas Pfeiffer, Ivan Vulic, Sebastian Ruder, Iryna Gurevych · ACL 2021 · 2021

Tokenizer quality affects task performance (days 41, 42).

Checked 2026-10-11: ACL Anthology 2021.acl-long.243 shows the title; arXiv 2012.15613 confirms authors. Abstract: besides pretraining data size, a dedicated monolingual tokenizer plays an equally important role in downstream performance.

Lingjiao Chen, Matei Zaharia, James Zou · arXiv preprint 2307.09009 (v3 revised 2023-10) · 2023

Supports insight 1512. Two snapshots of two models in 2023; not a general rate of drift.

Checked 2026-10-12: arXiv abstract page opened 2026-10-12: title, three authors. Abstract: GPT-3.5 and GPT-4 March 2023 and June 2023 versions changed substantially on several tasks (for example GPT-4 prime-number identification 84% to 51%, attributed partly to reduced responsiveness to chain-of-thought prompting); both models made more formatting mistakes in code generation in June; evidence that GPT-4's instruction following declined; the authors call for continuous monitoring of LLM services. Full text not read.

Lingjiao Chen, Matei Zaharia, James Zou · arXiv:2307.09009 (cs.CL) · 2023

Measures dated snapshots of a hosted service in 2023; drift size is specific to that study. Same paper as paper-chen-2023-chatgpt-behavior-drift (expansion-insights-llm-1); kept because insight-1583 cites this id.

Checked 2026-10-12: arXiv abstract page: title, three authors, v1 18 Jul 2023 (v3 31 Oct 2023). Abstract: March 2023 and June 2023 versions of GPT-3.5 and GPT-4 were compared on seven task types; performance and behaviour changed substantially; GPT-4 prime-versus-composite accuracy fell from 84% to 51%; more code formatting errors in June; the authors conclude LLM services need continuous monitoring.

Daniel Jaroslawicz, Brendan Whiting, Parth Shah, Karime Maamari · arXiv preprint 2507.11538 · 2025

Supports insight 1502. A synthetic keyword-inclusion task, so the 68% figure is not a threshold for other tasks.

Checked 2026-10-12: arXiv abstract page opened 2026-10-12: title, four authors, submitted 2025-07-15. Abstract: IFScale benchmark of 500 keyword-inclusion instructions inside a business-report writing task; 20 models from seven providers; best frontier models reach 68% accuracy at the maximum density of 500 instructions; three degradation patterns correlate with model size and reasoning capability; bias toward earlier instructions. Full text not read.

Hugging Face

Free course on Transformers, tokenizers and fine-tuning; optional reading for days 41-44.

Checked 2026-10-11: Chapter 1 introduction page loads (title "Introduction · Hugging Face"; page text labels it "LLM Course documentation").

Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever · OpenAI technical report · 2018

Decoder-only generative pretraining followed by supervised fine-tuning; supports day 43 (pretraining and fine-tuning).

Checked 2026-10-11: Opened the OpenAI-hosted PDF (≈0.5 MB); first page shows title and four authors (OpenAI). PDF has no printed date; year 2018 from the GPT-2 report, which cites "the OpenAI GPT model (Radford et al., 2018)". Not peer reviewed. The openai.com blog page returned HTTP 403.

Scott Deerwester, Susan T. Dumais, George W. Furnas, Thomas K. Landauer, Richard Harshman · Journal of the American Society for Information Science 41(6), 391-407 · 1990

SVD of the term-document matrix as an early dense semantic representation; supports day 38 and 41 (from sparse to dense).

Checked 2026-10-11: Crossref record for this DOI confirms title, five authors (Deerwester, Dumais, Furnas, Landauer, Harshman), JASIS vol 41 issue 6 pp 391-407, Sept 1990.

Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, Le Hou · 2023

IFEval uses about 500 prompts with 25 types of automatically checkable verifiable instructions (e.g., word counts, keyword frequency), avoiding human-evaluation cost and LLM-judge bias. Supports insight 0562.

Checked 2026-10-11: arXiv abs page (title, authors, abstract). No Comments or journal-ref; no venue confirmed (preprint).

Liang Wang et al. (intfloat)

One concrete example of per-model input formatting and silent truncation; other embedding models have different rules, so read each model's card.

Checked 2026-10-12: Model card opened. Each input should start with 'query: ' or 'passage: '; the FAQ says the prefix is required because of how the model is trained and omitting it degrades performance; 'passage: ' for the retrieved side of asymmetric retrieval; long texts are truncated to at most 512 tokens.

Anthropic · 2024

Vendor-reported, not peer reviewed; use as a hypothesis to test on your own golden set (day 48).

Checked 2026-10-11: Page title and date 2024-09-19. Method: prepend model-generated, chunk-specific context (about 50-100 tokens) to each chunk before embedding and BM25 indexing. Vendor-reported top-20 retrieval failure rate (1 - recall@20) on their test sets fell from 5.7% to 3.7% (contextual embeddings), 2.9% (plus contextual BM25) and 1.9% (plus reranking).

Christopher D. Manning, Prabhakar Raghavan, Hinrich Schütze · Cambridge University Press · 2008

Ch. 8 (evaluation in IR) for ranked-retrieval metrics; supports day 51.

Checked 2026-10-11: Companion site lists title, three authors, Cambridge University Press 2008, and links an HTML edition and chapter PDFs. Section 8.4 'Evaluation of ranked retrieval results' defines precision at k, R-precision, MAP and NDCG.

Harald Steck, Chaitanya Ekanadham, Nathan Kallus · Companion Proceedings of the ACM Web Conference 2024, 887-890 · 2024

Cosine vs dot product caveats (days 38, 48).

Checked 2026-10-11: Crossref record for DOI 10.1145/3589335.3651526 confirms title, authors and venue (pp 887-890, May 2024). Abstract: cosine can work better or worse than the unnormalized dot product; for regularized linear models cosine similarities can be arbitrary.

Shusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye, Weilin Liu, Zhiyu Mei, Guangju Wang, Chao Yu, Yi Wu · ICML 2024 · 2024

DPO can find solutions that exploit out-of-distribution responses and is sensitive to shift between model outputs and preference data; carefully tuned PPO beat DPO across the dialogue and code tasks studied. Supports insight 0568.

Checked 2026-10-11: arXiv abs page (journal-ref ICML 2024) and PDF body (Introduction, Section 4) for DPO exploiting out-of-distribution responses and sensitivity to distribution shift; abstract for carefully tuned PPO outperforming other methods in their experiments.

JSON Schema

The schema language used for tool inputs and structured outputs; supports days 47 and 49 (validation of model output).

Checked 2026-10-11: Page loads ("JSON Schema - Specification") and links the 2020-12 Core and Validation documents.

Pranav Rajpurkar, Robin Jia, Percy Liang · ACL 2018 (per arXiv comment) · 2018

Introduces SQuAD 2.0 and the requirement to abstain on unanswerable questions.

Checked 2026-10-11: arXiv export API: title, authors, first version 2018-06-11, comment 'ACL 2018'.