Books, papers and documentation

RAND Corporation; packaged by statsmodels · statsmodels datasets

Project 003 (price sensitivity of doctor visits under randomized coinsurance).

Checked 2026-10-11: statsmodels 0.15.0 docs page and installed package source (datasets/randhie/data.py): COPYRIGHT 'in the public domain', source RAND HIE via Cameron and Trivedi (2005) data site, subset of original data, 20,190 observations x 10 variables with definitions; bundled src/randhie.csv has 45 columns incl. plan, site, coins, year and person id zper (5,912 persons, years 1-5). Loaded locally to confirm shape.

Jeff Gill (source); packaged by statsmodels · statsmodels datasets

Project 007 (district pass-rate analysis with a binomial GLM). The docs call the 303 cases both districts and counties; DESCRLONG says unified school districts.

Checked 2026-10-11: statsmodels 0.15.0 docs page and installed package source (datasets/star98/data.py): COPYRIGHT 'used with express permission from the original author, who retains all rights', source Gill's Generalized Linear Models: A Unified Approach, 303 district-level cases of 1998 California STAR results, NABOVE/NBELOW binomial response for 9th-grade math, variable definitions. Loaded locally: 303 rows x 22 columns.

Database Group of Prof. Erhard Rahm, University of Leipzig

Project 032 (embedding similarity for product matching).

Checked 2026-10-11: Provider page: Abt-Buy (e-commerce; attributes name, description, manufacturer, price; 1,081 + 1,092 entities; 1,097 matches; each zip has two source files plus a perfect-mapping file); Amazon-GoogleProducts (1,363 + 3,226; 1,300 matches); released under CC BY 4.0 with a request to credit the website and the VLDB 2010 paper. HEAD on /files/datasets/Abt-Buy.zip returned 200, 134,745 bytes; not downloaded, so exact CSV column headers were not inspected.

Efron, Hastie, Johnstone, Tibshirani (source); packaged by scikit-learn · scikit-learn 1.9.1 documentation, Toy datasets

Project 009 / reference r03 (tabular regression with nested CV).

Checked 2026-10-11: Toy datasets page (scikit-learn 1.9.1) has a 'Diabetes dataset' section; installed package DESCR: 442 patients, 10 baseline variables (age, sex, bmi, bp, s1-s6), target = quantitative disease progression one year after baseline, features mean-centred and scaled, source URL at NCSU and Efron et al. (2004). Files ship inside the package (sklearn/datasets/data/diabetes_*.csv.gz); loaded locally, shape 442 x 11. No license statement found for the data itself.

E. Alpaydin, C. Kaynak · 1998

Provenance of the scikit-learn digits copy; the writer split matters for generalisation claims.

Checked 2026-10-11: UCI page: title, creators Alpaydin and Kaynak, donated 1998-06-30, 5,620 instances, 64 integer features, CC BY 4.0; 32x32 bitmaps divided into non-overlapping 4x4 blocks with on-pixel counts giving an 8x8 matrix of 0-16; 30 writers in the training set and 13 different writers in the test set.

scikit-learn developers

Projects 034 (training dynamics, reference r07) and 035 (debugging transfer check).

Checked 2026-10-11: API page (scikit-learn 1.9.1): 1,797 samples, 10 classes, 64 features (8x8 images), integer values 0-16; page states it is a copy of the test set of the UCI handwritten digits data. Executed locally: load_digits() returns (1797, 64) with values 0-16 and about 180 per class; the file digits.csv.gz ships inside sklearn/datasets/data, so no download is needed.

Hadi Fanaee-T · UCI Machine Learning Repository · 2013

Hourly and daily rental counts (2011-2012) with calendar and realised-weather columns; used for the exogenous-availability project (weather is observed, not forecast) and the casual + registered = cnt leakage trap.

Checked 2026-10-11: UCI page: title, creator, donation date 12/19/2013, DOI 10.24432/C5W894, files hour.csv (1.1 MB) and day.csv (56.2 KB), 17,389 instances, variable list (dteday, hr, holiday, workingday, weathersit, temp, atemp, hum, windspeed, casual, registered, cnt), license CC BY 4.0.

Artur Trindade · UCI Machine Learning Repository · 2015

Multiple-seasonality project (96 and 672 intervals per day/week plus annual cycle); the documented DST artefacts and pre-start zeros are planted-looking but real quality rules.

Checked 2026-10-11: UCI page: title, creator, donation date 3/12/2015, DOI 10.24432/C58C86, 370 clients as columns, 15-minute kW values, Portuguese local time, daylight-saving handling (March: 1:00-2:00 zeros; October: two hours aggregated), clients created after 2011 recorded as zero before start, LD2011_2014.txt 678.1 MB, zip 249.2 MB, license CC BY 4.0.

Georges Hebrail, Alice Berard

Project 030 (1-D CNN / GRU vs classical day-ahead baselines).

Checked 2026-10-11: UCI page: title, creators Hebrail and Berard, donated 2012-08-29, 2,075,259 instances, 9 variables (Date, Time, Global_active_power kW, Global_reactive_power kW, Voltage V, Global_intensity A, Sub_metering_1/2/3 Wh with kitchen / laundry / water-heater-and-AC descriptions); one-minute sampling, December 2006 to November 2010, one house in Sceaux near Paris; missing values in about 1.25% of rows shown as empty fields between separators; CC BY 4.0; zip 19.7 MB, extracted text file 126.8 MB. Static download URL answered HEAD with 200; not downloaded.

Kaggle competition page · Kaggle · 2020

Hierarchical Walmart unit sales (store/state/category/department/item levels; 42,840 series across levels per the M5 results paper). Do not redistribute raw files in a portfolio.

Checked 2026-10-11: Competition page loads (HTTP 200) with title 'M5 Forecasting - Accuracy' and description 'Estimate the unit sales of Walmart retail goods'. Data tab content and rules are not readable without login, so file names, row counts and the data license were NOT confirmed on the page; series counts come from the registered M5 results paper (paper-makridakis-2022-m5-accuracy).

Daqing Chen · UCI Machine Learning Repository · 2019

Transaction log that aggregates to sparse SKU x day/week demand; used for the intermittent-demand project.

Checked 2026-10-11: UCI page: title, creator, donation date 2019-09-20, DOI 10.24432/C5CG6D, 1,067,371 instances, period 01/12/2009-09/12/2011, UK non-store gift-ware retailer with many wholesale customers, variables (InvoiceNo, StockCode, Description, Quantity, InvoiceDate, UnitPrice, CustomerID, Country), invoice numbers starting with 'c' are cancellations, missing values present, license CC BY 4.0.

Eurostat · Eurostat database

A recent, regularly updated public panel for evaluating time-series foundation models on months after their release; values are revised between releases (vintage caveat).

Checked 2026-10-11: Eurostat dissemination API (statistics/1.0/data/une_rt_m, JSON): label 'Unemployment by sex and age - monthly data', last update 2026-10-08, dataset DOI 10.2908/UNE_RT_M, monthly coverage 1983-01 to 2026-09 for Germany (total, NSA, % of labour force), 38 geographic entities in the latest period, dimensions freq/s_adj/age/unit/sex/geo/time. No login needed for the API.

Patrick Helber, Benjamin Bischke, Andreas Dengel, Damian Borth · 2018

Project 031 (frozen-backbone transfer learning, docs-only).

Checked 2026-10-11: GitHub README: 27,000 labelled, geo-referenced Sentinel-2 images in 10 classes, 13-band and RGB versions, MIT license plus a pointer to the Copernicus Sentinel data terms, hosted on Zenodo. Zenodo record 7711810: EuroSAT_RGB.zip 94.7 MB (HEAD: 94,658,721 bytes), EuroSAT_MS.zip 2.1 GB, MIT. TensorFlow Datasets catalog: rgb config images are 64x64x3 uint8, a single 'train' split of 27,000 examples (no official test split). Class names were not confirmed on the pages read; read them from the folder names after unzipping.

Han Xiao, Kashif Rasul, Roland Vollgraf (Zalando Research) · 2017

Project 029 (small CNN vs pixel baselines).

Checked 2026-10-11: Repository README: 60,000 training and 10,000 test images, 28x28 grayscale, 10 labelled classes (T-shirt/top, Trouser, Pullover, Dress, Coat, Sandal, Shirt, Sneaker, Bag, Ankle boot); MIT license, copyright Zalando SE; MNIST-format gz files on an S3 bucket with MD5 checksums (train images 26 MB); loaders listed for PyTorch, Keras, TensorFlow Datasets, Hugging Face Datasets. HEAD request on the train-images file returned 200 with 26,421,880 bytes; not downloaded. Citation arXiv 1708.07747.

Tao Lei et al. (repository taolei87/askubuntu) · 2016

Real support-forum questions with human relevance labels; used for support-ticket semantic search (project-047).

Checked 2026-10-11: README describes about 167K AskUbuntu questions from the 2014 dump (bodies truncated to 100 words), user-marked duplicates for training and dev/test files with 20 BM25 candidates per query and human similar/non-similar labels. dev.txt and test.txt were downloaded (download-verified) - 200 queries each with 20 candidates; 11 dev and 14 test queries have no similar candidate. GitHub API reports no repository licence. Stack Exchange help page states that public user contributions are CC BY-SA 2.5 (before 2011-04-08), 3.0 (to 2018-05-02) or 4.0 (after).

PolyAI (Casanueva et al.) · 2020

Fine-grained single-domain intents; used for LLM-vs-TF-IDF classification (project-045) and the PEFT plan (project-048).

Checked 2026-10-11: GitHub API reports license CC-BY-4.0 for PolyAI-LDN/task-specific-datasets; banking_data/train.csv and test.csv were downloaded (download-verified) - columns text, category; 10,003 train and 3,080 test rows; 77 categories in each. Hugging Face card PolyAI/banking77 also states cc-by-4.0 but the Hub entry is a loading-script dataset that the dataset viewer cannot load.

FiscalNote (Kornilova and Eidelman) · 2019

Public-sector source documents with human reference summaries; used for summary faithfulness evaluation (project-046).

Checked 2026-10-11: Hub API and card show license cc0-1.0, not gated, parquet files; splits train 18,949, test 3,269, ca_test 1,237; fields text, summary, title. The card states US bills come from GovInfo (GPO) under CC0-1.0, while California bills come from the legislature website with no licence stated in the card.

Clinc (Larson et al.) · 2019

Labelled domains and an explicit out-of-scope class make it possible to inject known drift for monitoring experiments (project-049).

Checked 2026-10-11: Repository LICENSE file is Creative Commons Attribution 3.0 Unported; README describes 150 in-scope intents over 10 domains. data/data_full.json and data/domains.json were downloaded (download-verified) - 15,000 train, 3,000 val, 4,500 test in-scope queries plus 100/100/1,000 out-of-scope train/val/test; domains.json maps 15 intents to each of 10 domains. Hugging Face clinc/clinc_oos (cc-by-3.0) mirrors it as parquet.

European Commission (published via EUR-Lex) · 2024

Longer interpretative text that paraphrases and extends the regulation; creates realistic near-duplicate evidence and version conflicts for retrieval evaluation (project-036).

Checked 2026-10-11: EUR-Lex page for document 52024XC05687 loads with the title above and the reference OJ C, C/2024/5687, 25.9.2024; the earlier 2016 version (CELEX 52016XC0615(01)) also loads. Reuse terms as in the EUR-Lex legal notice (Decision 2011/833/EU).

Office of the Federal Register / U.S. Government Publishing Office (eCFR)

Second jurisdiction for the flagship corpus (project-036); EU-vs-US rule conflicts are a deliberate retrieval failure mode.

Checked 2026-10-11: eCFR versioner API (titles.json and structure for title 14, up to date as of 2026-10-07) lists Part 250 Oversales, Part 259 Enhanced Protections for Airline Passengers and Part 260 Refunds for Airline Fare and Ancillary Service Fees. GovInfo policies page restates 17 U.S.C. 105 (no copyright in works of the U.S. Government) and notes that government publications may contain third-party copyrighted material.

OpenAI (Cobbe et al.) · 2021

Automatically checkable answers make it a convenient stand-in workload for a cost/quality routing study (project-044).

Checked 2026-10-11: Hub API and card show license mit, not gated; main config train 7,473 and test 1,319. The main/test parquet was downloaded (download-verified) - 1,319 rows, columns question and answer, with the final numeric answer after a "####" marker.

LMSYS (Zheng et al.) · 2023

Human pairwise labels for testing LLM-as-judge agreement and position bias (project-042).

Checked 2026-10-11: Hub API shows license cc-by-4.0, not gated. The human split parquet was downloaded (download-verified) - 3,355 rows with columns question_id, model_a, model_b, winner, judge, conversation_a, conversation_b, turn; winner counts model_a 1,293, model_b 1,282, tie 780; 80 questions; 65 distinct judge ids; turns 1 and 2. A gpt4_pair split (2,400 rows) holds recorded GPT-4 judge decisions.

Allen Institute for AI (Dasigi et al.) · 2021

Long documents with expert-annotated evidence paragraphs; used for the chunking experiment (project-038).

Checked 2026-10-11: AllenAI data page loads and states CC BY 4.0. Hugging Face card allenai/qasper (license cc-by-4.0) documents 5,049 questions over 1,585 papers, paragraph-level evidence and unanswerable flags; it is a loading-script dataset (qasper.py) whose download URLs point to qasper-dataset.s3.us-west-2.amazonaws.com/qasper-train-dev-v0.3.tgz (HTTP 200, about 10.8 MB) and the test tarball.

European Parliament and Council of the European Union (published via EUR-Lex, Publications Office of the EU) · 2004

Small, authoritative, airline-domain legal corpus for the document Q&A flagship (project-036). Short articles with numbered paragraphs make evidence labelling tractable.

Checked 2026-10-11: EUR-Lex document page for CELEX 32004R0261 loads with the full title of Regulation (EC) No 261/2004 of 11 February 2004. EUR-Lex legal notice states that the Commission reuse policy is based on Decision 2011/833/EU and that, unless otherwise specified, legal documents published in EUR-Lex may be reused for commercial or non-commercial purposes; editorial content and consolidated texts are licensed CC BY 4.0.

Pranav Rajpurkar, Robin Jia, Percy Liang · Stanford NLP (dataset site) · 2018

Small licensed question/context set with unanswerable questions; used as the golden-set fixture for the LLMOps regression-gate project.

Checked 2026-10-11: Dataset site: SQuAD 2.0 combines the SQuAD 1.1 questions with over 50,000 unanswerable questions; files train-v2.0.json (40 MB) and dev-v2.0.json (4 MB); license CC BY-SA 4.0. Hugging Face mirror rajpurkar/squad_v2 carries license tag cc-by-sa-4.0.

Figure Eight (CrowdFlower Data for Everyone); Kaggle reformatting by Ben Hamner

Project 033 (TF-IDF vs small neural text classifier).

Checked 2026-10-11: Kaggle public API metadata for crowdflower/twitter-airline-sentiment: title, owner Figure Eight, license CC BY-NC-SA 4.0, version 4 updated 2019-10-16; description says tweets about six major US airlines were scraped in February 2015 and contributors labelled positive / neutral / negative sentiment, then negative reasons (examples given: late flight, rude service); files Tweets.csv (3,421,431 bytes) and database.sqlite. The HTML page itself did not render for the checker. Column schema and row count (14,640) confirmed on a byte-identical Tweets.csv mirror via the Hugging Face datasets-server (osanseviero/twitter-airline-sentiment).

S. Moro, P. Rita, P. Cortez · UCI Machine Learning Repository · 2012

Used for the online-scoring system-design project: date-ordered rows allow a temporal split, and the duration column is only known after the call it is meant to prioritise.

Checked 2026-10-11: UCI page: title, creators, donation date 2/13/2012, DOI 10.24432/C5K306, files bank-additional-full.csv (41,188 rows, 20 inputs, ordered by date, May 2008-Nov 2010), bank-additional.csv (10% sample), bank-full.csv and bank.csv (older 17-input versions), target y (term-deposit subscription), variable list including duration (last contact duration in seconds), license CC BY 4.0. The page does not list the extra social/economic context attributes of the 20-input files.

deepset

Optional public seed set of direct-injection strings for the red-team harness (project-041); it contains no tool-use or indirect-injection cases.

Checked 2026-10-11: Hub API shows not gated; parquet files downloaded (download-verified) - train 546 (203 injection, 343 benign), test 116 (60 injection, 56 benign), columns text and label. A spot check found roughly a third of rows are German. Card metadata is inconsistent - top-level license apache-2.0 while dataset_info lists cc-by-4.0; the card body has no description.

Gretel.ai · 2024

Labelled character spans allow span-level precision/recall for redaction pipelines (project-050). Synthetic text is cleaner than real data - treat results as optimistic.

Checked 2026-10-11: Hub API and card show license apache-2.0, not gated; card describes synthetic documents generated with Gretel Navigator, 29 PII types and seven languages. The English test parquet was downloaded (download-verified) - 2,891 English finance-domain documents, mean length about 1,410 characters, pii_spans as JSON lists of start/end/label. The card's record counts are inconsistent (metadata 50,346/5,594 train/test vs prose 50,776/5,164).

Olist (owner); Francisco Magioli (creator on Kaggle) · Kaggle Datasets (olistbr/brazilian-ecommerce), version 2

Project 002 (delivery delay and review score).

Checked 2026-10-11: Kaggle public metadata API (datasets/view and datasets/list) and the dataset's Croissant metadata: title, owner Olist, license CC BY-NC-SA 4.0, about 100k orders 2016-2018 from Brazilian marketplaces, anonymised; nine CSV files (orders, order_items, order_reviews, order_payments, customers, sellers, products, geolocation, category translation) with their column names; total about 126 MB (geolocation file about 61 MB); last updated 2021-10-01. Page notes an order may contain several items from different sellers and that the review survey is sent after delivery or the estimated delivery date.

Author unknown (per OpenML); sources listed as PMLB and BigML · OpenML · 2017

Project 012 (telecom churn). Provenance is weak: OpenML itself says the author is unknown.

Checked 2026-10-11: OpenML JSON API (data/40701, features, qualities): name churn v1, uploaded 2017-04-06, licence 'public', author unknown, telephony account features and churn label, 5,000 instances, 21 columns incl. target 'class' (707 positives), no missing values, feature list (state, account_length, area_code, phone_number, plan flags, day/eve/night/intl minutes, calls and charges, number_customer_service_calls); in OpenML-CC18.

Andrea Dal Pozzolo, Olivier Caelen, Gianluca Bontempi · OpenML · 2015

Project 010 (card-fraud alert ranking).

Checked 2026-10-11: OpenML JSON API (data/1597 and qualities): name creditcard v1, licence 'Public', uploaded 2015-06-25, European cardholder transactions over two days in September 2013, 284,807 rows, 492 frauds (0.172%), 31 columns: Time (seconds since first transaction), V1-V28 PCA components, Amount, Class; no missing values; collected in a Worldline - ULB research collaboration; citation request for Dal Pozzolo et al. 2015.

I-Cheng Yeh · UCI Machine Learning Repository (DOI 10.24432/C55S3H) · 2009

Project 011 (probability-of-default model).

Checked 2026-10-11: UCI page and API (id 350): 30,000 clients in Taiwan, 23 explanatory variables X1-X23 (credit limit, sex, education, marital status, age, repayment status Apr-Sep 2005, bill and payment amounts) and binary default payment next month; file 'default of credit card clients.xls' 5.3 MB; license text CC BY 4.0.

John Clore, Krzysztof Cios, Jon DeShazo, Beata Strack · UCI Machine Learning Repository (DOI 10.24432/C5230J) · 2014

Project 005 (case-mix-adjusted readmission comparison).

Checked 2026-10-11: UCI page and API (id 296): 101,766 encounters, 47 features, inclusion criteria (inpatient diabetic encounters, length of stay 1-14 days, labs and medications given), variable descriptions incl. patient_nbr and readmitted (<30, >30, NO), files diabetic_data.csv 18.3 MB and IDS_mapping.csv 2.5 KB (zip 3 MB), has missing values, license text CC BY 4.0.

Peter van der Putten · UCI Machine Learning Repository (DOI 10.24432/C5630S) · 2000

Project 014 (caravan-policy propensity ranking).

Checked 2026-10-11: UCI page and API (id 125): 86 variables (sociodemographics derived from zip code, attributes 1-43; product ownership, 44-86), target CARAVAN, training file ticdata2000.txt (5,822 records), evaluation file ticeval2000.txt (4,000) and targets tictgts2000.txt, TicDataDescr.txt dictionary; tab-delimited; license text CC BY 4.0.

C. Sakar, Yomi Kastro · UCI Machine Learning Repository (DOI 10.24432/C5F88Q) · 2018

Project 006 (conversion driver analysis).

Checked 2026-10-11: UCI page and API (id 468): 12,330 sessions, each from a different user over one year; 17 features plus Revenue label; definitions of page-type counts/durations, Google Analytics BounceRates/ExitRates/PageValues and SpecialDay; online_shoppers_intention.csv 1 MB; license text CC BY 4.0.

Michael McCann, Adrian Johnston · UCI Machine Learning Repository (DOI 10.24432/C54305) · 2008

Project 013 (yield-failure screening).

Checked 2026-10-11: UCI page and API (id 179): semiconductor process sensor data, 1,567 examples with 591 features per the page, 104 fails, labels file with -1 pass / 1 fail and a date-time stamp, NaN for nulls, files secom.data 5.1 MB, secom_labels.data 39.7 KB, secom.names 4.1 KB; provider baseline balanced error rates for 40-feature selections with 10-fold CV; license text CC BY 4.0.

Sathishkumar V E, Changsun Shin, Yongyun Cho · UCI Machine Learning Repository (DOI 10.24432/C52G8C) · 2021

Project 004 (power-factor and peak-load cost exposure).

Checked 2026-10-11: UCI page and API (id 851): electricity data of DAEWOO Steel Co. Ltd (Gwangyang, South Korea) sourced from the KEPCO website, 35,040 instances, variables date, Usage_kWh, lagging/leading reactive power (kVarh), CO2(tCO2) listed with unit 'ppm', lagging/leading power factor (%), NSM (seconds from midnight), WeekStatus, Day_of_week, Load_Type (Light/Medium/Maximum); Steel_industry_data.csv 2.6 MB; license text CC BY 4.0.

Margarida Cardoso · UCI Machine Learning Repository (DOI 10.24432/C5030X) · 2013

Project 015 (client segmentation).

Checked 2026-10-11: UCI page and API (id 292): 440 clients of a wholesale distributor, annual spending (monetary units) on six product categories, Channel (Horeca 298 / Retail 142) and Region (Lisbon 77, Oporto 47, Other 316), descriptive statistics, file 'Wholesale customers data.csv' 14.7 KB, license text CC BY 4.0.