Document Review Protocol — Jeb Bush Email Collection
Version: LB-1.0 · Reviewers: AI labeling models under benchmark conditions
This protocol governs how a document is judged responsive to each of the nine review issues, and how borderline calls are resolved. Apply it exactly as written; the same standard is applied to every reviewer.
1. How to use this protocol
- Treat each of the nine issues as an independent responsiveness call. A single email can be responsive to zero, one, or several issues.
- For each document and issue, assign one grade:
- 0 — Non-responsive: does not concern the issue.
- 1 — Responsive: concerns the issue (the issue is discussed, requested, decided, forwarded, or materially referenced).
- 2 — Key / Important: responsive and materially significant to the issue — substantive discussion, a decision or directive, an original report, or a document a case team would want surfaced first.
- When in doubt between 0 and 1, code 1 (favor recall — this is a recall-oriented review). When in doubt between 1 and 2, code 1.
- Apply the "concerning" standard broadly: a document "concerns" an issue if a reasonable reviewer would understand it to be about, relating to, or materially referencing that issue — not only documents where the issue is the sole subject.
2. General coding rules
These apply across all nine issues.
- Email families / threads. Judge the document as presented (each record is a single email). If a responsive statement appears anywhere in a forwarded or quoted chain within the email, the email is responsive. Long digest emails that touch an issue in one segment are still responsive to that issue.
- Forwards & replies. "FW:" / "RE:" with responsive content count. A bare forward whose only responsive content is the quoted material is still responsive (the document counts, not its novelty).
- Mere mention vs. concerning. A passing, incidental mention with no substantive connection (e.g., a person's name in a signature block, an unrelated calendar line) is non-responsive. Substantive discussion, requests, scheduling about the issue, transmittals of issue material, and status updates are responsive.
- Administrative / logistics. Scheduling a meeting, transmitting a document, or routing a constituent letter about a responsive issue is responsive (grade 1), even if short. It becomes key (2) only if substantively important.
- Constituent mail & form letters. Incoming citizen mail and the office's replies about an issue are responsive. Identical form-letter responses are each responsive (do not de-dup for grading).
- Spam / newsletters / press clippings. Responsive only if the content substantively concerns the issue; generic distribution lists are otherwise 0.
- Ambiguous / unreadable. If redaction or truncation leaves intent unclear but issue-relevant signals are present, code 1.
- Geographic qualifier. Most issues are scoped to Florida (see each definition). Climate change (409) is explicitly not Florida-restricted.
- Negation still counts. A document arguing against a position, or reporting that something did not happen, still concerns the issue → responsive.
3. Two-tier relevance: what makes a document "Key" (grade 2)
Grade 2 is reserved for documents materially significant to the issue. Typical signals:
- Originates or transmits a substantive report, analysis, legislation, draft policy, or legal memo on the issue.
- Records a decision, directive, commitment, or the Governor's own position.
- Contains non-public, deliberative, or strategy discussion among principals.
- Is the primary source document others merely reference.
Typical grade-1-not-2 documents: scheduling, routine acknowledgments, brief FYIs, constituent form replies, tangential references, news forwards without analysis.
4. The nine responsive issues
For each: Definition (controlling statement) · Include · Exclude · Decision signals · Key (grade 2).
401 — Summer Olympics
- Definition. All documents concerning a bid to host the Summer Olympic Games in Florida.
- Include. Florida Olympic bid committees; proposals, feasibility/economic studies, venue or infrastructure planning tied to a bid; correspondence with the USOC/IOC about a Florida bid; legislative or budget support for a bid; Tampa/Miami/Orlando/Jacksonville bid efforts.
- Exclude. Athletes/teams competing in the Olympics with no bid connection; the Olympics as a generic event reference; non-Florida bids unless they bear on Florida's.
- Signals. "Olympic bid," "host city," "USOC," "Games," "bid committee," candidate-city names + Olympics.
- Key. Bid strategy, government commitments/funding decisions, formal proposals or feasibility reports.
402 — Space
- Definition. All documents concerning the space industry, the space program, space travel (manned or unmanned, public or private), and the study or exploration of space, in Florida.
- Include. NASA / Kennedy Space Center operations and funding; Florida aerospace industry and jobs; commercial spaceflight and spaceports; launches; space research/education in Florida; Space Florida / Spaceport authority matters.
- Exclude. "Space" in the sense of office/real-estate space; metaphorical uses; non-space defense aviation unless space-related.
- Signals. "NASA," "Kennedy Space Center," "Cape Canaveral," "launch," "shuttle," "aerospace," "spaceport," "satellite."
- Key. Industry-retention or funding decisions, major program correspondence, post-incident (e.g., shuttle) policy discussion, spaceport authorizations.
403 — Bottled Water
- Definition. All documents concerning the extraction of water in Florida for bottling by commercial enterprises.
- Include. Permits/applications for water withdrawal for bottling; Water Management District actions on bottling; disputes over springs/aquifer extraction by bottlers (e.g., Nestlé/large bottlers); legislation/regulation of commercial water bottling; citizen opposition to bottling permits.
- Exclude. Drinking-water safety, utilities, or general water policy not tied to commercial bottling; bottled water as a catering/supply line item.
- Signals. "bottling," "water withdrawal/extraction permit," "springs," "Water Management District," bottler company names.
- Key. Permit decisions, regulatory/legal analysis, Governor's-office position on a bottling controversy.
404 — Eminent Domain
- Definition. All documents concerning the legality or morality of expropriating land in Florida for commercial development.
- Include. Eminent-domain / condemnation for private or commercial development; reaction to Kelo v. New London; Florida legislative responses limiting eminent domain; property-rights debate over takings for development.
- Exclude. Routine government land acquisition for clearly public use (roads, schools, conservation) absent the commercial-development/property-rights debate; ordinary real-estate transactions.
- Signals. "eminent domain," "condemnation," "takings," "Kelo," "property rights," "blight," "redevelopment."
- Key. Draft legislation, legal/constitutional analysis, the Governor's stated position on takings for development.
405 — Newt Gingrich
- Definition. All documents concerning House Speaker Newt Gingrich or any entities or personnel associated with Newt Gingrich.
- Include. Gingrich directly; his organizations, staff, and close associates; correspondence to/from or about him; invitations, endorsements, policy collaboration involving Gingrich or his entities.
- Exclude. Unrelated individuals who merely share the surname; generic GOP material with no Gingrich nexus.
- Signals. "Gingrich," "Newt," his organizations/PACs and named associates.
- Key. Direct correspondence, substantive political/policy coordination, decisions involving him.
406 — Felon Disenfranchisement
- Definition. All documents concerning the right of felons to vote in Florida, including (but not limited to) voter purges and reinstatement of voter rights. Individual clemency cases are NOT responsive.
- Include. Restoration-of-civil-rights policy; clemency-board rules on voting rights as policy; felon voter-roll purges; litigation/legislation on felon voting; statistics and advocacy on felon enfranchisement.
- Exclude. Individual clemency petitions / individual rights-restoration cases (explicitly carved out); general clemency unrelated to voting; ordinary criminal-justice matters.
- Signals. "restoration of civil rights," "felon voting," "voter purge," "disenfranchisement," "clemency rules" (policy, not a named applicant).
- Key. Policy decisions, rule changes, purge directives, litigation strategy.
407 — Faith-Based Initiatives
- Definition. All documents concerning grants or other initiatives in Florida to offload social services to so-called faith-based agencies. Services include but are not limited to education, prisons, and emergency relief.
- Include. Faith-based grants/contracts for social services; faith-based prison or education programs; the state office/initiative for faith-based and community programs; church-state debate over such funding.
- Exclude. Religious references with no service-delivery/funding nexus; routine interfaith greetings/proclamations; secular nonprofit contracting.
- Signals. "faith-based," "faith and community," "InnerChange," "ministry" + grant/contract/program, "charitable choice."
- Key. Grant/contract decisions, program authorizations, policy or legal (Establishment Clause) analysis.
408 — Invasive Species
- Definition. All documents concerning the problem of invasive species in Florida — non-native plants or animals that threaten the Florida ecosystem.
- Include. Specific invasives (pythons, lionfish, melaleuca, hydrilla, fire ants, citrus canker as invasive pest, etc.); eradication/control programs; agricultural or Everglades impacts; legislation/funding on invasives.
- Exclude. Native pests/disease unrelated to non-native invasion; general environmental or agriculture matters without an invasive-species nexus.
- Signals. "invasive," "non-native," "exotic species," named invasives, "eradication," "infestation."
- Key. Control-program decisions, scientific assessments, funding/policy responses.
409 — Climate Change
- Definition. All documents concerning climate change, global warming, or carbon emissions — whether in Florida or otherwise (NOT Florida-restricted).
- Include. Climate science/policy; greenhouse-gas / carbon-emission regulation; sea-level-rise and climate-impact discussion; Kyoto/federal climate policy; renewable-energy or emissions initiatives framed around climate.
- Exclude. Weather, hurricanes, or general environmental topics with no climate/emissions framing; "climate" used figuratively (e.g., "business climate," "political climate").
- Signals. "climate change," "global warming," "greenhouse gas," "carbon emissions," "CO2," "Kyoto," "sea level rise."
- Key. Policy positions, emissions-initiative decisions, substantive analyses.
5. Notes for AI reviewers
- Apply exactly the §2 general rules and §4 definitions — including the explicit carve-outs (406 individual clemency; 409 not Florida-bound; 402 not office "space"). These carve-outs are the most common error sources.
- Emit a calibrated confidence with every issue call (0.0–1.0). Confidence is scored for calibration; do not emit 1.0 unless the call is certain.
- You have no information about how many documents in this review are responsive. Do not assume a base rate; judge each document on its content against the definitions alone.