Due diligence · Technology

The demo is not the system

A tool that works in a conference room has been tested against an audience that wants it to work. A tool in a classroom is tested against a child, a network outage, a bored guide, and five years of records that outlive the company that made them.

What technical diligence actually asks

The common failure is to evaluate a product on its capability and ignore everything around it: who owns it, who funds it, where the data lands, what happens when it is wrong, and whether the company will exist in three years. Capability is the easiest thing to verify and the least predictive of harm.

Solvency and continuity: runway, revenue, ownership, and a written answer to what happens to records if the vendor is acquired, pivots, or fails.
Data flow, in full: every place a student utterance travels, every subprocessor, every retention window, and whether any of it becomes training data.
Model behaviour under pressure: not the happy path, but refusal behaviour, sycophancy, fabrication, and what the system does when a child says something alarming.
Evaluation you did not commission: independent testing, adversarial testing, and results from settings unlike your own.
Reversibility: can you export everything and leave in ninety days, in a format someone else can read?
Accessibility and equity of function: does it degrade for the student with the slowest connection, the thickest accent, or the least typing fluency?

Named cases

Failure — vendor collapse

Los Angeles Unified and “Ed” (2024)

In March 2024 the second-largest district in the United States launched a custom AI chatbot built by the startup AllHere at a cost of up to $6 million over five years; the superintendent called it a "game changer." By July the company had furloughed most of its staff and the district had switched the bot off. A former AllHere engineer had warned the district weeks earlier that the product's handling of sensitive student records violated basic data-privacy principles.

What diligence would have caught: vendor runway, subprocessor data handling, and a continuity clause. What it would not have caught: nothing. This was foreseeable.

Enforcement — COPPA

FTC v. Edmodo (2023)

The Federal Trade Commission obtained an order against the education platform Edmodo for collecting personal data from children without parental consent, using it for advertising, and unlawfully outsourcing its COPPA obligations to the schools themselves.

The transferable lesson is the last part. A vendor telling a school "you obtain the consent" is not a compliance arrangement; it is a liability transfer, and regulators have said so.

Enforcement — data security

FTC and Chegg (2022–2023)

The FTC finalised an order against Chegg over security practices that exposed personal information on millions of users and employees, including Social Security numbers and passwords; the complaint described personal data stored in plain text and outdated password encryption.

Diligence question this answers: "is it encrypted?" is not a yes/no question you accept on a sales call. Ask which data, at rest and in transit, under what algorithm, and who has ever audited the answer.

Failure — centralisation without consent

inBloom (2013–2014)

inBloom was a nonprofit student-data infrastructure effort backed by roughly $100 million in foundation funding. It shut down in April 2014 after sustained opposition from parents and advocates over data privacy, without ever having been shown to be unsafe in operation.

The technical design was defensible. The diligence failure was social: nobody established consent, or even comprehension, among the families whose children's records were the entire product. Legitimacy is a system requirement, not a communications problem.

Failure — hardware before pedagogy

LAUSD iPads (2013–2014)

A roughly $1 billion initiative to place a tablet with preloaded curriculum in the hands of every student was abruptly cancelled in August 2014, after implementation problems, incomplete curriculum, and questions about the procurement process.

The pattern: device counts are easy to specify, easy to fund, and easy to announce. Instructional change is none of those things. Buying the first does not produce the second.

Open risk — model behaviour

Sycophancy in conversational models

Published work on AI assistants documents sycophancy: a tendency to agree with the user's stated view rather than hold a position. For a general assistant this is an annoyance. For an instrument whose entire purpose is to press a child to defend a claim, it is a direct contradiction of function.

This is the single technical risk most specific to our method, and we treat it as unresolved rather than mitigated.

How this binds us

Hypothesis — untested in our setting

A constrained companion that returns questions, structure, counterexamples, and sources — and withholds conclusions — can resist sycophancy well enough to be safe in front of a five-to-seven-year-old under guide supervision. Falsified if logged sessions show the companion conceding to a student's incorrect claim at a rate a reviewing guide judges material, in which case the companion is withdrawn from student contact until the behaviour is fixed.

Commitment — binding on us

Before a single student session runs we will publish: every vendor in the chain, what each receives, retention terms, whether any processing occurs outside the United States, and the written continuity arrangement covering vendor insolvency.