theloop.felineunion.org
01 02 M
Case 03 — State & Compliance Systems WP-01 · The Conditioning Works

The state learned from the app.

Welfare reassessment portals, risk-scored policing dashboards, immigration check-in apps, automated benefit clawbacks: the public sector spent the 2010s and 2020s adopting the conditioning techniques pioneered in the private sector. It now uses them to manage compliance from populations that, unlike app users, cannot uninstall.

DomainState Compliance
Length~13 min read
Sources10 cited
StatusPublished

The clearest indicator that software-layer conditioning has become a mature governance technique is that governments — typically the slowest adopters of anything — have adopted it. Not in pilot. Not as a research project. As production infrastructure, in the welfare departments, the immigration enforcement agencies, the police services, the tax authorities, and the housing systems of every developed country. The same loop documented in Case 01 and Case 02 — instrument, score, feed the score back, adjust the weights — is now running on people whose participation is not optional.

The political marketing for this transition was always efficiency. The systems would reduce fraud, speed up processing, free up case workers for "complex cases." Some of these things happened, in part. What also happened, by design and by the operators' own performance metrics, was a redistribution of behavioural pressure away from the institution and onto the subject. The benefit claimant is now required to perform the labour the case worker used to do. The immigration applicant must monitor their own check-in cadence. The person flagged by a risk score now lives with the consequences of a number they cannot see and were never told the formula for.

§ 3.1The compliance app as enclosure

Begin with the most explicit example: ankle-monitor-replacement and check-in apps used in immigration enforcement and pretrial supervision. These are not metaphorical conditioning instruments. They are literal ones. They require the subject to open an app at scheduled times, prove their identity with a face scan, submit GPS data, and respond to push notifications within fixed windows. Missing a check-in is a violation. Violations escalate to detention.

The behavioural effect on the subject is total. They do not turn off their phone. They do not let the battery die. They organise their day around the app. They develop a quiet vigilance that does not switch off, because the system never switches off. This is, in the technical sense, conditioned hypervigilance — a state that the criminology and psychology literature has been describing in incarcerated populations for decades. The novelty is that it is now imposed on people who have not been convicted of anything, in their own homes, through an app on their own phone, at a fraction of the cost of physical custody.

Fig. 4
US ICE Alternatives to Detention enrolment
~190,000
Approximate number of people under app- or device-based ICE supervision in recent reporting years, the majority via the SmartLINK check-in app operated by BI Incorporated, a GEO Group subsidiary. The enrolled population has grown by roughly an order of magnitude over the past decade.

The vendor's marketing materials describe these systems as humane alternatives to detention. They are, in a narrow accounting sense, cheaper than detention. They are also, by the vendor's own performance reporting, extremely effective at producing compliance with court appearances and check-in schedules. The compliance is the product. The compliance is what's being sold to the agency. That the same compliance is produced through continuous psychological pressure on the subject is not a defect of the system; it is the system.

§ 3.2The welfare portal as a hostile interface

Move from the explicit case to the diffuse one. Most digitised welfare systems — Universal Credit in the UK, the various state-level reformed unemployment portals in the US, the Centrelink ecosystem in Australia, parts of Canada's CRA-administered benefits — do not look like conditioning systems. They look like ordinary websites. The conditioning is in the friction.

The portal requires periodic recertification. The recertification involves uploading documents. The document upload tool rejects formats without explaining why. The reassessment is automated and can trigger a clawback notice without human review. The appeals process is itself an online form that requires legalistic precision to fill out. Each of these is a small piece of friction. Stacked together they produce a measurable outcome: a substantial fraction of eligible claimants stop claiming, because the effort cost of continuing exceeds the benefit. The government's books improve. The technical eligibility numbers are unchanged. The number of people receiving the benefit goes down.

This is conditioning through the absence of a reward path. The claimant is not being told to stop claiming. The claimant is being shown that claiming is exhausting, uncertain, and may produce a sudden financial penalty months later. The behaviour the system selects for is dropout. This is precisely the design pattern the FTC named, in the commercial context, a "dark pattern." When deployed by the state against poor people, it does not yet have a settled name.

The portal does not deny your benefit. It exhausts you until you stop applying. The metric improves. The eligibility didn't change.

Tst. 2
Field testimony — UK Universal Credit claimant, 2023

"The journal updates at random hours. You have to log in to see if anything has changed. If you miss a message for forty-eight hours your claim can be sanctioned. I check the app eight, ten times a day. My partner does the same."

"I don't know anyone on UC who sleeps with their phone off."

§ 3.3The risk score as silent enforcement

Risk scoring is the most studied and least understood of the three patterns. Policing services in North America and Europe now use a layer of predictive analytics — gang affiliation scores, person-of-interest scores, "harm" scores, family-violence risk ratings — that produce a numeric output for an individual based on a basket of inputs. The individual does not know the score exists. The officer responding to a call does. The score shapes the encounter. The encounter generates new data. The new data feeds the score. The score is, in the most literal possible sense, a self-fulfilling prophecy with a data pipeline behind it.

Welfare fraud detection systems do the same on the benefit side: the Dutch SyRI system, the Australian Robodebt scheme, various US state-level analogues. In each case the score targeted historically over-policed populations, generated enforcement actions that produced harm at scale, and was eventually wound back by courts or political action only after the damage was substantially complete and the underlying behavioural conditioning of the targeted population — the chronic anxiety, the self-restriction, the pre-emptive withdrawal from services — had already settled in.

Fig. 5
Australian Robodebt scheme — final reckoning
$1.8B AUD
Approximate value of debts raised against welfare recipients via automated income-averaging, later found unlawful by the Federal Court. Royal Commission concluded the scheme caused substantial harm including documented suicides. The system "worked" — by the metric of debts raised — for years before it was stopped.

The Robodebt case is particularly clean because the Australian Royal Commission published a documentary record of what the operators knew, when. Senior officials understood the system was producing erroneous debts. The system continued. The reason it continued is the reason any conditioning system continues: by the metric the operators were judged on, it was working. It generated debts. It recovered money. The fact that it was also driving people to suicide was a property of the system that did not register in the operator's dashboard until it was forced to.

§ 3.4Why this case is different

Consumer platforms condition behaviour, but the user can — in principle — close the app. Gig workers are conditioned by the platforms they work for, but they can — in principle — find other work. The state-deployed conditioning systems documented above operate on populations who cannot exit. You cannot uninstall the immigration check-in app. You cannot decline to engage with the welfare portal if your rent is due. You cannot opt out of being scored by the local police service's risk-assessment vendor.

This is what makes the public-sector adoption qualitatively different from the private-sector pattern it copied. The same loop, deployed against a captive population, produces a kind of governance the political theory of the liberal democracies has no comfortable name for. It is not surveillance, exactly, although it includes surveillance. It is not coercion, exactly, although it includes coercion. It is the administrative application of operant conditioning to citizens, at scale, by institutions whose accountability mechanisms — courts, ombudsmen, freedom of information — were designed for an earlier kind of bureaucracy.

The accountability mechanisms have not caught up, and the operators know they have not caught up. This, more than the conditioning itself, is the political fact of the present moment.

§ 3.5The synthesis

Across the three cases, the same five elements appear. The behaviour is instrumented at fine resolution. A score or model summarises it. The score is fed back into the subject's environment in close to real time. The feedback shapes subsequent behaviour. The operator measures the shaping and tunes the parameters. None of this is novel as a method. What is novel is the cost. Operant conditioning used to require enormous human labour — wardens, foremen, social workers, editors, supervisors. Software has driven the marginal cost of running one of these loops on one more person to approximately zero.

When you drive the marginal cost of a powerful technique to zero, the technique stops being rationed by economics and starts being rationed only by what institutions consider acceptable. Right now, institutions consider an enormous amount of it acceptable, because there is no countervailing pressure that costs them anything. The countervailing pressure has to be built. Building it is the work of the next decade. The first step is admitting that the conditioning works, that the operators know it works, and that the gap between what the systems do and what the public has been told they do is now wider than at any point since the invention of mass advertising.

See also the Method note for the framing used across this paper.

Sources & Notes
  1. Eubanks, Automating Inequality (2018). Foundational work on US public-sector digital decision systems.
  2. Royal Commission into the Robodebt Scheme, Final Report (2023). Australian government inquiry; documentary record of the scheme's operation and harms.
  3. National Audit Office (UK), reports on Universal Credit implementation, 2018–2024. Quantitative record of recertification and sanctioning outcomes.
  4. Mijente, "Notes from the ICE Detention Cell in Your Pocket" (2021, updated). Investigative reporting on SmartLINK and ICE Alternatives to Detention.
  5. District Court of The Hague, NJCM et al. v. The State of the Netherlands (2020). The SyRI ruling: a European court striking down algorithmic welfare-fraud detection on human-rights grounds.
  6. Brayne, Predict and Surveil (2020). Ethnographic account of LAPD's adoption of person-based predictive policing.
  7. Citizen Lab (University of Toronto), ongoing reporting on surveillance technologies and immigration-enforcement apps.
  8. Whittaker, Crawford et al., AI Now Institute reports (2017–2022). Public-sector AI deployment surveys.
  9. Office of the Privacy Commissioner of Canada, reports and findings on automated decision systems in federal benefits administration.
  10. US Office of Science & Technology Policy, "Blueprint for an AI Bill of Rights" (2022). Documents the gap between deployed practice and proposed safeguards.
Closing — Method

How this paper frames what it sees.

Part of The Conditioning Works · Working paper No. 01 / 2026

Intro

Track 1 of 6 · auto-playing