Critical thinking used to be treated as a civic virtue. It is now an operational competency, because the cheapest way to influence a business decision is no longer to argue with the decision-maker but to supply them with convincing false inputs. The quality of a decision cannot exceed the quality of what it was based on.
Quick Overview
For most of the history of office work, a face and a voice were sufficient authentication. Synthetic media removed that shortcut faster than the habits built on it could adapt.
The reaction usually splits into a pair of unhelpful extremes: buy a detector and call the problem handled, or conclude that nothing can be trusted. Both treat the problem as a property of the content rather than of the decision the content is feeding.
What works is duller. Decide which decisions are worth verifying before anybody tries to manipulate them, attach a procedure that does not depend on anyone’s ability to spot a forgery, shift what you can onto provenance rather than detection, and train the reasoning habits that make people notice when a situation has been engineered to bypass the procedure.
This article is about that discipline. It is not a threat catalogue — which attacks exist, which tools generate them and which security controls block them is a separate subject. The question here is narrower and more personal: how a professional decides whether to act on something.
Why This Is a Professional Competency, Not a Media Topic
Disinformation is usually framed around public discourse, which makes it somebody else’s problem and explains why the corporate response is so often a poster campaign.
The business exposure is specific. An organisation decides on inputs it did not generate: supplier claims, market signals, internal reports stripped of their original context, an instruction arriving urgently from someone senior. Each is a place where a false input converts directly into a costly action: a payment leaves, a contract is signed, a statement is issued that has to be retracted.
The competency is therefore not scepticism as a personality trait. It is the ability to identify which inputs a decision rests on, to notice when one is unverified, and to say so before rather than afterwards — a professional skill in the same sense that reading a contract is. Media trust research such as the Digital News Report 2023 tracks how far confidence in information sources has eroded across markets; the organisational consequence is that the informal signals people once relied on — this looks official, this came through a familiar channel — no longer carry the weight they used to.
What Synthetic Media Changed, and What It Did Not
Forged documents and impersonation over the telephone are old crimes. What changed is the cost curve. Producing a convincing likeness of a specific person saying specific words used to require budget, time and specialist skill. It no longer requires them in the quantities that acted as a barrier, so the technique is now available against targets previously not worth the effort: a mid-sized company, a single accounts payable clerk.
That is the entire change, and it is enough. Defences that worked because attacks were rare and expensive stop working when attacks become cheap.
What did not change is more useful. The attacker still needs the target to take an action, and that action still passes through a process the organisation controls. The forgery has to be perfect only when the decision has no independent verification step; against a procedure that confirms instructions through a separate channel, the quality of the forgery stops mattering. It also helps to know what the generating systems actually do: produce output that is statistically plausible given what they were fitted on. The mechanism is laid out in first steps with machine learning in business, and understanding it prevents both credulity and panic.
The Attack That Arrives Wearing a Familiar Face
The characteristic pattern is not mass disinformation. It is a targeted, urgent, plausible request delivered through a channel that carries authority.
Someone in finance receives a call or a short video message from a person who looks and sounds like a senior executive. The matter is confidential, time-critical and slightly outside normal process for a credible reason. The request is for a payment, a credential or an approval.
Every element is engineered against a specific human tendency. Authority suppresses the question. Urgency removes the interval in which the question would occur. Confidentiality removes the colleague who would ask it instead. The forged likeness is the least important component: it exists to stop the target reaching for the one behaviour that ends the attack, which is putting the phone down and calling back on a number they already had. Incident classes catalogued in the ENISA Threat Landscape 2023 show the same shape, and the countermeasure is always the same and always unglamorous — for a defined class of instruction, verification through an independent channel is mandatory and not waivable by seniority.
Why Detection Heuristics Decay
Guidance on spotting synthetic video lists visual tells: unnatural blinking, mismatched lighting, blurring at the edge of a face. These were accurate when written and age badly, because each published tell becomes a defect fixed in the next generation of generators.
Automated detectors inherit the problem in sharper form. A detector is itself a model, and models can be attacked deliberately — inputs crafted to evade classification, training data manipulated to install a blind spot. The taxonomy in Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations describes exactly this class of failure, and it applies to detection tools as much as to anything else deployed.
Detection is therefore a probabilistic filter with a decaying half-life: useful for triage, useless as a final authority. Any procedure whose critical step is “check whether it looks fake” has put a decaying component where it needed a stable one. Teach the tells as a prompt to escalate, never as a verdict.
Provenance Is a Better Answer, and the Regulation Helps
The more durable direction reverses the question. Instead of asking whether content is fake, ask what it can prove about where it came from.
Content provenance standards attach cryptographically signed assertions to a file, recording what produced it and what edits followed. The specification for Content Credentials : C2PA Technical Specification defines that manifest and how it travels with the asset. The property is asymmetric in a useful way: absence of provenance proves nothing, but an intact and verifiable manifest is evidence that does not decay as generators improve. Adoption is uneven and metadata is easily stripped by the platforms most content passes through, so this is not yet a solution for arbitrary material found online. Inside an organisation’s own boundary it is entirely available — signing internal communications and requiring a verifiable chain for anything supporting a financial or legal action move the burden from perception to cryptography.
Regulation is pushing the same way, by placing obligations on the producers of synthetic content rather than on its victims. The Artificial Intelligence Act imposes transparency duties on systems that generate or manipulate content: material constituting a deepfake must be disclosed as artificially generated or manipulated, providers of generative systems must mark output in machine-readable form, and people interacting with an AI system must be told so unless it is obvious. A pair of limits deserves stating plainly, because compliance frameworks blur them. The obligations bind lawful actors, and whoever runs a fraud against your finance department is not one; and a disclosure duty is not a detection guarantee, so it removes nothing from your own verification responsibility. What it adds is internal legitimacy: insisting on provenance for material that will drive a decision is no longer one cautious employee’s preference.
Verification as a Procedure, Not an Instinct
The practical core is a small set of design choices made in advance, while nobody is under pressure.
- Classify the decisions, not the content. Identify actions where a false input causes irreversible loss: outbound payments, credential changes, contract execution, public statements. Those get a procedure; the rest do not, because a procedure applied to everything is abandoned within a month.
- Make the verification channel independent. Confirmation must travel by a route the attacker neither controls nor chose. Calling back a number from the internal directory qualifies; replying to the message does not.
- Remove the seniority override. The exception granted to an executive request is exactly the exception the attack is designed to invoke. A rule that authority can waive is decorative.
- Budget the delay explicitly. Verification costs time, and somebody will eventually argue the time was not available. Settle that in the policy, before the moment it is raised.
- Separate the sceptic from the actor. Where the stakes justify it, whoever verifies should not be whoever benefits from proceeding.
The Reasoning Habits Underneath the Procedure
Procedure covers the anticipated cases. Judgement covers the rest, and it is trainable.
Separate what is claimed from what is evidenced, out loud — a great deal of bad decision-making consists of a claim acquiring the status of a fact by being repeated in a meeting. Notice the emotional shape of a message: manufactured urgency, flattery and threatened loss are not packaging, they are the mechanism, and their presence is a signal regardless of whether the content is true. Ask who benefits if you believe this, routinely, of friendly sources as well as hostile ones. And construct the opposing case before committing — the most reliable correction for confirmation bias, and the habit least likely to survive time pressure, which is why it belongs in a procedure rather than in good intentions.
Building the Habit Into a Team
Awareness training consisting of a presentation and a quiz produces awareness and no behaviour change. What changes behaviour is practice under conditions resembling the real thing: simulated requests through real channels, reviewed afterwards without blame, focusing on whether the procedure was followed rather than on whether the individual was fooled.
The cultural precondition matters more than the content. Where questioning a senior person’s instruction is career-limiting, no verification procedure survives contact with one. Leaders demonstrating that they expect to be verified — and thanking whoever does it — is worth more than any module. It also pays to give technical staff a real understanding of how generative systems are built, because a team that knows what these models can and cannot do argues about the right things; hands-on work of the kind covered by advanced deep learning techniques with Keras and Python puts people close enough to the mechanism that the technology stops being magic in either direction.
Responding When Something Gets Through
Assume it eventually will, and decide the response in advance. Preserve evidence first — the message, the channel, the timestamps, the account — before anything is deleted. Assess exposure next: what decision was affected, what has already happened, and who else received the same approach, because a targeted attempt is rarely a single attempt.
Communicate deliberately: internally, whoever must act needs to know quickly, while correcting a falsehood publicly can amplify it. Then close the loop — the value of an incident is the change it produces in the procedure, and one that produces only a reprimand has been wasted.
Frequently Asked Questions
How can I tell a deepfake from a genuine video?
Visual tells such as unnatural blinking or mismatched lighting are worth knowing, but they degrade as generators improve, and automated detectors can themselves be evaded deliberately. Treat any such observation as a reason to escalate, never as a verdict. For decisions that matter, verification through an independent channel — a call back to a number you already had — settles the question regardless of how good the forgery is.
Does critical thinking training produce measurable business benefit?
The effects are indirect and take time to appear: fewer incidents where an instruction was acted on without verification, faster escalation when something looks wrong, and better decision records. Measure whether the procedure was followed rather than whether individuals felt confident. Confidence is not the outcome you want; it is frequently the symptom of the problem.
Which organisations are most exposed to synthetic media fraud?
Exposure follows the shape of the decision rather than the sector: any organisation where an instruction from a recognised person can move money, grant access or commit the business is exposed. Financial functions attract the most attempts because the payoff is immediate, and the same reasoning applies to teams whose judgement about risk and forecasting rests on inputs they did not produce, as discussed in AI for chief financial officers.
Where should an organisation start building resilience to disinformation?
Start by listing the decisions where a false input causes irreversible loss, rather than by buying a tool. Attach an independent verification step to that list, remove the seniority override, write down that the resulting delay is acceptable, and rehearse it. A short list of protected decisions that people actually follow beats a comprehensive policy waived the first time somebody important is in a hurry.
Does labelling of AI-generated content solve the problem?
It improves the baseline without removing the responsibility. Transparency obligations require providers of generative systems to mark output in machine-readable form and deepfakes to be disclosed, which makes unlabelled material more suspicious than it was. Those duties bind lawful actors, and whoever is targeting your finance department is not one. Labelling raises the cost of casual deception; it does nothing about a deliberate attack.