Artificial Intelligence (“AI”) has altered the way digital content is created, consumed, and trusted. Hyper-realistic deepfake videos, cloned voices, synthetic identities, and AI-generated communications are no longer experimental technologies. They are increasingly being used in fraud, extortion, impersonation, reputational attacks, financial scams, and misinformation campaigns across jurisdictions, including India.

Indian enforcement agencies have already begun dealing with “digital arrest” scams, manipulated political content, AI-assisted financial fraud, and fake audio recordings impersonating public officials, police officers, and family members. The growing sophistication of these technologies presents a difficult question for the criminal justice system: when digital evidence itself can be fabricated with alarming accuracy, how should courts determine authenticity, reliability, and admissibility?

The answer will likely shape the future of criminal litigation in India.

The Rise of Deepfake-Driven Criminal Activity

Deepfakes use machine learning systems to generate or manipulate visual, audio, or textual content in a manner that appears authentic. Voice cloning technology can replicate a person’s speech patterns with minimal source material, making impersonation scams increasingly difficult to detect.

The criminal implications are extensive:

• impersonation of corporate executives for financial fraud;
• fabricated confessional videos or audio clips;
• digitally altered evidence in criminal investigations;
• reputational blackmail and extortion;
• AI-generated pornography and sexual exploitation;
• manipulation of political or public-order sensitive content; and
• false evidence used in matrimonial, commercial, or criminal disputes.

India has already witnessed a sharp increase in cyber-enabled fraud involving impersonation through video calls, fake law enforcement communications, and synthetic voices. As these tools become more accessible, criminal litigation will inevitably confront disputes around manipulated evidence and fabricated digital trails.

Admissibility of AI-Generated Evidence Under Indian Law

Indian courts have historically accepted electronic evidence subject to compliance with statutory safeguards under the Indian Evidence Act, 1872. Sections 65A and 65B continue to govern admissibility of electronic records, while judicial precedents have clarified procedural requirements surrounding electronic certification.

However, AI-generated or AI-manipulated evidence introduces a fundamentally different challenge. Traditional electronic evidence disputes generally concern authorship, chain of custody, or device integrity. Deepfakes raise concerns about whether the content itself was ever genuine to begin with.

Courts may increasingly face situations involving:

• disputed audio recordings allegedly generated through voice cloning;
• edited CCTV footage;
• fabricated WhatsApp voice notes;
• manipulated video confessions;
• synthetic images used in extortion complaints; and
• AI-generated communications falsely attributed to accused persons.

In such cases, admissibility may no longer depend solely on technical compliance under Section 65B. Courts may also require stronger proof regarding authenticity, source verification, metadata integrity, and forensic validation.

This could gradually lead Indian courts toward stricter evidentiary scrutiny for digital content, particularly where allegations of AI manipulation are raised.

The Evidentiary Challenge: Can Digital Evidence Still Be Trusted?

The increasing realism of AI-generated content creates a broader institutional concern — the erosion of evidentiary trust.

Traditionally, courts have often treated photographs, video recordings, CCTV footage, and voice recordings as persuasive corroborative evidence. Deepfakes complicate this assumption. A fabricated video may appear visually convincing even to trained investigators.

This creates multiple litigation challenges:

Burden of Proof and Authenticity

A key issue will be determining who bears the burden of disproving manipulation. If an accused claims that a video confession or voice recording is AI-generated, prosecutors may need to establish forensic authenticity beyond ordinary evidentiary standards.

Conversely, fabricated allegations of “deepfake manipulation” could also become a defensive strategy to discredit genuine evidence.

Chain of Custody Concerns

Courts may place greater emphasis on chain-of-custody documentation for electronic evidence. Questions regarding device access, transfer history, cloud storage logs, and metadata preservation may become central to criminal trials involving digital evidence.

Metadata and Source Verification

Metadata analysis could emerge as a critical evidentiary tool. File creation timestamps, editing traces, compression inconsistencies, AI-generation artefacts, and transmission histories may become essential for determining authenticity.

Expert Testimony

Forensic experts will likely play a much larger role in criminal litigation involving AI-generated content. Courts may increasingly rely on digital forensic laboratories, cyber experts, and AI-detection specialists to assess manipulated evidence.

This may also increase litigation costs and procedural complexity in criminal prosecutions.

Forensic Authentication Will Become Central to Criminal Litigation

As deepfake technology evolves, forensic authentication may become indispensable in both investigation and trial stages.

Current forensic techniques used globally include:

• biometric facial mapping;
• audio waveform analysis;
• lip-sync inconsistency detection;
• AI artefact detection;
• metadata examination;
• blockchain-backed verification systems; and
• source-device tracing.

However, the problem is dynamic. AI-generation tools are evolving faster than detection technologies. This creates a continuous “arms race” between synthetic media generation and forensic verification.

India’s cyber forensic infrastructure may therefore require substantial strengthening. Investigative agencies, forensic laboratories, and judicial officers will need specialised technical training to assess AI-manipulated evidence effectively.

Without institutional capacity building, courts may struggle to distinguish genuine digital evidence from sophisticated fabrications.

Platform Liability and Regulatory Pressures

Another major legal issue concerns the liability of digital platforms hosting or distributing manipulated content.

India’s Information Technology Act, 2000 and the Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021 already impose due diligence obligations on intermediaries. However, deepfake-related harms may intensify demands for stricter platform accountability.

Potential regulatory developments may include:

• mandatory AI-generated content labelling;
• takedown obligations for synthetic media;
• traceability requirements;
• stronger identity verification systems;
• reporting obligations for AI-enabled fraud; and
• enhanced obligations for platforms hosting generative AI tools.

The challenge lies in balancing free speech protections, intermediary safe harbour protections, and the need to prevent criminal misuse of synthetic media.

Excessive regulation may raise concerns around censorship, surveillance, and over-removal of legitimate content. At the same time, weak enforcement frameworks may enable large-scale fraud and reputational harm.

Indian policymakers are likely to face increasing pressure to develop a dedicated regulatory framework for synthetic media and AI-generated content.

How Indian Courts May Respond

Indian courts have historically shown adaptability in dealing with technological evolution. Judicial recognition of electronic evidence, digital signatures, and cybercrime jurisprudence reflects this gradual evolution.

In the context of deepfakes and AI-generated evidence, courts may eventually move toward:

• heightened authentication standards for sensitive digital evidence;
• mandatory forensic examination in disputed cases;
• judicial guidelines for handling AI-generated content;
• adverse inferences for deliberate manipulation of evidence;
• stronger evidentiary safeguards in criminal prosecutions; and
• specialised protocols for cybercrime and synthetic media investigations.

Courts may also recognise that manipulated digital evidence can directly impact constitutional rights, including fair trial protections, privacy rights, and reputational interests.

The judiciary’s approach will likely seek a balance between technological realities and procedural fairness.

The Emerging Risk of “Digital Arrest” and Synthetic Authority Fraud

One particularly concerning trend involves “digital arrest” scams, where fraudsters impersonate police officers, regulators, judges, or investigative agencies through video calls and synthetic communications.

Victims are often psychologically coerced into transferring funds under the false belief that they are under investigation or facing imminent arrest.

Voice cloning and AI-generated visual impersonation significantly increase the credibility of such scams. These cases expose a broader enforcement challenge: criminal impersonation is no longer limited to forged documents or fake phone calls. AI now allows fraudsters to simulate institutional authority with remarkable realism.

This may require stronger coordination between law enforcement agencies, telecom providers, financial institutions, and digital platforms.

Conclusion

Deepfakes and voice cloning technologies are forcing criminal justice systems worldwide to reconsider foundational assumptions about evidence, authenticity, and trust.

For Indian courts, the challenge extends beyond technical verification. The rise of synthetic media raises broader concerns regarding due process, evidentiary reliability, cyber enforcement, and platform governance. As manipulated digital content becomes more sophisticated, criminal litigation may increasingly depend on forensic validation and technical expertise rather than visual credibility alone.

The legal system will ultimately need to adapt to a new reality — one where seeing is no longer believing, and where the authenticity of digital evidence may itself become the central issue in criminal trials.