Punjab Court Directs Social Media Platforms To Remove Deep Fake Videos Of CM Bhagwant Mann, Asks Google To Make It Non-Searchable

A Punjab court has directed Meta Platforms Inc. (Facebook) to immediately remove and block objectionable AI-generated content impersonating Chief Minister Bhagwant Mann, observing that the material appeared indecent, sordid, and potentially capable of disturbing public order.

Judicial Magistrate Sarveesha Sharma said, "This court has gone through the contents of objectionable material and is prima facie of the opinion that it is indecent and sordid at the very least. Secondly, considering the fact that the objectionable material is against a public official, Chief Minister of Punjab Sh. Bhagwant Mann, the tendency of the material to incite public disorder cannot be absolutely ruled out with certitude. Thirdly, at this stage, the apprehensions that the material could be AI-generated or synthetic cannot be said to be absolutely unfounded."

The order was passed on an application filed by the Station House Officer, State Cyber Cell, Punjab, seeking directions for urgent blocking of the impugned content under the Information Technology Act, 2000 and the Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021 (IT Rules, 2021).

The FIR, registered under Sections 340(2), 353(1), 353(2), 351(2), 356(4) of the Bharatiya Nyaya Sanhita (BNS) and Section 67 of the IT Act, was based on information received by SHO Gaganpreet Singh that a Facebook account named “Jagman Samra” had uploaded objectionable and vulgar AI-generated videos and images impersonating the Chief Minister.

Preliminary examination by the cyber monitoring cell revealed that the content was synthetic media created with malicious intent to mislead the public and harm the Chief Minister's reputation. The account, operated from Canada, was identified as belonging to Jagman Samra, against whom the FIR was subsequently lodged.

After perusing the material, including a pen drive containing the alleged content, the court noted that the videos were prima facie indecent and sordid, and considering that they targeted a public official, the possibility of public disorder arising from such posts could not be ruled out.

The court further observed that the apprehension regarding AI-generation of the material could not be dismissed at this stage and that expert opinion would be required during trial to determine its authenticity.

“If the rights of an individual are not protected at the earliest and the competent court were to finally conclude that the material was in fact AI-generated or defamatory, no purpose would be served as the right would become infructuous,” the Judge observed.

Court Reiterates Need For Responsible Digital Conduct

Emphasizing the growing menace of AI-manipulated media, the Magistrate remarked that swift action was essential to demonstrate that wrongful digital conduct would be remedied effectively. The court noted that while freedom of expression is a fundamental right, it cannot extend to the creation or dissemination of synthetic, defamatory, or indecent material designed to mislead the public.

“It would be appropriate to direct the blocking and taking down of the allegedly objectionable material as prima facie it appears to be indecent, immoral and beyond the bounds of responsible reporting,” the order stated.

Directions Issued To Meta And Google

Allowing the application, the court issued the following directions:

Meta Platforms Inc. (Facebook) shall remove and block the objectionable content hosted under the specific URLs mentioned in the FIR.

Meta shall preserve all data and associated records relating to the offending content to ensure evidence is not vitiated.

Meta shall also remove or disable access to all identical, mirror, or derivative versions of the content, as and when intimated by the State Cyber Crime Department.

Google has been directed to de-index and de-reference the offensive material, ensuring it remains non-searchable.

The court warned that failure to comply would result in loss of statutory immunity under Section 79(3)(b) read with Section 85 of the IT Act and Rules 3 and 7 of the IT Rules, 2021.

Mr. Hanvir Singh, APP for the State Complainant/IO in person

Title: State Vs. Jagmal Samra