OpenAI Sora restricts public viewing due to security concerns

OpenAI Sora limits the exposure of real people and takes other strong security measures to prevent misuse.

The video generator, announced Monday as part of its 12 Days of OpenAI event, has all kinds of editing capabilities for users to create and customize AI-generated videos. But there are certain things you’re not allowed to do with Sora, as users quickly find out.

The Tweet may have been deleted

According to its system card, “the ability to upload images of people will be available to a set of users,” meaning that most users cannot create videos of people based on an uploaded image. Those users are part of the “Likeness pilot” that OpenAI is testing with a select few. An OpenAI spokesperson said AI-generated human videos are limited to “address concerns about abuse of depth and similarity.” OpenAI will “actively monitor patterns of misuse, and when we find them we will remove content, take appropriate action with users, and use these early learnings to inform our security approach,” the spokesperson continued.

Limiting the exposure of people to Sora’s videos makes sense from a liability perspective. There are all kinds of ways the tool can be misused: unauthorized deepfakes, child exposure, scams, and misinformation to name a few. To combat this, Sora is trained to reject certain requests from text commands or image uploads.

It will prohibit the provision of NSFW (Not Safe For Work) and NCII (Non-Consensual Intimate Imagery) content and the production of realistic children, although fictional images are permitted. OpenAI added C2PA metadata to all Sora videos and made the visual watermark automatic, although it can’t be removed, and used internal reverse image search to check the video’s originality.

Despite the fact that many lines of caution have been put in place to prevent abuse, the question of how Sora will respond to severe stress testing. Currently, access to Sora is unavailable due to high demand.