Tell HN: Patreon banned AI porn, I lost 5K subs without warning

7 points

2 years ago

Hey HN, about a year ago I launched my AI porn site on Hacker News [1] and it reached the top of the front page. Initially, the site was free to use. I got a ton of traffic and was overwhelmed - I ended up shutting down the site. Dozens of users told me they would pay for the service, so I set up a Patreon and re-opened the site. Users could pay for "Pro mode" for additional benefits on the site, such as faster generations. It was a huge success and many people subscribed.

Fast forward a year later and I have since set up an alternative billing system. However, I still had about 5K subs on Patreon who did not switch. Just a few days ago, I got an email at 3AM [2] from Patreon: they shut down the site. I got no warning, they just deleted my page and cancelled all of my subscribers. AFAIK, they also haven't refunded any of the cancelled subscribers. So someone could have paid a day before the ban, then Patreon deleted the page, and the user never got their money back.

The reason was because "Due to updated credit card network guidelines, we are no longer able to support generative AI software that is focused on creating pornographic works. This is because this software can be used to create works that violate our guidelines on mature themes & pornography." For context, Patreon's previous terms said that pornography is allowed if it does not depict real people (i.e. art, comics, etc). AI porn was a grey area because although the content is photorealistic, the people are technically AI-generated.

My theory is that this ban was related to a controversy with another site called CivitAI. Recently, their cloud provider dropped them because CSAM and other unsafe content was generated on their site [3]. I have tried to avoid issues like this on my site by disabling free text prompting for image generation. Users have to choose from a set of preset, vetted tags. In addition, users cannot upload their own images to create deepfakes. I also have custom trained classifiers to detect unsafe content. And a human moderation team. I tried making AI porn as safe as possible, but it still wasn't enough for the credit card companies or Patreon...

Lessons learned:

- Don't use Patreon for any serious billing

- AI porn is still a rapidly evolving space, and credit card companies don't know what to do with it

- AI Safety is super important and most AI image generation services don't take it seriously enough. For example, you can just type in child, or the name of a celebrity, and it works.

[1] https://pornpen.ai/

[2] https://i.imgur.com/p6j9eec.png

[3] https://www.engadget.com/controversial-ai-image-platform-civitai-has-been-dropped-by-its-cloud-computing-provider-195530538.html

1 comment