‍

Trust and safety is a field most people never think about until something goes wrong. But for every platform where people upload content – from documents to images to reviews – there’s a team working to keep what doesn’t belong off the platform, and what does belong easier to find. Nicole Pauls is leading that effort at Scribd, Inc.

As head of content trust, Nicole oversees the policies and systems that govern what's allowed on a platform with hundreds of millions of documents, across dozens of languages and nearly every region of the world.

We sat down with Nicole to learn how she approaches that responsibility, what's changed as AI accelerates the threat landscape, and why the work is more than just enforcement.

How did you end up in content trust?

I came to it through IT security. I spent most of my career as a product manager in that space, and along the way I collected a lot of knowledge that turned out to apply more directly to trust and safety than I ever expected. There's a pattern that runs through both fields: you're constantly trying to find bad things and get rid of them. The methods differ, but the core problem is the same.

After taking time off during the pandemic to focus on my kindergartener, I came back and landed a trust and safety product management role. I had all of this built-up background in security, and suddenly it clicked. I'd always thought those experiences were unrelated to the field. Turns out I was wrong.

What did you do before Scribd, and how does it inform how you think about this?

Before Scribd, I worked at a two-sided marketplace with B2B customers. Marketplaces are a very active trust and safety environment – transactions happen constantly, person to person. Scribd has a different kind of problem. It's a knowledge-building environment. But the underlying questions are the same ones I deal with now. What do you need to detect? How do you find it? How do you handle legal compliance? Same context, different framework.

How do you describe what content trust actually means at a platform like Scribd?

The mission is to make harmful content as invisible as possible. When someone comes to Scribd, they should be able to trust that what they're finding is safe, vetted, and useful.

But it's not only about removing bad things. We also try to elevate things on the other end – good documents, quality content, making the best stuff easier to find. It's less about policing and more about raising the floor.

What does it feel like to be responsible for content on a platform with hundreds of millions of documents?

It's a big responsibility. The scale is one thing, but the diversity is what really hits you. People come from all over the world, for wildly different reasons, in dozens of languages. You're not just responsible to one community or one type of usage.

You have to stay focused on what actually matters most to the people using the platform. 

‍What's the hardest part of the job that nobody outside the team sees?

That my role is new, and there's a lot of building from the ground up. Scribd has been around for nearly 20 years and has been doing content moderation throughout. But now we're formalizing the team, the tooling, and the processes around it so we can move faster, more efficiently, and at scale.

In some ways that's harder than starting from scratch; you can't just create everything new. You're working alongside a legacy platform with real legacy decisions and real stakes. Scribd is also unique from its peers in social or marketplaces. The mix of documents, users, and use cases doesn't have a direct analog, so a lot of what we're building has to be invented independently.

We're also expanding the scope of the work to focus more on quality-of-the-corpus and not just  "catch the worst content and handle copyright." That's a bigger ambition, and it requires getting the team and infrastructure in place to support it.

What do you wish users understood about what this team actually does?

That it's more than removing spam. At the highest level, we're trying to help people find what they need faster, whether that means improving descriptions and metadata, better search, or fewer odd results showing up in recommendations. That involves a lot of teams across the company.

I'd also want people to understand why we can't simply remove all bad content immediately. We have to be careful. A document that sounds troubling might be a court record that someone genuinely needs. Removing everything that raises a flag would make the platform worse.

So we take a layered approach: elevating good content, filtering potentially harmful content, and removing when needed.

What's changed in the last couple of years — has AI made your job harder?

Harder, in several ways. AI and automation have made it faster and cheaper for bad actors to generate harmful content at scale – spam documents, bot farms, new domains that bypass detection. In the document world specifically, AI makes it easier to produce content that sounds legitimate, which makes it harder to identify. You can't just scan a document for obvious signals anymore.

It's a technology arms race. But what I've observed is that people running bad content at scale are also cheap and lazy. They'll take the path of least resistance. Our job is to make that path as expensive and difficult as possible – through identity verification, bot detection, and end-to-end partnerships – and keep making it harder.

What tools does the team use to manage content at that scale?

It's an ecosystem: large language models, machine learning, traditional data science, and human review working together.

The human element is something I think about a lot. You still need people to write the policy, define what's in and what's out, and handle the genuinely hard cases. A model might flag a document because it contains a description of a sexual assault. A person has to read it and figure out if that's a novel or a court case. One stays up. One doesn't. That's not a call you hand off to a classifier.

What surprised you most about the Scribd corpus when you first started looking at it?

The global scope. People don't always think about how many different reasons someone might come to Scribd, the topics they want to learn about, the knowledge they want to build, the documents they need to do their work.

Different regions use the platform in completely different ways. You have to account for all possible use cases, all possible languages, all possible contexts. A single policy doesn't translate cleanly across all of it.

What are your biggest priorities for the rest of the year?

Policy is a big one: making sure what we enforce is clearly documented, so we can move faster when we need to. Speed of detection is crucial to everything we do.

We're also deepening our participation in the trust and safety community, through organizations like the Trust & Safety Professional Association – we recently attended TrustCon. We have specific work underway on spam and copyright this half of the year. And we're working toward releasing our first transparency report, which I'm really excited about.

Across everything, we are working to formalize something that’s been happening in various forms for years. It’s a big project, but a really cool one. 

‍