TikTok Creators: Safety Tools
I turned enforcement-quality data into structured product recommendations, built the policy-to-product layer underneath safety surfaces, and designed the internal tooling and measurement moderation operations ran on. The direction that evidence argued for — prevention at the point of creation rather than review after it — is the one TikTok’s 2025 creator-safety tools follow
Role
Product Systems Designer & Product Operations
Industry
Social / Trust & Safety
Year
2023-2024
Platform
iOS, Android & Internal tools
Context
Moderation does not scale
In one quarter of 2024 TikTok removed 178.8 million videos. 98.2% went before anyone reported them, 144.4 million by automated systems, 93.5% within twenty-four hours. In the EU alone, roughly 18 million violative items were proactively removed in the second half of that year, across a base of 159 million users, at a stated automated-moderation accuracy of 99.1%. Every figure here is TikTok’s own published transparency data.
Read those numbers as a product brief and they say one thing. At that volume the review queue is not where quality problems get solved — it is where they arrive. A percentage point of prevention is worth more than any amount of review capacity you can hire, and the remaining work is almost entirely in the cases the model is unsure about.
I owned policy-to-product translation safety surfaces depend on, the internal tooling moderation and operations teams work in.
Challenge
— Policy is not a spec. Community Guidelines are written for people and have to be resolved into thresholds, taxonomies and states before any surface can act on them.
— Enforcement is legally consequential and regionally uneven. The EMEA remit carries the DSA; what is shippable in one market is not automatically shippable in another.
— Moderators are users, and their handling time is a business metric. One extra click is multiplied by millions of reviews.
— Automation carries the volume, so the design problem sits where the model is uncertain, not where it is confident.
— Nothing can be tested on a live account without consequence. There is no safe pilot for an enforcement decision.
— Almost none of it can be shown. This case argues from mechanism, because the alternative is showing nothing.
Discovery
The queue tells what the product failed to prevent
Quality analysis is a diagnostic instrument if you read it as one. Review enforcement decisions at volume and the same clusters keep returning: categories where moderators disagree with each other, rules where appeals succeed, thresholds where the model hesitates, creators who produce the same violation repeatedly. Each cluster has a location — a surface that let something through, a rule nobody could interpret the same way twice, or a creator who never knew the rule existed.
The finding that reframed the work: a large share of repeat volume was not adversarial. People did not knowthe rule, or found out only when something was removed. Those cases cannot be solved downstream at all.Review capacity can process them forever without reducing them by one.
That is the argument for moving the decision earlier — to before publish, to the creator’s own controls, and into the live moment where there is no afterwards.
178.8M
videos removed globally in a single quarter — TikTok public data
98.2%
removed proactively, before any user reported them — TikTok public data
~18M
violative items proactively removed in the EU in half a year — TikTok public data
99.1%
stated accuracy of automated moderation in the EU — TikTok public data
Core concept
Three places the decision could happen earlier
If the queue is where prevention already failed, the design question is where the decision could have been made instead. There are only a few honest answers, and each is a different point on the timeline.
01 - Before Publish
Tell a creator what will happen to a video while they can still change it, instead of after it is gone.
02 - Creator-Side Controls
Give creators real filters over their own comment space, so moderation is not the only defence against abuse.
03 - The Live Moment
LIVE has no post-hoc fix. Controls have to work inside the same seconds the harm does.
04 - In The Tools,Not The Policy Doc
Encode the rule where the decision is actually made, so interpretation is notre-derived by every reviewer.
TikTok LIVE screen filters: comment settings, mute rules, mute duration
Policy is not a spec until somebody makes it one
Most of the work was translation. A guideline says what is not allowed. A product needs a threshold, a taxonomy, a set of states and a copy string for each one. That layer was mine: decision trees turning a policy line into a question a reviewer can actually answer, categories stable enough to be measured across markets, and the states a safety surface needs — warned, limited, removed, appealable, restored — defined once instead of re-invented per feature.
The test for it is boring and decisive. Two reviewers in two markets, same content, same decision. When they do not agree, it is almost never a training problem.
The internal tooling is the same argument pointed inward. Review interfaces, queue structure, escalation paths and QA sampling are product surfaces with the highest usage-per-pixel in the company. Removing a step from a review flow, or making the uncertain cases visually distinct from the routine ones, moves handling time and decision consistency at a scale no user-facing change matches — and nobody outside the function ever sees it.
Validation
Measuring enforcement
Operations measurement is the closest thing this domain has to usability testing. Appeal and reinstatement rates say whether a rule is applied as written. Inter-rater agreement says whether it can be applied at all. Handling time says what the tooling costs. Repeat-violation rates say whether the creator ever understood what happened.
I ran that loop and fed what came out of it back as structured recommendations into specific in-app surfaces. It was the evidence product and engineering teams priced their decisions against, which in a function this size is the thing that actually moves them.
What the data kept saying
— Ambiguity in a rule shows up as disagreement between reviewers long before it shows up in appeals.
— Repeat violations concentrate among creators who never received a legible explanation.
— A rule that cannot be measured consistently across markets is not one rule. It is several.
— Most review effort goes to cases automation already flagged as uncertain — which is precisely where interface design pays.
What that changes
Recommendations of this kind do not ship as a feature with your name on it. They land as a threshold adjusted, two categories merged, a state added to a surface, a step cut from a review flow. I supplied the evidence and the structure. Product and engineering teams decided what to build with them. Anyone who tells you otherwise about a function this size is describing a different company.
Impact
Where the direction landed
I left TikTok in July 2024. In July 2025 the company announced Content Check, Creator Care Mode and LIVE comment muting. All three are the same move: put the decision before the harm and in the creator’s own hands, instead of in a review queue afterwards.
One public number is attached to it. The earlier seller version of Content Check produced a 27% reduction inlow-quality uploads — prevention, measured, at the point of creation.
I am not claiming those launches; the dates make that clear enough. What I am claiming is four years spent building the evidence and the systems layer that made that direction the obvious one, inside a function whose default answer to every problem is to add review capacity.
Other projects
2x conversion on the mobile homepage
Redesigned a top-10 Russian bank's mobile homepage from discovery to design language. Validated with eye-tracking and a staged A/B rollout: product-showcase conversion 4.2% → 8.14%, with no metric regression.
From one store to an omni-service
Turned one of Russia’s grocery chains’ app from a hypermarket delivery service into an omni-app covering five store formats. A/B tested to 30% of traffic: format switching 0 → 1.62%, loyalty engagement +25–30%, offline ARPU +1.99% — against a 1.36% dip in order conversion.
From LMS to streaming service
Rebuilt a Russian cultural-education platform — 157,000 users, 100+ courses — from an ageing LMS into a streaming product, then shipped iOS and Android off the same system in ten days with no additional design work. Purchase conversion +32% on day one, D7 retention 23% → 30%, authorisation 65% → 91%