Content Moderation
Livestream moderation
Enabling safe and engaging live-streaming experiences.
Real-time automated analysis of live video frames & audio to surface or auto-enforce against policy violations with minimal latency. Use this guide to enable, configure thresholds, act on incidents, and continuously tune signal quality.
Enable the Feature
1
Request Access
Contact support to enable livestream moderation.
2
Provisioning
Support confirms activation; feature appears in Console settings.
3
Shadow Phase (Recommended)
First 1–2 weeks: log results but keep automatic termination conservative.
Enable
Provision & price
Configure
Per-category thresholds
Monitor
Status & incident volume
Intervene
Stop / Delete streams
Tune
Threshold calibration
Audit
Review false positives
How Detection Works
The system continuously samples frames (and audio signals where applicable) and classifies content into risk categories. Each detection produces category confidence scores (0–100%). Your configured thresholds decide whether a stream:| Outcome | Trigger | System Action | User Experience |
|---|---|---|---|
| Pass | All confidences below Flagged thresholds | Continues | Normal playback |
| Flagged | Any category ≥ Flagged threshold but < Terminated | Marked for review (no auto-stop) | Appears normal to viewers |
| Terminated | Any category ≥ Terminated threshold | Broadcast stopped; post soft‑deleted | Appears as normally ended |
Detection Categories
Pornographic Content
Pornographic Content
Explicit sexual imagery/acts. Maintain strict thresholds to minimize exposure risk.
Violent Content
Violent Content
Graphic violence or severe physical harm. Sensitivity may vary by regional norms.
Prohibited Content
Prohibited Content
Illegal / compliance restricted material (e.g., weapons, contraband). Low tolerance.
Inappropriate Content
Inappropriate Content
Borderline or contextual material. Often noisier—calibrate carefully to reduce review overload.
Profanity Content
Profanity Content
Offensive language or slurs (audio/overlays). Tune to community standards.
Configuring Confidence Thresholds
Two thresholds per category: Flagged (human review) and Terminated (auto-stop). Flagged <= Terminated must always hold.
A threshold of 0 (or extremely low) will flood the queue with low-confidence matches, overwhelming moderators and increasing costs.
Tuning Playbook
When to Raise Flagged Threshold
When to Raise Flagged Threshold
Flag queue FPR > 30% or moderator backlog growing & SLA breach risk.
When to Lower Flagged Threshold
When to Lower Flagged Threshold
Confirmed late detections or missed early intervention opportunities.
When to Raise Terminated Threshold
When to Raise Terminated Threshold
Any unjustified auto-termination (precision drop) or creator complaints validated.
When to Lower Terminated Threshold
When to Lower Terminated Threshold
Severe violations frequently only reaching Flagged state; evidence of slow manual response.
Noise Isolation Strategy
Noise Isolation Strategy
Temporarily widen only the lowest-signal category (e.g., Inappropriate) while holding strict categories steady.
Acting on Incidents
Open flagged stream → verify frame samples → Decide: Stop, Delete, or Allow.
Key Metrics
| Metric | Definition | Target / Signal | Action Trigger |
|---|---|---|---|
| Flag Volume per Hour | Flagged streams / hour | Baseline after Week 1 | Sustained spike → investigate attack/content trend |
| False Positive Rate (Flagged) | (Flags cleared as OK) / Flags reviewed | < 30% | Higher → raise Flagged threshold |
| Termination Precision | Valid terminations / Total terminations | > 95% | Drop → raise Terminated threshold & audit samples |
| Miss Incidents | Confirmed violations not flagged/terminated | 0 critical misses | Any miss → lower relevant thresholds or escalate model gap |
| Avg Review Time | Time from flag to moderator decision | Within SLA (e.g., < 5 min) | Rising trend → staffing or threshold adjustment |
| Category Distribution | % flags by category | Stable pattern | Sudden skew → targeted abuse or threshold imbalance |
Best Practices
Shadow Launch
Shadow Launch
Run with only Flagged actions (no Terminated) for initial calibration if risk appetite allows.
Single Variable Changes
Single Variable Changes
Adjust one category per cycle to attribute impact.
Evidence Capture
Evidence Capture
Retain representative false positives & misses for model feedback.
Moderator Training
Moderator Training
Provide clear examples per category to align decisions & reduce subjective variance.
Creator Guidance
Creator Guidance
Publish community guidelines emphasizing live content do’s & don’ts.
Troubleshooting
| Issue | Likely Cause | Resolution |
|---|---|---|
| Overloaded Flag Queue | Flagged threshold too low / spike event | Raise threshold 5% or add temporary staffing; isolate noisy category |
| Repeated False Terminations | Terminated threshold aggressive or misclassified scenes | Increase threshold; review sample frames; report to support |
| Missed Severe Violation | Threshold too high or model blind spot | Lower thresholds; capture evidence & escalate for model improvement |
| Cost Surge | Increased broadcast minutes + high flag rate | Audit usage, enforce creator guidelines, optimize thresholds |
| Inconsistent Moderator Decisions | Lack of unified rubric | Establish decision matrix & training refresh |
| Slow Review Times | Understaffing or UI overload | Add shift coverage; refine filters; raise thresholds temporarily |