Problem Definition š¤
Introduction
For publishers hosting high volumes of user comments, maintaining community standards while fostering engagement presents significant challenges. Our legacy moderation system struggled with information overload, inefficient filtering for high-volume tasks, and limited transparency for tracking and verifying decisions.
š
The old moderation panel (had numerous UX & product flaws)
Who is it for
Moderators are focused on approving valid comments fast and preventing not safe comments from being published too long.
Moderation leads are focused on quality assurance of the moderators and automated moderation models performance and accuracy.
Research
My approach was rooted in user-centered research, prioritizing solutions that directly addressed moderators pain points and streamlined their workflow.
I conducted interviews with key business stakeholders and members of moderation teams, including moderators and moderation leads, to identify the most significant pain points and align them with business needs.
Key Insights
Moderators often click on incorrect moderation actions, which takes time to revert.
Tony Yemelyanov
Quick approval of non-toxic comments is crucial for engagement.
Tony Yemelyanov
Searching for comments with specific properties is challenging in the current product, but necessary for moderation QA.
Tony Yemelyanov
Comments are flagged by numerous models and tags, making it unclear why a comment requires moderation.
Tony Yemelyanov
Some moderation teams need to provide reasoning for approval or rejection due to legal regulations.
Tony Yemelyanov
Moderation work occurs from desktop devices only.
Tony Yemelyanov
Through interviews and usability testing with moderators, four key needs emerged.āØThe failure to meet these needs has led to slow approval times, inconsistent moderation, and an inability to scale effectively with traffic demands.
Enhanced Usability
to improve legibility and reduce time on fixing incorrect actions
Contextual Clarity
to improve legibility and reduce time on fixing incorrect actions
Efficient Filtering & Bulk Action Capabilities
to handle high comment volumes quickly.
Moderation Events Logging
to maintain accountability and support moderation quality assurance.
Business OKR
Reduce time to publish eligible comments
Based on interviews with key stakeholders I've found out that our current automation system effectively prevents ineligible comments from being published but often sends eligible comments to manual review. Until approved, these comments remain hidden, missing the window for engagement ā especially critical for news sites (most of our customers) where relevance is time-sensitive.
Delays in publishing lead to a decline in comment interactions, reducing future user participation and impacting overall site engagement. This drop affects key metrics for publishers, such as retention, time on site, ad viewability, and ultimately revenue. Thus, our primary business KPI in redesigning the moderation experience is to minimize the time required to publish eligible comments.
Feature Prioritization
Throughout several ideation sessions with product managers, developers and analysts, we came up with a variety of feature ideas to address each of the discovered needs but given limited development resources, we used a prioritization matrix to decide which features are most promising to implement in the first phase.
We chose to reserve the Automation and Smart Suggestions features for future phases, focusing instead on implementing the remaining prioritized features (green & yellow) using a lean approach where feasible.
This decision reflected a balance between the value these features bring to addressing moderatorsā pain points and achieving the business OKR, as well as their acceptable level of required effort.
Design Decisions š§©
The redesigned system provides a structured moderation experience where each component serves a specific purpose, collectively enhancing speed, accuracy, and transparency. Hereās how each component contributed.
Focus area #1
Comment cards in moderation
Moderators previously had to sift through excessive, unstructured information for each comment, leading to cognitive overload and decision fatigue.
Old comment cards interface in moderation panel didnāt provided easy access to valuable context
In the old design the moderation triggers were displayed in an unclear order under the comment, moderator had to hover to see valuable information and the tags that didnāt have enough space in the line, were hidden behind the āmoreā button making it even more difficult to discern the most critical issues and general moderation concerns reasoning.
This streamlined layout reduces decision time per comment, empowers moderators to make accurate choices quickly, and minimizes cognitive load by focusing only on essential information.
The new content tickets design in moderation displays the most valuable information for a quick decision while allows access to detailed information when a deep investigation is needed
The Comment Card consolidates critical information into an easily scannable layout, helping moderators quickly assess and act.
Key features include
1. Commenter Profile
Summarizes user credibility based on history, rejection rate, and engagement to support informed decisions.
2. Moderation Triggers
Organizes flags (user reports, restricted words, AI models) into separate tabs, showing at-a-glance why the comment needs review.
3. Safety Score
A risk indicator derived from multiple metrics, enabling faster prioritization of low-risk comments.
4. Rejection reasons
Allows moderators to select predefined rejection reasons or enter custom reasons when rejecting a comment. This flexibility is useful for meeting legal requirements in certain regions or saving time for teams that don't need detailed explanations.
Moderation Triggers Ordering
In the new solution we restructured the triggers into five main categories, each in its own tab at the bottom of the comment card.
Tabs are displayed based on the recency of the trigger, as the most recent event is often the most relevant.
The new content tickets design in moderation displays the most valuable information for a quick decision while allows access to detailed information when a deep investigation is needed
Reporter Details
Up to four reporter avatars and details are shown per reason. Hovering over an avatar reveals the reporter's approval rate, helping moderators assess the legitimacy of the report.
This structure helps moderators quickly determine whether a report is valid or if it's potentially an abuse of the reporting feature.
Efficiency Consideration
In over 90% of cases, comments are tagged by only one category. In such instances, the tab switcher is hidden to reduce cognitive load, displaying only the relevant trigger content.
Focus area #2
Filtering Tabs & Bulk Actions
With thousands of comments entering the moderation queue daily, moderators needed an efficient way to prioritize and process high-volume tasks. The existing system lacked advanced filtering capabilities and bulk action options, making it difficult to quickly address the most relevant comments. This led to bottlenecks, delayed approvals, and reduced overall engagement.
Filtering options in the old moderation panel lacked flexibility and required extensive scrolling to locate
In the old moderation panel, users could filter and enable bulk actions such as Approval or Rejection on selected comments. However, these filtering options lacked flexibility, and frequently used options often required extensive scrolling to locate.
The streamlined layout reduces decision time per comment, empowers moderators to make accurate choices quickly, and minimizes cognitive load by focusing only on essential information.
The new interface allows finding a very specific type of comments by mixing different types of parameters
In the new product we allow flexibility to mix anything with anything to filter results with a very specific conditions. Moderators can now create custom filters based on parameters like safety score, commenter history, and time in the queue.
Advanced filtering together with bulk moderation actions allow moderators to save time
Together, Filtering Tabs and Bulk Actions streamline the moderation process, helping teams prioritize critical content and process high volumes efficiently. This setup not only improves moderator productivity but also ensures timely publishing of relevant comments, driving higher user engagement and meeting business KPIs.
Focus area #3
Comment log
The legacy moderation system lacked a centralized, accessible history for each comment, making it difficult to track actions, verify decisions, and address appeals or quality assurance processes. This gap led to inconsistencies, reduced trust in moderation outcomes, and inefficiencies in error resolution.
The redesigned Comment Log serves as a comprehensive record for every comment, offering moderators the transparency and tools needed for effective decision-making.
Key features include
Detailed History
Captures every action taken on a comment, including approvals, rejections, reversals, and the rationale behind each decision.
Appeals Support
Provides clear records of user-submitted appeals and the moderation teamās responses, streamlining the process and ensuring fairness.
Audit Trail
Ensures compliance with quality standards and legal requirements by maintaining a transparent log of moderation activity.
Collaborative Access
Enables team members to view prior actions and decisions, fostering collaboration and reducing missteps.
The Comment Log design enhances transparency and accountability, empowering moderators to review, validate, and correct decisions with ease. It supports smoother workflows, ensures consistent application of moderation policies, and builds trust among teams and stakeholders.
Conclusion š”
The redesign of the moderation experience focused on solving real user pain points while aligning with business goals. By prioritizing clarity, efficiency, and transparency, we created an integrated system that empowers moderators to handle high comment volumes without compromising accuracy or speed.
Key outcomes of this project include
Comment Cards
Streamlined decision-making with context-rich layouts that reduce cognitive load and improve accuracy.
Filtering Tabs
Enabled moderators to create and save custom views for prioritizing and managing specific sets of comments, improving workflow efficiency.
Bulk Actions
Allowed moderators to process large volumes of comments simultaneously, drastically reducing manual effort.
Comment Log
Enhanced transparency and accountability with a comprehensive history for every comment.
Together, these solutions not only address the immediate challenges faced by moderation teams but also lay the foundation for future enhancements, such as automation and smart suggestions. The focus on reducing the time to publish eligible comments has a direct impact on engagement, retention, and overall publisher revenue, driving both user and business success.
This project highlights my ability to balance user-centered design with business strategy, delivering impactful, scalable solutions that address complex challenges. It demonstrates a strong collaboration with stakeholders and an iterative, user-centered approach to product and product design that ensures every decision adds value.
