FeedbackBot: An AI-Powered User Feedback Analysis Platform
How I built an AI-powered system that connected thousands of seller responses and surfaced product problems manual review missed
At a glance
- Challenge
- Thousands of seller responses were flowing into Slack, where keyword search and manual review made it difficult to recognize recurring problems, connect differently worded reports, or prioritize feedback by customer value.
- Approach
- Built an AI-powered feedback pipeline that classifies responses, groups related feedback by meaning and product area, synthesizes recurring themes, and routes bugs into Linear with the relevant customer context.
- Outcome
- Classified thousands of pieces of seller feedback and surfaced product problems that manual review had missed.
- Role
- Principal Product Manager and builder
- Timeline
- Released Jan 2026
The problem
I built an AI-powered feedback pipeline that turned a high-volume stream of unstructured seller responses into a classified, searchable, and actionable dataset. By clustering related feedback semantically and routing bugs directly into our issue tracker, the system helped the team spot patterns that manual review had missed and respond to high-value sellers faster.
Before FeedbackBot, every survey response landed in a Slack channel. Someone had to open each one, read it, and decide on the spot whether it meant anything. Slack search only matches exact words, so it worked fine if you already knew what to search for and it did nothing for you if you didn't. Insight lived in whoever happened to be paying attention to the channel, and in whatever word they thought to search for.
Take low stock alerts, as one example. One seller wanted a heads-up before something sold out completely. Another asked why there wasn't a warning when inventory got low. A third just wished the dashboard would flag anything running out. Read one at a time, when you happen to glance at the feed, it might seem like a low-volume request. But in reality the frustration has been building quietly without anyone realizing it.
How it works
Auto-classification
Each response is split into unique pieces of feedback tagged by type, plan, account age, and product area.
Semantic clustering
Vector embeddings group responses by meaning, not keyword — so differently-worded reports of the same issue land together.
Survey synthesis
Per-survey summaries turn each cluster into a readable narrative instead of an thread on slack.
Linear routing
Bugs are immediately routed to our team bug project in Linear with customer context for further triage.
Inside the product
Triage, clustering, ticket routing, and integration settings — the surfaces sellers' feedback flows through.
The scale of what needed reading
Why manual triage stopped working.
Signal 1: Reorder Products
Customers use different language to describe the same feature, making search difficult.
At the end of 2025, we released a complete overhaul of our seller admin and began inviting existing users to check it out. Due to a time crunch toward the end of the project, several legacy workflows weren't ported over right away. For four months, feedback about product reordering came in fragments. One seller said they couldn't move products around like the old version. Another asked what happened to drag-and-drop. A third called it unprofessional that their variants displayed out of order after adding a new one. Read one at a time, in a Slack channel full of other feedback, it was easy to miss the pattern.
In total, at least seventeen sellers complained about this feature, but what's even more important is who these sellers were. They were all sellers on our highest tier that had been with us for years, sellers it would be extremely painful to lose over such a silly miss. Fortunately, because the platform surfaced this for the team we were able to quickly restore the functionality and let each seller know personally.
Different words, same complaint
"I can't move products around like the old version."
"sort the product by order of display"
"drag and place would be a good feature"
Signal 2: Packing Slips
All twelve reports mentioned “packing slip,” but because they arrived over two months, nobody realized they traced back to the same regression.
The problem wasn’t finding the words “packing slip.” Each response described a different symptom—blank pages, cut-off details, overlapping text, or layouts split across multiple pages—and arrived amid a much larger stream of feedback. Read individually, they looked like isolated printing problems.
Once the platform was launched, it quickly grouped them together, we could see that the complaints began after the new admin launched and traced back to a single layout regression. The team was then able to identify the cause and quickly ship a fix.
In sellers' own words
"the top half of a packing slip will be on the bottom of a page and the bottom half on the top of the next page"
"I have to switch back to print the packing slips, because they come out so cut off"
"I've mentioned this 4 times now with no response"
Key Takeaways
Collecting feedback and learning from it are two different problems. FeedbackBot gave the team a way to move from isolated responses, to recurring patterns, to the customer segments affected, and finally to a product decision. In both examples above, the system surfaced problems that had been visible in individual messages but invisible as patterns.
More case studies
Rebuilding Big Cartel's discount engine to drive merchant adoption and conversion.
Five experiments that built the case for a trial-first acquisition model.
How lightweight fixes solved a duplicate-account spike without a multi-month rearchitecture.
A 17-skill AI system that processed 20,000+ pages of discovery and took the document-dump tactic off the table.
A photo and voice-first renovation punch list where AI does the filing and the app serves three tasks a day.
How solving a missing draft-product state and a decade of one-off themes shipped Big Cartel's strongest upgrade driver.