Skip to main content

How to analyze what your app users are actually complaining about

Three ways to find what's wrong: surface the themes and read the quotes, interrogate a hypothesis with Claude or ChatGPT, and tag the topics you're actively fixing.

A
Written by Axel Lavergne

Your rating slipped and you don't know why. You have thousands of reviews across the App Store and Play Store, in languages nobody on the team speaks, and reading them one by one isn't a plan. There are three ways to find out what's wrong, and they answer different questions:

  • Start blind when you have no idea what's broken. Surface the themes, then read the quotes.

  • Interrogate a hypothesis when you already suspect something. Ask Claude or ChatGPT to read the reviews and tell you if you're right.

  • Track what you're fixing when the work is underway. Tag the topic and watch the complaint volume move.

One thing you won't find below: exporting a CSV and running a topic clustering script. That advice is all over the internet and it's a waste of your time. Nobody hired you to tune an NLP pipeline, and doing it badly is worse than not doing it, because you end up trusting clusters that don't mean anything.

This is the job you buy a tool for. So here's how to do it with the reviews already sitting in your account.

Start blind: find the themes, then go read what people said

When you don't have a hypothesis, start at Reports, then Semantic analysis. Reviewflowz reads your reviews and builds the topic tree out of them: five topics, three subtopics each, with the positive and negative mentions counted over time. Nobody hands you a taxonomy, so the categories are the ones your users actually raise, not the ones an industry template thinks a dating app or a bank should care about.

Semantic analysis: the topic tree built from your own reviews, with performance over time

Now the part everyone skips. The chart tells you that a subtopic collected 89 negative mentions last month. It does not tell you what the complaint is. Charts give you a direction, and a direction is not a bug report. So once a subtopic spikes, stop looking at the chart and go read the sentences.

That's what Sentiment analysis under Reviews is for. Every quote in there is the exact sentence from a review that carries the feeling, tagged with its subtopic and scored positive or negative. Filter to the negative quotes for the subtopic that moved, sort by most recent, and read twenty of them. Twenty is usually enough: by the tenth you know whether it's the checkout, the sync, or the new tab bar. The quotes are translated too, so a spike in Portuguese is as readable as one in English.

The extracts list: every quote carries its subtopic and its sentiment

Read the quotes before you brief the team. A subtopic called "Bug Handling and Stability" trending negative is a mood. "The app logs me out every time I switch networks" is a ticket.

The mechanics of how topics and sentiment get built are in Mining Your Reviews for Business Insights.

Interrogate a hypothesis: have Claude or ChatGPT read the reviews for you

Sometimes you don't need to explore, you need an answer. You shipped a release on the 12th, support swears people hate the new onboarding, and you want to know whether the reviews back that up before you reprioritize a sprint.

A dashboard answers the questions it was built to answer. Your question is more specific than that, so ask it in plain English instead. Connect Claude or ChatGPT to your review data with the Reviewflowz MCP, and the model reads across your reviews to answer: what complaints appeared after the March release, which ones show up in 1-star reviews but not 3-star ones, what exactly do people say breaks when they mention login. Here's how to connect it.

Ask for the quotes, not just the summary. It's the same discipline as the first method: a model that hands you a tidy paragraph has done the reading for you, and you learn more from the three sentences it based that paragraph on. So end your question with "show me the reviews you're basing that on", and read them.

This is also the fastest way to kill a bad theory. Half the time the reviews say the onboarding is fine and the complaints are about the price you changed in the same release.

Track what you're fixing: tag the topics that matter to you

The first two methods have a blind spot. The topics Reviewflowz discovers are the ones your users raise, which is not the same as the ones your roadmap is about. If your team is spending the quarter on the onboarding flow, you want a line on a chart called onboarding, and you want it whether or not the AI decided onboarding was one of your top five topics.

That's what tags are for. Go to Settings, then Tags, and create a tag named after the thing you're working on. Tick "Use to tag reviews automatically" and the AI attributes that tag to your reviews based on the theme you named. So name the tag the way a user would complain, not the way your codebase is organised: "crash on launch" and "subscription price" work, "AuthModule" doesn't.

Then hit "Process all my reviews". This runs your tags across the reviews you already have, which matters more than it sounds: it gives the tag a history. Without it you can only see complaints from today forward, and "did the fix work" is a question you can only answer by comparing after to before.

Settings, Tags: the auto-tag toggle per tag, and Process all my reviews to backfill

Now you have the one number that counts after a release: is the complaint volume on that tag going down. Filter your reviews by the tag to read what's still coming in, and report on it over time to see the trend.

Tag creation is covered step by step in How to use review tags on Reviewflowz.

Which method, and when

  • You don't know what's wrong: start blind. Topic tree first, quotes second, and don't stop at the chart.

  • You suspect something specific: ask the model, and make it show you the reviews.

  • You're fixing something, or you'll care about it for the next six months: tag it and track it.

Most teams end up using all three, in that order. You explore once to find the fire, you interrogate whenever something smells, and you tag the things you're going to live with. What none of them require is that you become a data scientist first.

Once you know what to fix, tell the users who complained. Replying to App Store and Google Play reviews with AI closes that loop, and a reviewer who hears back often updates their rating.

Choosing a tool for this job? Here's what to look for, and how Reviewflowz approaches it.

Did this answer your question?