Chatbot Performance Report and AI Analysis

Overview #

The Performance Report in Noem.ai helps you understand how your chatbot is being used, how many conversations it is handling, and where your team may need to improve the chatbot experience.

Inside the Stats tab, you can view message activity, new conversation threads, average messages per thread, and Noem.ai handled activity over time. You can also enable AI Analysis, which reviews chatbot conversations and reports useful findings back to your Chatbot Inbox.

When AI Analysis is enabled, Noem.ai runs a nightly analysis on chatbot threads. Weekly, it can also email a summary report to the recipients you choose.

Additional token consumption may apply when AI Analysis is enabled.

What the Performance Report Helps You Understand #

The Performance Report is designed to help answer questions like:

• How many messages is my chatbot receiving?
• How many new conversation threads are being created?
• Are users actively engaging with the chatbot?
• Is the chatbot handling conversations consistently?
• Are there patterns that need review?
• What should my team improve next?

This makes the Stats tab useful not only for reporting, but also for improving your chatbot over time.

Step 1: Open the Stats Tab #

To view your chatbot’s performance report:

1. Log in to Noem.ai.
2. Go to My Projects.
3. Select the chatbot you want to review.
4. Click the Stats tab.

The Stats tab opens the Performance Report for that chatbot.

From here, you can view activity charts, enable AI reporting, refresh the dashboard, and email a report.

Step 2: Review Activity Over Time #

The main activity chart shows chatbot usage over a selected time period.

The chart may include:

• Messages: the total number of chatbot messages during the selected period
• New Threads: the number of new conversation threads started
• Total Messages: total message count for the reporting period
• Average Messages per New Thread: average conversation depth per new thread

You can use the activity selector to review different reporting windows, such as a 90-day view.

This helps you understand whether chatbot usage is increasing, decreasing, or staying flat.

Step 3: Use Auto Refresh #

The Auto Refresh toggle allows the Stats dashboard to update automatically.

Use Auto Refresh when you want to monitor recent activity without manually refreshing the page.

If you are reviewing older data or do not need live updates, you can leave Auto Refresh turned off.

Step 4: Enable Weekly AI Analysis Report #

To enable AI Analysis:

1. Open the Stats tab.
2. Find Enable Weekly AI Analysis report.
3. Click the gear icon next to the setting.
4. Open AI Analysis Settings.
5. Review the available settings.
6. Turn on Enable AI Analysis.
7. Click Save.

Once enabled, Noem.ai will run an AI analysis on chatbot threads and report findings in your Chatbot Inbox. Weekly, Noem.ai can also email an AI Analysis report to the recipients you configure.

Additional token consumption may apply.

Step 5: Add Weekly Report Recipients #

Inside AI Analysis Settings, use the Email Weekly Report to section to choose who should receive the weekly report.

To add a recipient:

1. Enter an email address.
2. Click the + button.
3. Repeat for any additional recipients.
4. Save your settings.

You can use this to send reports to team members such as support leads, marketing managers, customer success teams, or business owners.

Make sure all recipient emails are correct before saving.

Step 6: Enable AI Consolidation for Common Questions #

The AI consolidation for Common Questions option helps group repeated or similar questions across chatbot conversations.

This is useful when you want to identify what users keep asking about.

For example, AI consolidation can help surface patterns like:

• repeated pricing questions
• common login issues
• frequent product questions
• unclear support requests
• repeated confusion about a feature or service

Use this insight to improve your chatbot instructions, update your knowledge base, or create better website content.

Step 7: Customize Additional Training #

The Additional Training section lets you guide how Noem.ai analyzes chatbot threads.

If these options are turned on, your input is added to the AI Analysis run. If they are turned off, the default analysis behavior remains.

You can customize three types of guidance.

Action Plan Guidance #

Use Action Plan Guidance to tell Noem.ai what kind of recommendations matter most to your business.

This is helpful when you want the AI Analysis to focus on a specific improvement area.

Example guidance:

“Focus recommendations on improving the clarity of our services.”

Other examples:

• “Prioritize issues that may affect lead conversion.”
• “Focus on gaps in product explanations.”
• “Highlight confusing answers that could cause support tickets.”
• “Recommend improvements to demo booking flow.”

Use this field when you want the weekly report to produce more relevant next steps.

Review Guidance #

Use Review Guidance to control how the AI should review conversations.

This can help the report ignore noise or focus on the types of conversations that matter.

Example guidance:

“Ignore reviews from test conversations.”

Other examples:

• “Focus only on real customer conversations.”
• “Pay extra attention to support-related complaints.”
• “Ignore internal testing messages.”
• “Focus on conversations where users appear confused or frustrated.”

This helps keep the analysis cleaner and more useful.

Suspicious Guidance #

Use Suspicious Guidance to help Noem.ai decide what should or should not be treated as unusual, risky, or suspicious.

This is important because every business is different. A question that seems unusual for one business may be normal for another.

Example guidance:

“We sell survival equipment, so fire-starting questions are acceptable.”

Other examples:

• “Customers may ask about security settings because we sell IT services.”
• “Refund-related questions are normal and should not be flagged as suspicious.”
• “Ignore repeated test prompts from our internal QA team.”
• “Flag users asking for private customer data or account access.”

This helps reduce false positives and makes suspicious activity findings more accurate.

Step 8: Save AI Analysis Settings #

After configuring recipients, common question consolidation, and additional training, click Save.

If you want AI Analysis to run, make sure Enable AI Analysis is turned on before saving.

If this toggle is off, Noem.ai will not run the AI Analysis workflow, even if other fields are filled out.

Step 9: Email a Report Manually #

The Stats page includes an Email Report button.

Use this when you want to send or generate a report without waiting for the next weekly email cycle.

This can be useful before team meetings, client check-ins, reporting reviews, or chatbot optimization sessions.

How to Use the Report to Improve Your Chatbot #

The value of the Performance Report is not just seeing activity numbers. The goal is to use the information to improve the chatbot.

Here are practical ways to use the report.

If message volume is low #

Review whether the chatbot is visible enough on your website or deployed channels. You may need a better welcome message, a stronger chat button, or more relevant page placement.

If new threads are high but conversations are short #

Users may be opening the chatbot but not finding value. Review the first message, starter prompts, and the chatbot’s opening response.

If average messages per thread is high #

This may mean users are engaged, but it can also mean they are struggling to get an answer. Review long conversations to see whether the chatbot is being helpful or going in circles.

If common questions repeat often #

Update your knowledge base, website content, or chatbot instructions so the bot can answer those questions more clearly.

If AI Analysis finds suspicious or unusual activity #

Review those threads manually and decide whether the chatbot needs stronger guardrails, better routing, or clearer refusal behavior.

Troubleshooting #

I do not see any activity in the Stats tab #

If the chatbot is new or has not been deployed yet, there may be no activity to show.

Check that:

• the chatbot has been published live
• the chatbot is deployed to at least one channel
• users have interacted with it
• the selected date range includes actual activity

The weekly report was not received #

Check that:

• Enable AI Analysis is turned on
• the recipient email address is correct
• the report recipient was added with the + button
• the email is not in spam or junk
• enough activity exists for the report period

AI Analysis is not running #

Open AI Analysis Settings and confirm that Enable AI Analysis is turned on.

Also remember that AI Analysis may consume additional tokens. If your account has token or usage limitations, review your usage settings.

The report is too generic #

Add more specific guidance in the Additional Training section.

For example, instead of leaving Action Plan Guidance blank, tell the system what you care about:

“Focus on missed sales opportunities and unclear pricing explanations.”

The more specific your guidance, the more useful the analysis can become.

The report flags normal conversations as suspicious #

Update Suspicious Guidance to explain what is normal for your business.

For example:

“Customers often ask about account access and password resets. Do not treat those as suspicious unless they request another user’s private information.”

Best Practices #

Review the Performance Report weekly. Regular review helps you catch issues before they become bigger problems.

Use AI Analysis for improvement, not just reporting. The best reports should lead to specific changes in your chatbot instructions, knowledge, or deployment.

Add multiple report recipients when needed. If support, marketing, and operations all care about chatbot performance, include the right stakeholders.

Customize Additional Training. Generic analysis produces generic recommendations. Use Action Plan Guidance, Review Guidance, and Suspicious Guidance to make reports match your business.

Watch common questions closely. Repeated questions are often a sign that your website, knowledge base, or chatbot instructions need improvement.

Check conversations in the Inbox. AI findings are helpful, but reviewing real threads gives you the best understanding of what users are experiencing.

More stats, more valuable data #

The Chatbot Performance Report and AI Analysis tools help you understand how your chatbot is performing and where it can improve.

Use the Stats tab to monitor message activity and thread volume. Use AI Analysis to review conversations, surface patterns, identify common questions, and receive weekly recommendations.

When used consistently, this feature helps you move from guessing what users need to improving your chatbot based on real conversation data.