Automated Reporting
Summary
Intelligent Insights is the name given to the automated reporting for Cyara (previously for the company Spearline). All customers of the Voice Assure product in Cyara receive a monthly report highlighting benchmark reports of the best and worst performing network carriers and connection in their prospective countries around the world. The reports are curated by the Customer Success managers for their customers. They highlight the performance of business telephone numbers, SLAs, benchmarks and comparison with their peers in the telecommunication industry.
Goal
The goal of this project was to:
1. A way to deliver a report that Customer Success Managers can curate for their individual customers.
2. Layout the reports in a clear way that highlights insights, bad and good performance, relevant metrics, historical trends, benchmarking and comparison with peers in the industry.
3. Save time for the Customer Success Managers, whom some, spent a full working week manually creating reports.
4. Make the reports attractive to executives to buy into the product and the importance of how it can make a difference to their business's performance and SLAs.
Problem
The current process to generate a meaningful report for a customer is very difficult and time consuming. The final report that is sent to a customer is usually completed outside of the Platform by Cyara staff.
The information that is being pulled from the Voice Assure Platform needs to be redesigned into another format before being sent to a customer as there are no standardised reporting formats or templates that can be used to generate the reports.
There is also no targeted reporting generated from the Voice Assure Platform that could be presented to different areas of the customers business i.e.: at a management level through to technical users.
Discovery Phase
Methods used: Interviews, heuristic evaluation, sketching concepts, user testing, design thinking, wireframes, prototyping.
There were 3 areas of focus on this project:
1. The settings area for the Customer Success Manager - CS Manager to curate the monthly report and email it to the customer.
2. Displaying data analytics in a way that was easy to view insights and highlights.
3. The layout of the reports and the user experience of these reports.
I worked with the Product Manager Carl and (later Ola became the PM for this project and eventually saw it though) and the CTO Matt on the requirements of the automated reports - which we ended up calling Intelligent Insights.
Matt knew the customers very well having been the co-founder. He had the idea that there would be 5 separate reports and that each report would be targeted to different personas, from executive level to the engineering level. The first report would be high level; it highlights the most important data, showcasing benchmarks and worst performing countries by various metrics. The metrics related to the audio quality and the connection of business's telephone numbers.
The other 4 became more granular until the 5th report would report on 1 single number's performance for the previous month.
Interviewing Users
After speaking with Matt, I interviewed each of the CS Managers. I discovered their pain points. Creating reports for their customers was a very manual process. They had to download raw data from the platform and upload to Excel sheets where they calculated average data to highlight worst and best performing metrics and benchmarking metrics. They also screenshot charts from the platform to add to their reports. Looking at the reports, none of the CS Managers had a consistent look and feel. They did use brand guidelines but they all their style for creating reports. Reports would take them up to a week or more to create. This was time they could have spent on the customers needs and the business.
Data Analytics
I studied the reports that the CS Manager team sent out. I listed the main metrics and how they highlighted important issues. They also compared metrics with the previous month. Trending was important also to track performance.
I received Excel tables from Matt. At the time I didn't have access to a data analyst so I played with the chart types, together with design thinking to come up with the best way to display data in the most impactful way.
Ideation Phase
I approached this with a Design Thinking approach. I focused on the problems we were trying to solve. I worked closely with the CTO and CEO as they wanted to first report to have impact for the Executive Level.
Exploring layout, data visualisation of the various metrics and benchmarks met
Design Phase
I requested to get help from a graphic designer. What we discovered is graphic designers shy away from data visualisation, but luckily 1 designer at the time decided to work with us. She very quickly took the wireframes and discovery I had done as well as the company brand guidelines and created some mockups.
First draft designs of the Intelligent Insights reports with dummy data
User testing
I and the product manager demo'd the reports to 5 customers and 6 internal staff. We showcased to the different personas groups.
These persona groups were:
1. Contact Centre/BPO persona
2. Unified communications/Conference persona
3. Carrier Network persona
We spoke to Directors, Team Leads and we had very technical engineers and non technical users to make sure the reports suited all personas and technical levels.
Mostly feedback was positive. Positive feedback such as - “it hits on a key point” and “It's clear to understand”. Negative feedback "it’s only useful if comparing from the same point such as comparing with other conference companies." This feedback made sense as the reports were not segmented for the persona groups. The reports were targeted to all persona groups, however as there were 5 reports to choose from, this could solve the issue of targeting certain customers to make a tailor made report.
Getting Stakeholder approval
Getting feedback from the customers and internal users really benefited the design progression of the reports. However getting approval from the main stakeholder was not as easy. The CTO Matt rejected the designs. He mostly focused on the colours, which he found were too bright. It turned out Matt is colourblind so I made the strategic decision to use 'colour blind safe' colours. I sourced a palette from Venngage. We also decided to hire another graphic designer. Again we went through the design thinking process and demo'd the wireframes.
Design Approval
Taking the feedback on board from the users and the CTOs feedback, I worked again on the layout and colour-blind safe colours. With the assistance of a UX intern, we worked closely with the Graphic Design contractor.
Final Reports
The CEO and CTO finally approved of the designs. The colour-blind safe colours really brought the reports to a new level. It was great to make the reports accessible for colour blind users.
The developers made an amazing job of bringing the Figma files to life and allow the reports to be PDF generated.
Top Level Report
Targeted to Executive level personas. Has high level information, highlighting specific metrics, 6 month trends and comparison to their peers.
All Countries Breakdown
Targeted to management and engineering level. This gets more granular showing worse performing countries in specific metrics.
Individual Country Breakdown
Granular report on one country showing how that country performed the month previous and compare to the month before.
All Numbers Breakdown
Granular report that would be most useful to an engineer, showing the business telephone numbers and how they performed against specific metrics.
Individual Number Breakdown
Granular report that would be most useful to an engineer, showing one business telephone number against specific metrics and how it performed with average data per day for one month.