This project focused on the full-screen modal for resume customization. Two experiments had been conducted previously, but due to unsatisfactory results and persistent user issues, we decided to revisit and improve this area.

What bugged us

While conducting usability studies, we learned that despite two iterations, the appearance change modal still causes difficulties for users.

Additionally, our CV builder is available on several foreign markets and apart from a few problematic aspects that we observed during usability studies, the thing that was very important to us when preparing the third iteration of Mega Modal was the unification of the design version in all markets, because until now depending on the portal, different versions of the modal performed better.

The Goal

• Improve user experience

• Unify the components across markets

• Increase the CVR

The Team

My team, being the part of the whole design team was responsible for the building CV process and this time, the same as in every other experiments conducted in our CV builder, I was supported by the design team including PM, UX Designer and a UI Designer. We didn't go through the full project together, but this team was crucial during brainstormings or data analysis.

Scope of Work

Even though I was supported by my team members, as the experiment owner I was responsible for the main design decisions and selecting the main painpoints from the reseach stage to focus on.

Spoiler - results!

Depending on the market, the process of achieving statistical significance took varying numbers of days and required gathering different test groups. However, we finally managed to test the customization modal across all markets. 

The results? We won on every of the 7 countries we’ve run the test 🥳

Sounds great, but… how?

Usability Studies

To improve the modal’s performance, I needed to revisit what we had already learned in previous researches. By conducting usability tests with new participants, we receive their first impression with the product which is crucial for the final results.

Recordings Analysis

User interviews are priceless, but it’s also important to observe how users actually behave in a natural, unmoderated environment. This is why watching and analyzing recordings and heatmaps on Fullstory was a crucial part of preparing for a new iteration.

Data Analysis

To confirm our concerning observations, we turned to Mixpanel. The obtained charts clearly showed us that the problem is real and that there is significant potential for improvement in certain areas.

Other Brands' Insights

This stage of working on experiments can often be quite helpful. Sometimes, the ideas that are already forming in our minds have been tested by other employees on different platforms, allowing us to see how these ideas fared in the real world.

Brainstorming time

No extensive commentary is needed here. Depending on the project, I use different brainstorming methods, but the main goal is always the same: to extract the most interesting ideas from participants that can help us create a new, more effective design.

It's time to go Lofi…sh

When we translate ideas into an initial design. Although I often create actual low-fidelity sketches on paper or a tablet, in this case, we decided to go directly to mid-fidelity to better visualize the selected ideas.

Before we dive deep into designs

To avoid wasting time on a design that can’t be implemented, I consulted the entire plan with the development team after preparing the first clear visual forms and a detailed description of the expected changes.

Dev’s gave a green light for our ideas - let’s make magic happen! 👨‍🎨

Experiment started

We launched the experiment and continuously monitored the results across all markets. We checked and analyzed all differences in real time. Naturally, we started the process with one market, and after achieving positive results, we validated our design on the remaining ones.

Results

As you already know, the results were more than satisfying. Not only did we achieve significant business success by greatly increasing conversion among users in seven countries, but we also managed to unify this part of the product, which made our developers very happy 😃

Ok, numbers looks good - now what?

To baseline the experiment, the most important aspect is, of course, to observe an increase in conversion in the product, or if it's not significant, to note improvements in micro-conversions or other indicators that would demonstrate the superiority of the new variant over the control version. However, regardless of the size of the observed increase, to implement the variant into the product, three key elements of the experiment analysis must be checked:

Summary

It was quite a lengthy process, as evident from this case study, but we can clearly state that our three goals were achieved, and some increases in CVR even exceeded our expectations.

Can’t wait for more projects like this!