International Growth Experiments
Lead Product Designer, 2018-2019
+ Design & team strategy
+ Research, testing & fast-follow iteration
+ UX/UI design
+ Growth experimentation
+ Cross-functional relationship building and communication
+ Design systems collaboration
The International Growth team wanted to run tests on their pricing page and checkout flow in order to promote new monthly packages and pricing on the platform.
Status: All of these experiments have ended. Some were integrated into the product experience.
“Create a more targeted package offering, pricing page and checkout experience for our users in Italy, Spain and France.”
Overview
As our international efforts became more of a focus, the International Growth team was tasked with hitting an 18% year-over-year growth target in 2019 with expected acceleration in 2020.
In addition to this ambitious goal, the International Growth team had never worked with a dedicated product designer, so the majority of the experiments they had been running were based on copy and engineering improvements.
My role
I was already supporting two other teams, so my time had to be spent efficiently when working with this team. The team was almost completely made up of members who were new to the company: The growth director, Senior PM, Engineering lead, content strategist and engineers had all joined the company within the last six months. Additionally, two of the engineers had never worked directly with a product designer before.
Some of my contributions included:
Worked with leadership and teammates to develop rituals, cadence out workflows, and strengthen communication best practices to help the team better integrate into the company’s processes
Focused on designing experiences that were good enough to learn from in testing, but not too polished that it slowed down the team’s progress
Collaborated and built relationships with the engineering, product management and design systems team parters in order to keep forward progress as efficient as possible
Planned experiments by helping to identify key performance metrics, phasing tests to isolate variables and learnings, developed a thorough understanding of business goals and metrics and how to analyze test result
Business rationale
There was significant customer demand for monthly plans on higher-tier packages in the International market, and we believed that by offering new monthly packages, it would generate incremental new paid seats, increase out basic-to-paid rate, and have a positive effect on AOV by shifting demand for Standard Monthly packages to Advantage or Premier Monthly packages.
The success metric we focused on was the basic-to-paid rate for both existing and new users.
Design Strategy & User Research
I worked with the team to create some variations on the current user experience for paid plan sign-ups.
We needed to answer the following questions in our research:
Do users understand the differences between monthly and annual plans?
Is the toggle component the best way to navigate between the two categories of plans?
Does the ‘NEW’ label highlight the new packages effectively?
Do users feel they understand the annual saving percentages?
Onsite team training
I traveled to SurveyMonkey’s Dublin office for 10 days to work onsite with the team and we ran two sprints (team kickoff, designs, user testing, iteration, development, experiment launch, learnings, repeat).
Experiment metrics
Key performance metric: Conversion
Audience: France, Italy and Spain
Hypothesis: There is significant customer demand for monthly plans on higher tier packages in the Intl market, and we believe that these new monthly packages will generate incremental new paid seats, increase B2P rate, and have a positive effect on AOV by shifting demand for Standard Monthly to Advantage or Premier Monthly.
Success metric: Basic to paid rate
Results: We have seen a drop in advantage annual package mix (significant) so we are setting it to no new assignments.
Next steps: Finance to model and launching new designs to mitigate annual package mix drop
Experiment results
Our B2P (basic to paid user) upgrade rate was significantly higher in the treatment we tested. Additionally, ARPU (annual revenue per user) went up 9% and refunds went down to 0 (400%).
The next round of iterations would include:
In-product messaging to upsell advantage monthly to advantage annual
Highlighting features such as unlimited responses with advantage annual
Include US test in FR, IT, ES for annual savings on checkout
Add new currencies so we can test other lower B2P markets
Revisit the features for monthly packages to shift mix
Remove standard and rename advantage/premier so to just display 2 packages
Increase standard monthly price, test increments
Learnings
Communication is invaluable when working with a team; talk through scenarios, discuss things early and often, and work with a shared understanding of your goals
Nothing beats working onsite with a team
Clearly prioritize features for experiments so engineers know what it will take to ship something that’s good enough to learn from but not so polished that we’re losing time and resources to experiments that aren’t winning