For a startup, getting new customers is everything, but converting website visitors into customers is tricky business. Online writing course, Now Novel, set out to improve its conversion rate through a series of tests and tweaks. Below is a case study on what Now Novel tried, and the results thereof.
No ad to show here.
Brendan McNulty, co-founder of Now Novel, had this to say of the experience:
The fascinating thing about CRO (conversion rate optimization) is that it sits between art and science: a grasp of analytics allows you to understand the challenges that people face, using this with your customer feedback allows you to make hypotheses about what will help them, and great design and copy will help to move them along the conversion path. CRO is not about changing button colour or simply tweaking your call-to-action it’s the holistic process of understanding how your product can be most effectively sold to your customers.
Case Study
Now Novel, an online novel writing course, was launched 18 months ago by Bridget McNulty, Dagmar Timler and myself. We were really excited about how we were going to ‘kick ass’ and hoped people would find our product fantastic. Our users did enjoy Now Novel, but using our application was not as easy as expected. This case study will discuss what we discovered and how we set about testing our assumptions, as well as some of the missteps we made along the way.
Startup marketing blogger, Sean Ellis, has a ‘must have experience’ theory that aha moment when you realize how an application works, appreciate the value in it and then start using it on a regular basis (e.g. it’s connecting with 7 friends on Facebook and following 5 people on Twitter). This ‘must have experience’ for first time users is crucial and everything a startup does should be optimized towards achieving this. After undertaking user testing we found that Now Novel’s ‘must have experience’ was based on getting to the end of our mood creator (the mood creator is the first step of our process when you create the mood of your novel) and seeing your novel’s blueprint.
Signup flow
When optimising our website, the first thing we looked at was our signup flow. We initially looked at our homepage, running a series of 5-second tests. A 5 second test consists of showing strangers the homepage for 5 seconds and then asking questions about what they saw. We wanted to ensure people understood what the benefits we offered from the outset (i.e. was it immediately obvious what we offered to people just browsing?). From the test results, it seems that they had a decent comprehension of what we were about.
In parallel we analysed where people were clicking and whether that was matching our expectations. We laid out each of the possible touchpoints on the user journey (Google view-> homepage ->signup page-> etc.) and tried to see if we had the same ‘scent’ going through the whole sequence. ‘Scent’ is the process of providing a continuous experience and ensuring that our messaging was consistent throughout. Combined with the output from our Google Analytics funnels, we identified areas to focus on (see below, image)
Analysing heatmaps of our (long) homepage we could see that people were focusing on the top of the page, specifically on the call to action button ‘start your novel’, which was great. Users were also clicking on the ‘find out more’ button. This directed people through the ‘find out more’ tab to the ‘signup’ page, and not to a page which would give more information. This was potentially putting people off the scent, contributing to our poor 15% conversion off our signup page.
No registration
We did test allowing people to test the process without having to sign up. Our assumption was that this would decrease the barrier to trial and we would get more people interacting with the application. This test wasn’t a success, however having subsequently optimized the process towards our must have experience, we will be testing it again early in 2014.
Signup page
Our first attempt at using the benefits of our product on the signup page was a failure (21% less than our default). Our new candidate page didn’t do better than our default page (to compound the pain this was against a page that had “Sign up” as the headline.)
In order to improve this, we used a single question survey to ask people why they signed up for Now Novel on the registration confirmation page. We then sorted these answers into similar genres, prioritized the most prevalent answers and used the exact wording our customers were using to describe why they were signing up on our new test page. We added a recommendation from the Oprah magazine as an extra trust factor on the page. The final element to test was to drop the requirement for you password confirmation (i.e. you only have to fill in your password once.) Now Novel isn’t your bank and we don’t have to be super secure with a password confirmation. If you mistyped your password you can easily recover it through a lost password reminder.
We saw a pretty huge increase with this test, 25% increase over the default.
Further examination of heatmaps on the page showed that we were losing people to our site navigation on the top and bottom of the page. We ran a test to see if we increase conversion by remove all site navigation from the page (i.e. remove the site navigation from the header and footer from the page to remove the temptation to be distracted. This test had a 15% increase over our default, bringing the overall improvement to 81%.
First user experience
Looking at our Google Analytics data and our funnel from signup through confirmation to people using the application we saw a large drop off from confirmation to answering the first question of our mood creator. We did some user testing and saw that people didn’t know what to do once they had signed up: the next steps required to proceed weren’t obvious, and this was causing a dropoff.
Essentially our onboarding/first user experience wasn’t intuitive. Our hypothesis was that by adding a short tour of the application rather than putting people onto a discovery page (with a video, next steps and FAQ), that they had to click through to our dashboard page would get a better result and further progression towards our “must have experience”. Our onboarding tour is much like that of social games or a lot of web apps: it takes the most important parts of the service, shows you where they are and how to use them.
You can see our default page, and the onboarding tour that we integrated below.
Testing them we found that the onboarding tour had a 6% increase in conversion to answering the second question, so it has become our default.
Another hypothesis we had was that instead of using only short and punchy text in the onboarding flow we should reiterate the benefits of our program and continue to sell in this process. Although it only had a 90% (so not statistically significant enough) chance of a 4% increase in reaching the must have experience, we made it the default as it had a 25% increase on reaching the border of our freemium offer, and it gave us the opportunity to sell more effectively.
We have more tests lined up for this part of our product (as well as an interesting overhaul of our pricing page that we are currently testing).
Have you had any interesting test results that you’d like to share? Let me know in the comments.
