10 Tips To Improve Customer Experience During A Beta Test

Updated September 20th: I asked Adam Pennacchio (@ajpennacchio), Head Of Customer Success at Groove to add some of his thoughts to my initial list of 10 tips, after their extremely impressive Beta test and famous customer service. Groove is an awesome support desk product specifically designed for SaaS companies. Check them out (they’re also cheaper than Zendesk and Desk.com)

Your prototype probably stinks. It’s probably full of bugs. Lacks features. Probably uses twitter bootstrap. Writing more code is expensive. And in a Beta test, writing more code has diminishing returns.

But talking with customers. Building a brand. Getting feedback. Creating evangelists. This is the stuff that makes a successful Beta test. It’s on this initial group of people that you will declare your Beta period a success or failure.

How this group of users respond to your product could mean the difference between securing an investment of $250k, or going back to your old employer with your tail between your legs.

Never again in your startup will you experience the rapid prototyping, the excitement, the energy, the passion from customers, as you will during this time.

In case you didn’t get my point yet, your initial Beta testers are fucking important. Here’s some of the ways we’re looking after ours at trak.io:

1. Always be as personal as possible during support & onboarding

The biggest impact on your brand will be the support emails and calls you have with your users. What’s the tone of your messaging? What’s the personality of your emails? Your in-app messages?

Would you talk to a girl on a first date in a monotone robotic voice? Would you pull out a book of chat-up lines and start reciting them at her until she opened up her panties for you?

Of course not. You’d ask her about who she is. Then adapt your game. Build a plan that is personally tailored to who she is, what she wants. She tells you she’s extremely religious? OK, maybe extend your dating plans a little until you play the ‘closing card’. Is she into ‘bad boys’? Ok maybe leave out the part about your 2 white fluffy persian cats.

Craft personal messages as often as possible, and deliver communication in conversation format as often as possible.

A private Beta test for a B2B SaaS product is likely to have < 100 active users, and a public Beta < 500 active users. With these numbers, you should take every opportunity to talk to each user personally. Could you customize the setup instructions to suit that user’s specific business type? Share ways other users with similar business are using the product.

A B2C product will have 1,000 – 5,000 active users in a private Beta test. Personally emailing each user isn’t realistic, but you should make it extremely easy for users to move from an automated flow to a personal one. For example, having “Hit reply to speak with one of our team” in all of your emails. Video is another great way to deliver personality in a scaleable way.

Oh and never ask for secondary registration to a support desk – if you do this you deserve to locked in a windowless room with Justin Bieber records playing 24/7.

Think about it, customer experience is about giving the customer what they want, with as little friction as possible. You might ask them to make an account, but that’s not what they wanted. They want support. Not another account.

Adam: I think it’s important to have goals with your onboarding and send a clear message to your customers about what to expect.

This helps sets the tone for your product. A couple of goals that are important: Simply be human and yourself, always give a personal feel. Provide your users with a touch point, let them know they can contact you anytime and you’ll be there for them. Lastly, you want to help guide them to the next steps they should be taking. This reduces friction and leads to better engagement.

Support, support, support. Your competitors can always copy your product and features, but they can’t copy your customer experience. Providing your customers with an amazing support experience differentiates you from your competitors and leads to brand loyalty.  Be quick, be thorough, and be accurate. At Groove, we try to answer every support request within the first 15 minutes it comes in, and when you do that people notice.

2. Always make time for Skype calls

Emails are limited in bandwidth. People are just better at communicating in a face to face conversation. Often, a 10 email thread can be solved with a 2 minute video chat. So talk to people.

Make every excuse to try and get people on Skype. And if you’re lucky enough that a customer asks you for a Skype/Hangout, jump at the opportunity! Drop everything to take the call. You’ll learn more in 10 minutes on Skype than a full day reading Techcrunch.

Particularly if you’re building a B2B product, your customers are probably time-poor and are quite familiar with just jumping on a 10 minute call. They might not have time to read you’re documentation. Every page of your marketing site. They just want to know, will your product solve their problem, and if so, how do they get started.

Keep it informal. “Let’s jump on Skype”. Pissing around with Webex invites at this stage may be overkill. Everyone can jump into a Skype or Hangout with a minutes notice. Sure, this won’t scale for long. Soon you’ll need to do some scoring on leads, and prioritize some calls over others.

When you need to scale this for one-to-many, start doing good quality webinars – with a caveat: don’t use free or cheap webinar services with poor audio and video quality. This makes your brand feel cheap and budget and it’s a false economy in the long run.

Adam: This is a fantastic point! I can’t stress how valuable it is to get on the line with your users and talk to them. Email is great, but for customer development nothing is better than a one-on-one conversation. The amount of insight you can gain from a 10 minute call is more than you’ll often get in 10 email responses. Plus a lot of users don’t take the time to write detailed feedback, but are willing to jump on Skype for a few minutes.

During Groove’s beta we made it a point to personally contact every user and get on the phone with as many of them as possible. Using a canned reply in Groove we sent out a message to every new customer 5 days into their trial asking if they’d be interested in a Skype chat.

A quick tip for getting the most out of your customer development interviews is to ask for advice. No one wants to be sold, and most customers expect you to give them a sales pitch about your product. So instead,  when you get a customer on the line genuinely ask for their advice. Who doesn’t love giving advice. This demonstrates you respect their opinion and sets the expectations for the call that the customer will be talking and you’ll be listening.  

3. Never be afraid of sharing deadlines, and disappointments

Developers are petrified of deadlines. 37 Signals cautions against them. But sorry, your customers aren’t all developers. They might have no idea how agile works. They might be business people, trying to fit your tool into a project timeline that does have deadlines.

So strap on a pair.

If you know a particular feature will be done “within 2 weeks”, then tell them. If you know you’re probably not going to have it done in that time, then you should tell them that too. Why? Because they might absolutely require that feature before they can jump in. Rather than keeping them hanging around, you allow them to stop them wasting their valuable time and move on.

This is a hard shift in mindset, and of course you need to be smart. Don’t be more precise than you can backup. Don’t walk yourself into a corner. Don’t push your development team against unrealistic expectations. Never burn unnecessary midnight oil.

Some startups or software teams live by “make no promises, make no disappointments”. It’s cowardly. We prefer to take every opportunity to look awesome to our early adopters.

Adam: To this point, I’d add to never over promise anything to a customer. That just leads to disappointment and a poor experience. Be as transparent as possible and manage your customers expectations. Don’t be a afraid to say no. It’s important to build trust with your customers, and nothing undermines trust like overpromising and under delivering.

A great example is when customer recommends a feature that you never plan to develop. Lot’s of people just tell their customers it’s on our roadmap and we plan to release it, hoping the customer will forget about it. Well they don’t. Instead tell the customer the truth, but provide a detailed explanation of why this doesn’t fit into your product. In his post on why 37 Signals stopped support certain browser DHH does a terrific job of this.

4. Spend time to research every tester and build quality notes in your CRM

Not all early adopters are created equal. Some are 15 year old college students. Some are bigshot investors who have come to see what you’re made of. Some are just trolls from Hacker News wanting to kick your tyres so they can clone your software and put it on Github. Others are CMO’s for a large enterprise.

Make sure you’re learning everything about users as soon as they’re identified. A trick we do at trak.io is have a people filter setup to exclude anyone with a free hosted email address (yahoo.com, gmail.com etc.) and then we go through exploring the domains of users’ email addresses.

We have a lot of startup customers, so this is a really quick way to find out who they are, size of team etc. As soon as I figure this out, I use the “edit properties” feature on their trak.io profile and add new values like “team_size”, “funded (true/false)” etc. Then I know when I’m reaching out to that person, I have notes to help understand who they are. How they might use the product.

I can also use this data to segment drip email campaigns and newsletters. You might use a more sales focussed CRM like Salesforce. Whatever you use, make sure you have detailed notes on every Beta tester.

It’s a lot easier to look after a tester than it is to find a new one.

5. Make sure you’re speaking with the right people

The person who first signed up for your Beta might not actually be the right person to speak to. Or at least, you need to send the right information to the right person.

In our case, we have a technical integration which is definitely a job for a developer. So there’s no point sending a ton of techie tutorials to the marketing guy who just discovered us while reading TNW. Often, you’ll have no way of knowing this until you start a conversation. So ask early on, “Who in your team should I speak to about…”.

Managers, marketing, sales… they might want the end benefits from the product. Inform them about the “why, benefits, pricing”. Send them your awesome blog content. Your screencast guides on new features. Invite your testers to send email intro’s to the other relevant people in their team.

This has multiple benefits: being top of mind with more people in the company increases chances of desire, retention, and makes activation more efficient.

Adam: The best feedback comes from those who have just signed up or just cancelled their account (Jason Fried). If you’re not getting feedback from the people that bounced, you’re making a big mistake.

It’s the feedback from these users that helps you understand what your product is lacking and what needs to be improved upon. They left for a reason, your product didn’t deliver what they were looking for. People bounce for lots of reasons, but it usually comes down to missing features, friction when onboarding, over complexity in the product, or there simply isn’t a market for your product. Speaking to the customers that cancelled will give you this insight.

6. Remind, inspire & motive, but don’t pester

Drip email campaigns can be very powerful, but they can also piss people off. It’s ok to drop people a targeted reminder about a critical step they need to complete. But it’s much more powerful to emotionally inspire them, motivate them, to complete that step.

Don’t tell them they need to create their first project. Instead, demonstrate with the benefits to their business of managing their projects within the tool. Use case studies, screencasts, walkthroughs.

Pushing people and telling them to do something is completely missing the point of the new user experience. It’s ok to be visual and obvious as soon as they’re ready to take action, but you need to plant a seed of desire first.

7. Make your beta testers feel important, and get great qualitative research

Beta testers invest a ton of time and faith in you and your product. They need to invest the time to get setup, they invest time in clunky interfaces, they put up with bugs and downtime.

But another area where the sacred relationship dictates they invest effort, is in providing feedback.

So make it easy for them. Provide every opportunity to reach out to you. Direct email addresses. Contact widgets. Skype. But also give them a helping hand with what feedback you want.

14 days since registering and they still haven’t activated? Ask them about the setup process? What plugins would they have liked to see available. Missing a library for their favourite programming language?

14 days since they got setup, ask them about using the main features. The simple stuff. The stuff you’d experience if you walked around and kicked the tyres a bit.

60 days after activating and using the product quite a bit. Now ask them what features are missing. Which integrations would they like to see to take the product deeper into their company.

Ask for the right feedback, at the right time.

Also important is making testers feel like  their feedback is being put to use. Make a note of individuals who requested a particular feature or bug, and send them a personal email when that feature or fix goes live. It shows you care. It builds a relationship. Continues the conversation. Why would you not want an excuse to talk to one of your customers?

8. Send weekly & monthly product updates

Beta products are very raw, so it’s important that testers are aware of the current feature set right now, particularly if they’ve gone inactive. Startups often pivot on a penny. Clunky features can be ripped out in hours. Entirely new ideas can appear in days.

You might just pull them back by adding that ‘deal breaker’ feature they needed. Or you might hit that magic motivation they needed to chase their development team to get the product integrated today.

You only have a limited number of testers, so its a lot easier to retain them, or pull them back, than it is to find totally new testers.

9. Build a library of self help documentation from day 1, & use screencasts for visual learners

Disclaimer: trak.io doesn’t have any screencasts live yet! Yikes!

Documentation and ‘how to’ guides are not something “to sort out later”. It’s probably a LOT more important than the new feature your CTO wants to get stuck into!

At trak.io, we wrote our documentation before we wrote our product.

We knew we were a product for developers to setup. We knew the first thing we’d be asking them to do would be to setup our API. Before they completed this, they couldn’t explore our product and test out all the cool reporting features. Nothing was more important than the API documentation.

Even if you aren;t a developer tool, if no one can figure out how to use your tool then all of those features are a waste. Sure, many B2C products should strive for utility without needing any docs, but you best make sure than the second they have a problem, they can diagnose it themselves.

Self help resources are insanely scaleable, and much more convenient than submitting support tickets. Be sure to create screencasts and training videos for people who are visual learners.

10. Reward your beta testers for their faith with discounts and real world gifts!

They gave you their time. They gave you their insights. They even let you sport their logo on your homepage. Now it’s time to say thankyou!

Swag is awesome. Everyone loves swag. It might be a t-shirt if you’re a developer tool like trak.io. At a previous startup we gave iPads (unanounced!) to our champions within our enterprise clients. Swag is a great way to translate a digital relationship to the real world.

Also, make sure you grandfather in your beta testers with generous discounts and promotions if they choose to stay with the service. And if you don;t intend to offer a free plan after the Beta, make sure you provide an export utility. It’s their projects, photos, videos, data. Let them take it away with minimal friction.


That’s just 10 of the ways we’re delivering customer experience during our Beta at trak.io. Read any of these and think I’m bat shit crazy? Leave me a comment and tell me why! Our customer experience is a constantly evolving process and I’d love to hear how other people look after their Beta testers!

Adam: Most importantly add value. You do this by finding out what your target customers problem is and building a solution that solves those issues. This is a process, it takes time but along the way provide the best experience you can for your users by being transparent, honest, responsive and making them feel special. Do that and everything else will hopefully fall into place.

A huge thankyou to Adam Pennacchio from Groove!

Published by

Liam Gooding

Liam is the cofounder and CEO of Trakio. Previously an engineer, he writes about growing subscription companies using data-driven techniques and inside glimpses to Trakio's own growth journey. He wrote a book, "Growth Pirate!" which discusses data-driven growth strategies for startups.