For young entrepreneurs with small networks, it's hard to get feedback on your idea. We'd like to grab on any tool available to communicate with potential users. Survey is one of them.
This week, I was conducting surveys to validate my potential pivot. Through this experiment, I learned great potential as well as limitations of surveys. I want to share what I've learned how to take survey responses to validate your startup ideas.
Problem with Landing Page Validation
The best well known approach to validate your idea is to set up your landing page. Y Combinator recommends you to start collecting emails through a landing page as soon as possible. That is way before the product is launched. There is even a website builder dedicated for landing page feedback specifically (Launchrok).
It's very fast to set up. And, it helps to raise a flag if your potential solution is valuable to anyone.
However, there are many pitfalls in this landing page and email gathering approach.
Email does not say about the people
You want to understand who your potential customers are. This information is particularly important in the early stage where you want to understand your potential customers. What do they look like and where do they hang out? This is impossible to tell from the email list.
You could ask them directly via emails. But you'd need to have a massive email list to collect sufficient responses. For example, imagine the response rate is 5%, and you'd want to get responses from 30 people. That would require you to build a listing of 600 subscribers.
If you get subscribers from 10% of all visitors to your site, that amounts to the web traffic of 6000. I don't know how confident you are in marketing your landing page, but starting a landing page with 0 SEO set up would certainly make it challenging. If you are more confident in one to one approach asking for quantitative feedback, survey might be your better friend.
Strength of Surveys
1. Lets You Easily Qualify Responses
You can ask in the beginning if they would be the right fit for your audience. For example, you are selling a car seat, and get feedback from potential users. You need to know if the respondents have a car first. Surprisingly, if you ask people directly for feedback like "what do you think of this idea", everyone would give you all kinds of answers whether they have cars or drive infrequently. Listening to the wrong type of users would be detrimental to your product development.
In fact, this is the hardest lesson I learned in my startup journey. I first asked my friends about my startup ideas. It's the reading reflection tool on laptop. They were kind enough to give me feedback. But you have to value reading online and regularly spend time reading to be remotely interested in the products. It can be quite discouraging to hear it is a niche product, which simply translates to they don't see themselves using it. What I should have done instead is first to ask what they do now before jumping into asking for feedback. In reality though, people don't like to be disqualified. And treating your friends with respect and valuing their opinions is just nice things to do. That makes qualifying your friends difficult verbally.
In survey, asking for the current status is very standard practice, and it's very simple to set aside the first 2–3 questions for preliminary questions to find the feedback from the most qualified audience. I included how much they spend time reading every day in the beginning.
2. Measure Market Size
You can Google for potential audience size. For example, if I'm targeting students, I'd Google "students in US". The result would give me something like 50 million. The problem with that approach is it's too macro to start narrowing down the audience. There are students who read and those who watch YouTube. There are students who are ambitious and those who are party goers. Market segmentation data available online is too generic for startups.
You want to instead measure the problem approach. Y Combinator always suggests that you want to focus on the problem. It's defined by a simple formula:
frequency * pains = your solution's potential impact
For me, I desperately wanted to know in a quantified way how many people are having the pain of wanting to remember more from reading?
I asked 2 questions:
- When is the last time you wished you remember more from reading?
- Do you do anything to remember more from reading?
The first one checks for the frequency, and the second is for checking the pain point. If they are already doing something for enhancement, that shows the intensity of the pain.
After collecting answers, I could conclude 37% of my sample audience are doing something already to improve their memories from reading. It is so sample but it builds confidence in your potential problems.
3. Build a Real Persona
Most often, you build a persona out of real users. But, in the very beginning, you can utilize surveys to build a better persona. Most people (who've never talked with the potential audience before building the product) start off with broad persona: engineers, small business owners, startup people, female in 20s, etc. That's usually how outsiders describe your potential audience too if they ever give you advice. However, like I mentioned before that's just demographic. You want to get into the behavioral as much as possible.
Since Kaffae is an extension, I wanted to ask how many students use laptop and when. It turned out people do use laptop throughout the day for almost everything. Also, even though my focus is on laptop reading, those who showed interest in reading were also book readers. Likewise, people who don't read books at all would not show interest in reading online either. So the book reader's persona fits well in Kaffae.
4. Find a Marketing Channel
Finding a marketing channel is crucial for any company. That's especially true if you are tight in the marketing budget. You need to know where to focus your promotion effort. To do that, the pro marketer Gary Vee suggests first finding where your potential users hang out and work backward to fit your strategy. Are they hanging out at Medium, Twitter, or Reddit?
One way to do that in survey is through asking what services they use now. If there is a specific social media your potential audience uses more often, you can increase your social presence there. You can possibly build an integration with them also.
Unfortunately in my case, I couldn't find any commonality in users. I speculated heavy readers who are tuned to gather information effectively would be using news aggregator apps like Flipboard or Feedly. But there was no such relations in the types of apps people used and those who read more often. That's a good finding. I shouldn't focus on a specific integration at this stage.
5. Outsources Great Copywriting
As I wrote in another post, writing a copywriting is really hard. It's hard to translate your biased world into something understandable and attractive to people who've just landed on your page. That's why you get your potential audience to write one out for you.
In a survey, you'd always want to leave an open-ended question where they can write any answer. Sometimes from those answers, you can find great ways to phrase your solution. One response I've got was,
"I like the notion of being able to connect articles that have been read"
Yes, connect! I never phrased it that way, but this person knew how to use English! I was so delighted and immediately put this phrase in my landing page.
Don't Use Survey to Validate Idea
It's important to understand the limitation of surveying. First it is not good at validating your potential solution. At the end of the survey, I asked would you use such and such solution. You can still get a useful response like that for the copywriting sake. But even if they all say yes, it does not mean they'd actually use it.
One entrepreneur from Indiehacker was sharing his frustration about how he's built a mobile app after collecting survey response. He's asked his followers whether they wanted the mobile version of his service. And almost everyone said yes. But once he's built it, no one ended up using it. That's one bitter lesson.
One way to combat is to raise the bar for your survey audience. Instead of simply clicking yes or no, you could ask for emails or have them type out when and what for you'd like to use it. If people really wanted it, they'd happily fill it out.
I learned this in my survey journey as well. A few months ago, I was testing out the value prop of article recommendations. Of 19 respondents, 63% had said yes they'd use it. On another survey, I asked about reflection, only 44% said yes. In the first glance, article recommendations sound more appealing to people. However, in both cases I added an email column if they'd like to be notified if the product is ready. I got 0 email on the article recommendation responses, while the other had more emails.
The lesson here is if you took 63% literally and build the feature, you'd almost fail, because they don't care about the solution too much in their lives to leave emails. It's not that people are lying. But the solution is not appealing enough to move them from the current daily habit. The real excitement and what sounds good in papers are 2 different phenomenons.
Where to Get Survey
The world of survey community operates in exchanging favors. You fill out others' and they do the same for you. This makes the process of collecting responses very predictable.
There are various survey groups on Facebook that focus on students and researchers. There are also point base services called SurveyCircle and SwapSurvey. They are both free to use. All you have to do is to answer others' surveys, and yours will appear more often in the thread. The more you fill out surveys and get points, your surveys surface higher in ranking.
If your audience is other entrepreneurs, you should go to Indiehackers. There is a page for Landing Page feedback. Give them good feedback (value provide first) and ask for them to fill out your survey. In my experience, they'd always do you a favor in return.
A survey is a powerful tool anyone can use. While landing page is useful, I find it requires specific marketing skills to start with. Survey can help guide your early product development journey.