Public Radio Listener Survey Question Construction
Question construction needs to be thoughtful and tested. Even if you think you have a perfect question, designed to get the information you want, ask staff and volunteers for their response. It’s measure twice; cut once. I can’t tell you how many times I thought I had a great question that turned out to be a dud because I had one unclear word in it.
So what is it you want to know?
Do you really know what’s important to your audience? Answering that question is the over-all goal of any survey. Listeners don’t always make the distinction between your station’s programming and national programming. You should always construct your questions to focus your listeners minds on specific programming elements if that is important you.
Let’s go through a series of questions to get at what community issues are important to your respondents. This is a basic question sequence that can be used to ask about any part of your station including fundraising and news. We’ll cover the sequence using news focused questions. Remember too that this is a general sequence and it might take one, two, three or more questions on a topic to get the information you’re looking to acquire.
Question 1: Unaided Recall: What Three Issues facing our local community do you spend time thinking about? The order is not important, just that you spend time thinking about them.
The question should have three text boxes to capture the information. Notice the words “local community” are used. Local differentiates from national or state-wide issues and “community” so they focus
on their definition of local, not yours. Asking someone about issues affecting Baltimore is going to get you different data from respondents who live 30 miles outside the city. Once we get their home zip code later on in the survey then we can compare and contrast what different parts of our listening area.
Question 2: Prioritization: On the left are list of issues many would say are facing our local community. Drag each one to the box on the right and order them by dragging them up and down to put them in order
of you think are the most important to you to the least important.
You can take a few approaches to determine the list of issues for the left side. I advise that focus on topics from the beats that your station currently covers. This, combined with question 1, will give you an idea just how well your editorial is matching up with what your respondents want to know.
The topics on the right should be two or three words that can be “semi-specific.” “Transportation” has too wide a meaning. “Transportation Infrastructure” narrows it down some and might more in aligned with your content. “Public Transportation Infrastructure” makes it even more specific. I would choose “Transportation Infrastructure” as it would have similar meanings to drivers and public transportation riders. You may need to add a “how do you get to work question” later on the in the survey to determine if your listeners are more interested in road construction issues versus subway delays.
This question type will return a score and ranking from all of your respondents giving you a nice list of their priorities that you might use to guide the amount coverage you give each topic. You should also include a free form text box for comments. “Is there anything you’d like us to know about these topics? It could be something going on in your community or something about how we covered one of these topics.”
Question 3: Performance: This is the question that gets to the heart of the matter. Here we ask respondents for their opinion about how well your station fills their needs. I like to ask this one in a table of statements and use a 1-5 strongly disagree to strongly agree scale. Generally your listeners are going to give you high marks for your coverage but we’ll talk about that in a future post about analyzing your data but generally, any statement that averages less than 3.75 should be a topic of conversation in the station.
“Below are a series of statements about WXXX. Please tell us how much you agree with each.”
“WXXX covers the kind of stories I need to know about”
“WXXX reporters work to tell stories without injecting their own opinions”
“WXXX covers stories that no other TV or radio station covers”
“WXXX stories have facts and figures that can’t be disputed”
“WXXX stories are clear and easy to understand”
You can see that you can offer soft or hard statements about your coverage. I would also include a comments box here for feedback. “Do you have any insightful comments about our coverage that you think would be helpful in the future?”
I’ll demonstrate how this sequence can be constructed to shed light on your fundraising program in the next post.
So now that you’ve answered some questions about WHY you want to conduct a public radio listener survey, it’s time to pick the right tool.
I want to go back to a line David Giovannoni use chant during the Audience 98 study. “Free is not a value.” There are plenty of free survey tools on the web but they often will not provide you with enough options to fully collect and analyze your data.
My Personal Favorite
I’ve used Survey Gizmo for more than 10 years and it keeps getting better. It has two versions. The Business version of an industrial strength version used by Fedex and others. The Consumer or small business version is perfect for most station who want to accomplish a listener survey, listener polls and online forms. I even used Survey Gizmo for job applicants to fill out and upload their resumes. It made the EEO reports a breeze at the end of the year.
You can experiment with a free account and the decide which options (mostly more complex question types) you want to add. I like the $85/month level since it gives the most question types. The $25/month version has enough features for first time survey construction.
Survey Monkey Beyond The Free
A lot of folks liked Survey Monkey because it was free. But everyone knew you were using Survey Monkey so it really didn’t work for listener surveys where you wanted to present a professional image.
The paid plans offer a similar, but not identical, number of features as Survey Gizmo. You’ll need to compare the product feature tables on their websites to determine which features you really have to have.
I find Survey Gizmo is a little less expensive and has more question types than Survey Monkey. But Survey Monkey includes quizzes in their mid-priced product and Survey Gizmo doesn’t. Quizzes could be a really cool tool for station promotion.
Aside from collecting basic information Google Forms isn’t going to supplant a full service survey product. You don’t have much control over the look and feel of the form so everyone knows you’re using Google Forms. Reporting is super basic but if you’ve never done a survey before and just want to tool around for an afternoon then Google Forms could be your note pad.
There are bunch of other survey programs out there and you should do a google search and play around with them to determine which one is best suited to you—but I think you’ll come back to Survey Gizmo.
Microstrategy offers a free desktop client for their extremely expensive system that allows you to do some basic analysis. It allows you to do some nifty stuff. For example, I use it to come with the Public Triple Play list. I copy in each station’s play list and then matches up all the song names from each station and totals the plays from all of the stations.
It’s a great tool that you can play around with on a lonely Friday night with a bottle of wine.
Don’t underestimate the power of Excel. Most of us don’t learn the advanced features, but we really should.
Here’s a list of other software products you might want to investigate.
In the next installment we’ll tackle the fun stuff: Question Construction.
Don’t call them Public Radio Listener Surveys
Every once in a while you’ll see a post on one of our list servs or Facebook pages asking for examples or help in formulating public radio listener surveys. I’ll often offer advice based the surveys we did at my station which I think provided us with helpful information as well as the opportunity to cultivate non or expired members.
These really aren’t ‘Listener’ surveys. If you do them right, they’ll tell you more about your market and why non listeners don’t listen and why non-member listeners don’t contribute. They’ll also tell you what issues are important to your community so you can adjust your content to better serve them.
The first question you have to yourself is why you want to do a survey. After talking to several folks who asked for advice; many folks are trying to prove their position in an internal argument or they don’t believe Neilson ratings or other research products. Obviously these are not good reasons to conduct a survey.
The best reasons to conduct a survey is to add information to the other research your station has at its disposal. Maybe you’re not sure that your editorial team is covering all the issues your listeners are concerned about. Maybe you’re wondering why more listeners aren’t becoming members. Maybe you’re wondering why attendance at your events are lackluster. You may also want to try to quantify the financial impact of your events on your community.
There’s a lot you can learn. There is one caveat. The survey you’re going to do is not going to scientifically (unless you’re going to hire a research firm at a big price tag) sound. The information is used to inform your judgement and maybe even be a good sales tool for your underwriting staff, but it should not be used exclusively to make changes in your station’s focus, schedule or other decision.
A Few Don’ts for Public Radio Listener Surveys
Don’t ask “When Do You Listen?” questions. Your Nielsen ratings will tell you that. Of course if you’re in an unmeasured market then you might consider asking this question.
Don’t ask “What other stations do you listen to?” Same reason, you have that information already. We’re not looking for behaviors–we’re looking for the why behind the behaviors.
Don’t ask for feedback about specific reporters or hosts. Under NO circumstances should you share any comments about personnel with staff. It will cause an HR nightmare not matter if the feedback is positive or negative. Been There… Done That…. Got The Scars.
Don’t ask which malls or shopping centers they visit. You have this information in Tapscan or Media Audit. If your station doesn’t subscribe, use the effort your own survey would take to convince your GM to buy one of these products.
A Few Dos for Public Radio Listener Surveys
Do ask questions about the important issues facing them personally. Don’t be surprised you’ve overlooked an important issue in your editorial planning.
Do ask them about their satisfaction level with your station along with a space for feedback. You’re going to get very high ranks here since your pool of responses is filled with listeners who love you. The feedback will be helpful to learn what you’re doing well. You’ll need a way to quantify the intensity of the comments.
You might ask them about your events and how much and where they spend money before and after the event to demonstrate the financial impact your station brings to an event.
In the next post we’ll tackle the tools you’ll need to accomplish Public Radio Listener Surveys and then we’ll tackle question construction.