Brad:
For Letter Tycoon I’ve been recording the play length, scores, and which letter tiles were bought by each player. Comparing the scores and letter tile choices helps me see if any of the letter tile powers are over/under powered. Early on I was tracking how often players opted to discard and redraw instead of playing a word. I assumed that discarding might indicate a non-optimal letter distribution but a couple of players would discard strategically to try and build bigger words so gathering that statistic didn’t prove useful.
Peter:
For What the Food?! I collected a bunch of chicken scratch on legal pads per session, which eventually formalized into name, length, date and final scores at the very least. I wrote a bunch of notes and feedback – whatever players shouted out, to parse after.
The names were key for play test credits, but it also showed me who had played multiple times, which I then used to target certain players and ask questions on whether or not the game became more or less fun. In the final stages, I formed a core group of 6 super fans who played every week and I was tracking individual character wins and some notes about advanced powers to finish balancing. I also discovered that I was able to keep better notes if I was not playing, although that’s not always possible.
Professor What!?!?
- Date of birth
- Mother’s maiden name
- Social security number
…and any financial institutions that they have accounts with.
Mark:
I don’t have a form or standard set of questions. Most of the information I “collect” is my observations about player behavior. I look for what people do, what they don’t do, and how they discover the gameplay. I generally open discussion up to voluntary comments after a game, and if there was something specific I was testing that session, I’ll ask the players how they felt about it then.
Christina:
We should probably be better about formally recording this kind of stuff, but it’s mostly been just taking a note of general impressions and balance issues to think about. When I remember, I’ll take a picture of the group playing and reference it later for a timestamp and how many people/who was playing, but it’s very informal.
Luke:
I usually like to just let people talk freely. Though I have used forms, I think forms they often get in the way of organic expression (though they’re good at getting people to say things they were afraid to say!).
As I listen, I ask follow up questions to try to dig for the root feelings about the game and various aspects of it. Knowing how it felt to play the game, knowing what kinds of frustrations and what kinds of excitement they experienced is vital, and doesn’t always make it on to a feedback form. Ultimately, I’m building my games for other people to enjoy, and I want to try to understand the how they felt about the experience of playing my game.
Kelsey:
Lots of notes on lots of paper! It always starts with a summary of what rules that particular playtest is employing (since they change so often while in development). Then we’ll record number of players, play time, scores and as many details of how the game played out as possible. During and after the game is done, we love to allow people to freely talk about the game, give impressions, even make suggestions. We love dialogue to get to the root of why something “wasn’t fun.”
We write down every question, every perceived problem, every suggestion – for us to go back to later and discuss as a design team. We steal the “Yes, and” principle from improv acting and say, “no suggestion is a bad suggestion. Yes, I see what you’re saying, and I will consider it!” The questions often expose holes in the rules. The actual bad suggestions work themselves out eventually, but allowing people the freedom to say anything without rebuke opens them up to more in-depth and creative thinking – and that’s when the suggestions become genius innovations.
Norv:
If I’m play testing with a non-designer group, such as at a convention here’s the form I used at Strategicon GameX which worked pretty well for me:
Chris:
What Mark said!
—
And there you have it. The biggest suggestion seems to be open discussion, followed closely by score keeping of course, length of session and finally social security numbers/maiden name, if applicable. And snap a picture if you can – you never know when it might be useful in a blog post.
And now YOUR turn. What do you collect?
The following two tabs change content below.
Brad Brooks
iOS Developer
Brad designed Letter Tycoon (a word game for capitalists) which won the 2015 Mensa Select award, and the upcoming Rise of Tribes. He is currently in need of a time machine to address his idea vs. execution imbalance.
4 Readers Commented
Join discussion“Was length of game play good?” would probably give more meaningful data if the options were something like “much too short”, “a bit too short”, “about right”, “a bit too long” and “much too long”. Otherwise it looks a lot like the Qs I ask.
Thanks for the interesting article! It’s neat to see each person’s personality come out in their answers.
I actually do a combination of what Mark and Kelsey said. The player interactions are very important to me. So, I track how people are playing, having fun, etc. On the other hand, I’ll document what new or conditional rules are in place so that I can track how they affect the game and player behavior. Even though I am one of those silly engineer, number types, I’ll usually only track final scores if I already think there may be a balance issue. In that case I’ll actually record point gathering so I can look for above average spikes. It doesn’t usually concern me because regular play testing will pretty quickly reveal over-powered processes, combinations, or strategies (especially if you encourage testers to break your game every chance they get).
I record the date, how many players and their names. As the game goes on, I note trouble areas and ideas that come up as play goes on. Later, I divide the notes into changes I will make immediately (in any) and ideas that may or may not make the cut later.
I’m definitely going to start taking down play time, how players achieve victory and how players interact with each other.
Great article. Thank you.
The type of feedback you want and need is so dependent on where you are in your development of the game and who your play-testers are.
I feel like you need to “cast” your testers correctly. Start with other designers you respect earlier in your design process and then move toward less savvy testers as you become more confident of your design and shift your focus toward rules comprehension, balance, and new player experience.