I got in touch with Christopher Gibbs just before PGConnects London 2017. I looked for great games by indie game developers to feature in my talk Indie Analytics. After talking with Chris, who had just released Smart Numbers, I was intrigued how he worked closely with feedback from 100+ beta testers for 6 months – to make a game to be loved – and featured in all App Stores within just a couple of weeks.
Tell us a little bit about yourself, what is your background in gaming?
Well, I first became interested in video games when I saw a ZX80 for the first time. I was about 14. It was a black and white home computer and you programmed it using BASIC. I was transfixed by the way you could make shapes move across the screen, and how it could respond back to you when you typed things in. It felt like magic!
That sense of wonder continued and I quickly became a hobbyist programmer. With 4 friends I started a games studio, called Attention To Detail, and now, almost 30 years later, making games is all I’ve ever done. I still have that feeling of magic when I see games come alive.
So, with 15yrs as an Indy at ATD, and then another 10yrs as an Exec at EA, I guess I’m an industry old-timer now. Or I prefer to say, ‘veteran’. 🙂
When and why did you decide to make your own games full time?
I guess I’ve made that decision twice now, both for different reasons. The first was after graduating in 1988, with 4 friends/fellow students, who all shared a passion for seeing what we could do with computers. It was such an unexplored area back then. I dreamt of making a number one game/fame/fortune/all the normal motivations at that time in your life! And I really didnt want to go into banking/consultancy like everyone else on my course with a computing degree! I wanted to be a rock star 🙂
The second was more recent, in 2014. I had just finished a 1 yr sabbatical – a detox for the brain – and as I came through that I clearly felt the desire to make a game for myself – kind of like a personal labour of love. I wanted to just enjoy the creative process, with no corporate interference, no team politics, no milestones, no business plan. Just wake up every morning thinking ‘how can i make the game better today?’ and jump right into it – no meetings/presentations/conf calls – just creation.
And I had a simple philosophy… I only worked if I felt inspired – if I didnt I would go outside/see friends/play music… and inevitably the brain kept crunching in the background, and in a day or 2 inspiration would return with a new set of ideas…
So Smart Numbers is the most personal game I’ve ever made.
What projects are you currently working on?
Smart Numbers is my whole world right now. It’s in its infancy, so many new things to add and try, so much to learn from the players…
But I am seeing this more than a single game. I believe it could be the start of a family of games, (or Franchise), which all have fun with numbers, but in different ways. For example, Smart Numbers 4 Kids (tailored to young teens), Smart Numbers Infinity (endless play instead of levels), Smart Numbers Pro (More advanced math) and Smart Numbers Rivals (Turn-based head-to-head). I think its a pretty new area of gameplay to explore. Each game in the Smart Numbers franchise would be differentiated, complimentary, but united in delivering the promise of ‘playing smarter’ which I think a lot of people are keen to do these days.
How did the idea of Smart Numbers evolve?
It began in a small Hawaiian village called Pahoa. After breakfast in a local diner, my girlfriend turned to the back of the newspaper looking for a sudoku. Instead it just had a crossword. She hates crosswords! And she said ‘what would it look like if you crossed crosswords with sudoku?’ That really got me thinking!! We went back to our holiday villa, found some paper, pencil and a dice, and started playing a ‘paper prototype’. We simulated what was to become the core gameplay! As soon as we got back to Montreal, I coded up a quick demo, and found it surprisingly engaging. Just the act of swiping to add numbers felt ‘fun’ and you instinctively created objectives for yourself. I guess i had enough experience of making games to recognise that maybe something special was here…!
Evolution is a very accurate way of describing the process over the next 2 yrs! I would build several levels and the first tutorial, and give to friends without any instruction. And then watch their reaction. It took many many iterations to reach a point where people ´got it’ without any prompting.
A great source of unfiltered and honest feedback were Helene’s clients! She is a nail technician. Every day, women from 18 to 70 sit opposite her for an hour with one hand occupied and one free. On the desk was an ipad always running the latest prototype! Helene would tell me every evening what reaction her clients had. Huge learnings! Often they would call me as they were playing, and i could ask questions on how they were experiencing things. This was especially good in proving out and then fine-tuning the on-boarding.
This continued for months. It improved the game tons plus gave me early insights into how addictive it could be, and why. For instance, as soon as I added Grades (c to a+) instead of stars, people started refusing to go to the next level until they got a+. They cared about grades! Being told you got a B was more emotionally powerful than getting 2 out of 3 stars. So i designed the front end screen to showcase the grades front and centre.
After 6 months i ran a private beta using testflight, with around 50 people. At that time the levels were structured within the context of the US schooling system. You began in kindergarten, and worked thru grade school, high school, university, ivy league etc…. but feedback persuaded me to drop this ( system did not translate well for european players).
After 1 yr I did a public beta with 200 players. I engaged via email regulary during a 4 week period, and ran an online survey at the end. Tons of great data. Insights into level difficulty, need for variety, lack of clarity in places. I dropped interstitial ads and went to rewarded videos. I revisited tutorials. Changed graphic style. It was a big evolution. But the fundamentals were the same. The game was changing style/tone/clarity but it wasn’t changing its ‘shape’ or its ‘X’.
It soft launched in Oct, and continues to evolve – roughly one update every 3 weeks, and a longer gap now as perfect a major new update.
Who is Smart Numbers for?
Great question! I can tell you who I think its for, but I’m gonna learn over the coming months what the reality is! As i said earlier, I designed it for me first of all. I love casual puzzle games. I was intrigued by Threes! And 2048. And loved Two Dots. I wanted to have a challenge that used my brain more than most casual games – but still be a 2 minute fix. I figured there must be other people who wanted that too!
So I see the game is for anyone who enjoys numbers, see’s themselves as a bit smart, wants to train their brain a little. Maybe thats the sudoku crowd? I figure they must have played so many zillion sudoku’s by now, they are ready for something fresh!?
Interestingly, judging from player feedback i get, women between 30 and 60 seem to be over-indexing in the audience, and I also see a significant number a males, 20-30, who are competitve high-achiever puzzle gamers looking to prove their worth.
But its early days really to be concrete about the audience. One thing i am curious to learn is what is the youngest age of player that can handle this degree of math/logic…
How did you work with beta testers, and what are your techniques and tips for doing so?
I used the Apple TestFlight beta testing feature. I created a Facebook page and website for the game, and used these as platforms to recruit beta testers and keep the community of testers up-to-date on progress.
I was lucky that having been in gaming so long I had many friends/colleagues around the world who were gaming professionals. And so I built a group of 100 or so testers within a few days. But then I got requests to join from friends of friends, people i didnt know, and I ended up with about 250 bets testers that way!
Many people sent emails with their thoughts. Email seems quite an easy place for people to be ‘frank’! Much more so than in conversation! This was great as it gave me honest feedback.
So i encouraged this by making an email mailing list and regularly sent everyone my learnings to date/asked questions and requested preferences. Along with facebook posts, this generated a ton of back and forth dialogue, and I saw a wide variety of perspectives. It forced me to make real choices. For example, to drop interstitial ads. And to add a ReDo button for accidental adds/subtracts.
Finally i ran an online survey using Survey Monkey, focussed very much on learning what caused people to stop playing. Difficulty/lack of Variety/lack of sense of progress, I recall as being the key reasons. I spent a further 6 months in the end refining the game to respond to all the beta feedback!
How was it perceived in soft-launch and full-launch? Did it meet your expectations?
Soft-launch reaction, in terms of star rating, engagement and retention KPI’s, was amazing, beating my expectations for sure! 4 stars, 45 mins a day played, D1, D7, and D30 all at acceptable industry levels for a good game.
And people were spending! It is quite an exhilarating moment when you see your first IAP happen. I remember seeing in realtime in Unity Analytics the very first $0.99 being spent and thinking ´WOW – someone somewhere in Canada just bought 5 swipes’. It was the best dollar I’ve ever made!!
Plus Apple featured it in new board and new puzzle games in Canada, which was a wonderful piece of personal validation for me.
WW Launch reaction has been very good too, but a little more mixed. Star ratings have been astonishing – over 90% are all 5 or 4 star, with 1-stars being linked mainly to crashes. And the comments have really convinced me of the game’s potential. Engagement was also high as in soft launch. But retention, especially D7, was only half what i saw at soft-launch. Monetisation though has gone up nicely in terms of ARPDAU – definitely hitting expectations for this early stage.
You mentioned that retention was significally lower in full launch than soft-launch, why?
I do believe the golden cohort effect of friends/family made a significant difference. The data doesn’t much support the theory that its due to lack of localisation. Improving/reducing difficulty has also had a minimal effect on increasing retention – really surprised me that it wasn’t more. That is still a work-in-progress!
Maybe I just got lucky in soft launch – with only 30-40 installs a day the data was pretty spiky! So harder to be accurate with what it is telling you. So I have more advanced features coming up to address retention directly.
You mentioned that retention was significally lower in full launch than soft-launch, why?
It trended up a few percentage points as i tuned difficulty across levels 1-30. I could see in my Funnel which levels had most drop off, and it coincided with win/loss data – IE if early levels were lost too often, people gave up fast.
But I think because the improvement was minimal even after several rounds of making levels easier, there is something else going on.
Maybe one thing that you have suggested: the steps between levels might be too ‘high friction’ and need refining. Plus I see mini-tutorial steps at level 16 also causes drop-off. I believe the top games spend months and months in soft launch refining these kinds of things… the devil is in the details!
What will you focus on for the next three months?
– Daily Training. A 7-day calendar which increasingly rewards the player for doing training levels every day, on a 7 day cycle. Give free Diploma’s
– Reduced number of steps/friction between levels
– Social features for sharing/competing
– Rewards for getting more friends in the game
– Ad mediation platform, more ad providers
– Increasing opportunities for rewarded vids (right now only 15% players are exposes to them)
– Selling ‘training packs’ of levels, player controls difficulty, and chooses type of objective.