Creating the Game V: There Is Not Enough Testing

Testing part belongs to essentials of creating the software and applications at all, not only of games. And you won´t believe it (or rather will) but it takes enormous time. It is so because the purpose of testing does not cover only looking for bugs. Well performed testing part should improve aspects like understanding of controlling the game, user experience, difficulty and give us feedback related to marketing and business potential.

Selftesting

Let´s say we divide testing into few parts. First one is quite natural – we test on ourselves. Every colleague or good friend is kind a forced to try the game. Fortunately, they like it. We have to test in every stage of the game, even the prototype. This has an advantage – we can find and kill bugs in the very beginning and thanks this we do not waste time by programming something what just does not work.

By the way, fixing one bug can sometimes generate additional 2 so do not suppose that everything is over. Just test, test and test.

The problem with selftesting is that everybody in our team has to play the game many times. Too many times and sooner or later it will lead to allergic reactions on the game. More dangerous is fact that we fully know our product and we cannot depersonalize our perspective. We can miss something what is totally obvious for person who is not incorporated in development.

User Testing

Which bring us to second phase of testing – players. Or shall I be more business oriented and write “target groups”. I used the plural intentionally because it is quite normal targeting on various groups. That can be good in terms of potential market but brings more work to be done in testing phase while we must test the game on selected members of each target group. Maybe it sounds like banality but behavior of each group can be totally different. In other words you just cannot depend on game designer intuition.

informační architekturaWorld of Cheese for example. We tested the game with parents and some adults. Result was clear – some levels have to be redesigned and decreased in difficulty. How can small kid solve the problem in which his/her parent is lost? To our astonishment kids (our target group) passed the same levels without any problem fast and easily. On the contrary, tasks which were supposed to be really easygoing kept them in confusion. Except holes in our game design we could also see the differences between thinking of kids and adults. Adults sometimes see things just too complicated.

The form of testing itself can differ from case to case but there is necessity to follow some rules.

The most important is to explain testing persons that we are testing the game not them or their skills. Therefore you have to keep in mind to avoid any humiliation, even if you do not mean it. Especially shouts like “it is pretty obvious that you have to click on this bulb!” are not very comfortable for testers. Besides, if tester cannot see that bulb must be activated, it is definitely not obvious.

Except face to face testing we also use TestFlight service. Thanks this we can easily transfer testbuild to anyone who accepts our invitation. We use TestFlight mainly for balancing technical issues received in reports sent after crashing the game, for instance. Also, TestFlight is great tool to deliver game to reviewers in case you are out of promocodes (Apple allows up to 100 promocodes per build. You can generate them only for paid apps and games) or want to send a preview build to reviewers.

Closed Betatesting

We are about to release Defend Your Life! and few weeks ago we prepared kind of soft launch version for closed testing of web build. For gathering and interpreting data we used analytic tool GameAnalytics and our own questionnaire.

Tests were conducted on hundreds of players. We could see which levels are problematic, how far players can go, what type of buildings they like most etc. Thanks this we could optimize and balance difficulty of game as well as impact of buildings, units or bonus upgrades. Heat maps showed where were players clicking and if they do not miss some navigation buttons.

Purpose of questionnaire was simple – we wanted to know the general impression, feedback regarding to Defend Your Life! We asked players to answer series of questions, both – closed and opened as well. Do not forget you cannot ask everything. Trying to find out something about every detail in the game can be tempting but it is first of all time consuming and secondly it can be senseless. And last but not least, it can scare the player and leads to simple closing of the form.

Testing Before Releasing

We mentioned it in previous article – we use various kinds of devices, we are focusing mainly on the most used hardware from relevant platforms (recently iOS, Android and Windows Phone). We test the functionality of hardware features and if the screen resolution is okay.

Before clicking on “Release” button we do last checks of analytic codes, localizations, in-apps, review and social media features, etc. And trust me there is no only one last check.

We would like to also mention that testing can be made automatically. We use it mainly in cases of changing the code. So we just define sequence of tasks (as similar to human behavior as possible) and wait for result. This can spare pretty much time.

Testing After Releasing

“This is the end…” or not. We are still tracking what is going on in game, if and when does game crush and so on.

Except this we have to keep eyes opened on marketing part – we can simply find out what devices users use, how often players play, how long, reviews and if we have some advertisement system implemented how does it go (CTR, ARPU…).

You may get a feeling that games like Save the Snail or World of Cheese could be pretty easy to testing. We thought the same but we were wrong. No time for testing is wasted!

To Be Continued

Next time the game will be online. We will talk about promotion and marketing. Cheers!

Alda Games, 2.10.2014 | Brno, Czech Republic