But Is It Research

I came across an interesting article about World of Warcraft a few days ago. Blizzard CEO Mark Morhaime revealed, in an interview, that only about 30% of players in World of WarCraft actually go past the level 10 mark in the game. In fact, directly quoting him from the article:

"Our research shows that trial players who play World Of Warcraft past level 10 are much more likely to stick with the game for a long time. Currently, only about 30% of our trial players make it past this threshold. So anything we can do to improve the new player experience is a huge opportunity for us," he said.

Since this blog is about research and that claim is about research, I thought it might be worthy of comment. That is an interesting statistic and I don't doubt that it is true. But is it research? And what can you do with it?

First, on the question of "is it research", I would say no. It is not research. Earlier in this blog I referred to a concept introduced by Francis Bacon called Idols of the Marketplace. It is the common place usage of terms that have more precise definitions in the advancement of knowledge. Research is one of those terms. In common place usage, we might say, "I did a lot of research before I bought my car." In common place meaning that is perfectly acceptable. We understand it to mean that we gathered information before making a decision. However, in the world of research this term has a very different meaning. Research must advance knowledge. Typically, this means a well defined claim, based upon well defined concepts where the claim is either supported or refuted advancing our understanding of the concepts involved.

Since this is not research, it is unclear what should be done, if anything, about the derived number.

What Blizzard has done is some data analysis which yielded a parameter the meaning of which is entirely unclear. The quote suggests that the problem can be addressed by improving new player experience. But is that the case? Is there really a problem? Let's say the 70% who don't make it to level ten are regular players who download the trial version to amuse themselves while they are in a lab, or when their regular server is down, or for a class, or a demo. If that is the case, nothing needs to be done. What if the 70% include people who use trial version to get used to the game. So, they set up a trial account, use it for ten days, set up a new one and so on. Let's say, for the sake of argument, that each player does this three times before achieving a level 10 and converting to a regular account. If this is the case then the trial period should be extended. Finally, let's say that the 70% are people who play some other game and are trying WoW just so they can say they tried it. If this is the case then nothing should be done.

The point here is that we think the 30% parameter is meaningful information and it is not. We don't know what it means and we don't know what was measured. Hence, we really do not know what to do. If Blizzard were worried about retention, ease of use or some other factor then it would be possible to set up a research study to get to the bottom of it. But as it is, they don't have research. They only have a number, the meaning of which is entirely unclear.

Video Games as Social Interaction Technology

I came across an article about a Professor in Indiana using World of Warcraft as a means of communicating with his students.I found this interesting from two perspectives. First, thinking of a video game as a social interaction technology has potentially profound implications.

As we spend more an more time in virtual worlds and video games, people know where they can find us. Certainly, if someone carries a cell phone you always know where to find them as well. But there is a cultural difference. A cell phone call is an interruption where as an IM in a virtual world is not. When you call someone, you have no idea what you might be interrupting. However, if you find someone in a virtual world, they are most likely just hanging out. They may be building something, or writing a script, or slaying a monster, or talking to friends. But the culture of virtual worlds has evolved very differently from the culture of cell phones. And it seem to facilitate informal communications.

This is significant because an earlier technology, email, had a tendency to flatten out organizations. Whereas before you might have to make an appointment to see someone a level or two above you, with email you could just shoot them a note. Video games provide a common gaming experience that may flatten things out even more.

On the other hand, it further dilutes the meaning of "video game" and makes the possibility of studying video games in their entirety even more remote.

Conceptual Bleeding

I started this blog attempting to provide some philosophical grounding for video game research. I discussed concept analysis and how it applied to the concept of games. I explored the notion of play and how it is one of the few, if not the only, activity that we pursue for its own sake. I took that a step further and introduced the concept of flow and explained how video games are a quintessential flow activity. So far so good. But then my focus began to drift. This is not a lament, only an observation. You can't ignore all the other things that are going on. At the same time you cannot be too distracted by them. Video games can be seen as art. They can be used for education. The can provide channels for advertising. They can provide critical commentary. In fact, video games are useful for a whole lot of things.

I mention this because it is a good example of the kind of conceptual bleeding that plagues researchers. We begin by trying to develop a crisp essential concept of the phenomenon that we are trying to study. Over time that phenomenon begins to bleed into other areas. If we stick to our essential definition then, over time, we begin to appear irrelevant. If we expand the scope of our phenomenon to include new uses or applications the concept begins to appear somewhat fuzzy over time which ultimately threatens our rigor.

As I said, this is not a lament. Only an observation. It is a good example of a conceptual problem that many researchers encounter. So, what am I going to do about it? Well, probably nothing. I intend to keep looking at new uses of video games and I intend to keep trying to nail down the essential concepts. And I intend to keep reflecting on what I am doing. I am probably not going to write the book on video games nor am I going to have the last word. This is a blog and the best I can hope for is to make a few interesting observations along the way.