Game Analytics - Big Data And Business Intelligence(BI)

Games generate more data then an average application because of the game state machine. Terabytes  of data can be accumulated in a short period of  time in a gaming environment. Essentially, a player can enter a game and cycle through a series of scenarios for a prolonged period resulting in the opportunity to generate considerably more data about a player and  game play characteristics than a standard waterfall application.  This unique  characteristic of games  results in an opportunity to capture more data throughout the game cycle and to potentially add this  data  to a depository  for the purpose of business intelligence(BI).  Ironically, this perceived gold mine for project managers, financial analysis, game designers and business managers creates its own set of challenges.  Be careful what you wish for!

Data Capture, Format Of Data And Player Experience - There is price to pay for adding data capture for the full cycle of a game. The data itself has to be identified in a way to make sure that each data point is uniquely identified and can be used to generate BI on a game, its players, traffic sources, revenue, etc. 

The data capture has be executed in a way that does not impact game play. The speed of game play, transitions in a game and messaging in the game have a significant influence over the popularity of a game. Data capture has to avoid any impact on these game mechanics.

The  game data has to be in a form that allows for the data to be retrieved and evaluated in the context of the information requirements of different stakeholders in an organization. The challenge is to maintain a balance between the objectives of a player and the objectives of the organization's desire to learn as much from game play as it can. 

There has to be a place to put the data and a way to handle the spikes that can occur in game data generation. Social game developers and platform providers were some of the first adopters of the NoSQL model because they could quickly store game data in a simplified format and at the same time capture data about game play. The problem with this approach is that often the format of that data and the storage  depository makes it very difficult to retrieve and make any sense out of this data. Things are changing with more emphasis on the ultimate goal of using the data for BI.  There are companies attempting to create a bridge between NoSQL and SQL reporting databases.

Retrieving Data, Reporting And Business Intelligence - Business intelligence has  emerged as a very important aspect of operating a successful game company.  Internet gambling, MMOG and RPG game providers were early adopters of traditional business intelligence tools to help them understand cost of acquisition, player retention, revenue per player,  game mechanics factors,  identification of important players in the game community, etc.   Social game companies have followed adding NoSQL strategies to cope with high data volume. The social game companies have pushed the envelope or data capture and reporting due to the potential for social games to grow large communities of gamers quickly and for these  communities to shrink quickly. NoSQL products are evolving based on the experience gained from the early adoption of NoSQL by social game companies.

A number of traditional BI companies, SQL and NoSQL providers and newly minted data reporting platform providers  focused on the social and casual gaming sector have entered the market to address the specific data and reporting needs of the gaming sector. This is an indication that new innovations and tools will become available for game companies. It also indicates that were are not quite there yet. Very often unique BI, analytics, data capture and retrieval techniques are being created by the game companies themselves because no ultimate data capture and reporting tool kit is available.

In summary, the game data and intelligence world is evolving quickly. Competition within gaming sectors and the cross over of players between different gaming sectors  is  requiring any company desiring to be competitive in the gaming market to have efficient data capture, report  centered data stores and a full menu of key performance indicators and adhoc reporting to adjust to ever changing situations and to remain competitive in the marketplace. 

Game Analytics Blogs

Game Analytics Series

Game Analytics Overview

Game Analytics - The Classic Funnel And The State Machine

Game Analytics - The Social Connection

Game Analytics - Big Data And Business Intelligence(BI)

Kevin Flood is the CEO of Gameinlane, Inc. Kevin writes extensively about online games and their impact and integration into iGaming and E-commerce environments. Kevin is a frequent speaker at online game events and conferences in Asia, Europe and the US. Kevin and his Gameinlane team are currently working with online gambling, social gaming and e-commerce companies integrating social gaming with online gaming operations and integrate game mechanics into e-commerce applications.  

   

   

Internet Games Driving "Big Data" Requirements

Venture capitalists are all over the "Big Data(BD)" category investing heavily in a sector they feel is the next big tech sector that will grow quickly and consolidate into few big winners over the next two years.

The Internet gaming space is one of the verticals that ran up against the deficiencies of the traditional DB and BI(data analytics) providers early on requiring them to look for alternatives and to create their own solutions and to take risks with newly minted BD solution providers.  Given the early adopter status of Internet gaming as it relates to BD we can speculate about products and services that BD vendors will have to provide based on the Internet gaming experience.

The challenges that BD will have to solve boils down to solving the standard challenges that have faced computing and especially data solution providers from the beginning of computing.  Latency and storage are the two areas that all data solution providers  must have an answer for. 

Latency - Latency comes down to response for a game. In gaming it is very important for a game to keep up with a player's expected pace of a game. If it does not, players will become frustrated and in some cases the game will become inoperable because the game requires a certain cadence to be considered functional. In mulitplayer games high volumes of players trying to play or compete in a game can put a heavy strain on infrastructure if all parts of the infrastructure are not capable of handling high concurrent usage volumes. This means that "latency" caused by an portion of the game stack or the  infrastructure supporting the game could be disastrous for a game. In social games that depend on their success in the first 48 hours of game launch, any issue that causes the game to be unappealing could sink the game. 

Data Storage- Games that have high volumes of players or associated players can generate extremely large amounts of data in short periods of time because there are many transaction events that occur within a game. We use the term terabytes as a measure of data storage. However, we are getting to a point where terabytes  squared or to the power of x could be a reality if games and especially social games, continue to grow in sophistication and popularity.

The latency and storage challenges associated with games set the gaming sector apart from other verticals such as banking operations, web applications and mobile apps that are not gaming related. This is not to say that applications in these categories are not also driving  BD  requirements. The "gamification" of conventional applications are beginning to transform applications into games. With this transformation comes consumer game play expectations and potentially bigger data storage requirements for other verticals..

BD is not a new challenge. Social game providers experienced the DB storage and retrieval issue very early  after they began to launch games in Facebook. With the domination of Facebook as the leading social game platform the issue has become exacerbated because of the K  or virality effect that can result in a very quick ramp of game usage and in short period of time. This means that a game publisher/developer has to have an out of the box answer to large data requirements.   

NoSQL Data Depositories - The social game companies were  early adopters of NoSQL data storage  providers such as Cassandra, Membase, Hadoop, etc. because they experienced the "how do I handle all of this data problem" in the early days of game deployment in Facebook.  NoSQL DB's as the name implies do not use a table structure to organize and store data.  Instead the NoSQL data depositories use a key value combination to store and identify data. Because there is no "sophisticated" organization of the data and no immediate need to retrieve it or even report on it (we will get back to this point later) these DB's can handle transaction rates with less overhead and more efficient data storage relative to conventional data base products. 

Analysis - Gaming companies and especially Internet gambling companies have historically conducted sophisticated data analysis on game play, player acquisition, player retention, etc. to maintain a competitive advantage and to keep their businesses  growing. Cost of acquisition is extremely high for gambling companies requiring them to be especially careful about retaining existing players. In many cases the success of an online gambling site rests with only a few players that contribute high amounts of revenue. Finding and keeping those players is a very high priority which requires just in time data analysis mining a relatively large data, slicing and dicing the data in a number of different dimensions.

Social gaming companies have a different challenge. Social game companies have a  low revenue per player number relative to Internet gambling or even e-commerce companies. This requires them to drive high numbers of players, which causes the data storage issues previously mentioned, and the need to fully understand how and if players are inviting and associating with other social gamers that are or could drive revenue. Social gamers do have their "wales" just like gambling companies. However, the ratio of total social game players to "high rollers" is significantly different then gambling companies. Social game publishers need data analysis to keep this all in balance.

Both Internet gambling companies and social game companies need almost instantaneous data analysis to react quickly to changing traffic patterns, take advantage of successful marketing programs, retain "high rollers" and to control player drop off.

Unfortunately the current crop of NoSQL data storage solutions does not easily allow someone to analyze that data leaving an entire depository of data off limits to BI.  In a perfect work you would like to be able to combine NoSQL and SQL depositories in a BI roll up to give you the complete picture your business. There are BI vendors attempting to do this as this blog is being written. 

The opportunity and the challenge for BD investors, infrastructure providers, DB vendors, DB analytic engines is to address data capacity and latency issues created by larger data storage requirements and data formats that do not fit conventional SQL structured data . More importantly, creating an analytic feedback loop that farms NoSQL and SQL data,  satisfies standard marketing funnel analysis, game mechanics optimization and social interaction optimization needs to be in place to get real value out of BD. Getting this right will create the wealth effect investors, stockholders and entrepreneurs are looking for. If the BD vendors can solve the Internet gaming companies challenge they will most likely be successful in other verticals.

 Kevin Flood is the CEO of Gameinlane, Inc. Kevin writes extensively about online games and their impact and integration into iGaming and E-commerce environments. Kevin is a frequent speaker at online game events and conferences in Asia, Europe and the US. Kevin and his Gameinlane team are currently working with online gambling, social gaming and e-commerce companies integrating social gaming with online gaming operations and integrate game mechanics into e-commerce applications.