Venture capitalists are all over the "Big Data(BD)" category investing heavily in a sector they feel is the next big tech sector that will grow quickly and consolidate into few big winners over the next two years.
The Internet gaming space is one of the verticals that ran up against the deficiencies of the traditional DB and BI(data analytics) providers early on requiring them to look for alternatives and to create their own solutions and to take risks with newly minted BD solution providers. Given the early adopter status of Internet gaming as it relates to BD we can speculate about products and services that BD vendors will have to provide based on the Internet gaming experience.
The challenges that BD will have to solve boils down to solving the standard challenges that have faced computing and especially data solution providers from the beginning of computing. Latency and storage are the two areas that all data solution providers must have an answer for.
Latency - Latency comes down to response for a game. In gaming it is very important for a game to keep up with a player's expected pace of a game. If it does not, players will become frustrated and in some cases the game will become inoperable because the game requires a certain cadence to be considered functional. In mulitplayer games high volumes of players trying to play or compete in a game can put a heavy strain on infrastructure if all parts of the infrastructure are not capable of handling high concurrent usage volumes. This means that "latency" caused by an portion of the game stack or the infrastructure supporting the game could be disastrous for a game. In social games that depend on their success in the first 48 hours of game launch, any issue that causes the game to be unappealing could sink the game.
Data Storage- Games that have high volumes of players or associated players can generate extremely large amounts of data in short periods of time because there are many transaction events that occur within a game. We use the term terabytes as a measure of data storage. However, we are getting to a point where terabytes squared or to the power of x could be a reality if games and especially social games, continue to grow in sophistication and popularity.
The latency and storage challenges associated with games set the gaming sector apart from other verticals such as banking operations, web applications and mobile apps that are not gaming related. This is not to say that applications in these categories are not also driving BD requirements. The "gamification" of conventional applications are beginning to transform applications into games. With this transformation comes consumer game play expectations and potentially bigger data storage requirements for other verticals..
BD is not a new challenge. Social game providers experienced the DB storage and retrieval issue very early after they began to launch games in Facebook. With the domination of Facebook as the leading social game platform the issue has become exacerbated because of the K or virality effect that can result in a very quick ramp of game usage and in short period of time. This means that a game publisher/developer has to have an out of the box answer to large data requirements.
NoSQL Data Depositories - The social game companies were early adopters of NoSQL data storage providers such as Cassandra, Membase, Hadoop, etc. because they experienced the "how do I handle all of this data problem" in the early days of game deployment in Facebook. NoSQL DB's as the name implies do not use a table structure to organize and store data. Instead the NoSQL data depositories use a key value combination to store and identify data. Because there is no "sophisticated" organization of the data and no immediate need to retrieve it or even report on it (we will get back to this point later) these DB's can handle transaction rates with less overhead and more efficient data storage relative to conventional data base products.
Analysis - Gaming companies and especially Internet gambling companies have historically conducted sophisticated data analysis on game play, player acquisition, player retention, etc. to maintain a competitive advantage and to keep their businesses growing. Cost of acquisition is extremely high for gambling companies requiring them to be especially careful about retaining existing players. In many cases the success of an online gambling site rests with only a few players that contribute high amounts of revenue. Finding and keeping those players is a very high priority which requires just in time data analysis mining a relatively large data, slicing and dicing the data in a number of different dimensions.
Social gaming companies have a different challenge. Social game companies have a low revenue per player number relative to Internet gambling or even e-commerce companies. This requires them to drive high numbers of players, which causes the data storage issues previously mentioned, and the need to fully understand how and if players are inviting and associating with other social gamers that are or could drive revenue. Social gamers do have their "wales" just like gambling companies. However, the ratio of total social game players to "high rollers" is significantly different then gambling companies. Social game publishers need data analysis to keep this all in balance.
Both Internet gambling companies and social game companies need almost instantaneous data analysis to react quickly to changing traffic patterns, take advantage of successful marketing programs, retain "high rollers" and to control player drop off.
Unfortunately the current crop of NoSQL data storage solutions does not easily allow someone to analyze that data leaving an entire depository of data off limits to BI. In a perfect work you would like to be able to combine NoSQL and SQL depositories in a BI roll up to give you the complete picture your business. There are BI vendors attempting to do this as this blog is being written.
The opportunity and the challenge for BD investors, infrastructure providers, DB vendors, DB analytic engines is to address data capacity and latency issues created by larger data storage requirements and data formats that do not fit conventional SQL structured data . More importantly, creating an analytic feedback loop that farms NoSQL and SQL data, satisfies standard marketing funnel analysis, game mechanics optimization and social interaction optimization needs to be in place to get real value out of BD. Getting this right will create the wealth effect investors, stockholders and entrepreneurs are looking for. If the BD vendors can solve the Internet gaming companies challenge they will most likely be successful in other verticals.
Kevin Flood is the CEO of Gameinlane, Inc. Kevin writes extensively about online games and their impact and integration into iGaming and E-commerce environments. Kevin is a frequent speaker at online game events and conferences in Asia, Europe and the US. Kevin and his Gameinlane team are currently working with online gambling, social gaming and e-commerce companies integrating social gaming with online gaming operations and integrate game mechanics into e-commerce applications.