What are Live Operations (Live-Ops)?
A Live-Op is a feature added to the game, over and above what is presented to players as part of the standard content, that presents extra challenges for the player, and at the same time offers rewards when specific progress milestones are achieved.
They are usually activated at specific times, for example, over a weekend or regional holiday period, so that players would see the content and engage with it only during those periods.
We periodically offer the player alternative gameplay experiences in addition to the usual game progression. A good example is the Peek-a-Boo Reward Live-Op while playing Bubble Witch 3 Saga.
The need for real-time tracking
Before Live-Ops are deployed, they are configured to be activated and visible on player devices on a specific date and time for a fixed duration. During these times, it is not only assumed that the Live-Op is delivered successfully that is, that they receive notifications, they can successfully trigger the Live-Op to begin a challenge, rewards appear when they are meant to and milestones can be progressively achieved.
In order to mitigate the uncertainty of this assumption and verify that things are well in the Live environment, we track various aspects of the lifetime of the Live-Op. Using Corcyra, a tool created specifically for this purpose, a game team can visualize key metrics.
From the chart depicted above, we see (as colour-coded chart legends) the specific metrics we are interested in. The graph moves from right to left, depending on the frequency of measurement.
The graph’s horizontal access represents a fixed trailing time-span (for example, 1 minute) from the current time. Therefore the right-most edges always represent the current time. Also handy, to the right of the main graph, is a differential cardiogram (or activity tracer). It depicts the rate at which the individual metric values are being received.
Below the time-span setting is the ability to control what metrics to display on the graph. By taking advantage of Corcyra, any King game can track Live-Ops activities in real-time. Corcyra can be configured to work with any deployment environment, such as QA or Dev.
A scheduler (browser-based) UI is used to configure individual Live-Op verifications. It is an interface to let Corcyra know that we need to track a Live-Op. The time between which it beings to stream graphical information and the time it stops is typically the same as the time in which the Live-Op was originally configured to operate under.
For example, here is a typical scenario of a valid configuration.
The “Stream start” depicts the time at which the Corcyra begins streaming metrics for the LiveOp while the “L start” depicts the time at which the Live-Op is meant to start. The reason for the difference might be that the game team wants to verify that the Live-Op was not accidentally configured to be visible before a certain time.
Similarly, the “L end” time represents when the Live-Op is meant to stop being available for all players of the game. The “Stream end” time is the time at which Corcyra is meant to stop streaming any more metric statistics for that Live-Op. The advantage in the time difference is that we can visually verify that, for example, no more Live-Op notifications are received for new players, past the “L end” data/time.
What happens when Corcyra is offline and a Live-Op is already active?
When Corcyra is brought online, and a visual depiction of the statistics is shown, the absolute values (number of players for each metric) will be incorrect (since we did not begin to track those statistics before, or even at the time when the Live-Op became operational).
Luckily, a Recovery module exists as part of the tool. Once Corcyra is online, it will automatically check for the date/time when it was last active; then, seeing that the time difference is significant. it will proceed to fetch all “missing” data for each metric; once complete, it will automatically update the main graph with the values. Therefore, when there are periods Corcyra is offline, it will not hinder the charts from showing the correct information, once Corcyra is back online.
Teams that take advantage of this tool can now be more confident in ensuring that issues surrounding Live-Op deployment will not hide true user experiences when the Live-Ops are actually activated.