world team championships 2014 part 3

World Team Championships 2015 is nearly upon us, so high time to complete the analysis for 2014!

banner-website

Previously using the PlayerRatings package we were able to get some player scores from a relatively small number of games. We also got a warcaster/warlock rating, but this did not take into account rock/paper/scissors relationships. This is going to get a little bit less rigorous, since we’re estimating a large number of parameters; one for each warcaster/warlock pairing. There are some tools I have added to GitHub as WTCTools to save posting too much code here!

> library(devtools)
> install_github("CSJCampbell/WTCTools")
> library(WTCTools)
> data(wtc2014)
> head(wtc2014, n = 3)
  game_id round TP victory_condition           player1
1       1     1  1          Scenario Anthony Ferraiolo
2       2     1  1          Scenario       Brian White
3       3     1  1       Caster Kill       Will Pagani
           team1     list1               faction1 CP1 AP1
1 Team USA Stars Deneghra2                   Cryx   5   9
2 Team USA Stars Reclaimer Protectorate of Menoth   5  40
3 Team USA Stars    Kromac         Circle Orboros   2  50
            player2               team2  list2
1       Petr Cermak Team Czech Republic Saeryn
2   Josef Skládanka Team Czech Republic  Siege
3 Daniel Bogdanoski Team Czech Republic  Irusk
              faction2 CP2 AP2
1 Legion of Everblight   0   0
2               Cygnar   2  16
3               Khador   0  20

This dataset has been slightly reshaped from the previous analysis where each dataset was represented twice for player 1 and player 2. In this dataset, each game is won by player 1.

Previously we used the PlayerRatings package to estimate player and warcaster rankings.

> rating <- steph(x = wtc2014[, 
+     c("round", "player1", "player2", "TP")])
> head(rating$ratings)
             Player   Rating Deviation Games Win Draw Loss Lag
1     Jake VanMeter 2676.402  157.0390     6   6    0    0   0
2       Brian White 2667.702  159.1511     6   6    0    0   0
3        Ben Leeper 2592.218  158.1336     6   6    0    0   0
4 Anthony Ferraiolo 2590.775  155.9631     6   5    0    1   0
5        Colin Hill 2587.563  165.7409     6   6    0    0   0
6       Will Pagani 2563.477  161.2261     6   5    0    1   0

Player ratings are often based on the result of a match. However, control points (CP) for scenario scoring and army points (AP) for attrition advantage, is also available. To increase the information available for scoring the strength of each player we could create a new variable based on these scores. To get the scenario effectiveness we could estimate this based on the control points scored.

> # ratio of score
> wtc2014$scorefracCP <- 
+     wtc2014$CP1 / (wtc2014$CP1 + wtc2014$CP2)
> # if player 2 scored 0
> wtc2014$scorefracCP[wtc2014$CP2 == 0] <- 
+    0.5 + wtc2014$CP1[wtc2014$CP2 == 0] / (5 * 2)
> # if player 1 scored 0
> wtc2014$scorefracCP[wtc2014$CP1 == 0] <- 
+    0.5 - wtc2014$CP2[wtc2014$CP1 == 0] / (5 * 2)
> quantile(wtc2014$scorefracCP)
       0%       25%       50%       75%      100% 
0.0000000 0.5000000 0.6000000 0.8333333 1.000000

The army points scored take a little bit of tweaking since the number of army points available is variable by list.

> quantile(wtc2014$AP1)
   0%  25%  50%  75% 100% 
    0   24   36   48   71
> quantile(wtc2014$AP2)
   0%  25%  50%  75% 100% 
    0    8   16   26   58

After a bit of trial and error, a flat value of 61 points gave a good range of fractional army point scores.

> # fractional army points (approximately)
> wtc2014$AP1 <- wtc2014$AP1 / 61
> wtc2014$AP2 <- wtc2014$AP2 / 61
> # ratio of score
> wtc2014$scorefracAP <- wtc2014$AP1 / 
+     (wtc2014$AP1 + wtc2014$AP2)
> # if player 2 scored 0
> wtc2014$scorefracAP[wtc2014$AP2 == 0] <- 
+     0.5 + wtc2014$AP1[wtc2014$AP2 == 0] / 2
> # if player 1 scored 0
> wtc2014$scorefracAP[wtc2014$AP1 == 0] <- 
+     0.5 - wtc2014$AP2[wtc2014$AP1 == 0] / 2
> quantile(wtc2014$scorefracAP)
        0%        25%        50%        75%       100% 
0.03846154 0.56802826 0.67962264 0.78260870 1.0000000

The army and control points are skewed above 0.5 since the data is structured that player 1 won.

> library(ggplot2)
> ggplot() + 
+     geom_histogram(data = wtc2014, 
+         aes(x = scorefracCP), binwidth = 1/12) + 
+     theme_economist()
> ggplot() + 
+     geom_histogram(data = wtc2014, 
+         aes(x = scorefracAP)) + 
+     theme_economist()

wtc_scorefracAP

wtc_scorefracCP

We can combine these to get a more granular estimate of a player’s performance.

> # combine the scores
> wtc2014$scorefrac <- 0.5 * wtc2014$scorefracCP + 
+     0.5 * wtc2014$scorefracAP

In previous posts we used the PlayerRatings package to calculate Elo ratings for players and warcasters. To try to combine the effect of both at the same time I wanted to try minimizing the scorefrac for each caster pairing. First we can create a matrix which has a value for each warcaster pairing.

> wtc2014 <- na.omit(wtc2014)
> # all observed combinations
> pairLookup <- initializeLookup(data = wtc2014)
> pairLookup[1:3, 1:5]
           Absylonia2 Ashlynn Asphyxious Asphyxious2 Asphyxious3
Absylonia2          0       0          0           0           0
Ashlynn             0       0          0           0           0
Asphyxious          0       0          0           0           0

This can then be updated by calculating the Elo rating for all players. As normal, we can run this on a training dataset, then use it to predict on a test dataset.

> isTrain <- wtc2014$round < 6
> wtc2014t <- wtc2014[isTrain, ]
> pairLookup <- updateLookup(data = wtc2014t, 
    pairlookup = pairLookup, result = "TP")

Not all pairs occurred during these games. Also, many matchups only occurred only a few times. To account for this, the maximum penalty allowed was restricted based on the number of games that occurred.

> # the warcaster-self matchup was not minimized, 
> # since the list itself was not being minimized
>  getMatrixVal(list1 = "Haley2", 
+     list2 = c("Asphyxious2", "Feora2", "Haley2"), x = pairLookup)
[1]  29.96216 234.15133   0.00000
> # the opposite is the negative score
> getMatrixVal(list1 = "Asphyxious2", list2 = "Haley2", x = pairLookup)
[1] -29.96216

The statistic for minimized is calculated by getStat.

> getStat(data = wtc2014t, pairlookup = pairLookup, result = "TP")
[1] 19.18362

These results can then be used to get player ratings while attempting to account for warcaster pairings. The gamma correction for the Elo ranking is based on the direction of the matchup for each warcaster pair.

> # ratings based on pairings with selected caster pairings
> rating <- steph(x = wtc2014t[, c("round", "player1", "player2", "TP")], 
+     gamma = getMatrixVal(
+         list1 = wtc2014t[, "list1"], 
+         list2 = wtc2014t[, "list2"], 
+         x = pairLookup))
> rat < - rating$ratings
> head(rat)
             Player   Rating Deviation Games Win Draw Loss Lag
1       Brian White 2653.070  164.8256     5   5    0    0   0
2       Tomek Tutaj 2617.604  169.3150     5   5    0    0   0
3     Jake VanMeter 2598.001  168.6811     5   5    0    0   0
4 Andrzej Kasiewicz 2593.161  170.4032     5   5    0    0   0
5   Robert Cantrell 2591.786  176.8392     5   5    0    0   0
6    Stefan Riegler 2575.240  167.7219     5   5    0    0   0

This is somewhat interesting as the top six players have changed somewhat. Although the number of games for this minimization is very small, this may be some indication that it is possible to separate the effect of list matchup and player performance.

> names(rat) <- casefold(names(rat), upper = FALSE)
> rat <- rat[, c("player", "rating", "deviation")]
> colnames(rat) <- c("player1", "steph_player1_rating", "steph_player1_deviation")
> # ratings based on pairings only
> wtc2014 <- merge(wtc2014, rat)
> colnames(rat) <- c("player2", "steph_player2_rating", "steph_player2_deviation")
> wtc2014 <- merge(wtc2014, rat)
> wtc2014$pstephplayer1 <- predict(rating, wtc2014[, c("round", "player1", "player2")], 
+     tng = 0, gamma = getMatrixVal(
+         list1 = wtc2014[, "list1"], 
+         list2 = wtc2014[, "list2"], 
+         x = pairLookup))
> wttest <- wtc2014[wtc2014$round == 6, ]
> # probability of player 1 winning
> psteph12 <- wtc2014$pstephplayer1[wtc2014$round == 6]
> quan <- seq(from = 0, to = 1, by = 0.1) 
> pQuantiles <- quantile(psteph12, probs = quan, na.rm = TRUE) 
> pQuantiles 
0% 10% 20% 30% 40% 50% 
0.1177160 0.3278433 0.4020137 0.4959527 0.5372217 0.5844014 
60% 70% 80% 90% 100% 
0.6438665 0.7139579 0.7649379 0.8375189 0.9501363

This looks fine, but actually isn’t terribly well calibrated.

> ggplot() + 
+     geom_point(data = data.frame(Quantile = pQuantiles, Quantilex = quan), 
+         aes(x = Quantilex, y = Quantile)) + geom_abline() +
+     theme_economist()

wtc_quantiles
However, this analysis does give us an alternative view on assessing the difference between game matchup and player skill. For example, if we find the total of the gamma effect we get an estimate of a warcaster’s effectiveness (although slightly convolved with popularity for this dataset).

> scores <- apply(X = pairLookup, MARGIN = 1, FUN = sum)
> head(sort(scores, decreasing = TRUE))
Krueger2   Haley2   Skarre   Kromac    Haley     Rask
653.5357 521.8946 279.6237 271.7100 254.9814 250.2347

By this simple statistic, Krueger2 performed better than Haley2. However, Kruger2 may have seen more favourable matchups than Haley2.

> x <- apply(X = pairLookup, MARGIN = 1, FUN = function(x) { any(x == maxGamma[1]) })
> y <- apply(X = pairLookup, MARGIN = 2, FUN = function(x) { any(x == maxGamma[1]) })
> pairLookup[x, y, drop = FALSE]
          Harbinger
Morvahna2  269.9422

So Morvahna2 had the most consistent result against Harbinger, Krueger2 had a very consistent result against Vyros2, Haley2 against Feora2 and so on.

For now I will call this experiment a success. There are so many opportunities for extending this. I hope to have some more time for this before WTC2015.

Advertisements

One thought on “world team championships 2014 part 3

  1. Pingback: world team championships 2015 part 3 | analytical gaming

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s