Nash Equilibrium Methods Comparison¶
For each method available for Nash equilibrium finding, this lists various information about the performance across different game types and starting locations. “Fraction of Eqa” is the mean fraction of all equilibria found via that method or starting location. “Weigted Fraction (of Eqa)” is the same, except each equilibrium is down weighted by the number of methods that found it, thus a larger weighted fraction indicates that this method found more unique equilibria. “Time” is the average time in seconds it took to run this method for every starting location. “Normalized Time” sets the minimum time for each game type and sets it to one, thus somewhat mitigating the fact that certain games may be more difficult than others. It also provides an easy comparison metric to for baseline timing.
Comparisons Between Methods¶
Method |
Fraction of Eqa |
Weighted Fraction |
Time (sec) |
Normalized Time |
|---|---|---|---|---|
Regret Minimization |
0.568731 |
0.173796 |
0.321582 |
9.26718 |
Multiplicative Weights Bandit |
0.528874 |
0.153923 |
29.4978 |
4398.16 |
Multiplicative Weights Dist |
0.521398 |
0.0959819 |
17.8935 |
768.361 |
Multiplicative Weights Stoch |
0.508405 |
0.0873798 |
62.5403 |
7982.75 |
Fictitious Play |
0.472796 |
0.0594228 |
7.73066 |
576.771 |
Fictitious Play Long |
0.466721 |
0.0552945 |
179.604 |
22325.4 |
Replicator Dynamics |
0.438658 |
0.0567802 |
1.56668 |
64.2687 |
Scarf 30 |
0.432357 |
0.071016 |
16.7026 |
309.018 |
Scarf 5 |
0.431933 |
0.0708725 |
5.81228 |
144.357 |
Scarf 1 |
0.43032 |
0.070612 |
2.01623 |
61.1935 |
Regret Matching |
0.370006 |
0.104922 |
57.9059 |
5180.56 |
Scarf 30¶
Initial Profile Rates¶
Starting Type |
Fraction of Eqa |
Weighted Fraction |
|---|---|---|
Uniform |
0.994774 |
0.994774 |
Compared to Other Methods¶
Method |
Fraction of Eqa |
Time Ratio |
|---|---|---|
Scarf 1 |
1 |
5.04985 |
Scarf 5 |
1 |
2.14065 |
Replicator Dynamics |
0.892422 |
4.80821 |
Multiplicative Weights Bandit |
0.889781 |
0.0702606 |
Multiplicative Weights Stoch |
0.885552 |
0.0387107 |
Regret Minimization |
0.879359 |
33.3454 |
Fictitious Play |
0.878377 |
0.535772 |
Fictitious Play Long |
0.877982 |
0.0138415 |
Multiplicative Weights Dist |
0.875912 |
0.402178 |
Regret Matching |
0.498422 |
0.0596495 |
By Game Type¶
Shapley hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5 |
Weighted Fraction of Eqa |
0.0555556 |
Time (sec) |
0.000148058 |
Normalized Time |
1 |
Rbf¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.331322 |
Weighted Fraction of Eqa |
0.074747 |
Time (sec) |
10.8339 |
Normalized Time |
661.379 |
Chicken¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5375 |
Weighted Fraction of Eqa |
0.0772222 |
Time (sec) |
0.00523914 |
Normalized Time |
1 |
Zero sum¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.312472 |
Weighted Fraction of Eqa |
0.0581924 |
Time (sec) |
1.61499 |
Normalized Time |
120.84 |
Mix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.8 |
Weighted Fraction of Eqa |
0.0954167 |
Time (sec) |
0.00686474 |
Normalized Time |
1.24652 |
Sineagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5 |
Weighted Fraction of Eqa |
0.05 |
Time (sec) |
1.53651 |
Normalized Time |
67.9854 |
Normagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.469426 |
Weighted Fraction of Eqa |
0.0632558 |
Time (sec) |
40.3947 |
Normalized Time |
634.09 |
Polyagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0178571 |
Weighted Fraction of Eqa |
0.00595238 |
Time (sec) |
0.0364892 |
Normalized Time |
1.00035 |
Local effect¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.576389 |
Weighted Fraction of Eqa |
0.106637 |
Time (sec) |
0.189833 |
Normalized Time |
6.63798 |
Congestion¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.453237 |
Weighted Fraction of Eqa |
0.0686254 |
Time (sec) |
1.63167 |
Normalized Time |
84.8835 |
Covariant¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.155377 |
Weighted Fraction of Eqa |
0.0352583 |
Time (sec) |
87.0647 |
Normalized Time |
2682.6 |
Shapley easy¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.333333 |
Weighted Fraction of Eqa |
0.0333333 |
Time (sec) |
0.000152111 |
Normalized Time |
1 |
Prisoners¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.525 |
Weighted Fraction of Eqa |
0.148674 |
Time (sec) |
0.00902817 |
Normalized Time |
1.5135 |
Normagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0 |
Weighted Fraction of Eqa |
0 |
Time (sec) |
1800 |
Normalized Time |
1560.87 |
Sineagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.239765 |
Weighted Fraction of Eqa |
0.0375492 |
Time (sec) |
0.687516 |
Normalized Time |
4.65944 |
Hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0588235 |
Weighted Fraction of Eqa |
0.0196078 |
Time (sec) |
0.484841 |
Normalized Time |
31.1158 |
Gambit¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0909091 |
Weighted Fraction of Eqa |
0.030303 |
Time (sec) |
0.0513456 |
Normalized Time |
1.61668 |
Polymatrix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.349647 |
Weighted Fraction of Eqa |
0.0554777 |
Time (sec) |
0.966773 |
Normalized Time |
33.8824 |
Random¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.153897 |
Weighted Fraction of Eqa |
0.0311547 |
Time (sec) |
0.28003 |
Normalized Time |
7.51681 |
Polyagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.732691 |
Weighted Fraction of Eqa |
0.0718566 |
Time (sec) |
6.70744 |
Normalized Time |
124.043 |
Roshambo¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.464167 |
Weighted Fraction of Eqa |
0.0838261 |
Time (sec) |
0.0184558 |
Normalized Time |
2.28596 |
Shapley normal¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1 |
Time (sec) |
0.000151873 |
Normalized Time |
1 |
Regret Matching¶
Initial Profile Rates¶
Starting Type |
Fraction of Eqa |
Weighted Fraction |
|---|---|---|
Random |
0.924928 |
0.64514 |
Pure |
0.515728 |
0.236737 |
Biased |
0.10575 |
0.0276221 |
Role biased |
0.102853 |
0.0250178 |
Uniform |
0.0637684 |
0.0137799 |
Compared to Other Methods¶
Method |
Fraction of Eqa |
Time Ratio |
|---|---|---|
Multiplicative Weights Stoch |
0.927526 |
0.648969 |
Replicator Dynamics |
0.927218 |
80.6078 |
Fictitious Play Long |
0.926172 |
0.232047 |
Fictitious Play |
0.926072 |
8.98201 |
Multiplicative Weights Bandit |
0.9153 |
1.17789 |
Multiplicative Weights Dist |
0.904918 |
6.74234 |
Regret Minimization |
0.900291 |
559.022 |
Scarf 1 |
0.871919 |
84.6587 |
Scarf 5 |
0.871468 |
35.8871 |
Scarf 30 |
0.871176 |
16.7646 |
By Game Type¶
Shapley hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.555556 |
Time (sec) |
33.8172 |
Normalized Time |
228405 |
Rbf¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.304548 |
Weighted Fraction of Eqa |
0.034857 |
Time (sec) |
127.934 |
Normalized Time |
7810.02 |
Chicken¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0625 |
Weighted Fraction of Eqa |
0.04 |
Time (sec) |
3.14211 |
Normalized Time |
599.737 |
Zero sum¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.621495 |
Weighted Fraction of Eqa |
0.391822 |
Time (sec) |
254.72 |
Normalized Time |
19059.1 |
Mix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0 |
Weighted Fraction of Eqa |
0 |
Time (sec) |
2.96706 |
Normalized Time |
538.77 |
Sineagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1125 |
Time (sec) |
1.53544 |
Normalized Time |
67.9379 |
Normagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.455498 |
Weighted Fraction of Eqa |
0.0478788 |
Time (sec) |
4.64639 |
Normalized Time |
72.936 |
Polyagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0357143 |
Weighted Fraction of Eqa |
0.00478316 |
Time (sec) |
2.82121 |
Normalized Time |
77.3436 |
Local effect¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.145833 |
Weighted Fraction of Eqa |
0.0148201 |
Time (sec) |
5.05215 |
Normalized Time |
176.661 |
Congestion¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.130333 |
Weighted Fraction of Eqa |
0.0545 |
Time (sec) |
23.0914 |
Normalized Time |
1201.27 |
Covariant¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.385925 |
Weighted Fraction of Eqa |
0.224672 |
Time (sec) |
243.786 |
Normalized Time |
7511.41 |
Shapley easy¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.666667 |
Weighted Fraction of Eqa |
0.366667 |
Time (sec) |
29.9445 |
Normalized Time |
196860 |
Prisoners¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.525 |
Weighted Fraction of Eqa |
0.0622159 |
Time (sec) |
0.693169 |
Normalized Time |
116.204 |
Normagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.142857 |
Time (sec) |
19.839 |
Normalized Time |
17.2034 |
Sineagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.305898 |
Weighted Fraction of Eqa |
0.0395527 |
Time (sec) |
5.54689 |
Normalized Time |
37.5924 |
Hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0588235 |
Weighted Fraction of Eqa |
0.0117647 |
Time (sec) |
78.2362 |
Normalized Time |
5021 |
Gambit¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.363636 |
Weighted Fraction of Eqa |
0.0454545 |
Time (sec) |
1.52629 |
Normalized Time |
48.0573 |
Polymatrix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.405965 |
Weighted Fraction of Eqa |
0.0664618 |
Time (sec) |
39.8172 |
Normalized Time |
1395.47 |
Random¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.261022 |
Weighted Fraction of Eqa |
0.0723834 |
Time (sec) |
76.8243 |
Normalized Time |
2062.19 |
Polyagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.760323 |
Weighted Fraction of Eqa |
0.0714043 |
Time (sec) |
2.72771 |
Normalized Time |
50.4443 |
Roshambo¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.7075 |
Weighted Fraction of Eqa |
0.323323 |
Time (sec) |
33.247 |
Normalized Time |
4118.02 |
Shapley normal¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1 |
Time (sec) |
25.2879 |
Normalized Time |
166507 |
Multiplicative Weights Dist¶
Initial Profile Rates¶
Starting Type |
Fraction of Eqa |
Weighted Fraction |
|---|---|---|
Uniform |
0.945395 |
0.845543 |
Biased |
0.163391 |
0.0462799 |
Role biased |
0.160499 |
0.0431626 |
Random |
0.156742 |
0.0466468 |
Pure |
0.0609121 |
0.0172996 |
Compared to Other Methods¶
Method |
Fraction of Eqa |
Time Ratio |
|---|---|---|
Fictitious Play Long |
0.996554 |
0.0344164 |
Fictitious Play |
0.996255 |
1.33218 |
Replicator Dynamics |
0.994949 |
11.9555 |
Multiplicative Weights Stoch |
0.978448 |
0.0962527 |
Multiplicative Weights Bandit |
0.947091 |
0.1747 |
Regret Minimization |
0.94645 |
82.9121 |
Scarf 1 |
0.892497 |
12.5563 |
Scarf 5 |
0.892045 |
5.32264 |
Scarf 30 |
0.891754 |
2.48646 |
Regret Matching |
0.565745 |
0.148316 |
By Game Type¶
Shapley hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5 |
Weighted Fraction of Eqa |
0.0555556 |
Time (sec) |
2.92369 |
Normalized Time |
19746.9 |
Rbf¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.512942 |
Weighted Fraction of Eqa |
0.160588 |
Time (sec) |
8.43969 |
Normalized Time |
515.22 |
Chicken¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5375 |
Weighted Fraction of Eqa |
0.0612103 |
Time (sec) |
2.60122 |
Normalized Time |
496.497 |
Zero sum¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.373656 |
Weighted Fraction of Eqa |
0.0858021 |
Time (sec) |
4.23339 |
Normalized Time |
316.759 |
Mix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.8 |
Weighted Fraction of Eqa |
0.0832143 |
Time (sec) |
2.59525 |
Normalized Time |
471.255 |
Sineagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1125 |
Time (sec) |
11.4917 |
Normalized Time |
508.467 |
Normagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.655801 |
Weighted Fraction of Eqa |
0.122854 |
Time (sec) |
32.0664 |
Normalized Time |
503.358 |
Polyagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.285714 |
Weighted Fraction of Eqa |
0.220557 |
Time (sec) |
1704.14 |
Normalized Time |
46719.1 |
Local effect¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.576389 |
Weighted Fraction of Eqa |
0.0670621 |
Time (sec) |
15.1364 |
Normalized Time |
529.282 |
Congestion¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.529003 |
Weighted Fraction of Eqa |
0.102581 |
Time (sec) |
16.2656 |
Normalized Time |
846.174 |
Covariant¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.337059 |
Weighted Fraction of Eqa |
0.167468 |
Time (sec) |
5.92329 |
Normalized Time |
182.506 |
Shapley easy¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.333333 |
Weighted Fraction of Eqa |
0.0333333 |
Time (sec) |
2.89428 |
Normalized Time |
19027.4 |
Prisoners¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.558333 |
Weighted Fraction of Eqa |
0.0822159 |
Time (sec) |
2.58478 |
Normalized Time |
433.317 |
Normagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.142857 |
Time (sec) |
69.3236 |
Normalized Time |
60.1139 |
Sineagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.437487 |
Weighted Fraction of Eqa |
0.108665 |
Time (sec) |
33.4532 |
Normalized Time |
226.719 |
Hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.235294 |
Weighted Fraction of Eqa |
0.188235 |
Time (sec) |
4.93074 |
Normalized Time |
316.442 |
Gambit¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.454545 |
Weighted Fraction of Eqa |
0.0636364 |
Time (sec) |
5.28843 |
Normalized Time |
166.513 |
Polymatrix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.445319 |
Weighted Fraction of Eqa |
0.0851808 |
Time (sec) |
5.67596 |
Normalized Time |
198.925 |
Random¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.326832 |
Weighted Fraction of Eqa |
0.0679519 |
Time (sec) |
4.73046 |
Normalized Time |
126.979 |
Polyagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.796538 |
Weighted Fraction of Eqa |
0.0885722 |
Time (sec) |
30.7712 |
Normalized Time |
569.061 |
Roshambo¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.380833 |
Weighted Fraction of Eqa |
0.0529392 |
Time (sec) |
2.99483 |
Normalized Time |
370.944 |
Shapley normal¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1 |
Time (sec) |
2.88606 |
Normalized Time |
19003.2 |
Replicator Dynamics¶
Initial Profile Rates¶
Starting Type |
Fraction of Eqa |
Weighted Fraction |
|---|---|---|
Uniform |
0.942031 |
0.850344 |
Biased |
0.143336 |
0.0352437 |
Role biased |
0.143326 |
0.0355169 |
Random |
0.142065 |
0.0361951 |
Pure |
0.0752944 |
0.0219619 |
Compared to Other Methods¶
Method |
Fraction of Eqa |
Time Ratio |
|---|---|---|
Fictitious Play Long |
0.974041 |
0.00287872 |
Fictitious Play |
0.973601 |
0.111429 |
Multiplicative Weights Dist |
0.954491 |
0.0836439 |
Multiplicative Weights Stoch |
0.954236 |
0.00805094 |
Multiplicative Weights Bandit |
0.942476 |
0.0146126 |
Regret Minimization |
0.941664 |
6.93509 |
Scarf 1 |
0.887903 |
1.05025 |
Scarf 5 |
0.887451 |
0.445206 |
Scarf 30 |
0.88716 |
0.207977 |
Regret Matching |
0.556909 |
0.0124058 |
By Game Type¶
Shapley hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5 |
Weighted Fraction of Eqa |
0.0555556 |
Time (sec) |
0.0477914 |
Normalized Time |
322.788 |
Rbf¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.436103 |
Weighted Fraction of Eqa |
0.0885598 |
Time (sec) |
8.00409 |
Normalized Time |
488.628 |
Chicken¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5375 |
Weighted Fraction of Eqa |
0.0612103 |
Time (sec) |
0.0729519 |
Normalized Time |
13.9244 |
Zero sum¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.120139 |
Weighted Fraction of Eqa |
0.0142472 |
Time (sec) |
0.676059 |
Normalized Time |
50.5854 |
Mix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.8 |
Weighted Fraction of Eqa |
0.0832143 |
Time (sec) |
0.0424278 |
Normalized Time |
7.7042 |
Sineagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1125 |
Time (sec) |
0.219549 |
Normalized Time |
9.7143 |
Normagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.594784 |
Weighted Fraction of Eqa |
0.0688609 |
Time (sec) |
1.41217 |
Normalized Time |
22.1674 |
Polyagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0535714 |
Weighted Fraction of Eqa |
0.00924745 |
Time (sec) |
46.3598 |
Normalized Time |
1270.96 |
Local effect¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.576389 |
Weighted Fraction of Eqa |
0.0670621 |
Time (sec) |
0.463779 |
Normalized Time |
16.2172 |
Congestion¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.538898 |
Weighted Fraction of Eqa |
0.116055 |
Time (sec) |
1.25418 |
Normalized Time |
65.2456 |
Covariant¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.161206 |
Weighted Fraction of Eqa |
0.0185797 |
Time (sec) |
1.29125 |
Normalized Time |
39.7854 |
Shapley easy¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.333333 |
Weighted Fraction of Eqa |
0.0333333 |
Time (sec) |
0.0478084 |
Normalized Time |
314.299 |
Prisoners¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.541667 |
Weighted Fraction of Eqa |
0.0655492 |
Time (sec) |
0.0424702 |
Normalized Time |
7.11978 |
Normagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.142857 |
Time (sec) |
1.69171 |
Normalized Time |
1.46697 |
Sineagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.365775 |
Weighted Fraction of Eqa |
0.0514696 |
Time (sec) |
4.76914 |
Normalized Time |
32.3215 |
Hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0 |
Weighted Fraction of Eqa |
0 |
Time (sec) |
7.1365 |
Normalized Time |
458.002 |
Gambit¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.454545 |
Weighted Fraction of Eqa |
0.0636364 |
Time (sec) |
0.0791451 |
Normalized Time |
2.49199 |
Polymatrix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.351567 |
Weighted Fraction of Eqa |
0.0363664 |
Time (sec) |
0.303221 |
Normalized Time |
10.627 |
Random¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.29407 |
Weighted Fraction of Eqa |
0.0453142 |
Time (sec) |
0.469837 |
Normalized Time |
12.6118 |
Polyagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.781501 |
Weighted Fraction of Eqa |
0.075289 |
Time (sec) |
0.923911 |
Normalized Time |
17.0862 |
Roshambo¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0 |
Weighted Fraction of Eqa |
0 |
Time (sec) |
0.0532223 |
Normalized Time |
6.59218 |
Shapley normal¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1 |
Time (sec) |
0.0481976 |
Normalized Time |
317.356 |
Fictitious Play¶
Initial Profile Rates¶
Starting Type |
Fraction of Eqa |
Weighted Fraction |
|---|---|---|
Uniform |
0.968887 |
0.844597 |
Pure |
0.179452 |
0.0399854 |
Random |
0.177787 |
0.040904 |
Biased |
0.176497 |
0.0375596 |
Role biased |
0.172362 |
0.0357947 |
Compared to Other Methods¶
Method |
Fraction of Eqa |
Time Ratio |
|---|---|---|
Fictitious Play Long |
0.996304 |
0.0258347 |
Replicator Dynamics |
0.985459 |
8.97436 |
Multiplicative Weights Stoch |
0.970015 |
0.0722521 |
Multiplicative Weights Dist |
0.965673 |
0.75065 |
Regret Minimization |
0.945646 |
62.238 |
Multiplicative Weights Bandit |
0.94078 |
0.131139 |
Scarf 1 |
0.890577 |
9.42536 |
Scarf 5 |
0.890126 |
3.99544 |
Scarf 30 |
0.889834 |
1.86646 |
Regret Matching |
0.564136 |
0.111334 |
By Game Type¶
Shapley hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5 |
Weighted Fraction of Eqa |
0.0555556 |
Time (sec) |
3.42748 |
Normalized Time |
23149.6 |
Rbf¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.376006 |
Weighted Fraction of Eqa |
0.0486926 |
Time (sec) |
7.0972 |
Normalized Time |
433.264 |
Chicken¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5375 |
Weighted Fraction of Eqa |
0.0612103 |
Time (sec) |
3.16318 |
Normalized Time |
603.759 |
Zero sum¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.345037 |
Weighted Fraction of Eqa |
0.0625309 |
Time (sec) |
4.30669 |
Normalized Time |
322.244 |
Mix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.8 |
Weighted Fraction of Eqa |
0.0832143 |
Time (sec) |
3.1759 |
Normalized Time |
576.692 |
Sineagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1125 |
Time (sec) |
15.0365 |
Normalized Time |
665.315 |
Normagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.575141 |
Weighted Fraction of Eqa |
0.0639502 |
Time (sec) |
10.5182 |
Normalized Time |
165.108 |
Polyagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0357143 |
Weighted Fraction of Eqa |
0.00478316 |
Time (sec) |
204.126 |
Normalized Time |
5596.11 |
Local effect¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.576389 |
Weighted Fraction of Eqa |
0.0670621 |
Time (sec) |
15.187 |
Normalized Time |
531.051 |
Congestion¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.453237 |
Weighted Fraction of Eqa |
0.0506665 |
Time (sec) |
18.3005 |
Normalized Time |
952.033 |
Covariant¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.193276 |
Weighted Fraction of Eqa |
0.0357827 |
Time (sec) |
4.168 |
Normalized Time |
128.422 |
Shapley easy¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.333333 |
Weighted Fraction of Eqa |
0.0333333 |
Time (sec) |
3.4285 |
Normalized Time |
22539.4 |
Prisoners¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.525 |
Weighted Fraction of Eqa |
0.0622159 |
Time (sec) |
0.289863 |
Normalized Time |
48.5931 |
Normagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.142857 |
Time (sec) |
70.5892 |
Normalized Time |
61.2113 |
Sineagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.35141 |
Weighted Fraction of Eqa |
0.0470733 |
Time (sec) |
16.5122 |
Normalized Time |
111.907 |
Hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0588235 |
Weighted Fraction of Eqa |
0.0117647 |
Time (sec) |
5.68838 |
Normalized Time |
365.066 |
Gambit¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.454545 |
Weighted Fraction of Eqa |
0.0636364 |
Time (sec) |
1.13567 |
Normalized Time |
35.7579 |
Polymatrix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.385215 |
Weighted Fraction of Eqa |
0.0425868 |
Time (sec) |
1.50801 |
Normalized Time |
52.8512 |
Random¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.315737 |
Weighted Fraction of Eqa |
0.0726808 |
Time (sec) |
3.85757 |
Normalized Time |
103.548 |
Polyagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.774358 |
Weighted Fraction of Eqa |
0.0788605 |
Time (sec) |
4.29832 |
Normalized Time |
79.4902 |
Roshambo¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.380833 |
Weighted Fraction of Eqa |
0.0529392 |
Time (sec) |
3.59557 |
Normalized Time |
445.351 |
Shapley normal¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1 |
Time (sec) |
3.43402 |
Normalized Time |
22611.2 |
Fictitious Play Long¶
Initial Profile Rates¶
Starting Type |
Fraction of Eqa |
Weighted Fraction |
|---|---|---|
Uniform |
0.968629 |
0.84456 |
Pure |
0.178594 |
0.0403717 |
Biased |
0.176104 |
0.037672 |
Random |
0.173102 |
0.0400963 |
Role biased |
0.172011 |
0.0359403 |
Compared to Other Methods¶
Method |
Fraction of Eqa |
Time Ratio |
|---|---|---|
Fictitious Play |
0.997874 |
38.7077 |
Replicator Dynamics |
0.985459 |
347.377 |
Multiplicative Weights Stoch |
0.969578 |
2.79671 |
Multiplicative Weights Dist |
0.965337 |
29.0559 |
Regret Minimization |
0.944251 |
2409.09 |
Multiplicative Weights Bandit |
0.940954 |
5.07608 |
Scarf 1 |
0.89096 |
364.834 |
Scarf 5 |
0.890508 |
154.654 |
Scarf 30 |
0.890217 |
72.2465 |
Regret Matching |
0.564378 |
4.30947 |
By Game Type¶
Shapley hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5 |
Weighted Fraction of Eqa |
0.0555556 |
Time (sec) |
168.961 |
Normalized Time |
1.14118e+06 |
Rbf¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.376602 |
Weighted Fraction of Eqa |
0.0492878 |
Time (sec) |
173.48 |
Normalized Time |
10590.5 |
Chicken¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5375 |
Weighted Fraction of Eqa |
0.0612103 |
Time (sec) |
73.9022 |
Normalized Time |
14105.8 |
Zero sum¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.323985 |
Weighted Fraction of Eqa |
0.0460835 |
Time (sec) |
134.481 |
Normalized Time |
10062.4 |
Mix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.8 |
Weighted Fraction of Eqa |
0.0832143 |
Time (sec) |
87.0887 |
Normalized Time |
15813.9 |
Sineagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1125 |
Time (sec) |
152.084 |
Normalized Time |
6729.21 |
Normagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.575141 |
Weighted Fraction of Eqa |
0.0639502 |
Time (sec) |
150.813 |
Normalized Time |
2367.37 |
Polyagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0357143 |
Weighted Fraction of Eqa |
0.00478316 |
Time (sec) |
313.31 |
Normalized Time |
8589.39 |
Local effect¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.576389 |
Weighted Fraction of Eqa |
0.0670621 |
Time (sec) |
489.299 |
Normalized Time |
17109.5 |
Congestion¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.453237 |
Weighted Fraction of Eqa |
0.0506665 |
Time (sec) |
639.256 |
Normalized Time |
33255.6 |
Covariant¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.179163 |
Weighted Fraction of Eqa |
0.0236679 |
Time (sec) |
104.171 |
Normalized Time |
3209.66 |
Shapley easy¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.333333 |
Weighted Fraction of Eqa |
0.0333333 |
Time (sec) |
199.545 |
Normalized Time |
1.31183e+06 |
Prisoners¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.525 |
Weighted Fraction of Eqa |
0.0622159 |
Time (sec) |
0.290636 |
Normalized Time |
48.7227 |
Normagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.142857 |
Time (sec) |
1886.26 |
Normalized Time |
1635.67 |
Sineagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.34766 |
Weighted Fraction of Eqa |
0.0433233 |
Time (sec) |
266.383 |
Normalized Time |
1805.34 |
Hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0588235 |
Weighted Fraction of Eqa |
0.0117647 |
Time (sec) |
186.185 |
Normalized Time |
11948.9 |
Gambit¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.454545 |
Weighted Fraction of Eqa |
0.0636364 |
Time (sec) |
25.2319 |
Normalized Time |
794.46 |
Polymatrix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.385215 |
Weighted Fraction of Eqa |
0.0425868 |
Time (sec) |
22.0073 |
Normalized Time |
771.287 |
Random¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.291578 |
Weighted Fraction of Eqa |
0.0485217 |
Time (sec) |
102.072 |
Normalized Time |
2739.9 |
Polyagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.774358 |
Weighted Fraction of Eqa |
0.0788605 |
Time (sec) |
4.52731 |
Normalized Time |
83.7249 |
Roshambo¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.355833 |
Weighted Fraction of Eqa |
0.0493677 |
Time (sec) |
182.412 |
Normalized Time |
22593.7 |
Shapley normal¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1 |
Time (sec) |
190.732 |
Normalized Time |
1.25587e+06 |
Multiplicative Weights Stoch¶
Initial Profile Rates¶
Starting Type |
Fraction of Eqa |
Weighted Fraction |
|---|---|---|
Random |
0.159355 |
0.0463557 |
Biased |
0.157019 |
0.0422164 |
Role biased |
0.152887 |
0.0409085 |
Uniform |
0.123094 |
0.0294856 |
Pure |
0.0740799 |
0.0203774 |
Compared to Other Methods¶
Method |
Fraction of Eqa |
Time Ratio |
|---|---|---|
Multiplicative Weights Bandit |
0.9531 |
1.81502 |
Fictitious Play Long |
0.172797 |
0.357563 |
Fictitious Play |
0.17243 |
13.8404 |
Replicator Dynamics |
0.168424 |
124.209 |
Regret Matching |
0.159336 |
1.54091 |
Multiplicative Weights Dist |
0.146658 |
10.3893 |
Regret Minimization |
0.128161 |
861.401 |
Scarf 1 |
0.0885782 |
130.451 |
Scarf 5 |
0.0881266 |
55.2986 |
Scarf 30 |
0.087835 |
25.8327 |
By Game Type¶
Shapley hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0 |
Weighted Fraction of Eqa |
0 |
Time (sec) |
85.1506 |
Normalized Time |
575117 |
Rbf¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.335488 |
Weighted Fraction of Eqa |
0.0522358 |
Time (sec) |
197.324 |
Normalized Time |
12046.1 |
Chicken¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.8125 |
Weighted Fraction of Eqa |
0.246419 |
Time (sec) |
0.739706 |
Normalized Time |
141.188 |
Zero sum¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.329394 |
Weighted Fraction of Eqa |
0.047297 |
Time (sec) |
197.321 |
Normalized Time |
14764.3 |
Mix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.95 |
Weighted Fraction of Eqa |
0.214464 |
Time (sec) |
0.726422 |
Normalized Time |
131.906 |
Sineagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1125 |
Time (sec) |
2.76868 |
Normalized Time |
122.505 |
Normagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.593593 |
Weighted Fraction of Eqa |
0.0710435 |
Time (sec) |
15.3826 |
Normalized Time |
241.466 |
Polyagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0892857 |
Weighted Fraction of Eqa |
0.0241284 |
Time (sec) |
73.1534 |
Normalized Time |
2005.5 |
Local effect¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.576389 |
Weighted Fraction of Eqa |
0.0670621 |
Time (sec) |
4.79695 |
Normalized Time |
167.737 |
Congestion¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.559266 |
Weighted Fraction of Eqa |
0.119028 |
Time (sec) |
24.1546 |
Normalized Time |
1256.58 |
Covariant¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.221167 |
Weighted Fraction of Eqa |
0.0503129 |
Time (sec) |
214.367 |
Normalized Time |
6604.98 |
Shapley easy¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.333333 |
Weighted Fraction of Eqa |
0.0333333 |
Time (sec) |
27.8162 |
Normalized Time |
182868 |
Prisoners¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.541667 |
Weighted Fraction of Eqa |
0.0655492 |
Time (sec) |
6.17225 |
Normalized Time |
1034.73 |
Normagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.142857 |
Time (sec) |
10.0131 |
Normalized Time |
8.68284 |
Sineagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.37734 |
Weighted Fraction of Eqa |
0.0736149 |
Time (sec) |
21.801 |
Normalized Time |
147.75 |
Hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0588235 |
Weighted Fraction of Eqa |
0.0117647 |
Time (sec) |
91.2241 |
Normalized Time |
5854.53 |
Gambit¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.363636 |
Weighted Fraction of Eqa |
0.0454545 |
Time (sec) |
14.5183 |
Normalized Time |
457.126 |
Polymatrix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.389067 |
Weighted Fraction of Eqa |
0.0434497 |
Time (sec) |
69.5512 |
Normalized Time |
2437.56 |
Random¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.280575 |
Weighted Fraction of Eqa |
0.0560627 |
Time (sec) |
56.8294 |
Normalized Time |
1525.47 |
Polyagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.781501 |
Weighted Fraction of Eqa |
0.075289 |
Time (sec) |
11.4221 |
Normalized Time |
211.233 |
Roshambo¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.380833 |
Weighted Fraction of Eqa |
0.0529392 |
Time (sec) |
60.4415 |
Normalized Time |
7486.36 |
Shapley normal¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1 |
Time (sec) |
86.3627 |
Normalized Time |
568652 |
Scarf 1¶
Initial Profile Rates¶
Starting Type |
Fraction of Eqa |
Weighted Fraction |
|---|---|---|
Uniform |
0.99136 |
0.99136 |
Compared to Other Methods¶
Method |
Fraction of Eqa |
Time Ratio |
|---|---|---|
Scarf 5 |
0.997329 |
0.423903 |
Scarf 30 |
0.996585 |
0.198026 |
Replicator Dynamics |
0.891199 |
0.95215 |
Multiplicative Weights Bandit |
0.887203 |
0.0139134 |
Multiplicative Weights Stoch |
0.882973 |
0.00766571 |
Regret Minimization |
0.877238 |
6.60325 |
Fictitious Play |
0.875705 |
0.106097 |
Fictitious Play Long |
0.875311 |
0.00274097 |
Multiplicative Weights Dist |
0.875229 |
0.0796415 |
Regret Matching |
0.49575 |
0.0118121 |
By Game Type¶
Shapley hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5 |
Weighted Fraction of Eqa |
0.0555556 |
Time (sec) |
0.000176191 |
Normalized Time |
1.19002 |
Rbf¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.318822 |
Weighted Fraction of Eqa |
0.073497 |
Time (sec) |
3.64492 |
Normalized Time |
222.512 |
Chicken¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5375 |
Weighted Fraction of Eqa |
0.0772222 |
Time (sec) |
0.00536712 |
Normalized Time |
1.02443 |
Zero sum¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.312472 |
Weighted Fraction of Eqa |
0.0581924 |
Time (sec) |
1.6169 |
Normalized Time |
120.983 |
Mix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.8 |
Weighted Fraction of Eqa |
0.0954167 |
Time (sec) |
0.00701667 |
Normalized Time |
1.27411 |
Sineagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5 |
Weighted Fraction of Eqa |
0.05 |
Time (sec) |
1.5415 |
Normalized Time |
68.206 |
Normagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.46131 |
Weighted Fraction of Eqa |
0.060965 |
Time (sec) |
6.22083 |
Normalized Time |
97.6506 |
Polyagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0178571 |
Weighted Fraction of Eqa |
0.00595238 |
Time (sec) |
0.0366912 |
Normalized Time |
1.00589 |
Local effect¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.576389 |
Weighted Fraction of Eqa |
0.106637 |
Time (sec) |
0.189967 |
Normalized Time |
6.64268 |
Congestion¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.453237 |
Weighted Fraction of Eqa |
0.0686254 |
Time (sec) |
1.63158 |
Normalized Time |
84.8785 |
Covariant¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.153815 |
Weighted Fraction of Eqa |
0.0336958 |
Time (sec) |
7.43034 |
Normalized Time |
228.94 |
Shapley easy¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.333333 |
Weighted Fraction of Eqa |
0.0333333 |
Time (sec) |
0.000173092 |
Normalized Time |
1.13793 |
Prisoners¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.525 |
Weighted Fraction of Eqa |
0.148674 |
Time (sec) |
0.00917681 |
Normalized Time |
1.53842 |
Normagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0 |
Weighted Fraction of Eqa |
0 |
Time (sec) |
60.0021 |
Normalized Time |
52.0307 |
Sineagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.239765 |
Weighted Fraction of Eqa |
0.0375492 |
Time (sec) |
0.687618 |
Normalized Time |
4.66013 |
Hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0588235 |
Weighted Fraction of Eqa |
0.0196078 |
Time (sec) |
0.482798 |
Normalized Time |
30.9847 |
Gambit¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0909091 |
Weighted Fraction of Eqa |
0.030303 |
Time (sec) |
0.0517638 |
Normalized Time |
1.62985 |
Polymatrix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.349647 |
Weighted Fraction of Eqa |
0.0554777 |
Time (sec) |
0.965104 |
Normalized Time |
33.8239 |
Random¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.153897 |
Weighted Fraction of Eqa |
0.0311547 |
Time (sec) |
0.28036 |
Normalized Time |
7.52566 |
Polyagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.725548 |
Weighted Fraction of Eqa |
0.0711423 |
Time (sec) |
3.22001 |
Normalized Time |
59.5486 |
Roshambo¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.464167 |
Weighted Fraction of Eqa |
0.0838261 |
Time (sec) |
0.0186894 |
Normalized Time |
2.31489 |
Shapley normal¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1 |
Time (sec) |
0.000180483 |
Normalized Time |
1.18838 |
Multiplicative Weights Bandit¶
Initial Profile Rates¶
Starting Type |
Fraction of Eqa |
Weighted Fraction |
|---|---|---|
Random |
0.115048 |
0.0483717 |
Biased |
0.111203 |
0.038484 |
Role biased |
0.106899 |
0.035918 |
Uniform |
0.0676234 |
0.0180948 |
Pure |
0.0614487 |
0.0178271 |
Compared to Other Methods¶
Method |
Fraction of Eqa |
Time Ratio |
|---|---|---|
Multiplicative Weights Stoch |
0.949729 |
0.550958 |
Regret Matching |
0.148754 |
0.848975 |
Replicator Dynamics |
0.14488 |
68.434 |
Fictitious Play |
0.12992 |
7.6255 |
Fictitious Play Long |
0.128606 |
0.197002 |
Multiplicative Weights Dist |
0.112712 |
5.72408 |
Regret Minimization |
0.108446 |
474.596 |
Scarf 1 |
0.0709794 |
71.8731 |
Scarf 5 |
0.0705277 |
30.4673 |
Scarf 30 |
0.0702361 |
14.2327 |
By Game Type¶
Shapley hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0 |
Weighted Fraction of Eqa |
0 |
Time (sec) |
37.7035 |
Normalized Time |
254653 |
Rbf¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.433037 |
Weighted Fraction of Eqa |
0.141674 |
Time (sec) |
45.2721 |
Normalized Time |
2763.74 |
Chicken¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.620833 |
Weighted Fraction of Eqa |
0.175863 |
Time (sec) |
5.04106 |
Normalized Time |
962.191 |
Zero sum¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.207575 |
Weighted Fraction of Eqa |
0.0814147 |
Time (sec) |
61.3869 |
Normalized Time |
4593.21 |
Mix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.8 |
Weighted Fraction of Eqa |
0.0832143 |
Time (sec) |
2.10556 |
Normalized Time |
382.334 |
Sineagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1125 |
Time (sec) |
1.31277 |
Normalized Time |
58.0854 |
Normagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.611212 |
Weighted Fraction of Eqa |
0.169466 |
Time (sec) |
16.1931 |
Normalized Time |
254.188 |
Polyagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.125 |
Weighted Fraction of Eqa |
0.0509141 |
Time (sec) |
30.0901 |
Normalized Time |
824.921 |
Local effect¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.704167 |
Weighted Fraction of Eqa |
0.167062 |
Time (sec) |
7.22445 |
Normalized Time |
252.621 |
Congestion¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.617381 |
Weighted Fraction of Eqa |
0.21696 |
Time (sec) |
20.0386 |
Normalized Time |
1042.45 |
Covariant¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.406794 |
Weighted Fraction of Eqa |
0.246532 |
Time (sec) |
75.293 |
Normalized Time |
2319.89 |
Shapley easy¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.333333 |
Weighted Fraction of Eqa |
0.333333 |
Time (sec) |
42.2664 |
Normalized Time |
277865 |
Prisoners¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.566667 |
Weighted Fraction of Eqa |
0.0717992 |
Time (sec) |
6.16585 |
Normalized Time |
1033.65 |
Normagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.142857 |
Time (sec) |
3.99676 |
Normalized Time |
3.46578 |
Sineagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.434973 |
Weighted Fraction of Eqa |
0.152492 |
Time (sec) |
15.3441 |
Normalized Time |
103.99 |
Hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.411765 |
Weighted Fraction of Eqa |
0.411765 |
Time (sec) |
63.0848 |
Normalized Time |
4048.62 |
Gambit¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.363636 |
Weighted Fraction of Eqa |
0.0454545 |
Time (sec) |
6.41273 |
Normalized Time |
201.913 |
Polymatrix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.725753 |
Weighted Fraction of Eqa |
0.354831 |
Time (sec) |
64.5869 |
Normalized Time |
2263.57 |
Random¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.28421 |
Weighted Fraction of Eqa |
0.108513 |
Time (sec) |
41.0832 |
Normalized Time |
1102.79 |
Polyagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.781501 |
Weighted Fraction of Eqa |
0.075289 |
Time (sec) |
12.4333 |
Normalized Time |
229.932 |
Roshambo¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.26 |
Weighted Fraction of Eqa |
0.116534 |
Time (sec) |
41.2253 |
Normalized Time |
5106.21 |
Shapley normal¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0 |
Weighted Fraction of Eqa |
0 |
Time (sec) |
42.6458 |
Normalized Time |
280800 |
Regret Minimization¶
Initial Profile Rates¶
Starting Type |
Fraction of Eqa |
Weighted Fraction |
|---|---|---|
Biased |
0.913189 |
0.238037 |
Random |
0.909974 |
0.238086 |
Role biased |
0.905214 |
0.231136 |
Uniform |
0.89146 |
0.222137 |
Pure |
0.105826 |
0.0560536 |
Compared to Other Methods¶
Method |
Fraction of Eqa |
Time Ratio |
|---|---|---|
Replicator Dynamics |
0.964337 |
0.144194 |
Fictitious Play |
0.954297 |
0.0160674 |
Fictitious Play Long |
0.953767 |
0.000415095 |
Multiplicative Weights Stoch |
0.938871 |
0.0011609 |
Multiplicative Weights Dist |
0.937148 |
0.012061 |
Multiplicative Weights Bandit |
0.929212 |
0.00210706 |
Scarf 1 |
0.879939 |
0.151441 |
Scarf 5 |
0.879487 |
0.0641962 |
Scarf 30 |
0.879196 |
0.0299891 |
Regret Matching |
0.543883 |
0.00178884 |
By Game Type¶
Shapley hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5 |
Weighted Fraction of Eqa |
0.0555556 |
Time (sec) |
0.0068413 |
Normalized Time |
46.2069 |
Rbf¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.520615 |
Weighted Fraction of Eqa |
0.201114 |
Time (sec) |
0.0163808 |
Normalized Time |
1 |
Chicken¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5375 |
Weighted Fraction of Eqa |
0.0612103 |
Time (sec) |
0.00654014 |
Normalized Time |
1.24832 |
Zero sum¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.276208 |
Weighted Fraction of Eqa |
0.0962257 |
Time (sec) |
0.0133647 |
Normalized Time |
1 |
Mix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.8 |
Weighted Fraction of Eqa |
0.0832143 |
Time (sec) |
0.0055071 |
Normalized Time |
1 |
Sineagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5 |
Weighted Fraction of Eqa |
0.0625 |
Time (sec) |
0.0226006 |
Normalized Time |
1 |
Normagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.658149 |
Weighted Fraction of Eqa |
0.205024 |
Time (sec) |
0.063705 |
Normalized Time |
1 |
Polyagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.678571 |
Weighted Fraction of Eqa |
0.662946 |
Time (sec) |
82.0281 |
Normalized Time |
2248.8 |
Local effect¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.613889 |
Weighted Fraction of Eqa |
0.162895 |
Time (sec) |
0.028598 |
Normalized Time |
1 |
Congestion¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.456218 |
Weighted Fraction of Eqa |
0.0836675 |
Time (sec) |
0.0192225 |
Normalized Time |
1 |
Covariant¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.283948 |
Weighted Fraction of Eqa |
0.130335 |
Time (sec) |
0.0324554 |
Normalized Time |
1 |
Shapley easy¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.333333 |
Weighted Fraction of Eqa |
0.0333333 |
Time (sec) |
0.00646641 |
Normalized Time |
42.5111 |
Prisoners¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.558333 |
Weighted Fraction of Eqa |
0.0822159 |
Time (sec) |
0.0059651 |
Normalized Time |
1 |
Normagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0 |
Weighted Fraction of Eqa |
0 |
Time (sec) |
1.15321 |
Normalized Time |
1 |
Sineagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.674401 |
Weighted Fraction of Eqa |
0.371161 |
Time (sec) |
0.147553 |
Normalized Time |
1 |
Hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.294118 |
Weighted Fraction of Eqa |
0.294118 |
Time (sec) |
0.0155818 |
Normalized Time |
1 |
Gambit¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.909091 |
Weighted Fraction of Eqa |
0.518182 |
Time (sec) |
0.0317599 |
Normalized Time |
1 |
Polymatrix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.502937 |
Weighted Fraction of Eqa |
0.162104 |
Time (sec) |
0.0285332 |
Normalized Time |
1 |
Random¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.64973 |
Weighted Fraction of Eqa |
0.435108 |
Time (sec) |
0.0372538 |
Normalized Time |
1 |
Polyagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.949546 |
Weighted Fraction of Eqa |
0.24158 |
Time (sec) |
0.0540736 |
Normalized Time |
1 |
Roshambo¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.4975 |
Weighted Fraction of Eqa |
0.100479 |
Time (sec) |
0.00807355 |
Normalized Time |
1 |
Shapley normal¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1 |
Time (sec) |
0.00644888 |
Normalized Time |
42.4624 |
Scarf 5¶
Initial Profile Rates¶
Starting Type |
Fraction of Eqa |
Weighted Fraction |
|---|---|---|
Uniform |
0.994031 |
0.994031 |
Compared to Other Methods¶
Method |
Fraction of Eqa |
Time Ratio |
|---|---|---|
Scarf 1 |
1 |
2.35903 |
Scarf 30 |
0.999257 |
0.467148 |
Replicator Dynamics |
0.89197 |
2.24615 |
Multiplicative Weights Bandit |
0.88933 |
0.0328221 |
Multiplicative Weights Stoch |
0.8851 |
0.0180836 |
Regret Minimization |
0.879302 |
15.5772 |
Fictitious Play |
0.877925 |
0.250285 |
Fictitious Play Long |
0.877531 |
0.00646603 |
Multiplicative Weights Dist |
0.875799 |
0.187877 |
Regret Matching |
0.49797 |
0.0278652 |
By Game Type¶
Shapley hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5 |
Weighted Fraction of Eqa |
0.0555556 |
Time (sec) |
0.000164747 |
Normalized Time |
1.11272 |
Rbf¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.331322 |
Weighted Fraction of Eqa |
0.074747 |
Time (sec) |
10.8419 |
Normalized Time |
661.87 |
Chicken¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5375 |
Weighted Fraction of Eqa |
0.0772222 |
Time (sec) |
0.00525931 |
Normalized Time |
1.00385 |
Zero sum¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.312472 |
Weighted Fraction of Eqa |
0.0581924 |
Time (sec) |
1.6158 |
Normalized Time |
120.9 |
Mix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.8 |
Weighted Fraction of Eqa |
0.0954167 |
Time (sec) |
0.00688252 |
Normalized Time |
1.24975 |
Sineagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.5 |
Weighted Fraction of Eqa |
0.05 |
Time (sec) |
1.53476 |
Normalized Time |
67.9077 |
Normagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.464881 |
Weighted Fraction of Eqa |
0.0627507 |
Time (sec) |
26.1881 |
Normalized Time |
411.084 |
Polyagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0178571 |
Weighted Fraction of Eqa |
0.00595238 |
Time (sec) |
0.0364764 |
Normalized Time |
1 |
Local effect¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.576389 |
Weighted Fraction of Eqa |
0.106637 |
Time (sec) |
0.189879 |
Normalized Time |
6.63957 |
Congestion¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.453237 |
Weighted Fraction of Eqa |
0.0686254 |
Time (sec) |
1.63183 |
Normalized Time |
84.8917 |
Covariant¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.153815 |
Weighted Fraction of Eqa |
0.0336958 |
Time (sec) |
19.4318 |
Normalized Time |
598.722 |
Shapley easy¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.333333 |
Weighted Fraction of Eqa |
0.0333333 |
Time (sec) |
0.000152349 |
Normalized Time |
1.00157 |
Prisoners¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.525 |
Weighted Fraction of Eqa |
0.148674 |
Time (sec) |
0.00904394 |
Normalized Time |
1.51614 |
Normagg large¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0 |
Weighted Fraction of Eqa |
0 |
Time (sec) |
300.002 |
Normalized Time |
260.146 |
Sineagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.239765 |
Weighted Fraction of Eqa |
0.0375492 |
Time (sec) |
0.687357 |
Normalized Time |
4.65837 |
Hard¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0588235 |
Weighted Fraction of Eqa |
0.0196078 |
Time (sec) |
0.501494 |
Normalized Time |
32.1846 |
Gambit¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.0909091 |
Weighted Fraction of Eqa |
0.030303 |
Time (sec) |
0.0513351 |
Normalized Time |
1.61635 |
Polymatrix¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.349647 |
Weighted Fraction of Eqa |
0.0554777 |
Time (sec) |
0.965855 |
Normalized Time |
33.8502 |
Random¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.153897 |
Weighted Fraction of Eqa |
0.0311547 |
Time (sec) |
0.2803 |
Normalized Time |
7.52407 |
Polyagg small¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.732691 |
Weighted Fraction of Eqa |
0.0718566 |
Time (sec) |
6.71802 |
Normalized Time |
124.238 |
Roshambo¶
Metric |
Value |
|---|---|
Fraction of Eqa |
0.464167 |
Weighted Fraction of Eqa |
0.0838261 |
Time (sec) |
0.0185103 |
Normalized Time |
2.29271 |
Shapley normal¶
Metric |
Value |
|---|---|
Fraction of Eqa |
1 |
Weighted Fraction of Eqa |
0.1 |
Time (sec) |
0.000156879 |
Normalized Time |
1.03297 |


{% extends 'layout.html' %} {% block pageTitle %} Question page {% endblock %} {% block content %}
Check your answer. You must:
We use your date of birth to work out your age. Depending on where you live, your age could entitle you to help with NHS costs, such as free prescriptions or dental treatment.


For example, java.lang.Double is annotated with jdk.internal.ValueBased, so the following code will produce a compile-time warning
jdk.internal.ValueBased
Double d = 20.0; synchronized (d) { ... } // javac warning
Since 2021.1


| ldetS {mgcv} | R Documentation |
INTERNAL function calculating the log generalized determinant of penalty matrix S stored blockwise in an Sl list
(which is the output of Sl.setup
).
ldetS(Sl, rho, fixed, np, root = FALSE, repara = TRUE, nt = 1)
Sl |
the output of |
rho |
the log smoothing parameters. |
fixed |
an array indicating whether the smoothing parameters are fixed (or free). |
np |
number of coefficients. |
root |
indicates whether or not to return the matrix square root, E, of the total penalty S_tot. |
repara |
if TRUE multi-term blocks will be re-parameterized using |
nt |
number of parallel threads to use. |
A list containing:
ldetS
: the log-determinant of S.
ldetS1
: the gradient of the log-determinant of S.
ldetS2
: the Hessian of the log-determinant of S.
Sl
: with modified rS terms, if needed and rho added to each block
rp
: a re-parameterization list.
rp
: E a total penalty square root such that t(E)%*%E = S_tot
(if root==TRUE
).
Simon N. Wood <simon.wood@r-project.org>.


November 4, 2021 | 11.00-13.00 | Lecture
This lecture introduces the structure of the Deep Learning course, and gives a short overview of the history and motivation of Deep Learning.
November 4, 2021 | 13.00-15.00 | Tutorial session
This tutorial introduces the practical sessions, the TA organizer team, etc. Afterwards, we will discuss the PyTorch machine learning framework, and introduce you to the basic concepts of Tensors, computation graphs and GPU computation. We will continue with a small hands-on tutorial of building your own, first neural network in PyTorch.
We also provide a crash-course for working with the Lisa cluster, and how to setup your account for Lisa.
November 5, 2021 | 11.00-13.00 | Lecture
This lectures introduces basic concepts for Deep Feedforward Networks such linear and nonlinear modules, gradient-based learning and the backpropagation algorithm.
November 11, 2021 | 11.00-13.00 | Lecture
This lecture series discusses advanced optimizers, initialization, normalization and hyperparameter tuning.
November 11, 2021 | 13.00-15.00 | Tutorial session + TA Q&A
In this tutorial, we will discuss the role of activation functions in a neural network, and take a closer look at the optimization issues a poorly designed activation function can have.
After the presentation, there will by a TA session for Q&A for assignment 1, lecture content and more.
November 12, 2021 | 11.00-13.00 | Lecture
This lecture series discusses advanced optimizers, initialization, normalization and hyperparameter tuning.
November 18, 2021 | 11.00-13.00 | Lecture
This lecture series covers convolutional neural networks for image processing.
November 18, 2020 | 13.00-15.00 | Tutorial session + TA Q&A
In this tutorial, we will discuss the importance of proper parameter initialization in deep neural networks, and how we can find a suitable one for our network. In addition, we will review the optimizers SGD and Adam, and compare them on complex loss surfaces.
After the presentation, there will by a TA session for Q&A for assignment 1, lecture content and more.
November 19, 2021 | 11.00-13.00 | Lecture
This lecture series covers modern ConvNet architecture.
November 25, 2021 | 11.00-13.00 | Lecture
This lecture series covers Recurrent Neural Networks
November 25, 2020 | 13.00-15.00 | Tutorial session + TA Q&A
In this tutorial, we will implement three popular, modern ConvNet architectures: GoogleNet, ResNet, and DenseNet. We will compare them on the CIFAR10 dataset, and discuss the advantages that made them popular and successful across many tasks.
After the presentation, there will by a TA session for Q&A for assignment 2, lecture content and more.
November 26, 2021 | 13.00-15.00 | Online Lecture
Petar Veličković lecture on Graph Neural Networks.
December 2, 2021 | 11.00-13.00 | Lecture
Ivona's lecture on Attention and Transformers.
December 2, 2020 | 13.00-15.00 | Tutorial session + TA Q&A
In this tutorial, we will discuss the implementation of Graph Neural Networks. In the first part of the tutorial, we will implement the GCN and GAT layer ourselves. In the second part, we use PyTorch Geometric to look at node-level, edge-level and graph-level tasks.
After the presentation, there will by a TA session for Q&A for assignment 2, lecture content and more.
December 3, 2021 | 15.00-17.00 | Lecture
This lecture series discusses Generative Adversarial Networks.
December 9, 2021 | 11.00-13.00 | Lecture
This lecture series introduces the framework of variational inference and Variational Autoencoders (VAEs).
December 9, 2020 | 13.00-15.00 | Tutorial session + TA Q&A
In this tutorial, we will discuss the relatively new breakthrough architecture: Transformers. We will start from the basics of attention and multi-head attention, and build our own Transformer. We will perform experiments on sequence-to-sequence tasks and set anomaly detection.
After the presentation, there will by a TA session for Q&A for assignment 3, lecture content and more.
December 10, 2021 | 15.00-17.00 | Online Lecture
Deep Learning Generalization.
December 16, 2021 | 11.00-13.00 | Lecture
Geometry and Structure Regularized Deep Learning.
December 16, 2020 | 13.00-15.00 | Tutorial session + TA Q&A
In this tutorial, we will discuss deep convolutional autoencoders and their applications. In the practical and lecture, you will see variational autoencoders (VAE), which add a stochastic part to vanilla autoencoders. Both have their advantages and applications, of which we visit image retrieval and compression for the vanilla auotoencoder.
After the presentation, there will by a TA session for Q&A for assignment 3, lecture content and more.
December 17, 2021 | 11.00-13.00 | Lecture
Details will follow soon.


Data Analysis: Moral Foundations Theory
Professor Amy Tick
Moral Foundations Theory (MFT) hypothesizes that people’s sensitivity to the foundations is different based on their political ideology: liberals are more sensitive to care and fairness, while conservatives are equally sensitive to all five. Here, we’ll explore whether we can find evidence for MFT in the campaign speeches of 2016 United States presidential candidates. For our main analysis, we’ll go through the data science process start to finish to recreate a simplified version of the analysis done by Jesse Graham, Jonathan Haidt, and Brian A. Nosek in their 2009 paper “Liberals and Conservatives Rely on Different Sets of Moral Foundations”. Finally, we’ll explore other ways to visualize and use this data in rhetorical analysis.
Estimated Time: 50 minutes
Topics Covered
- Word count using a dictionary
- Data visualization with pandas
- Graph interpretations
Table of Contents
1 - Data Set and Test Statistic
1.1 - 2016 Campaign Speeches
1.2 - Moral Foundations Dictionary
2 - Data Analysis
2.1 - Calculating Perceptages
2.2 - Filtering Table Rows
2.3 - Democrats
2.4 - Republicans
2.5 - Democrats vs Republicans
4 - Assignment: Run Analysis with Your Dictionary
Dependencies:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
import json
from nltk.stem.snowball import SnowballStemmer
import os
import re
Intro: The Data Science Process
Module 01 defined data science as an interdisciplinary field, combining statistics, computer science, and domain expertise to understand the world and solve problems. The data science process can be thought of like this:
This module walks through a simplified version of the process to explore speech data and probe Moral Foundations Theory. Steps done in this module are in bold.
- Raw Data Collection: speech data is collected into csv files via web-scraping.
- Data Processing/Cleaning: speech data is transformed to enable analysis. Some processing/cleaning has already been done.
- Exploratory Data Analysis: transform, visualize, and summarize data with the goal of understanding the data set, finding possible issues, and looking for potential questions to explore further.
- Models and Algorithms: develop and test a model- a theory of how the data was generated (in this case, Moral Foundations Theory).
- Communicate, Visualize, Report: to be discussed in Module 03.
Part 1: Speech Data and Foundations Dictionary
In Part 1, we’ll get familiar with our data set and determine a way to answer questions using the data.
2016 Campaign Speeches
Run the cell below to load the data.
# load the data from csv files into a table.
speeches = pd.read_csv('campaign_2016.csv', index_col=0)
# show the first 5 rows of the table
speeches.head()
| Candidate | Party | Type | Date | Title | Speech | |
|---|---|---|---|---|---|---|
| 0 | Jeb Bush | R | c | June 15, 2015 | Remarks Announcing Candidacy for President at ... | Thank you all very much. I always feel welcome... |
| 1 | Jeb Bush | R | c | July 30, 2015 | Remarks to the National Urban League Conferenc... | Thank you all very much. I appreciate your hos... |
| 2 | Jeb Bush | R | c | August 11, 2015 | Remarks at the Ronald Reagan Presidential Libr... | Thank you very much. It's good to be with all ... |
| 3 | Jeb Bush | R | c | September 9, 2015 | Remarks in Garner, North Carolina | Thank you very much. I appreciate your hospita... |
| 4 | Jeb Bush | R | c | November 2, 2015 | Remarks in Tampa, Florida | Thank you. It's great to be in Tampa with so m... |
Take a moment to look at this table. Before doing any analysis, it’s important to understand:
- the size of the table (how much data does it contain?)
- the structure of the table (how is the data organized?)
- what information it contains (what are the aspects of each record described in columns? what does each record (row) represent?)
# use this cell to expore the speeches DataFrame
# the `shape` attribute is useful to get the number of rows and columns
speeches.shape
(430, 6)
Moral Foundations Dictionary
In “Liberals and Conservatives Rely on Different Sets of Moral Foundations”, one of the methods Graham, Haidt, and Nosek use to measure people’s use of Moral Foundations Theory is to count how often they use words related to each foundation. This will be our test statistic for today. To calculate it, we’ll need a dictionary of words related to each moral foundation.
The dictionary we’ll use today comes from a database called WordNet, in which “nouns, verbs, adjectives and adverbs are grouped into sets of cognitive synonyms (synsets), each expressing a distinct concept.” By querying WordNet for semantically related words, it was possible to build a dictionary automatically using a Python program.
Run the cell below to load the dictionary and assign it to the variable ‘mft_dict’.
# Load a dictionary into the mft_dict variable
# The path is the argument for the open function. It gives the location of the dictionary file.
# To use the Wordnet dictionary from the Module 02 lecture, set the path to '../mft_data/foundations_dict.json'
# To use your hand-coded dictionary, set the path to '../mft_data/my_dict.json'
with open('../mft_data/foundations_dict.json') as json_data:
mft_dict = json.load(json_data)
# Stem the words in your dictionary (this will help you get more matches)
stemmer = SnowballStemmer('english')
for foundation in mft_dict.keys():
curr_words = mft_dict[foundation]
stemmed_words = [stemmer.stem(word) for word in curr_words]
mft_dict[foundation] = stemmed_words
We can see the keys of the dictionary using the .keys() function:
keys = mft_dict.keys()
list(keys)
['authority/subversion',
'care/harm',
'fairness/cheating',
'liberty/oppression',
'loyalty/betrayal',
'sanctity/degradation']
And we can look up the entries associated with a key by putting the key in brackets:
mft_dict
{'authority/subversion': ['respect',
'esteem',
'regard',
'subver',
'say-so',
'offic',
'disrespect',
'valu',
'obedi',
'assur',
'honor',
'disesteem',
'agenc',
'corrupt',
'honour',
'domin',
'author',
'observ',
'confid',
'defer',
'bureau',
'authori',
'sure',
'sanction'],
'care/harm': ['hurt',
'scath',
'precaut',
'concern',
'attent',
'damag',
'care',
'manag',
'impair',
'worri',
'harm',
'trauma',
'guardianship',
'aid',
'tend',
'caution',
'forethought',
'tutelag',
'injuri',
'upkeep',
'mainten',
'charg'],
'fairness/cheating': ['equiti',
'fair',
'cuckold',
'unsportsmanlik',
'screw',
'dirti',
'candour',
'cheat',
'proport',
'balanc',
'inequ',
'chican',
'betray',
'candor',
'adult',
'chous',
'unsport',
'unfair',
'two-tim',
'foul',
'shaft',
'fair-mind'],
'liberty/oppression': ['self-direct',
'self-suffici',
'autonomi',
'conquest',
'burdensom',
'independ',
'subjug',
'oner',
'oppress',
'subject',
'self-r',
'liberti',
'conquer',
'heavi'],
'loyalty/betrayal': ['traitor',
'disloyalti',
'treason',
'betray',
'commit',
'dedic',
'commit',
'consign',
'perfidi',
'truth',
'subver',
'allegi',
'trueness',
'veriti',
'inscript',
'treacheri',
'fealti',
'loyalti',
'committ',
'falsiti'],
'sanctity/degradation': ['pure',
'guilt',
'respect',
'impur',
'reward',
'disrespect',
'deba',
'honor',
'sanctitud',
'white',
'sanctiti',
'honour',
'holi',
'degrad',
'adult',
'dross',
'observ',
'innoc',
'natur',
'ingenu',
'aba',
'dishonor',
'puriti',
'abject',
'unholi',
'sinless',
'humili']}
Try looking up the entries for the other keys by filling in for ‘…’ in the cell below.
# look up a key in mft_dict
...
There’s something odd about some of the entries: they’re not words! The entries in this dictionary have been stemmed, meaning they have been reduced to their smallest meaningful root.
We can see why this is helpful with an example. Python can count the number of times a string can be found in another string using the string method ‘count’:
# Counts the number of times the second string appears in the first string
"Data science is the best major, says data scientist.".count('science')
1
It returns one match, for the second word. But, ‘scientist’ is very closely related to ‘science’, and many times we will want to match them both. A stem allows Python to find all words with a common root. Try running the count again with a stem that matches both ‘science’ and ‘scientist’.
# Fill in the parenthesis with a stem that will match both 'science' and 'scientist'
"Data science is the best major, says data scientist.".count('...')
0
Another thing you might have noticed is that all the entries in our dictionary are lowercase. This could be a problem when we do our text analysis. Try counting the number of times ‘rhetoric’ appears in the example sentence.
# Fill in the parenthesis to count how often 'rhetoric' appears in the sentence
"Rhetoric major says back: NEVER argue with a rhetoric student.".count('...')
0
We can clearly see the word ‘rhetoric’ appears twice, but the count function only returns 1. That’s because Python differentiates between capital and lowercase letters:
'r' is 'R'
False
To get around this, we can use the .lower() function, which changes all letters in the string to lowercase:
"Rhetoric major says back: NEVER argue with a rhetoric student.".lower()
'rhetoric major says back: never argue with a rhetoric student.'
Let’s add a column to our ‘speeches’ table that contains the lowercase text of the speeches. The clean_text
function lowers the case of the text in addition to implementing some of the text cleaning methods seen in Module 01, like removing the punctuation and splitting the text into individual words.
def clean_text(text):
# remove punctuation using a regular expression (not covered in these modules)
p = re.compile(r'[^\w\s]')
no_punc = p.sub(' ', text)
# convert to lowercase
no_punc_lower = no_punc.lower()
# split into individual words
clean = no_punc_lower.split()
return clean
speeches['clean_speech'] = [clean_text(s) for s in speeches['Speech']]
speeches.head()
| Candidate | Party | Type | Date | Title | Speech | clean_speech | |
|---|---|---|---|---|---|---|---|
| 0 | Jeb Bush | R | c | June 15, 2015 | Remarks Announcing Candidacy for President at ... | Thank you all very much. I always feel welcome... | [thank, you, all, very, much, i, always, feel,... |
| 1 | Jeb Bush | R | c | July 30, 2015 | Remarks to the National Urban League Conferenc... | Thank you all very much. I appreciate your hos... | [thank, you, all, very, much, i, appreciate, y... |
| 2 | Jeb Bush | R | c | August 11, 2015 | Remarks at the Ronald Reagan Presidential Libr... | Thank you very much. It's good to be with all ... | [thank, you, very, much, it, s, good, to, be, ... |
| 3 | Jeb Bush | R | c | September 9, 2015 | Remarks in Garner, North Carolina | Thank you very much. I appreciate your hospita... | [thank, you, very, much, i, appreciate, your, ... |
| 4 | Jeb Bush | R | c | November 2, 2015 | Remarks in Tampa, Florida | Thank you. It's great to be in Tampa with so m... | [thank, you, it, s, great, to, be, in, tampa, ... |
Part 2: Exploratory Data Analysis
Now that we have our speech data and our dictionary, we can start our exploratory analysis. The exploratory analysis in this module will be more focused than in most cases since we already have a model in mind- Moral Foundations Theory.
To get a sense of how Moral Foundations words were used in campaign speeches, we’ll do three things:
- Count the occurances of words from our dictionary in each speech
- Calculate how often words from each category are used by each political party
- Plot the percents on a bar graph
Think about what you know about Moral Foundations Theory. If this data is consistent with the theory, what should our analysis show for Republican candidates? What about for Democratic candidates? Try sketching a possible graph for each political party, assuming that candidates’ speech aligns with the theory.
Calculating Percentages
We’re interesting in knowing the percent of words that correspond to a Moral Foundation in speeches- in other words, how often candidates use words related to a specific foundation.
(Bonus question: why don’t we just use the number of Moral Foundation words instead of the percent as our test statistic?)
To calculate the percent, we’ll first need the total number of words in each speech.
# create a new column called 'total_words'
speeches['total_words'] = [len(speech) for speech in speeches['clean_speech']]
speeches.head()
| Candidate | Party | Type | Date | Title | Speech | clean_speech | total_words | |
|---|---|---|---|---|---|---|---|---|
| 0 | Jeb Bush | R | c | June 15, 2015 | Remarks Announcing Candidacy for President at ... | Thank you all very much. I always feel welcome... | [thank, you, all, very, much, i, always, feel,... | 2284 |
| 1 | Jeb Bush | R | c | July 30, 2015 | Remarks to the National Urban League Conferenc... | Thank you all very much. I appreciate your hos... | [thank, you, all, very, much, i, appreciate, y... | 2638 |
| 2 | Jeb Bush | R | c | August 11, 2015 | Remarks at the Ronald Reagan Presidential Libr... | Thank you very much. It's good to be with all ... | [thank, you, very, much, it, s, good, to, be, ... | 3735 |
| 3 | Jeb Bush | R | c | September 9, 2015 | Remarks in Garner, North Carolina | Thank you very much. I appreciate your hospita... | [thank, you, very, much, i, appreciate, your, ... | 1880 |
| 4 | Jeb Bush | R | c | November 2, 2015 | Remarks in Tampa, Florida | Thank you. It's great to be in Tampa with so m... | [thank, you, it, s, great, to, be, in, tampa, ... | 2550 |
Next, we need to calculate the number of matches to entries in our dictionary for each foundation for each speech.
Run the next cell to add six new columns to speeches
, one per foundation, that show the number of word matches.
#Note: much of the following code is not covered in these modules. Read the comments to get a sense of what it does.
# do the following code for each foundation
for foundation in mft_dict.keys():
# create a new, empty column
num_match_words = np.zeros(len(speeches))
stems = mft_dict[foundation]
# do the following code for each foundation word
for stem in stems:
# find synonym matches
wd_count = np.array([sum([wd.startswith(stem) for wd in speech]) for speech in speeches['clean_speech']])
# add the number of matches to the total
num_match_words += wd_count
# create a new column for each foundation with the number of foundation words per speech
speeches[foundation] = num_match_words
speeches.head()
| Candidate | Party | Type | Date | Title | Speech | clean_speech | total_words | authority/subversion | care/harm | fairness/cheating | liberty/oppression | loyalty/betrayal | sanctity/degradation | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Jeb Bush | R | c | June 15, 2015 | Remarks Announcing Candidacy for President at ... | Thank you all very much. I always feel welcome... | [thank, you, all, very, much, i, always, feel,... | 2284 | 4.0 | 4.0 | 3.0 | 0.0 | 7.0 | 4.0 |
| 1 | Jeb Bush | R | c | July 30, 2015 | Remarks to the National Urban League Conferenc... | Thank you all very much. I appreciate your hos... | [thank, you, all, very, much, i, appreciate, y... | 2638 | 8.0 | 2.0 | 7.0 | 0.0 | 4.0 | 9.0 |
| 2 | Jeb Bush | R | c | August 11, 2015 | Remarks at the Ronald Reagan Presidential Libr... | Thank you very much. It's good to be with all ... | [thank, you, very, much, it, s, good, to, be, ... | 3735 | 12.0 | 5.0 | 1.0 | 0.0 | 4.0 | 5.0 |
| 3 | Jeb Bush | R | c | September 9, 2015 | Remarks in Garner, North Carolina | Thank you very much. I appreciate your hospita... | [thank, you, very, much, i, appreciate, your, ... | 1880 | 3.0 | 1.0 | 1.0 | 0.0 | 1.0 | 4.0 |
| 4 | Jeb Bush | R | c | November 2, 2015 | Remarks in Tampa, Florida | Thank you. It's great to be in Tampa with so m... | [thank, you, it, s, great, to, be, in, tampa, ... | 2550 | 8.0 | 3.0 | 1.0 | 1.0 | 0.0 | 7.0 |
To calculate the percentage of foundation words per speech, divide the number of matched words by the number of total words and multiply by 100.
for foundation in mft_dict.keys():
speeches[foundation] = (speeches[foundation] / speeches['total_words']) * 100
speeches.head()
| Candidate | Party | Type | Date | Title | Speech | clean_speech | total_words | authority/subversion | care/harm | fairness/cheating | liberty/oppression | loyalty/betrayal | sanctity/degradation | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Jeb Bush | R | c | June 15, 2015 | Remarks Announcing Candidacy for President at ... | Thank you all very much. I always feel welcome... | [thank, you, all, very, much, i, always, feel,... | 2284 | 0.175131 | 0.175131 | 0.131349 | 0.000000 | 0.306480 | 0.175131 |
| 1 | Jeb Bush | R | c | July 30, 2015 | Remarks to the National Urban League Conferenc... | Thank you all very much. I appreciate your hos... | [thank, you, all, very, much, i, appreciate, y... | 2638 | 0.303260 | 0.075815 | 0.265353 | 0.000000 | 0.151630 | 0.341168 |
| 2 | Jeb Bush | R | c | August 11, 2015 | Remarks at the Ronald Reagan Presidential Libr... | Thank you very much. It's good to be with all ... | [thank, you, very, much, it, s, good, to, be, ... | 3735 | 0.321285 | 0.133869 | 0.026774 | 0.000000 | 0.107095 | 0.133869 |
| 3 | Jeb Bush | R | c | September 9, 2015 | Remarks in Garner, North Carolina | Thank you very much. I appreciate your hospita... | [thank, you, very, much, i, appreciate, your, ... | 1880 | 0.159574 | 0.053191 | 0.053191 | 0.000000 | 0.053191 | 0.212766 |
| 4 | Jeb Bush | R | c | November 2, 2015 | Remarks in Tampa, Florida | Thank you. It's great to be in Tampa with so m... | [thank, you, it, s, great, to, be, in, tampa, ... | 2550 | 0.313725 | 0.117647 | 0.039216 | 0.039216 | 0.000000 | 0.274510 |
Filtering table rows
To examine the data for a particular political party, it is necessary to filter out rows of our table that correspond to speeches from the other party, something we can do with Boolean indexing.
A Boolean is a Python data type. There are exactly two: True
and False
. A Boolean expression is an expression that evaluates to True
or False
. Boolean expressions are often conditions on two variables; that is, they ask how one variable compares to another (e.g. is a
greater than b
? Does a
equal c
?).
# These are all Booleans
True
not False
6 > 0
"Ted Cruz" == "zodiac killer"
False
Note that Python uses ==
to check if two things are equal. This is because the =
sign is already used for variable assignement.
Filtering out DataFrame rows can be broken into three steps:
- identify the correct feature column
- specify the desired condition for that column
- index the Dataframe with that condition in square brackets
Here’s an example of how to create a new table with only Bernie Sanders’ speeches.
# find the column
speech_col = speeches['Candidate']
# specify the condition
sanders_condition = speech_col == 'Bernie Sanders'
# index the original DataFrame by the condition
sanders_speeches = speeches[sanders_condition]
sanders_speeches.head()
| Candidate | Party | Type | Date | Title | Speech | clean_speech | total_words | authority/subversion | care/harm | fairness/cheating | liberty/oppression | loyalty/betrayal | sanctity/degradation | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 5877 | Bernie Sanders | D | c | April 30, 2015 | Interview with Ed Schultz of MSNBC Regarding S... | Schultz: A gentleman who has appeared on the p... | [schultz, a, gentleman, who, has, appeared, on... | 3645 | 0.329218 | 0.082305 | 0.054870 | 0.027435 | 0.109739 | 0.109739 |
| 5878 | Bernie Sanders | D | c | April 30, 2015 | Interview with Wolf Blitzer of CNN Regarding S... | Blitzer: I want to move to politics right now ... | [blitzer, i, want, to, move, to, politics, rig... | 1764 | 0.566893 | 0.000000 | 0.056689 | 0.170068 | 0.000000 | 0.453515 |
| 5879 | Bernie Sanders | D | c | April 30, 2015 | Interview with Andrea Mitchell of MSNBC Regard... | Sanders (from video clip): I believe that in a... | [sanders, from, video, clip, i, believe, that,... | 976 | 0.204918 | 0.409836 | 0.204918 | 0.614754 | 0.000000 | 0.204918 |
| 5880 | Bernie Sanders | D | c | May 6, 2015 | Interview with Chris Cuomo of CNN's "New Day" | CUOMO: Senator Sanders, welcome to the race. G... | [cuomo, senator, sanders, welcome, to, the, ra... | 1561 | 0.128123 | 0.000000 | 0.192184 | 0.000000 | 0.064061 | 0.064061 |
| 5881 | Bernie Sanders | D | c | May 11, 2015 | Interview with Andrea Mitchell of MSNBC | MITCHELL: Vermont Senator and Democratic presi... | [mitchell, vermont, senator, and, democratic, ... | 910 | 0.219780 | 0.000000 | 0.000000 | 0.109890 | 0.000000 | 0.219780 |
Democrats
Let’s start by looking at Democratic candidates. First, we need to make a table that only contains Democrats using boolean indexing.
# Filter out non-Democrat speeches
party_col = speeches['Party']
dem_cond = party_col == 'D'
democrats = speeches[dem_cond]
democrats.head()
| Candidate | Party | Type | Date | Title | Speech | clean_speech | total_words | authority/subversion | care/harm | fairness/cheating | liberty/oppression | loyalty/betrayal | sanctity/degradation | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 436 | Lincoln Chafee | D | c | June 3, 2015 | Remarks Announcing Candidacy for President at ... | Thank you, Bob. Thank you, Bob and Mark, very ... | [thank, you, bob, thank, you, bob, and, mark, ... | 5512 | 0.235849 | 0.181422 | 0.090711 | 0.036284 | 0.145138 | 0.108853 |
| 437 | Lincoln Chafee | D | c | July 17, 2015 | Remarks at the Iowa Democrats Hall of Fame Din... | Congratulations to the Hall of Fame Inductees.... | [congratulations, to, the, hall, of, fame, ind... | 745 | 0.268456 | 0.134228 | 0.268456 | 0.000000 | 0.805369 | 0.268456 |
| 438 | Lincoln Chafee | D | c | October 23, 2015 | Remarks Announcing the End of Presidential Cam... | Once again it is a pleasure to join so many De... | [once, again, it, is, a, pleasure, to, join, s... | 939 | 0.212993 | 0.212993 | 0.000000 | 0.106496 | 0.425985 | 0.106496 |
| 570 | Hillary Clinton | D | c | January 20, 2007 | Video Transcript: Presidential Exploratory Com... | HILLARY CLINTON: I announced today that I am f... | [hillary, clinton, i, announced, today, that, ... | 349 | 0.286533 | 0.573066 | 0.000000 | 0.286533 | 1.719198 | 0.286533 |
| 571 | Hillary Clinton | D | c | January 22, 2007 | Remarks in a "Let the Conversation Begin Webcast" | SENATOR CLINTON: Hi, everyone, and welcome to ... | [senator, clinton, hi, everyone, and, welcome,... | 5349 | 0.355207 | 0.261731 | 0.037390 | 0.018695 | 0.149561 | 0.093475 |
We have our percentages for the Democratic party, but it’s much easier to understand what’s going on when the results are in graph form. Let’s start by looking at the average percents for Democrats as a group.
# select the foundations columns and calculate the mean percent for each
avg_dem_stats = (democrats.loc[:, list(mft_dict.keys())]
.apply(np.mean)
.to_frame('D_percent'))
avg_dem_stats
| D_percent | |
|---|---|
| authority/subversion | 0.321495 |
| care/harm | 0.292418 |
| fairness/cheating | 0.082472 |
| liberty/oppression | 0.030215 |
| loyalty/betrayal | 0.145686 |
| sanctity/degradation | 0.204513 |
Now, create a horizontal bar plot by calling the .plot.barh()
method on avg_dem_stats
.
avg_dem_stats.plot.barh()
<matplotlib.axes._subplots.AxesSubplot at 0x1271e9860>
Take a look at this graph. What does it show? How does it compare with the predictions of MFT?
Republicans
Now, let’s repeat the process for Republicans. Replace the ellipses with the correct code to select only Republican speeches, then run the cell to create the table.
(Hint: look back at how we made the ‘democrats’ table to see how to fill in the ellipses)
# Filter out non-Republican speeches
# select 'Party' column from 'speeches'
party_col = speeches['Party']
# create a condition (boolean expression) that checks if a party is Republican
republican_cond = party_col == 'R'
# index `speeches` using `republican_cond`
republicans = speeches[republican_cond]
# uncomment the next line to show the first 5 rows of the `republican` DataFrame
republicans.head()
| Candidate | Party | Type | Date | Title | Speech | clean_speech | total_words | authority/subversion | care/harm | fairness/cheating | liberty/oppression | loyalty/betrayal | sanctity/degradation | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Jeb Bush | R | c | June 15, 2015 | Remarks Announcing Candidacy for President at ... | Thank you all very much. I always feel welcome... | [thank, you, all, very, much, i, always, feel,... | 2284 | 0.175131 | 0.175131 | 0.131349 | 0.000000 | 0.306480 | 0.175131 |
| 1 | Jeb Bush | R | c | July 30, 2015 | Remarks to the National Urban League Conferenc... | Thank you all very much. I appreciate your hos... | [thank, you, all, very, much, i, appreciate, y... | 2638 | 0.303260 | 0.075815 | 0.265353 | 0.000000 | 0.151630 | 0.341168 |
| 2 | Jeb Bush | R | c | August 11, 2015 | Remarks at the Ronald Reagan Presidential Libr... | Thank you very much. It's good to be with all ... | [thank, you, very, much, it, s, good, to, be, ... | 3735 | 0.321285 | 0.133869 | 0.026774 | 0.000000 | 0.107095 | 0.133869 |
| 3 | Jeb Bush | R | c | September 9, 2015 | Remarks in Garner, North Carolina | Thank you very much. I appreciate your hospita... | [thank, you, very, much, i, appreciate, your, ... | 1880 | 0.159574 | 0.053191 | 0.053191 | 0.000000 | 0.053191 | 0.212766 |
| 4 | Jeb Bush | R | c | November 2, 2015 | Remarks in Tampa, Florida | Thank you. It's great to be in Tampa with so m... | [thank, you, it, s, great, to, be, in, tampa, ... | 2550 | 0.313725 | 0.117647 | 0.039216 | 0.039216 | 0.000000 | 0.274510 |
Then, calculate the averages.
# select the foundations columns and calculate the mean percent for each
avg_rep_stats = (republicans.loc[:, list(mft_dict.keys())]
.apply(np.mean)
.to_frame('R_percent'))
avg_rep_stats
| R_percent | |
|---|---|
| authority/subversion | 0.393636 |
| care/harm | 0.176955 |
| fairness/cheating | 0.066454 |
| liberty/oppression | 0.039025 |
| loyalty/betrayal | 0.080041 |
| sanctity/degradation | 0.191499 |
Finally, create a bar plot of avg_rep_stats
using the .plot.barh()
method.
# your code here
avg_rep_stats.plot.barh()
<matplotlib.axes._subplots.AxesSubplot at 0x12cd937b8>
How does this plot compare with Moral Foundations Theory predictions?
Democrats vs Republicans
Comparing two groups becomes much easier when they are plotted on the same graph.
First, combine avg_dem_stats
and avg_rep_stats
into one DataFrame with the join
function. join
is called on one table using .join()
, takes the other table as its argument (in the parentheses), and returns a table with the indices matched.
Here’s an example of a simple join:
peanut_butter = pd.DataFrame(data=[2.99, 3.49], index = ['Trader Joes', 'Safeway'], columns=['pb_price'])
peanut_butter
| pb_price | |
|---|---|
| Trader Joes | 2.99 |
| Safeway | 3.49 |
jelly = pd.DataFrame(data=[4.99, 3.59], index = ['Trader Joes', 'Safeway'], columns=['jelly_price'])
jelly
| jelly_price | |
|---|---|
| Trader Joes | 4.99 |
| Safeway | 3.59 |
jelly.join(peanut_butter)
| jelly_price | pb_price | |
|---|---|---|
| Trader Joes | 4.99 | 2.99 |
| Safeway | 3.59 | 3.49 |
Now, write the code to join avg_dem_stats
with avg_rep_stats
.
# fill in the ellipses with your code
all_avg_stats = avg_dem_stats.join(avg_rep_stats)
all_avg_stats
| D_percent | R_percent | |
|---|---|---|
| authority/subversion | 0.321495 | 0.393636 |
| care/harm | 0.292418 | 0.176955 |
| fairness/cheating | 0.082472 | 0.066454 |
| liberty/oppression | 0.030215 | 0.039025 |
| loyalty/betrayal | 0.145686 | 0.080041 |
| sanctity/degradation | 0.204513 | 0.191499 |
Then, make a horizontal bar plot for `all_avg_stats’.
# your code here
all_avg_stats.plot.barh()
<matplotlib.axes._subplots.AxesSubplot at 0x11325f3c8>
It can be hard to make comparison judgments if the bar lengsth are very similar. The next cell creates a plot of only the difference in average foundation word usage of Democrats and Republicans. A positive value means Democrats use the word more frequently; a negative value indicates Republicans use it more frequently.
# uncomment the next two lines to plot the difference in percent of foundations words per speech by party
party_diffs = pd.DataFrame(data = avg_dem_stats['D_percent'] - avg_rep_stats['R_percent'],
columns = ["dem_rep_pct_diff"],
index = mft_dict.keys())
party_diffs.plot.bar()
<matplotlib.axes._subplots.AxesSubplot at 0x12eebbac8>
Part 3: Additional Visualizations
Many different graphs can be generated from the same data set to facilitate different comparisons. For example, we can compare the average use of foundation words by individual Democrats…
dem_indivs = (democrats.loc[:, list(mft_dict.keys()) + ['Candidate']]
.groupby('Candidate')
.mean())
dem_indivs.plot.barh(figsize=(8, 8))
<matplotlib.axes._subplots.AxesSubplot at 0x107d55ac8>
…or individual Republicans.
rep_indivs = (republicans.loc[:, list(mft_dict.keys()) + ['Candidate']]
.groupby('Candidate')
.mean())
rep_indivs.plot.barh(figsize=(8, 20))
<matplotlib.axes._subplots.AxesSubplot at 0x12f041cf8>
We can also examine how a candidate uses foundation words over time. The following plot shows foundation word usage for Donald Trump in the weeks leading up to the election.
# select Trump's speeches and drop unnecessary columns
trump = (republicans[republicans['Candidate'] == "Donald Trump"]
.loc[:, list(mft_dict.keys()) + ['Date']])
# set the speech dates as the table index
trump['Date'] = pd.to_datetime(trump['Date'])
trump = (trump.set_index('Date')
.loc['2016-07-01':])
# plot the data
trump.plot(figsize = (10, 6))
<matplotlib.axes._subplots.AxesSubplot at 0x12f716f60>
What other kinds of plots could be generated from this data? What other questions might we be able to explore with these or other plots?
Part 4: Run Analysis with Your Dictionary
One of the advantages of coding is how easy it is to repeat one method of analysis with different parameters. For instance, changing a single line of code means that all of the word counts, proportions, and graphs in the above sections can be recalculated using a different dictionary of Moral Foundations words.
To change what dictionary is loaded to the mft_dict
variable, go to Part 1.2: Moral Foundations Dictionary
and follow the instructions in the first code cell.
Once the dictionary load code has been changed, the easiest way to regenerate all the tables, percents, and graphs is to go to the Cell
menu and click Run all
. This ensures that all the statistics used to make the graphs will be recalculated with the new dictionary.
For this assignment, answer the following three questions about the graphs made using your hand-coded dictionary:
- What does each graph show?
- How are these graphs different from the ones made using the Wordnet dictionary?
- Do these graphs support Moral Foundations Theory?
Bibliography
- Election documents scraped from http://www.presidency.ucsb.edu/2016_election.php
- Graham, J., Haidt, J., & Nosek, B. A. (2009). Liberals and conservatives rely on different sets of moral foundations. Journal of personality and social psychology, 96(5), 1029. http://projectimplicit.net/nosek/papers/GHN2009.pdf, October 9 2017.
Notebook developed by: Keeley Takimoto, Sean Seungwoo Son, Sujude Dalieh
Data Science Modules: http://data.berkeley.edu/education/modules


This guide will lead you through the steps required to get AppSensor configured on your system. You'll have the server-side analysis components ready so you can start doing real-time application intrusion detection.
At the end of this process, you'll be ready to move on to the user guide in order to prepare your application(s) to interact with the AppSensor system.
The first step is to download AppSensor. You can conveniently get access through any of the following mechanisms.
The current mechanism for retrieving binary files is to download them directly from the maven central repository. You can find all subprojects at AppSensor at Maven Central Please use version 2.0.0 or higher.
The reference implementation currently supports 5 execution modes and we'll discuss each in turn.
The REST model is the most common model for APIs today, and it is easy to understand why. The model is simple, lightweight and effective, while still allowing for flexibility. It's also supported by almost every networking framework available, making integration effort much easier.
Choose this model if any/all of the following are true:
Avoid this model if any/all of the following are true:
The Thrift model uses Thrift, which is a binary protocol that allows fast communication and supports many languages.
Choose this model if any/all of the following are true:
Avoid this model if any/all of the following are true:
The SOAP model is still in heavy use, but more often within the enterprise. There is significant tooling available. It's a well-known and well understood protocol. Integration points are often available, and there are a fair number of supporting frameworks.
Choose this model if any/all of the following are true:
Avoid this model if any/all of the following are true:
The RabbitMQ model uses RabbitMQ, which is an AMQP implementation for doing messaging that supports a variety of deployment patterns and languages.
Choose this model if any/all of the following are true:
Avoid this model if any/all of the following are true:
The Kafka model uses Kafka, which is a ""publish-subscribe messaging system rethought as a distributed commit log"". It supports a variety of deployment patterns and languages.
Choose this model if any/all of the following are true:
Avoid this model if any/all of the following are true:
The local model simply means that the client and server portions of appsensor are "embedded" within a single application. When an event is triggered, all of the communication occurs within the JVM as native calls, as opposed to network requests. This model was the only one available in AppSensor version 1. There are drawbacks to this approach, but it is still a good model for some.
Choose this model if any/all of the following are true:
Avoid this model if any/all of the following are true:
AppSensor v2 has been built with pluggability in mind. Ever major component of the system can be swapped out, including the analysis engines themselves. While this allows for great flexibility, it comes with the cost that you have to perform some up-front configuration in order to get going. For most situations, this will be a trivial one-time step.
The following section describes each of the components you'll interact with in order to get your system setup:
Core represents the central domain and interface specifications for the system. You'll need this, but you often don't have to specify it as an explicit dependency, since pretty much every other part of the system depends on this component as well.
The analysis engines are the core implementation components. These components decide when an event becomes an attack and how to respond accordingly.
There is currently only the reference implementation (policy driven), but there could be more in the future.
The storage component does what you'd expect: stores the data for the system.
There are several implementations, including in-memory, file-based, and sql/no-sql databases. You'll need to select a storage implementation based on your needs.
The configuration component of the system represents your client (connect to server) and server (client info, detection points) configurations. While there are other possible implementations, the only current implementation is XML (Stax). You'll need to include this component in order to allow your configuration information to be processed properly.
The access controller component simply ensures that the client that is making requests has permission to do so.
There is currently only a single access controller implementation (reference), which implements a simple role-based access control check.
This component is necessary if you are using either the REST or SOAP configurations.
Reporting is an important part of the AppSensor capability. The analysis engine on the server interacts with the client to allow responses to be sent and processed by the client, but that's the extent of data provided. In order to get any other information about the system, you'll want to use the reporting subsystem.
There are multiple implementations of the reporting component (logging, websocket, REST/SOAP), and you can select one appropriate to your environment.
While many of the implementations available in AppSensor do not require further configuration, some will.
Some require configuration and for others, configuration is optional. An example is in the storage component:
There are various components that have configuration options. The necessary specific information regarding these components can be found in the documentation on GitHub or in the JavaDoc for that component.
While reporting is technically just another component of AppSensor, it's a common need, so I'll call it out separately here.
There are various implementations of the reporting subsystem, such as:
Setting up reporting is an important part of a complete AppSensor system. The reporting components as well as sample dashboards are included in the source at GitHub.
Congratulations ... If you've finished this guide, you've got AppSensor setup! Now, you'll want to get your application configured to use AppSensor. Have a look at the user guide to start that process.
See the User Guide »


Delete
Are you sure you want to delete this cart?
Request Firm Quote
Are you sure you want to request a firm quote for this cart?
My Carts
My Firm Quote Requests


Judge rebuffs Trump’s bid to delay woman’s defamation suit
by The Associated PressNEW YORK — President Donald Trump has lost his latest bid to delay a former “Apprentice” contestant’s defamation suit as he faces a Jan. 31 deadline to undergo sworn pretrial questioning.
A New York judge rejected Trump’s request Monday, meaning that Summer Zervos’ case can continue. Trump can pursue other legal avenues to try to hold it off.
Zervos’ lawyer, Mariann Wang, declined to comment. A request for comment was sent to Trump’s lawyers.
Zervos appeared on “The Apprentice” in 2006.  His lawyers have argued his statements weren’t defamatory.
Trump’s attorneys have repeatedly tried to get the case dismissed, or postponed until after his presidency.
The Associated Press


Bootstrap to WordPress
Earn An Extra $1k - $5k a Month by Learning to Code Your Very Own Responsive & Custom WordPress Website with Bootstrap
Earn An Extra $1k - $5k a Month by Learning to Code Your Very Own Responsive & Custom WordPress Website with Bootstrap
Subscribe to our mailing list. We'll send something special as a thank you.

WordPress developers have options. Many developers make a generous living off of creating custom WordPress themes and selling them on websites like ThemeForest. Freelance designers and developers can also take on WordPress projects and make an extra $1,000 - $5,000+ per month.
Graphic designers are extremely talented, but ask them to code their designs and they'll freeze up! This leaves them with no other choice but to hire a web developer. Any professional graphic designers knows web developers can be expensive.
If you’re a designer, learning to code your own WordPress websites can change your business entirely!  This puts you in a position to make an extra $1,000 - $5,000 per project.
Entrepreneurs have big dreams, and in many cases, shoestring budgets. In order to survive in the cut-throat world of the Startup company, it’s a necessity to have a world-class website. However, world-class websites come with a large price tag.
If you can learn how to build a high-quality startup website by yourself, then you’ve just saved yourself a lot of cash, tens of thousands of dollars in many cases.
Any company knows the education & training of their employees is key to a thriving team.
Depending on the type of company you work for, if you understand how to code, and can develop CMS driven websites, that gives you negotiating power for a better position, or a higher salary.
It’s fun to learn challenging new skills. Code hobbyists can add dynamic websites to their arsenal of tools to play with — you can even sell WordPress themes and plugins for cash! The possibilities are truly endless.
Are you out of work? Looking for a more rewarding job? Desire a career that can allow you to work almost anywhere in the world? Becoming a Web Developer might be the answer for you.
Web developers are paid well, anywhere from $33,000 to more than $105,000 per year. They get to work at amazing companies that are changing the world, or they enjoy the ability to start their own companies, become location-independent and work from home, from coffee shops, in an airplane, on the beach, or wherever they want!
Throughout this entire course, you work towards building an incredibly beautiful website. Want to see the website you are going to build? You're looking at it! The website you're using right now is the website you will have built entirely by yourself, by the end of this course.
You get to work with a modern, professional quality design & layout.
You'll learn how hand-craft a stunning website with valid, semantic and beautiful HTML5 & CSS3.
Allow your clients to easily update their websites by converting your static websites to dynamic websites, using WordPress.
A highly skilled professional, Brad Hussey is a passionate and experienced web designer, developer, blogger and digital entrepreneur.
Hailing from North Of The Wall (Yellowknife, Canada), Brad made the trek to the Wet Coast (Vancouver, Canada) to educate and equip himself with the necessary skills to become a spearhead in his trade of solving problems on the web, crafting design solutions, and speaking in code.
Brad's determination and love for what he does has landed him in some pretty interesting places with some neat people. He's had the privilege of working with, and providing solutions for, numerous businesses, big & small, across the Americas.
Brad builds custom websites, and provides design solutions for a wide-array of clientele at his company, Brightside Studios. He regularly blogs about passive income, living your life to the fullest, and provides premium quality web design tutorials and courses for tens of thousands of amazing people desiring to master their craft.
These videos are well created, concise, fast-paced, easy to follow, and just funny enough to keep you chuckling as you're slamming out lines of code. I've taken 3 courses from this instructor. Whenever I have questions he is right there with a simple solution or a helpful suggestion to keep me going forward with the course work. — Brennan, graduate of all of Brad's courses
I found Brad to be a great teacher, and a very inspiring person. It's clear he is very passionate about helping designers learn to code, and I look forward to more courses from him! — Ben, graduate of Build a Website from Scratch with HTML & CSS
Brad is amazing and I honestly think he's the best tutor of all the courses I have taken on Udemy. Will definitely be following him in the future. Thanks Brad! — AJ, graduate of Code a Responsive Website with Bootstrap 3
Brad is an excellent instructor. His content is super high quality, and you can see the love and care put into every section. The tutorials are the perfect length, and you feel like your doing something right out the gate! I really can't believe this is free. I highly recommend taking advantage of this course. — Ernest, graduate of Code Dynamic Websites with PHP


edu.biu.scapi.comm.twoPartyComm
Interfaces
DestroyDestinationUtil
PartyData
TwoPartyCommunicationSetup
Classes
ActiveMQCommunicationSetup
ActiveMQCommunicationSetup.ActiveMQDestroyer
LoadQueueParties
LoadSocketParties
NativeChannel
NativeSocketCommunicationSetup
NativeSocketListenerThread
PlainTCPSocketChannel
PlainTCPSocketChannel.Message
QueueCommunicationSetup
QueuePartyData
SocketCommunicationSetup
SocketPartyData
SSLActiveMQCommunicationSetup
SSLSocketChannel
SSLSocketCommunicationSetup
TwoPartySocketConnector
Enums
NativeChannel.State