Simon Dräger<EMAIL_ADDRESS>
Jannik Dunkelau<EMAIL_ADDRESS>
Heinrich Heine University Düsseldorf, Germany

# Evaluating the Impact of Loss Function Variation in Deep Learning for
Classification

###### Abstract

The loss function is arguably among the most important hyperparameters for a
neural network. Many loss functions have been designed to date, making a
correct choice nontrivial. However, elaborate justifications regarding the
choice of the loss function are not made in related work. This is, as we see
it, an indication of a dogmatic mindset in the deep learning community which
lacks empirical foundation. In this work, we consider deep neural networks in
a supervised classification setting and analyze the impact the choice of loss
function has onto the training result. While certain loss functions perform
suboptimally, our work empirically shows that under-represented losses such as
the KL Divergence can outperform the State-of-the-Art choices significantly,
highlighting the need to include the loss function as a tuned hyperparameter
rather than a fixed choice.

###### keywords:

Deep Learning, classification, loss function

##  1 Introduction

In recent years, Deep Neural Networks (DNNs) such as Convolutional Neural
Networks (CNNs; [LeCun et al. (1989)]) have been used extensively for learning
tasks, including classification. In a supervised setting, neural networks
vitally require an understanding of goodness of their prediction which they
receive via the _loss function_. Many loss functions have been proposed so
far, some of which are tailored to specific problems [Wang et al. (2019),
Diakogiannis et al. (2020), Demirkaya et al. (2020), Feng et al. (2018), Lin
et al. (2017)] while we can more generally distinguish between loss functions
for _classification_ and loss functions for _regression_ [Muthukumar et al.
(2021)]. In this work, we are only concerned with the classification case.

Usually, researchers select a loss function and assume it throughout their
publication. We highlight the use of seemingly axiomatic statements such as
“We adopt Mean-Square Error (MSE) as the loss function [ …], who [sic] could
be rather sensitive to outliers.” [Wang et al. (2016)], “The _Cross Entropy_
is used as loss function [ …]” [Tabik et al. (2017)] and “For parameter
optimization, we use the Adam optimizer with cross-entropy loss function.” [An
et al. (2020)]. Some work [Chauhan et al. (2018), Ciresan et al. (2011)] is
cited hundred- and thousand-fold but does not mention which loss measure was
used.

The loss measure arguably does not receive extensive attention with regards to
optimization in deep learning research but is rather subject to a dogmatic
choice. We opine however, that this dogmatic mindset is misplaced in deep
learning due to it being deeply ingrained into the learning process [Yessou et
al. (2020)] Thus, we conjecture that a poor choice of loss measure leads to
converse effects like a bad loss surface, local optima and stalled or slowed
optimization. This can in turn lead to an increase in required epochs and
model complexity which pose a higher computational demand on the computing
infrastructure. In this analysis, we vary the loss function across our
experiments in order to explore the impact this has on the training outcome.

##  2 Considered Loss Functions

Consider the scenario of classifying data points from a data set
$\mathcal{D}=\\{(x_{i},y_{i})\\}_{i=1}^{N}$ into $c\in\mathbb{N}$ classes.
Using a neural network, a function $f_{\theta}$ that maps each data point onto
its corresponding class is learned over a set of weights $\theta$. To do so,
the predicted output and its class are compared by a loss function
$\mathcal{L}$ which indicates how good the prediction was. Subsequently, the
loss function is derived wrt. $\theta$ in order to move $\theta$ into the
direction of steepest descent [Rumelhart et al. (1985)]. Formally, we are
interested in

$\operatorname*{arg\,min}_{\theta\,\in\,\Theta}\
\operatorname*{\mathbb{E}}_{x,y}\quantity[\mathcal{L}(f_{\theta}(x),y)].$

However, we do not know the data’s joint distribution
$P(\mathcal{X},\mathcal{Y})$ and thus have to resort to the expected value’s
empirical formulation

$\operatorname*{arg\,min}_{\theta\,\in\,\Theta}\
\frac{1}{N}\sum\limits_{i=1}^{N}\mathcal{L}(f_{\theta}(x_{i}),y_{i}).$

In this work, we consider each label $y_{i}$ to be _one-hot encoded_ and thus
assume that $f_{\theta}(x_{i})\in\mathbb{R}^{c}$ returns a vector of
probabilities corresponding to the model’s confidence in each class which can
be achieved via the Softmax function.

Our analysis incorporates four loss functions (listed in Table 1) that have
been widely used in machine learning. Intuitively, these have different
properties, for example, the MAE is Lipschitz continuous with Lipschitz
constant $L=1$ [Qi et al. (2020)]. Besides that, it inherits the absolute
value function’s discontinuity at 0. Because the MSE squares its arguments
component-wise, large differences in the components lead to outliers being
penalized stronger. The MAE, on the other hand, is insensitive to outliers
because it weighs all data points equally [Chai and Draxler (2014)]. The CCE’s
gradients tend to be steep which is conjectured to be a reason it converges
quickly [Bosman et al. (2020)]. Both CCE and KLD are not symmetric whereas the
MSE and MAE are. The CCE and KLD have information theoretic interpretations:
according to Murphy (2012), the CCE is the mean number of bits required to
encode a source distribution (label distribution) using a model distribution
(prediction distribution). Furthermore, the KLD decomposes into

$\mathcal{L}_{\mathrm{KLD}}(\hat{y},y)=\mathcal{L}_{\mathrm{CCE}}(\hat{y},y)-\mathbb{H}(y)$
(1)

with $\mathbb{H}(\cdot)$ being the entropy Murphy (2012). Therefore, the KLD
is the mean number of _additional_ bits required to encode the data.

Table 1: Loss functions used in this work. Models using these are implemented, trained and evaluated in the Experiments section. Loss Function | Abbreviation | Expression
---|---|---
Mean Square Error Bickel and Doksum (2015) | MSE | $\sum\limits_{k=1}^{c}\left(\hat{y}^{(k)}-y^{(k)}\right)^{2}$
Mean Absolute Error Ghosh et al. (2017) | MAE | $\sum\limits_{k=1}^{c}\absolutevalue{\hat{y}^{(k)}-y^{(k)}}$
(Categorical) Cross-Entropy Murphy (2012) | (C)CE | $\sum\limits_{k=1}^{c}-y^{(k)}\log\left(\hat{y}^{(k)}\right)$
Kullback-Leibler Divergence Kullback and Leibler (1951) | KLD | $\sum\limits_{k=1}^{c}y^{(k)}\log\left(\frac{y^{(k)}}{\hat{y}^{(k)}}\right)$

##  3 Related Work

Similar to our work, Janocha and Czarnecki (2017) compared 12 loss functions
in a classification regime. They investigated synthetic data as well as MNIST
LeCun et al. (1989). The experimental results obtained by Janocha and
Czarnecki (2017) and Demirkaya et al. (2020) yielded intriguing insight into
loss functions for sufficiently complex data sets: less common, non-classical
losses such as the _Tanimoto loss_ Diakogiannis et al. (2020) and the
_Correct-Class Quadratic Loss_ (CCQL; Demirkaya et al. (2020)) achieved very
good performances. In an effort to improve the MAE, Wang et al. (2019)
proposed the Improved Mean Absolute Error (IMAE) which, compared to the MAE,
led to an increase in training accuracy by 18.8 % on CIFAR-10.

Despite the success in employing the IMAE, Tanimoto loss and CCQL, we focus on
the most popular loss functions which is why less popular (but well-
performing) functions are not incorporated in our analysis. However, we
applied our analysis onto more datasets to measure a more generalized impact.

##  4 Experiments

In this section, we present an array of experiments regarding the loss
functions from Table 1. In order to avoid results occurring by stochasticity,
we generate 6 outcomes (iterations) for each combination of data set and loss
function and compute the respective mean metrics. Figure 1 shows a conceptual
diagram of our approach. Training took place using 2 $\times$ NVIDIA RTX 8000
+ 12 $\times$ Intel Xeon + 32 GB RAM. All models are trained for 100 epochs
using the Adam optimizer Kingma and Ba (2014).

Data Set MNIST CharFontIm CIFAR-10 Fashion-MNIST Phishing Mushroom Model
Architecture CNN CNN CNN CNN MLP MLP Loss Function CCEMSE KLD MAE

Figure 1: Our experiments as a network diagram.

### Data Sets.

We used six data sets as presented in Table 2, three of which are popular
benchmarking data sets. The other three (Mushroom, Phishing and CharFontIm)
are known to a lesser extent. Mushroom and Phishing are tabular data sets. We
performed feature analysis via PCA Abdi and Williams (2010) and concluded that
for Mushroom, nearly 100 % of the data’s variance can be explained by 15
features. Thus, we reduced Mushroom to its 15 most important features, which
are, by index: 0, 1, 3, 6, 7, 8, 10, 11, 13, 14, 16, 17, 18, 19, 20.

Table 2: Summary of our data sets, including the train/validation/test split. The column PCA indicates whether any feature reduction via PCA took place; a squiggly arrow in the Features column indicates the reduction result. The Train Split column gives the percentages of the resulting split into train/validation/test set, respectively.  Data Set | $N$ | $c$ | Features | PCA | Batch Size | Train Split (%)
---|---|---|---|---|---|---
Mushroom Dua and Graff (2017) | 8,124 | 2 | 22 $\rightsquigarrow$ 15 |  | 16 | 50 / 10 / 40
Phishing Dua and Graff (2017) | 11,055 | 2 | 29 |  | 16 | 50 / 10 / 40
MNIST LeCun et al. (1989) | 70,000 | 10 | $28\times 28$ |  | 64 | 76 / 10 / 14
Fashion-MNIST Xiao et al. (2017) | 70,000 | 10 | $28\times 28$ |  | 64 | 76 / 10 / 14
CIFAR-10 Krizhevsky and Hinton (2009) | 60,000 | 10 | $32\times 32\times 3$ |  | 64 | 74 / 10 / 16
CharFontIm Dua and Graff (2017) | 745,000 | 153 | $20\times 20$ |  | 128 | 60 / 10 / 30

### Hyperparameters.

We perform a broad hyperparameter search across the learning rate ($\lambda$)
and $\ell_{2}$ weight decay ($\eta$). The model for each combination is
trained for 20 epochs. Consequently, we choose the parameters that led to the
highest validation accuracy. In total, we sample 30 candidates for $\lambda$
and $\eta$ from the log-uniform distribution over $[$110-5$,$110-1$]$.

Table 3: Hyperparameter configuration for each model and loss function. We round each parameter to the nearest most significant digit. $\lambda^{\ast}$ and $\eta^{\ast}$ denote the best learning rate and weight decay coefficient, respectively. | MSE | MAE | CCE | KLD
---|---|---|---|---
Data Set | $\lambda^{\ast}$ | $\eta^{\ast}$ | $\lambda^{\ast}$ | $\eta^{\ast}$ | $\lambda^{\ast}$ | $\eta^{\ast}$ | $\lambda^{\ast}$ | $\eta^{\ast}$
Mushroom | $310-3$ | $210-3$ | $310-3$ | $110-4$ | $910-4$ | $210-5$ | $510-4$ | $910-4$
Phishing | $710-5$ | $410-5$ | $110-5$ | $210-5$ | $310-5$ | $110-5$ | $710-4$ | $410-3$
MNIST | $210-3$ | $110-5$ | $210-4$ | $310-5$ | $110-5$ | $910-5$ | $510-4$ | $410-3$
Fashion-MNIST | $510-4$ | $210-4$ | $110-5$ | $210-5$ | $610-5$ | $310-3$ | $610-4$ | $210-4$
CIFAR-10 | $110-2$ | $210-5$ | $510-2$ | $110-2$ | $210-4$ | $210-3$ | $310-5$ | $310-5$
CharFontIm | $210-5$ | $310-3$ | $910-3$ | $310-2$ | $410-5$ | $110-3$ | $110-4$ | $410-5$

### Architectures.

Two convolutional architectures are obtained from related work Chauhan et al.
(2018); Çalik and Demirci (2018) while the one used on CharFontIm is our own.
We chose these because they performed reasonably well on CIFAR-10 and MNIST
(80.17 % and 99.6 % training accuracy, respectively). We use a Multi-Layer
Perceptron (MLP) architecture for the tabular data sets, inspired by previous
work on categorical classifiers Vashisth et al. (2020); Potghan et al. (2018);
Desai and Shah (2021); Wan et al. (2018); Yulita et al. (2018); Das et al.
(2018). For visualizations of the CNNs/MLP, we refer to Appendix A.

### Evaluation Metrics.

We use three metrics in order to assess the trained classifiers’ performances.
(I) Test accuracy: we observe the progression of accuracy per elapsed epoch.
The final attained test accuracy is compared across all loss functions. (II)
Matthew’s Correlation Coefficient ($\phi$) Yule (1912); Matthews (1975). (III)
Area Under Curve (AUC) Collinson (1998).

### Results.

Table 4 shows the mean test accuracies across $M=6$ iterations of an
experiment. In Table 5, we present the attained mean performances for $\phi$
and AUC. Our results show that, performance-wise, the KLD outperforms all
other loss functions in 4 out of 6 experiments. Remarkably, the KLD attains
100 % test accuracy on all 6 iterations for Mushroom. While the State-of-the-
Art for classification (CCE) performs slightly worse than the MSE, it is more
stable. Unexpectedly, the MAE performs best and most stably on MNIST solely.
However, as we transition to more challenging data sets, test accuracy
degrades into pathological cases for the MSE and MAE; on CIFAR-10, they were
only able to ever predict one class (10 % accuracy). During training, the MAE
almost always kept an offset in test accuracy to the other losses (cf. Figure
4 in Appendix B). We hypothesize that this is due to its bad gradient
properties. On MNIST, its performance was best which we attribute to the
possibility that MNIST may be classifiable by learning its conditional median
$\operatorname*{\mathbb{E}}\quantity[\absolutevalue{f_{\theta}(x)-y_{i}}\mid
x=x_{i}]$, $i\in\\{1,\dots,N\\}$.

Table 4: Mean attained test accuracies and corresponding standard error of
mean (SEM) after 100 epochs. The best accuracies $\pm$ SEMs for each row are
boldfaced. A higher mean accuracy is better. A lower SE is better except if
the attained accuracy is pathological. Here, the SE should be understood as a
measure of stability across $M$ iterations of an experiment.

Data Set | MSE | MAE | CCE | KLD
---|---|---|---|---
Mushroom | $99.69\%$ | $\pm$ | $0.26\%$ | $95.83\%$ | $\pm$ | $0.27\%$ | $99.67\%$ | $\pm$ | $0.10\%$ | $\mathbf{100.00}\%$ | $\pm$ | $\mathbf{0.00}\%$
Phishing | $91.41\%$ | $\pm$ | $\mathbf{0.08}\%$ | $88.70\%$ | $\pm$ | $0.32\%$ | $90.29\%$ | $\pm$ | $0.11\%$ | $\mathbf{92.07}\%$ | $\pm$ | $0.26\%$
MNIST | $98.53\%$ | $\pm$ | $0.17\%$ | $\mathbf{99.30}\%$ | $\pm$ | $\mathbf{0.01}\%$ | $98.99\%$ | $\pm$ | $0.03\%$ | $98.84\%$ | $\pm$ | $0.04\%$
Fashion-MNIST | $87.96\%$ | $\pm$ | $\mathbf{0.04}\%$ | $82.04\%$ | $\pm$ | $0.09\%$ | $88.60\%$ | $\pm$ | $0.11\%$ | $\mathbf{90.47}\%$ | $\pm$ | $0.11\%$
CIFAR-10 | $10.00\%$ | $\pm$ | $0.00\%$ | $10.00\%$ | $\pm$ | $0.00\%$ | $\mathbf{78.13}\%$ | $\pm$ | $\mathbf{0.28}\%$ | $70.05\%$ | $\pm$ | $0.30\%$
CharFontIm | $11.30\%$ | $\pm$ | $0.00\%$ | $3.37\%$ | $\pm$ | $1.52\%$ | $69.21\%$ | $\pm$ | $0.97\%$ | $\mathbf{76.62}\%$ | $\pm$ | $\mathbf{0.02}\%$

Table 5: Performance metrics for the data sets. The best entries for each row
are boldfaced. Recall that a higher $\phi$ and a higher AUC is better. For
MNIST, no AUC column is boldfaced since all attain equal values. For
pathological cases (MSE/MAE for CIFAR-10 and CharFontIm), no entry is
boldfaced.

MSE MAE CCE KLD Data Set $\phi$ AUC $\phi$ AUC $\phi$ AUC $\phi$ AUC Mushroom
0.99 0.99 0.91 0.94 0.99 1.00 $\mathbf{1.00}$ $\mathbf{1.00}$ Phishing 0.82
0.96 0.77 0.94 0.80 0.96 $\mathbf{0.83}$ $\mathbf{0.97}$ MNIST 0.98 0.99
$\mathbf{0.99}$ 0.99 0.98 0.99 0.98 0.99 Fashion-MNIST 0.86 0.98 0.80 0.97
0.87 $\mathbf{0.99}$ $\mathbf{0.89}$ $\mathbf{0.99}$ CIFAR-10 0.00 0.50 0.00
0.50 $\mathbf{0.75}$ $\mathbf{0.97}$ 0.66 0.95 CharFontIm 0.00 0.50 0.00 0.50
0.67 0.98 $\mathbf{0.75}$ $\mathbf{0.99}$

##  5 Future Work

Our work leaves improvements for future research. Firstly, we believe that
more data sets need to be incorporated into loss function variation analysis.

Secondly, since we only considered classification, our work leaves open the
question of how the impact of loss function variation expresses in regression
settings. How significant and nontrivial is the selection of the loss function
in regression? Besides our restriction to four loss functions, we mentioned
novel losses such as the IMAE, Tanimoto loss and CCQL which outperformed
classical losses. Therefore, we recommend more work be dedicated to designing
custom loss measures. A hypothesis regarding the KLD’s very good performance
may be that the class of $f$-divergences provides good loss functions.
Examples of $f$-divergences are the KLD, the Bhattacharyya divergence
Bhattacharyya (1943) and Hellinger distance Hellinger (1909).

Thirdly, an expansion to sequence-based models would be interesting. To date,
several well-performing neural architectures, such as the Transformer Vaswani
et al. (2017), have been proposed.

##  6 Conclusions

In this work, we explored the impact of varying the loss function in different
deep learning architectures on the training process and its performance. We
considered the case of classifying data points into $c$ classes using
probabilistic predictions with one-hot encoded labels. We have demonstrated
empirically that on MNIST, the MAE can achieve a better classifier performance
than the CCE or KLD. Our designation is that if rapid convergence is needed,
the MSE can be favored for a low amount of classes. Our main finding is that
the KLD outperforms the CCE on 4 out of 6 data sets. Along with very good test
accuracies, two performance metrics are underpinnings of this result. Thus,
our recommendation is to increase the KLD’s usage in binary and multi-class
classification regimes. Due to its very bad performance on color images and
overall subpar performance, we advise that using the MAE in a classification
setting should be considered carefully. Finally, we encourage a paradigm shift
that steers away from axiomatically fixating loss functions but considers them
part of hyperparameter optimization.

### Acknowledgements.

Computational infrastructure and support were provided by the Center for
Information and Media Technology at Heinrich Heine University Düsseldorf.

## References

  * Abdi and Williams (2010) Hervé Abdi and Lynne J. Williams.  Principal component analysis.  _WIREs Computational Statistics_ , 2(4):433–459, 2010.  https://doi.org/10.1002/wics.101.
  * An et al. (2020) Sanghyeon An, Minjun Lee, Sanglee Park, Heerin Yang, and Jungmin So.  An ensemble of simple convolutional neural network models for MNIST digit recognition.  _arXiv preprint arXiv:2008.10400_ , 2020.
  * Bhattacharyya (1943) Anil Bhattacharyya.  On a measure of divergence between two statistical populations defined by their probability distributions.  _Bull. Calcutta Math. Soc._ , 35:99–109, 1943.
  * Bickel and Doksum (2015) Peter J. Bickel and Kjell A. Doksum.  _Mathematical Statistics: Basic Ideas and Selected Topics, Volumes I–II Package_.  Chapman and Hall/CRC, 2015.
  * Bosman et al. (2020) Anna Sergeevna Bosman, Andries Engelbrecht, and Mardé Helbig.  Visualising basins of attraction for the cross-entropy and the squared error neural network loss functions.  _Neurocomputing_ , 400:113–136, 2020.
  * Çalik and Demirci (2018) Rasim Caner Çalik and M. Fatih Demirci.  Cifar-10 image classification with convolutional neural networks for embedded systems.  In _2018 IEEE/ACS 15th International Conference on Computer Systems and Applications (AICCSA)_ , pages 1–2. IEEE, 2018.
  * Chai and Draxler (2014) Tianfeng Chai and Roland R. Draxler.  Root mean square error (RMSE) or mean absolute error (MAE)? – arguments against avoiding RMSE in the literature.  _Geoscientific Model Development_ , 7(3):1247–1250, 2014.
  * Chauhan et al. (2018) Rahul Chauhan, Kamal Kumar Ghanshala, and R. C. Joshi.  Convolutional neural network (CNN) for image detection and recognition.  In _2018 First International Conference on Secure Cyber Computing and Communication (ICSCCC)_ , pages 278–282. IEEE, 2018.
  * Ciresan et al. (2011) Dan Claudiu Ciresan, Ueli Meier, Jonathan Masci, Luca Maria Gambardella, and Jürgen Schmidhuber.  Flexible, high performance convolutional neural networks for image classification.  In _Twenty-Second International Joint Conference on Artificial Intelligence_ , 2011.
  * Collinson (1998) P. Collinson.  Of bombers, radiologists, and cardiologists: time to ROC.  _Heart_ , 80(3):215–217, 1998.
  * Das et al. (2018) Anup Das, Francky Catthoor, and Siebren Schaafsma.  Heartbeat classification in wearables using multi-layer perceptron and time-frequency joint distribution of ECG.  In _Proceedings of the 2018 IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies_ , pages 69–74, 2018.
  * Demirkaya et al. (2020) Ahmet Demirkaya, Jiasi Chen, and Samet Oymak.  Exploring the role of loss functions in multiclass classification.  In _2020 54th Annual Conference on Information Sciences and Systems (CISS)_ , pages 1–5. IEEE, 2020.
  * Desai and Shah (2021) Meha Desai and Manan Shah.  An anatomization on breast cancer detection and diagnosis employing multi-layer perceptron neural network (MLP) and convolutional neural network (CNN).  _Clinical eHealth_ , 4:1–11, 2021.
  * Diakogiannis et al. (2020) Foivos I. Diakogiannis, François Waldner, Peter Caccetta, and Chen Wu.  ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data.  _ISPRS Journal of Photogrammetry and Remote Sensing_ , 162:94–114, 2020.
  * Dua and Graff (2017) Dheeru Dua and Casey Graff.  UCI machine learning repository, 2017.  URL http://archive.ics.uci.edu/ml.
  * Feng et al. (2018) Zhen-Hua Feng, Josef Kittler, Muhammad Awais, Patrik Huber, and Xiao-Jun Wu.  Wing loss for robust facial landmark localisation with convolutional neural networks.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 2235–2245, 2018.
  * Ghosh et al. (2017) Aritra Ghosh, Himanshu Kumar, and PS Sastry.  Robust loss functions under label noise for deep neural networks.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 31, 2017.
  * Hellinger (1909) Ernst Hellinger.  Neue begründung der theorie quadratischer formen von unendlichvielen veränderlichen.  _Journal für die reine und angewandte Mathematik_ , 1909(136):210–271, 1909.
  * Janocha and Czarnecki (2017) Katarzyna Janocha and Wojciech Marian Czarnecki.  On loss functions for deep neural networks in classification.  _arXiv preprint arXiv:1702.05659_ , 2017.
  * Kingma and Ba (2014) Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.
  * Krizhevsky and Hinton (2009) Alex Krizhevsky and Geoffrey Hinton.  Learning multiple layers of features from tiny images.  2009\.
  * Kullback and Leibler (1951) Solomon Kullback and Richard A. Leibler.  On information and sufficiency.  _The Annals of Mathematical Statistics_ , 22(1):79–86, 1951.
  * LeCun et al. (1989) Yann LeCun, Bernhard Boser, John S. Denker, Donnie Henderson, Richard E. Howard, Wayne Hubbard, and Lawrence D. Jackel.  Backpropagation applied to handwritten zip code recognition.  _Neural Computation_ , 1(4):541–551, 1989.
  * Lin et al. (2017) Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár.  Focal loss for dense object detection.  In _Proceedings of the IEEE International Conference on Computer Vision_ , pages 2980–2988, 2017.
  * Matthews (1975) Brian W. Matthews.  Comparison of the predicted and observed secondary structure of t4 phage lysozyme.  _Biochimica et Biophysica Acta (BBA)-Protein Structure_ , 405(2):442–451, 1975.
  * Murphy (2012) Kevin P. Murphy.  _Machine Learning: A Probabilistic Perspective_.  MIT Press, 2012.
  * Muthukumar et al. (2021) Vidya Muthukumar, Adhyyan Narang, Vignesh Subramanian, Mikhail Belkin, Daniel J. Hsu, and Anant Sahai.  Classification vs regression in overparameterized regimes: Does the loss function matter?  _Journal of Machine Learning Research_ , 22:222:1–222:69, 2021.
  * Potghan et al. (2018) Sneha Potghan, R. Rajamenakshi, and Archana Bhise.  Multi-layer perceptron based lung tumor classification.  In _2018 Second International Conference on Electronics, Communication and Aerospace Technology (ICECA)_ , pages 499–502. IEEE, 2018.
  * Qi et al. (2020) Jun Qi, Jun Du, Sabato Marco Siniscalchi, Xiaoli Ma, and Chin-Hui Lee.  On mean absolute error for deep neural network based vector-to-vector regression.  _IEEE Signal Processing Letters_ , 27:1485–1489, 2020.
  * Rumelhart et al. (1985) David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams.  Learning internal representations by error propagation.  Technical report, University of California San Diego, La Jolla Institute for Cognitive Science, 1985.
  * Tabik et al. (2017) Siham Tabik, Daniel Peralta, Andres Herrera-Poyatos, and Francisco Herrera Triguero.  A snapshot of image pre-processing for convolutional neural networks: case study of MNIST.  2017\.
  * Vashisth et al. (2020) Shubham Vashisth, Ishika Dhall, and Shipra Saraswat.  Chronic kidney disease (CKD) diagnosis using multi-layer perceptron classifier.  In _2020 10th International Conference on Cloud Computing, Data Science & Engineering (Confluence)_, pages 346–350. IEEE, 2020.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  _Advances in Neural Information Processing Systems_ , 30, 2017.
  * Wan et al. (2018) Shaohua Wan, Yan Liang, Yin Zhang, and Mohsen Guizani.  Deep multi-layer perceptron classifier for behavior analysis to estimate parkinson’s disease severity using smartphones.  _IEEE Access_ , 6:36825–36833, 2018.
  * Wang et al. (2019) Xinshao Wang, Yang Hua, Elyor Kodirov, and Neil M. Robertson.  IMAE for noise-robust learning: Mean absolute error does not treat examples equally and gradient magnitude’s variance matters.  _arXiv preprint arXiv:1903.12141_ , 2019.
  * Wang et al. (2016) Zhangyang Wang, Shiyu Chang, Yingzhen Yang, Ding Liu, and Thomas S. Huang.  Studying very low resolution recognition using deep networks.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 4792–4800, 2016.
  * Xiao et al. (2017) Han Xiao, Kashif Rasul, and Roland Vollgraf.  Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms.  _arXiv preprint arXiv:1708.07747_ , 2017.
  * Yessou et al. (2020) Hichame Yessou, Gencer Sumbul, and Begüm Demir.  A comparative study of deep learning loss functions for multi-label remote sensing image classification.  In _IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium_ , pages 1349–1352. IEEE, 2020.
  * Yule (1912) G. Udny Yule.  On the methods of measuring association between two attributes.  _Journal of the Royal Statistical Society_ , 75(6):579–652, 1912.
  * Yulita et al. (2018) Intan Nurma Yulita, Rudi Rosadi, Sri Purwani, and Mira Suryani.  Multi-layer perceptron for sleep stage classification.  In _Journal of Physics: Conference Series_ , volume 1028, page 012212\. IOP Publishing, 2018.

##  Appendix A Architectures

[CNN architecture for MNIST. Dropout is applied after the 1 and 2 conv layer
($p=0.2$), as well as after the dense layer ($p=0.2$). The 1 conv layer has
$5\times 5$ filters.]

[CNN architecture for CIFAR-10. Dropout is applied after both Max-Pool layers
($p=0.25$), after the 4 conv layer ($p=0.25$) and after the dense layer
($p=0.5$). The 1 and 2 conv layer have $5\times 5$ filters.]

[CNN architecture for CharFontIm. Dropout is applied after the 2, 3 and 4 conv
layer ($p=0.25$), as well as after the 1 and 2 dense layers ($p=0.5$). Filters
have size $5\times 5$ for the 1 and 2 conv layers.]

Figure 2: CNN architectures. Red layers are Max-Pool layers of size $2\times
2$. If not specified, the filter size of a conv layer is $3\times 3$.
$\vdots$$\vdots$$\vdots$4 layersOutput layer$\vdots$$\vdots$ReLU activations
$\cdots$Softmax$\vdots$$\bm{o}$$\vdots$$x_{i}^{(1)}$$\vdots$$x_{i}^{(d)}$$\hat{y}_{i}^{(1)}$$\vdots$$\hat{y}_{i}^{(c)}$
Figure 3: Our dense architecture. Hidden layers have a teal color. The first
dense layer has 32 neurons, the second and third 64, and the fourth 32. This
is followed by a fifth dense layer (red-ish colored) having 2 output neurons.
Dropout is applied after the first 3 hidden layers ($p=0.4\rightarrow
0.2\rightarrow 0.2$). Note that this representation is not confined to a fixed
number of classes or neurons; the vertical dots indicate that it is indeed
conceptual.

##  Appendix B Accuracy Progressions

Figure 4: Test accuracies on each data set (across 6 iterations). Points of
interest are inset.


# Kinetic Models of Wealth Distribution Having Extreme Inequality: Numerical
Study of Their Stability Against Random Exchanges

Asim Ghosh<EMAIL_ADDRESS>Department of Physics, Raghunathpur
College, Raghunathpur, Purulia 723133, India.  Suchismita Banerjee  Economic
Research Unit, Indian Statistical Institute, Kolkata 700108, India.  Sanchari
Goswami  Department of Physics, Vidyasagar College, Kolkata 700006, India.
Manipushpak Mitra  Economic Research Unit, Indian Statistical Institute,
Kolkata 700108, India.  Bikas K. Chakrabarti  Economic Research Unit, Indian
Statistical Institute, Kolkata 700108, India.  Saha Institute of Nuclear
Physics, Kolkata 700064, India.

###### Abstract

In view of some persistent recent reports on a singular kind of growth of the
world wealth inequality, where a finite (often handful) number of people tend
to possess more than the wealth of the planet’s 50% population, we explore
here if the kinetic exchange models of the market can ever capture such
features where a significant fraction of wealth can concentrate in the hands
of a countable few when the market size $N$ tends to infinity. One already
existing example of such a kinetic exchange model is the Chakraborti or Yard-
Sale model, where (in absence of tax redistribution etc) the entire wealth
condenses in the hand of one (for any value of $N$), and the market dynamics
stops. With tax redistribution etc, its steady state dynamics have been shown
to have remarkable applicability in many cases of our extremely unequal world.
We show here that another kinetic exchange model (called here the Banerjee
model) has intriguing intrinsic dynamics, by which only ten rich traders or
agents possess about 99.98% of the total wealth in the steady state (without
any tax etc like external manipulation) for any large value of $N$. We will
discuss in some detail the statistical features of this model using Monte
Carlo simulations. We will also show, if the traders each have a non-vanishing
probability $f$ of following random exchanges, then these condensations of
wealth (100% in the hand of one agent in the Chakraborti model, or about
99.98% in the hands ten agents in the Banerjee model) disappear in the large
$N$ limit. We will also see that due to the built-in possibility of random
exchange dynamics in the earlier proposed Goswami-Sen model, where the
exchange probability decreases with an inverse power of the wealth difference
of the pair of traders, one did not see any wealth condensation phenomena.
These aspects of the statistics of these intriguing models have been discussed
here.

##  I INTRODUCTION

The first successful theory of classical many-body physics or classical
condensed matter systems has been about one and a quarter centuries old
kinetic theory of the (classical) ideal gas, composed of Avogadro number
(about $10^{23}$) order constituent atoms or molecule (each following
Newtonian dynamics). It remains still a robust, versatile and extremely
successful foundation of classical many-body physics. Social systems, economic
markets in particular, are intrinsically many-body dynamical systems composed
of a lesser number of constituents (order of $10^{2}$ for a village market to
the order of $10^{10}$ for a global market). One Robinson Crusoe in an island
can not develop a market or a society for that matter and markets are
intrinsically many-body systems. It is no wonder that the kinetic exchange of
money or wealth models had therefore been conjectured early on (e.g., by Saha
and Srivastava Saha1931  in 1931, Mandelbrot Mandelbrot1960  in 1960) and
resurrected recently (e.g., by Chakrabarti and Marjit Chakrabarti1995  in
1995, Dragulecu and Yakovenko Dragulescu2000  in 2000, Chakraborti and
Chakrabarti Chakraborti2000  in 2000, Chatterjee, Chakrabarti and Manna
Chatterjee2004a  in 2004).

The kinetic exchange models of trades and their statistics have been quite
successful in capturing several realistic features of wealth distributions
among the agents in the societies (see e,g., Yakovenko2009 ; Chakrabarti2013
). The beneficial effects of the agent’s saving propensity in reducing the
social inequality has been studied extensively Chakraborti2000 ;
Chatterjee2004a ; Chakrabarti2013 . The choice of the poorest trader as
mandatory in each trade (the other trade partner being randomly chosen) leads
to the remarkable self-organized poverty line, below which none remains in the
steady state (see e.g., Pianegonda2004 ; Iglesias2010 ; Ghosh2011 ; Paul2022
). This model was inspired by some crucial observations by the economists (see
e.g., Iglesias2010 ) and suggests the built-in (self-organized) remedies for
reducing social inequality. Though it must be admitted, such intriguing self-
organizing properties of the kinetic exchange models have not been
investigated extensively yet.

Contrarily, the recent focus has moved to the unusual rate of growth of social
inequality in the post world war II period (see e.g., Pickety2014 ;
Iglesias2021 ; Danial2022 ; Banerjee2023 ), which in some countries seem to
have crossed significantly above the 80-20 Pareto limit and have reached a
steady state with 87% percent wealth accumulated in the hands of 13% people.
This has indeed been argued, following an analogy with the inequality index
values for the avalanche burst statistics in self-organized sand-pile models
near their respective critical points, to be the natural limit in all social
competitive situations, where the welfare mechanisms (helping those who fail
to participate properly in such self-organizing dynamics) are either absent or
removed (see e.g., a recent review Banerjee2023 ).

Although the Pareto-like inequality mentioned above where a small fraction of
people (say 13%) possess a large fraction (say 87%) of wealth, can already be
devastating, more annoying kind of inequalities are being reported recently.
For example, the Oxfam Report Oxfam2020  of January, 2020 in Davos said “The
world’s 2,153 billionaires have more wealth than the 4.6 billion people who
makeup 60 percent of the planet’s population.” In other words, a handful
number (about $10^{3}$) of rich people possess more than about 60% (or
$10^{9}$ order) poor people’s wealth of this planet. This dangerous trend in
the world as a whole, is repeatedly mentioned in various recent reports.

The Pareto-type inequality mentioned above have long been investigated (see
e.g., Chatterjee2004a ; Chatterjee2007 ) using the kinetic exchange models
with non-uniform saving propensities of the traders (see e.g., Chakrabarti2013
, Pareschi2014  for reviews). One may naturally ask the question, does the
kinetic exchange theory allow for possible models where only a handful of
traders (say, about 10 in number) possess a significant fraction (say, above
50%) of the total wealth considered in the model, even when its population $N$
tends to infinity?

The answer is yes. The Chakraborti model Chakraborti2002 , widely known today
as the Yard-Sale model, starting with Hayes2002 , have attracted a lot of
attention (see e.g., Cardoso2023 ; Boghosian2019 ; Julian2022 ). In its barest
form Chakraborti2002  in the Chakraborti model (called C-model here), two
randomly chosen traders at any point of time come and participate in an
exchange trade when the richer one saves the excess over the poorer’s wealth
and goes for a random exchange of the total available wealth (double of the
poorer’s). The slow but inevitable attractor fixed point of the trade dynamics
arrive when all wealth ends up in the hand of just one trader, no matter how
big the population ($N$) is. Because of the particular form of savings during
any trade, whenever one becomes pauper, nobody trades with him and gradually
all condense to that state where one trader acquires the entire wealth and the
trade dynamics stop (see also Cardoso2023 ). External perturbations like
regular redistribution of tax collections by the central government (or any
non-playing agent) can help relieving Boghosian2019 ; Julian2022  the
condensation phenomenon and this seems to fit well with many observed
situations Boghosian2019 . We will show here numerically that if each of the
traders has a finite probability ($f$) of playing Dragulecu and Yakovenko (DY)
Dragulescu2000  type random exchanges, then for any $f>0$, the condensation of
wealth in the hand of one trader disappear and the steady state distribution
of wealth becomes exponentially decreasing, as in the DY model.

In the Goswami-Sen (or GS) model Goswami2014 , one considers a kinetic
exchange mechanism where the interaction (trade) probability among the trade
partners ($i$ and $j$) decreases with their wealth difference
($|m_{i}-m_{j}|$) at that instant of trading (time), following a power law
($|m_{i}-m_{j}|^{-\alpha}$). Of course, for $\alpha$ = 0, the model reduces to
that of DY. Their numerical results showed that for $\alpha$ values less than
about 2.0, the steady-state wealth distribution among the traders are still
DY-like (exponentially decaying with increasing wealth). For higher values
(beyond 2.0) of $\alpha$, power-law (Pareto-law) decays occur. No condensation
of wealth in the hands of a finite number of traders or agents are observed,
because of the inherent DY-like exchange probability in the dynamics of the
model (checked by extrapolating with $N$ the fraction of total wealth
possessed in the steady state by the richest ten traders).

We finally consider here a seemingly natural version of the kinetic exchange
model, called here the Banerjee (B) model Banerjee2021 , where the intrinsic
dynamics of the model lead to another extreme kind of inequality in the steady
state in the sense that precisely ten traders (out of the $N$ traders in the
market; $N\rightarrow\infty$) possess (99.98 $\pm$ 0.01)% of the total wealth.
These fortunate traders are not unique and their fortune does not last for
long (residence time on average is about 66 time units with the most probable
value around 25 time units, counted in units of $N$ trades or exchanges, for
any value of $N$) and it decreases continuously with increasing fraction ($f$)
of random trades or interactions. Unlike in the Chakraborti or Yard-Sale model
Chakraborti2002 ; Hayes2002 , where the dynamics stops after the entire wealth
goes to one (unless perturbed externally), here the trade dynamics continue
with the total wealth circulating only within a handful (about ten) traders in
the steady state. In this model, after each trade, the traders are ordered
from lowest wealth to highest and each of the traders trade only with their
nearest-in-wealth traders, richer or poorer compared to own, with equal
probability. Even if by chance the entire wealth goes to one trader, the
dynamics of random exchanges do not stop in this model as all the paupers
become the only nearest neighbors (wealth-wise) of this trader and random
exchange among them occurs! The process continues. Apart from the steady state
wealth distributions and most probable wealth amounts of the top few rich
traders, we will show that in this model the condensation of almost the entire
(99.98%) wealth occurs in the hands of 10 traders (no matter how big $N$ is).
We will show here again that this condensation disappears when a finite
fraction $f$ of time each of the traders go for DY type random exchanges, and
eventually the DY-type exponentially decaying wealth distribution sets-in,
after a power law region for low values of $f$.

##  II MODELS and NUMERICAL STUDIES FOR THEIR STATISTICS

Figure 1: Distributions of the fraction of total wealth ($M=N$) ending up in
the hands of the richest three traders. The error estimation is based on 10
runs. The typical errors in the distribution of wealth are seen to grow with
$N$ near the most-probable value of the wealth fraction and are indicated for
$N$ = 800 for all the three traders. Far away from the most-probable values,
the errors are less than the data point symbol sizes.

We study here numerically the statistical features of the three kinetic
exchange models introduced in the Introduction. We begin with the B (Banerjee
Banerjee2021 ) model. Next, we consider the C (Chakraborti, or Yard Sale)
model Chakraborti2002 ; Hayes2002  and then the GS (Goswami-Sen) model
Goswami2014 . In order to explore the stability of the condensation of wealth
feature in these models, we study the steady-state wealth distributions $P(m)$
in each of these models and the fraction of total wealth concentrated in the
hands of a few (say ten) traders or agents (whenever meaningful), allowing
each trader to have a nonvanishing probability $f$ (the faction of tradings or
times) to go for DY (Dragulecu and Yakovenko Dragulescu2000 ) type random
exchanges.

Most of the numerical (Monte Carlo) studies of the dynamics of these models
are studied with four sets of numbers $N$ of the agents or traders: $N$ = 100,
200, 400, and 800 and at each time step $t$, the dynamics runs over all the
$N$ order traders. We consider total money ($M$), to be distributed among the
agents is equal to $N$ and we denote the money of any agent $i$ at time $t$ by
$m_{i}(t)$ and, as such $M=\sum_{i}m_{i}(t)=N$. When the steady state is
reached after the respective relaxation times when the average quantities do
not change with time (relaxation time typically much less than $10^{5}$
trades/interactions for the $N$ values considered here), the statistical
quantities are evaluated from averages over $10^{5}$ post-relaxation time
steps or so.

###  II.1 Banerjee model results

Figure 2: Distribution of the total wealth fraction possessed by the ten
richest (at any time in the steady state and for different $N$ values). The
inset shows that the average of this total wealth fraction of the ten richest
(for any time and any value of $N$) in the steady state is very close to
0.9998. It may be noted, although the wealth share fractions of the richest
ten traders have considerable fluctuations (see Fig. 1), the sum total of
their wealth fractions have hardly any fluctuation (much less than the symbol
size in the inset). The error estimation is based on 10 runs. The typical
error in the distribution of total wealth of the ten richest are seen to be
more than the data point symbol sizes only near the most-probable value, where
it is indicated. Figure 3: Wealth distribution $P(m)$ among all the agents
against the wealth $m$ in the B model for different probability $f$ of DY
random exchanges. Note that the fluctuations appear to grow more for the lower
values of the distribution of wealth due to the log scale used in the y-axis.

In this B-model, when the DY fraction ($f$) is set equal to zero, no wealth
distribution $P(m)$ across the population is meaningful, because of wealth
condensation in the hands of a few. We first study the distributions (see Fig.
1) of total wealth fraction in the hands of the richest three. Note, these
three are not unique, and once they become so rich, their residence time (in
unit of $N$) is finite (about 66) and in case these positions are lost, the
return time also is finite.

Though the distributions of the total wealth fraction in the hands of the few
richest (shown in Fig. 1) are rather wide (each one spread over more than 30%
of the total wealth and does not $N$), the distribution of the total wealth
fraction possessed by the ten richest (at any time in the steady state) is
extremely narrow and spreads over 0.1% only (see Fig. 2). At any time in the
steady state its value is much more robust in this B model (with $f$ = 0) and
its value is less than unity, but very close to 0.9998.

Next, we consider the B model with a nonvanishing probability $f$ of each
trader to follow DY trades or exchanges. We see, immediately, the wealth
condensation disappears and with increasing values of $f$, the wealth gets
Boltzmann (exponentially) distributed among all the agents (see Fig. 3),
starting with Pareto-like power law distribution for lower values of $f$ (see
the inset of Fig. 3). Indeed, when we consider the limiting values (for large
$N$) of the average fraction of total wealth ($M=N$) possessed by the ten
richest traders in the steady state, they all seem to vanish (see Fig. 4) for
any non-zero value of $f$ (and remains a constant 0.9998 for $f$ = 0, the pure
B model).

For the wealth condensation in B-model (with $f$ = 0), we show next in Fig.
5(a), the distribution of residence-time (in unit of $N$) of the 10 fortunate
traders and (in the inset) the variation of the most probable and average
values of the residence-time ($\tau$, in unit of $N$). For the same model with
$f$ = 0, we show in Fig. 5(b) the distribution of return-time to fortune
(become one of the 10 richest starting from the 20th rank) and (in the inset)
the variation of the most probable and average values of the residence-time
with market size $N$.

Figure 4: To get the limiting values (for large $N$) of the average fraction
of total wealth ($M=N$) possessed by the ten richest traders in the steady
state, we plot the fraction against $1/N^{2}$ (as with DY type trades each of
N traders interacts with (N - 1 other trader). The extrapolated values all
seem to approach zero for any non-zero value of $f$ (but remains a constant
0.9998 for $f$ = 0, as in the pure B model). The error estimation is based on
10 runs. Typical sizes of error bars are indicated. Figure 5:  (a) The
distribution of residence-time (in units of $N$) of the 10 fortunate traders
and (in the inset) the variation of the most probable and average values of
the residence time. (b) The distribution of return-time to fortune (become one
of the 10 richest, starting from the 20th rank) and (in the inset) the
variation of the most probable and average values of the return-time (in units
of $N$). The error estimation is based on 10 runs. The typical errors in the
distribution of both the residence and return times are seen to grow with $N$
near the most-probable values of the respective quantities and are indicated
for $N$ = 400 here, when they are bigger than the symbol sizes.

###  II.2 Chakraborti or Yard-Sale model results

The C Model or Yard sale model is well-studied. However, in order to check the
stability of the condensation of wealth (entire money $M=N$ going to the hand
of one trader only, we added a nonvanishing probability $f$ of each trader to
follow DY trades or exchanges. We see, immediately, the wealth condensation
disappears for any $f>0$ (see Fig. 6) and the wealth gets distributed in the
Boltzmann form (exponentially decaying with increasing wealth) among all the
agents. The inset shows that for any nonzero value of $f$, the steady state
wealth distribution is exponentially decaying (and there is a power law
region) in this extended model C. Also, when we consider the limiting values
(for large $N$) of the average fraction of total wealth ($M=N$) possessed by
the ten richest traders the steady state (see Fig. 7), they all seem to vanish
from the unit value in the original C model (with $f=0$) for any non-zero
value of $f$.

Figure 6: Wealth distribution $P(m)$ among all the agents against the wealth
$m$ in the C model for different probability $f$ of DY random exchanges. Note
that the fluctuations appear to grow more for the lower values of the
distribution due to the log scale used in the y-axis. Figure 7: The limiting
values (for large $N$) of the average fraction of total wealth ($M=N$)
possessed by the ten richest traders in the the steady state of the C model
with $f$ fraction of DY-like trades. For $f=0$, the entire money goes to one
agent and the other nine agents contribute nothing. When we plot the fraction
against $1/N^{2}$ (as with DY type trades each of N traders interacts with
$(N-1)$ other traders), the extrapolated values all seem to approach zero for
any non-zero value of $f$. The error estimation is based on 10 runs. Typical
sizes of error bars are indicated.

###  II.3 Goswami-Sen model results

Figure 8: Wealth distribution $P(m)$ among all the agents against the wealth
$m$ in the GS model for different values of $\alpha$. Note that the
fluctuations appear to grow more for the lower values of the distribution due
to the log scale used in the y-axis.

Here the interaction (trade) probability among the trade partners ($i$ and
$j$) decreases with their wealth difference ($|mi–mj|$) at that instant of
trading (time), following a power law ($|mi–mj|^{-\alpha}$). As such in the GS
model, there is always a finite (but small) probability of random exchanges.
We do not need to consider the additional fraction of DY interaction in this
model. Of course, for $\alpha$ = 0, the model reduces to that of DY. Our
numerical results confirm (see Fig. 8) that for $\alpha$ values less then
about 2.0, the steady-state wealth distribution among the traders are still
DY-like (exponentially decaying). For higher values (beyond 2.0) of $\alpha$,
power-law (Parto-like) decays occur (but no condensation of wealth). Though
the model leads to extreme inequality, there is no condensation of wealth in
the hands of a few traders for any (larger) value of $\alpha$. In order to
check that we studied again the average fraction of total wealth ($M=N$)
possessed by the ten richest traders in the steady state of the GS model with
$\alpha$. When we plot the fraction against $1/N^{2}$ (see Fig. 9), the
extrapolated values of the fraction all seem to approach zero for any non-zero
value for any of the $\alpha$ values considered.

Figure 9: Plot of the fraction of total wealth ($M=N$) against $1/N^{2}$ for
different values of $\alpha$ in the GS model. The extrapolated (with $N$)
values of the fraction all seem to approach zero for any non-zero value of
$\alpha$. The error estimation is based on 10 runs. Typical sizes of error
bars are indicated. Figure 10: The DY fraction $f$ dependence of the bare
residence time (in units of interactions or exchanges) at different $N$ values
for B-model (Fig. 10a) and for C-model (Fig. 10b) are shown. Their power law
fits with $f$ for $N$ = 400 are shown (for other $N$ vales, the respective
prefactors change linearly with $N$). The insets show the $f$ dependence of
the residence-time $\tau$ (in units of $N$). Note that the limiting values of
$\tau$ at $f$ = 0 is about 66 for the B-model, while it goes to infinity for
the C-model.

##  III SUMMARY and DISCUSSION

In view of the observed extreme income or wealth inequalities in society, the
suitability of the kinetic exchange models Chakrabarti2013  to capture them,
at least qualitatively, have been investigated here. We distinguish between
two types of such extreme inequalities: One (Pareto) type Banerjee2023  where
a small fraction (typically 13%) of the population possess about 87% of the
total wealth (following a power law distribution) of the respective country.
The other more recently observed (and reported by Oxfam Oxfam2020 ) truly
extreme nature of income and wealth inequalities worldwide, where only a
handful number (say a few hundreds to thousands) of super-rich people of the
world accumulate more than the total wealth of 50 to 60 percent poor people.

Several kinetic exchange models (see e.g, Chatterjee2004a ; Chakrabarti2013 )
have been developed to analyze Pareto type of inequalities. We have
investigated here the statistics of some kinetic exchange models where, even
in the $N$ going to infinity limit, only one person can grab the entire wealth
(as in the Yard-sale or Chakraborti or C model Chakraborti2002 ; Hayes2002 ),
or only 10 people can accumulate about 99.98% of the total wealth (as in the
Banerjee or B model Banerjee2021 , see Fig. 2). We investigate how these
extreme inequalities in these kinetic models get softened to the Dragulescu-
Yakovenko (DY) Dragulescu2000  type exponentially decaying wealth
distributions among all the traders or agents, when the traders each have a
non-vanishing probability $f$ of DY-type random exchanges. These condensations
of wealth (100% in the hand of one agent in the C model Chakraborti2002 , or
about 99.98% in the hands of ten agents in the B model) then disappear in the
large $N$ limit (clearly seen when extrapolated against $1/N^{2}$, as in DY
type random exchanges, each of $N$ agents interact or exchange with all
others; see Figs. 4 and 7). We also showed that due to the built-in
possibility of DY-type random exchange dynamics in the Goswami-Sen or GS model
Goswami2014 , where the exchange probability decreases with an inverse power
of the wealth difference of the pair of traders, one does not see any wealth
condensation phenomena. In both GS and B model (with $f>0$ fraction DY
interactions or exchanges) no wealth condensation occurs, though strong
Pareto-type power-law wealth distribution $P(m)$ or inequalities occur for
large values of $\alpha$ and smaller values of $f$ in GS and B models
respectively (see Figs. 3 and 8). For the wealth condensation in the B model,
for $f$ = 0, we additionally find that the fortunate top ten traders are not
unique and their fortune does not last for long (residence-time $\tau$ to
fortune on average is about 66 time units with its most probable value around
25 time units, when counted in units of $N$ trades or exchanges; see Fig. 5a).
The most probable ‘return-time’ to such a fortune (of the 20th rank holder to
come within the group of fortunate 10), is found to be about 20 (again in
units of $N$; see Fig. 5b). It may be noted that with $f=0$, in the C-model,
the residence-time $\tau$ is infinity for the only fortunate one accumulating
the entire wealth in the system. Indeed with increasing values of DY fraction
$f$, the values of $\tau$ in both the cases decrease rapidly (see Fig. 10),
following inverse power laws with $f$. We further note, for $f$ = 0 in the
B-model near the most-probable values of the wealth fractions (Figs. 1 and 2)
and residence or return times (Fig. 5), the fluctuations tend to grow with
$N$, indicating a possible divergence there in the macroscopic limit of $N$.
We plan to explore its significance later.

Our studies for the B, C, and GS kinetic exchange models, using Monte Carlo
techniques code , suggest that the potential condensation type extreme
inequality can disappear in all of them if a non-vanishing probability of
random exchanges are allowed, and converge to Pareto-type power law inequality
(for B and GS model) which in turn converges to Gibbs like (exponentially
decaying) wealth distribution for larger values of $f$ in the B model, smaller
values of $\alpha$ in the GS model, or any values of $f>0$ in the C model.
These observations may help to formulate public welfare policies.

## acknowledgement

SB acknowledges the support from DST INSPIRE. BKC is grateful to the Indian
National Science Academy for their Senior Scientist Research Grant support.

## References

  * (1) M. N. Saha and B. N. Srivastava, B.N. A Treatise on Heat, p. 105, Indian Press: Allahabad, India (1931)
  * (2) Mandelbrot, B. B., The Pareto-Levy law and the distribution of income. International Economic Review, 1, 79-106 (1960)
  * (3) B. K. Chakrabarti and S. Marjit, Self-organisation and complexity in simple model systems: Game of life and economics, Indian Journal of Physics, 69B, 681–698 (1995).
  * (4) A. Dragulescu, V. M. Yakovenko, Statistical mechanics of money. European Physical Journal B - Condensed Matter and Complex Systems 2000, 17, 723–729. (2000)
  * (5) A. Chakraborti and Chakrabarti, B.K. Statistical mechanics of money: How saving propensity affects its distribution, European Physical Journal B - Condensed Matter and Complex Systems, 17, 167–170 (2000)
  * (6) A. Chatterjee, B. K. Chakrabarti and S. S. Manna, Pareto law in a kinetic model of market with random saving propensity. Physica : Statistical Mechanics and its Applications, 335, 155–163 (2004)
  * (7) V. M. Yakovenko and J. B. Rosser, Statistical mechanics of money, wealth, and income, Rev. Mod. Phys. 81, 1703 (2009)
  * (8) B. K. Chakrabarti, A. Chakraborti, S. R. Chakravarty and Chatterjee, Econophysics of Income and Wealth Distributions, Cambridge Univ. Press (2013)
  * (9) Pianegonda, S.; Iglesias, J.R. Inequalities of wealth distribution in a conservative economy, Physica A, 42, 193–199 (2004)
  * (10) J. R. Iglesias, How simple regulations can greatly reduce inequalities, Science and Culture, 76, 437–443 (2010) [arXiv:1007.0461]
  * (11) A. Ghosh, U. Basu, A. Chakraborti, and B. K. Chakrabarti, Threshold-induced phase transition in kinetic exchange models. Physical Review E, 83(6):061130 (2011
  * (12) S. Paul et al., Kinetic Exchange Income Distribution Models with Saving Propensities: Inequality Indices and Self-Organised Poverty Level, Philosophical Transactions of the Royal Society A, 380, 20210163 (2022)
  * (13) T. Pickety, Capital in Twenty First Century, Harvard University Press, Massachusetts (2014)
  * (14) J. R. Iglesias, B.-H. F. Cardoso and S. Gonçalves, Inequality, a scourge of the XXI century, Communications in Nonlinear Science and Numerical Simulation, 95, 105646 (2021)
  * (15) L. Danial and V. M. Yakovenko, Physics-inspired analysis of the two-class income distribution in the USA in 1983–2018, Phil. Trans. R. Soc. A, 380, 20210162 (2022) [http://doi.org/10.1098/rsta.2021.0162]
  * (16) S. Banerjee, S. Biswas, B. K. Chakrabarti, A. Ghosh and M Mitra, Sandpile Universality in Social Inequality: Gini and Kolkata Measures. Entropy, 25, 735 (2023) [https://doi.org/10.3390/e25050735]
  * (17) Oxfam International Report (2020) [https://www.oxfam.org/en/press-releases/worlds-billionaires-have-more-wealth-46-billion-people]
  * (18) A. Chatterjee and B. K Chakrabarti. Kinetic exchange models for income and wealth distributions. The European Physical Journal B, 60, 135–149 (2007)
  * (19) L. Pareschi and G. Toscani, Interacting Multiagent Systems, Oxford University Press, Oxford (2014)
  * (20) A. chakraborti, Distributions of money in model markets of economy, International Journal of Modern Physics C, 13, 1315–1321 (2002)
  * (21) B. Hayes, Follow the Money, American Scientist, 90, 400-405 (2002)
  * (22) B.-H. F. Cardoso, S. Gonçalves and J. R. Iglesias, Why equal opportunities lead to maximum inequality? The wealth condensation paradox generally solved, Chaos, Solitons and Fractals, 168, 113181 (2023)
  * (23) B. Boghosian, Is Inequality Inevitable?, Scientific American, 321, 70-77 (2019)
  * (24) N. Julian, C. B.-H. F. Cardoso, L. M. Fabiana, S. Gonçalves and J. R. Iglesias, Study of taxes, regulations and inequality using machine learning algorithms, Philosophical Transactions of the Royal Society A, 380 20210165 (2022)
  * (25) S. Goswami and P. Sen, Agent based models for wealth distribution with preference in interaction, Physica A: Statistical Mechanics and its Applications, 415, 514-524 (2014)
  * (26) S. Banerjee, Role of Neighbouring Wealth Preference in Kinetic Exchange model of market, arXiv:2305.16238 (2023)
  * (27) Code will be available from the corresponding author upon request.