# Loss Modeling for Multi-Annotator Datasets

Uthman Jinadu
Department of Computer Science
Georgia State University
Atlanta, GA 30303
<EMAIL_ADDRESS>
&Jesse Annan
Department of Mathematics
Georgia State University
Atlanta, GA 30303
<EMAIL_ADDRESS>
Shanshan Wen
Department of Computer Science
Georgia State University
Atlanta, GA 30303
<EMAIL_ADDRESS>
&Yi Ding
Department of Computer Science
Georgia State University
Atlanta, GA 30303
<EMAIL_ADDRESS>
All correspondences to the authors should be addressed to this author.

###### Abstract

Accounting for the opinions of all annotators of a dataset is critical for
fairness. However, when annotating large datasets, individual annotators will
frequently provide thousands of ratings which can lead to fatigue.
Additionally, these annotation processes can occur over multiple days which
can lead to an inaccurate representation of an annotator’s opinion over time.
To combat this, we propose to learn a more accurate representation of diverse
opinions by utilizing multitask learning in conjunction with loss-based label
correction. We show that using our novel formulation, we can cleanly separate
agreeing and disagreeing annotations. Furthermore, we demonstrate that this
modification can improve prediction performance in a single or multi-annotator
setting. Lastly, we show that this method remains robust to additional label
noise that is applied to subjective data.

##  1 Introduction

Samples in a dataset are often created using multiple annotations to enhance
label accuracy, a practice supported by research from Snow et al. (2008). The
reason is due to the assumption that individual annotators make mistakes and a
collection of annotators can provide better-quality labels. Traditional
methods for reconciling differences are to use techniques such as majority
voting, averaging, or expert opinions Waseem and Hovy (2016). These methods
are designed to arrive at a single ’ground truth’ for training supervised
learning models. However, in tasks of a subjective nature where a single
’correct’ answer may not exist, these practices can compromise the diverse
perspectives of individual annotators, as highlighted by Cheplygina and Pluim
(2018)

To improve the fair representation of annotators, datasets have been built to
include fine-grained annotations from each annotator. This enables capturing
the opinions of annotators who would otherwise have been removed due to the
aggregation function. Multitask learning is one way to model this data as
demonstrated by Mostafazadeh Davani et al. (2022), where each task is to
predict an annotator’s individual label. However, these methods do not account
for possible mistakes that individual annotators might make during the
annotation process. Datasets such as the GabHateCorpus by Kennedy et al.
(2018) and GoEmotions by Demszky et al. (2020) can contain thousands of
annotations by a single annotator. Mistakes such as misidentifying emotions
due to annotation exhaustion would not be surprising. Furthermore, annotation
of a large number of samples may occur over multiple days and introduce
temporal distribution shifts of opinion.

We propose to address this problem by introducing loss-based label correction
into a multitask learning setting. A fundamental property of loss-based label
correction works by exploiting the network memorization effect by Arpit et al.
(2017). They found that networks tended to learn simple patterns before
learning more complex ones. This has been used extensively within the noisy
learning community such as in Arazo et al. (2019). These methods first utilize
the memorization effect to detect mislabeled instances. Then they correct
these possible sources of mislabeling using techniques such as incorporation
of a network’s own guess.

Motivated by this, we propose a novel formulation of multitask learning with
label correction. While the original purpose of the technique is to detect
incorrect samples, we find that in our novel formulation, the loss separates
cleanly into agreeing versus disagreeing samples. This enables us to adjust
the degree of conforming predictions by strengthening or weakening a model’s
guess of the true label. We show that this is useful when modeling subjective
datasets with differing agreement properties. We highlight our contributions
as follows:

  * •

We present a novel formulation of multitask learning with loss-based noise
correction.

  * •

We demonstrate that with our formulation, we can cleanly separate agreeing and
disagreeing annotations.

  * •

We demonstrate that this modification can improve prediction performance in a
single or multi-annotator setting. Additionally, this method remains robust to
additional label noise that is applied to subjective data.

  * •

We introduce a hyperparameter to control the degree of label correction due to
variability in label properties for different annotation tasks. We show that
this has a large effect on performance.

##  2 Background

###  2.1 Multitask Learning for Disagreement

Disagreement among annotators, a common occurrence in the annotation process,
signifies differing opinions or interpretations when labeling a specific task.
Addressing annotator disagreement is a fundamental part of enhancing machine
learning model accuracy. Variations in data labeling underscore the need for
sophisticated approaches to model training that account for the nuances in
human-coded datasets. Ensuring consistency in annotations directly influences
the reliability of predictive models, thus necessitating strategies that can
effectively harness the richness of human annotations while minimizing their
discordance. Researchers have extensively studied various underlying sources
of such disagreements, focusing on two primary types: random variation and
systematic disagreement Mostafazadeh Davani et al. (2022).

Random variation encompasses the inherent unpredictability in human judgments,
which can introduce randomness into machine learning algorithms, impacting
their outcomes. This type of disagreement might stem from individual biases or
the subjective nature of a task such as due to demographic properties Ding et
al. (2022).

On the other hand, systematic disagreement is more nuanced, arising from
consistent inaccuracies or biases in the machine learning models themselves.
Systematic disagreement can also arise from multiple sources or due to
multiple reasons. As demonstrated by Krippendorff, this can result from
systematic factors that skew the models, leading to persistent errors in
interpretation Krippendorff (2008).

Understanding both types of disagreement is vital for developing robust
machine learning models, as recognizing and mitigating these factors can lead
to more reliable annotations. The ongoing research in this area continues to
refine the techniques used to identify and minimize disagreements, enhancing
the quality and reliability of annotated data Mostafazadeh Davani et al.
(2022).

Classifiers’ performance and resilience vary based on different levels of
annotators’ agreement when selecting training and test data. Leonardelli et
al. (2021)

###  2.2 Annotator Noise

Noise correction refers to the systematic process of identifying and
rectifying errors or inconsistencies within a dataset, often termed error
correction. Its primary objective is to enhance both the quality and
reliability of the data by either eliminating or minimizing the influence of
noise Zhan et al. (2019).

This noise may include random errors, artifacts, or inaccuracies that can
significantly distort the underlying patterns and relationships within the
data. By employing various correction techniques tailored to the specific type
and source of the noise, researchers can obtain a cleaner, more accurate
representation of the information.

Traditional categorical cross-entropy loss is not well-suited for this task as
it tends to fit the noise rather than the underlying data distribution (Zhang
et al. (2016)). The hard bootstrapping loss introduced by Reed et al. (2015)
offers a solution by augmenting the standard cross-entropy loss with a
perceptual term. This addition helps adjust the training objective, making it
more robust to label noise.

Effective noise correction is a crucial step in data preprocessing,
particularly in fields such as machine learning, where the integrity and
precision of the data can directly affect the performance and interpretability
of the models. It’s an ongoing challenge that requires a delicate balance of
removing genuine noise without erasing essential variations or introducing new
biases Arazo et al. (2019).

##  3 Methods

We are given dataset denoted as $D$, comprising text samples $X$, and an
annotation matrix $Y\in\mathcal{R}^{2\times A}$. Here, $X$ represents
individual text entries, $A$ signifies the number of annotators, and $Y$ is a
matrix where each cell $y_{i}$ stores one hot value of a label class. Due to
the non-uniform distribution of annotator input across text instances, missing
values in $Y$ are prevalent. Our goal is to deduce a consensus label for each
text instance.

###  3.1 Baseline

A standard way for optimizing a model for multi-class classification is to
utilize the cross entropy loss function. For an input sample $x$, the cross
entropy loss is given by:

$L_{CE}=-\sum_{m=1}^{M}y_{m}\cdot log(p_{m}(x,\theta))$ (1)

where m represents the number of classes and the $y_{i}$ represents the truth
for the ith class. $\theta$ signifies some model.

###  3.2 Multi-task learning to account for different opinions

Individual annotators’ opinions can be incorporated into learning the model
via multi-task learning. Suppose there are $A$ annotators, we can create $A$
different task heads (fully connected layers) to predict each of the
individual annotations. This changes the cross-entropy loss to a summation of
the individual outputs for each task head:

$L_{MT}=-\frac{1}{A}\sum_{a=1}^{A}\sum_{m=1}^{M}y_{a,m}\cdot
log(p_{a,m}(x,\theta)).$ (2)

Here, the ground truth values for each sample form a $A\times M$ matrix
$\mathbf{y}$. The model will also output a prediction distribution for each
annotator. Note that MT loss reduces to the CE loss when the number of
annotators is 1.

###  3.3 Noise Correction via Loss Modeling

Arazo et al. (2019) found that the distribution of losses corresponds to its
correctness with other annotators. That is, when learning with imperfect data,
lower losses corresponded to the correct samples, and higher losses were
correlated with incorrect annotations. This leads to a two-step algorithm.
First, the loss distribution was modeled using a two-component mixture model
to generate a weight $w$. The weight provides an indication of how likely a
particular sample, based on its loss, is likely to be mislabeled. The
intuition is that a model’s guess should be integrated for possibly mislabeled
data. This gives us a loss function which scaled by this weight:

$L_{LC}=-\left((1-w)\sum_{m=1}^{M}y_{i}\cdot
log(p_{m}(x,\theta))+w\sum_{m=1}^{M}z\cdot log(p_{m}(x,\theta))\right)$ (3)

The new label is re-parameterized as a weighted sum of a network’s guesses
with its ground truth. Here, $z=p_{m}(x,\theta)$, and represents a network’s
prediction based on a single round of evaluation and is detached during a
training cycle of the network.

###  3.4 Multi-task learning with loss-based noise correction

(a) Distribution of losses on the GabHateCorpus for a single annotator after
training with multitask learning.

(b) Distribution of losses on the GoEmotions for a single annotator after
training with multitask learning.

Figure 1: Modeling the loss distribution for annotators in multitask learning.
Arpit et al. (2017) demonstrated that networks tended to learn correct samples
first before memorizing the incorrect samples. In our problem, the semantics
of this changes to agreeing versus disagreeing samples. As can be seen, there
is a clear separation of samples based on loss. The samples in the Agree
Category for (a) is 63.67% and the samples in the Disagree category for (b)
are 36.32%. Additionally, this demonstrates that our method is preserving
disagreement and diverse opinions. This also conforms with the previous
understanding that networks tend to be over-confident with predictions. This
overconfidence appears to extend to a multi-task setting where the network is
more confident in the majority of predictions.

We can now combine equations 2 and 3 from above to create a multi-task label
correction loss:

$\displaystyle L_{MLC}$
$\displaystyle=\frac{1}{A}\sum_{a}^{A}-\left((1-w)\sum_{m=1}^{M}y_{i}\cdot
log(p_{a,m}(x,\theta))+w\sum_{m=1}^{M}z_{a,m}\cdot
log(p_{a,m}(x,\theta))\right)$ (4)
$\displaystyle=-\frac{1}{A}\sum_{a=1}^{A}\sum_{m=1}^{M}(1-w)y_{a,m}\cdot
log(p_{a,m}(x,\theta))-\frac{1}{A}\sum_{a=1}^{A}\sum_{m=1}^{M}wz_{a,m}\cdot
log(p_{a,m}(x,\theta))$ (5) $\displaystyle=(1-w)*(L_{MT})+(\psi*w*L_{G})$ (6)

Additionally, we propose to introduce a subjectivity parameter $\psi$
parameter to tune the impact of a network’s self-guess loss term $\psi\cdot
L_{G}$. Intuitively, as can be seen in Figure 1, higher values would push more
disagreeing labels to the majority class. This can be tuned as a way to
balance agreement with disagreement and noise based on dataset properties.

##  4 Experimental Setup

We evaluate each of the four different optimization methods above in these
experiments.

  * •

Baseline corresponds to a network with a single output head optimized to the
majority label using $L_{CE}$.

  * •

Baseline + Loss Correction injects our loss correction technique onto the
majority label optimization method ($L_{LC})$.

  * •

Multitask involves adding $A$ fully-connected layers to the last layer of the
network. Each output is trying to predict the annotations of a unique
annotator. Since the datasets we are using are missing annotators, these loss
terms are zero-ed out and not back-propagated. This corresponds to the
($L_{MT}$) loss.

  * •

Multitask + Loss Correction is our proposed method for integrating loss
correction ($L_{MLC}$) into a multitask setting.

###  4.1 Datasets

In our experiments, we made use of GoEmotions and GabHateCorpus datasets to
annotate our subjective tasks.

GoEmotions Demszky et al. (2020) is a large-scale dataset consisting of 58K
Reddit comments annotated with 27 different emotion categories, including
neutral labels. The dataset is annotated by 82 crowdsourced workers and label
noise is minimized by removing labels that have fewer than one annotator. The
dataset is designed for multi-class and multi-label emotion classification,
however, 83% of examples have a single label. The dataset is pre-split into a
training set with 43,410 entries, a validation set containing 5,426 entries,
and a testing set comprised of 5,427 entries. We evaluate the six basic Ekman
emotion labels in this work.

GabHateCorpus (GHC): The Gab Hate Corpus (GHC) hate speech dataset developed
by Kennedy et al. (2018), and consists of 27,665 text entries collected from
Gab.com. This public dataset was annotated by 18 specialized annotators, with
each entry evaluated by at least 3 different experts. This rigorous process
ensures a high level of accuracy and consistency within the GHC. This property
is unique due to its emphasis on label quality when given label diversity.

###  4.2 Implementation Details

The classification models were built using the HuggingFace transformers
library (version 4.38) (Wolf et al., 2020). Our experimental setup closely
resembled that of Mostafazadeh Davani et al. (2022). For the GabHateCorpus
experiments, we trained the models for five epochs using a learning rate of
1e-7. As for the GoEmotions dataset, we utilized a learning rate of 5e-6 and
trained the models for five epochs. We used this to obtain the baseline and
multitask for each of the datasets and then we introduced the label correction
similar to how Arazo et al. (2019) was implemented. We used the pre-trained
Bert base-uncased model as the underlying architecture. As we do not have
access to the original multi-task code some results are different than
Mostafazadeh Davani et al. (2022). Optimization was conducted using Stochastic
Gradient Descent (SGD) with a momentum of $0.9$, weight decay of $0.01$, and a
batch size of $64$.

Additionally, we integrate some components that are necessary to train for
loss correction. A warm-up period using an entropy-based penalty on the
confidence term has been found useful to prevent noise overfitting during
warm-up periods Pereyra et al. (2017). We designated a warm-up period of $2$
epochs for the GoEmotions and GabHateCorpus datasets. This was necessary to
stabilize the training, mitigate poor initialization, and make it start with a
smaller learning rate and gradually increase it. We utilized regularization
following Tanaka et al. (2018) and Arazo et al. (2019) aiming to deter the
allocation of all samples to a singular class. We did not implement the full
Arazo et al. (2019) to minimize possible confounding effects of auxiliary
techniques such as mixup. Unless otherwise specified, $\psi$ is set to $1$.

###  4.3 Noise Injection

The datasets used in the evaluation are for binary classification. Therefore,
we simply flip the labels by sampling for a subset of data. In this paper, we
used 20% noise rate for the noise experiments. For samples where the labels
are from multiple annotators, e.g. 4 votes for true and 1 vote for false, we
swap the labels for the samples where we are injecting noise. Not all
annotators annotated every sample. We ignored these annotators when injecting
noise and thus these did not have an impact on the loss.

##  5 Results

Due to the nature of this problem, it is important to consider these results
within the context of data collection processes. GoEmotions annotators were
crowdworkers who spoke English from India. GHC annotation was a two-step
process that involved expert opinions. The annotators of GHC were trained and
the annotation process was much more controlled. Furthermore, the annotator
population was recruited from a smaller sample and consisted of undergraduate
students. These processes would cause GHC to be less variable, due to likely
similar backgrounds of annotators. On the other hand, using untrained
crowdworkers from a foreign country would likely lead to more variability.

With this in mind, we first discuss the comparison of performance metrics in
evaluated methods. We then examine the effect of injecting additional noise.
Lastly, we examine the effect of adjusting the subjectivity parameter and its
effect on results.

###  5.1 Comparison of techniques

| No Noise | 20% Noise
---|---|---
Method | Precision | Recall | F1 | Precision | Recall | F1
Baseline | 46.09 | 49.96 | 47.95 | 49.87 | 49.78 | 41.67
Baseline + Label Correction | 48.04 | 50.14 | 48.41 | 49.89 | 50.14 | 43.79
Multitask | 49.56 | 49.86 | 45.57 | 47.70 | 47.95 | 45.62
Multitask + Label Correction | 54.01 | 50.94 | 48.58 | 50.48 | 50.23 | 49.47
Table 1: Performance comparison on the GabHateCorpus dataset. Our proposed
method of incorporating label correction into multitask learning had the
highest score. Our method is also robust to label noise under conditions of
label subjectivity. No Noise, $\psi=.5$
---
Emotion | Baseline | Baseline + LC | Multitask | Multitask + LC
Anger | 71.46 | 71.53 | 58.58 | 55.23
Disgust | 73.73 | 72.72 | 58.72 | 60.01
Fear | 73.16 | 74.76 | 62.69 | 65.43
Joy | 75.51 | 76.21 | 59.66 | 61.76
Sadness | 73.36 | 73.61 | 63.04 | 64.12
Surprise | 72.67 | 70.47 | 62.58 | 61.96
Table 2: Performance comparison on the GoEmotions dataset. The binary
classification accuracy is reported for each emotion. For four out of the six
emotions, we demonstrate an improved performance when $\psi$ is fixed to .5.

We present the GabHateCorpus results in Table 1 and the results of GoEmotions
in Table 2. All metrics reported are compared to the majority label on the
ground truth. As can be seen in the GHC results, our method demonstrates
improvements across all metrics when compared against the baselines.
Additionally, we see that baseline methods based on the majority also perform
worse than baseline + label correction. These results indicate that regardless
of the number of annotators, it may be beneficial to incorporate label
correction.

For GoEmotions results, we see that for Multitask+LC, four out of the six
cases exceed the multitask results. This indicates that the loss correction is
having some effect. Most notably, there is a large drop in performance between
the majority and annotator (multitask) conditions. We hypothesize that this is
due to two factors. One plausible explanation for this phenomenon may be the
inconsistency in the number of annotators who labeled each text instance in
the dataset. This pattern was also noted by Mostafazadeh Davani et al. (2022).
Such variance could introduce an additional layer of complexity or uncertainty
into the model, affecting its ability to efficiently generalize from the data.
This opens up avenues for future research to understand how the number of
annotators, and the possible divergence in their annotations, influences the
effectiveness of loss correction methods in multi-task learning setups.
However, while this drop exists, our multitask results still exceed the
results presented in the previous work. Secondly, additional investigation is
warranted for the effect of the initial model which further confounds the
results.

###  5.2 Classification of Subjective Labels in the Presence of Noise

We compare the effect of noise for each classification task on GHC as well as
GoEmotions. In this paper, we conduct experiments with a 20% noise rate
injected based on the process discussed in 4.3. As can be seen in Table 1 our
method is robust to noise injection even when accounting for multiple
annotators. It is also interesting to note that both the majority-based
methods for aggregating labels (Baseline Majority and Baseline + Label
Correction), showed a large drop in performance. This highlights the
importance of training multiple annotators as a way to mitigate the effect of
noise.

We see that for the results of GoEmotions in Table 3, there is a significant
drop in the majority label annotations. However, the multitask cases all
showcase a smaller drop in performance than the majority label techniques.
This highlights the need to account for multiple annotator opinions.

20% Noise
---
Emotion | Baseline | Baseline + LC | Multitask | Multitask + LC
Anger | 45.63 | 46.32 | 43.64 | 43.95
Disgust | 44.34 | 44.24 | 43.96 | 42.45
Fear | 44.25 | 44.24 | 44.06 | 45.28
Joy | 46.41 | 46.69 | 44.28 | 45.38
Sadness | 45.62 | 45.44 | 43.82 | 41.85
Surprise | 44.89 | 44.61 | 43.76 | 42.94
Table 3: Performance comparison on the GoEmotions dataset when injected with
20% label noise.

###  5.3 Effect of Subjectivity Parameter

| No Noise | 20% Noise
---|---|---
Label type |  $\psi=$ 1 |  $\psi=$ 0.5 |  $\psi=$ 0.25 |  $\psi=$ 1 |  $\psi=$ 0.5 |  $\psi=$ 0.25
GHC | 48.58 | 47.06 | 45.98 | 49.47 | 44.77 | 45.04
Anger | 57.89 | 55.23 | 57.84 | 44.23 | 43.95 | 44.39
Disgust | 59.02 | 60.01 | 59.13 | 41.56 | 42.45 | 40.65
Fear | 64.14 | 65.43 | 54.62 | 45.01 | 45.28 | 45.21
Joy | 58.43 | 61.76 | 60.75 | 44.56 | 45.38 | 45.41
Sadness | 64.12 | 64.12 | 64.35 | 42.94 | 41.85 | 42.15
Surprise | 59.64 | 61.96 | 62.31 | 41.35 | 42.94 | 42.31
Table 4: Comparison of accuracy for GHC and GoEmotions for different
parameters of $\psi$. Adjusting $\psi$ has a clear effect on GHC and
GoEmotions. GHC appears to benefit from greater amounts of noise correction
($\psi=1$). GoEmotions on the other hand appears to perform the best in
general for $\psi=.5$. However, the optimal parameter is different for
individual emotions. This indicates a difference in the label patterns for
subjectivity and noise.

We present the results exploring the subjectivity parameter $\psi$ in Table 4.
As can be seen, there is a large effect on the performance of different sets
of labels. For GHC, reducing the noise correction tends to decrease the
performance. This results in more tolerance of label disagreement and would
indicate that the test set has labels that conform more with the majority
labels. For GoEmotions, we see that $\psi=.5$ appears to have the best
performance overall. This supports our hypothesis that it is important to tune
the degree of label correction depending on the dataset type. Furthermore, we
see that for each type of label in GoEmotions, the best $\psi$ parameter to
use is changing. This indicates that there are nuances to the label
distribution which require further investigation. For example, surprise and
joy benefit from a lower $\psi$ while fear performs better with higher $\psi$
values.

###  5.4 Loss Separation

We showcase a clear bimodal distribution in Figure 1. We see that for a
specific annotator, there are peaks for agreeing and disagreeing samples. Note
that these are semantically different than the noisy label’s definition of
correct and incorrect. In this paper, we refer to incorrect annotations as a
label that the annotator would not typically ascribe to a sample. We assume
that incorrect answers are more likely to fall in the distribution of
disagreeing labels. We calculate the proportion of labels within each mode to
see how many of them agree with the majority annotation and how many disagree.

Text | Annotator Rating | Agree With Majority?
---|---|---
Yeah, OK then, get your nurse to give you some more of those nice tablets that make you sleep. | Not Hate | Yes
This creature was predicated his actions were predicted etc etc etc now the media are outraged as predicted…. this is proof that the level of evil we are dealing with is not a person but a large and well financed group | Not Hate | Yes
Farcebook? Should have tanked a long time ago. | Not Hate | Yes
IDK, my favorite was "Icelandic Women are Whores" Happy New Year and Hail Victory | Not Hate | Yes
Christian cake baker who won SCOTUS case is now being harassed by demented Satanist homosexuals, backed by the state of Colorado. | Not Hate | No
GabBrasil an enterprising capitalist might do well setting up an online course/forum teaching Portuguese to Deplorable US Patriots. And/or vice versa… Just saying. | Hate | No
Table 5: Samples from the GabHateCorpus Dataset which shows the text, the annotation on the text, and if it agrees with the majority based on the Agree and Disagree distribution in Figure 1. Exploring this sample dataset extract, the first entry’s ‘Not Hate’ annotation matches the majority view, while the final entry shows a dissenting ‘Hate’ annotation. Accounting for such individual variations is vital to accurately reflect the annotator’s perspectives. Text | Annotator Rating | Agree With Majority? | Emotion
---|---|---|---
But also: fuck the daily mail. | Anger | Yes | Anger
You blew it. They played you like a fiddle | Not Disgust | Yes | Disgust
The sin was refusing to impregnate his brother | Not Disgust | No | Disgust
Ok now that was epic! | Not Fear | Yes | Fear
This is creepy | Not Fear | No | Fear
I can see why! I would totally be offended too! | Not Joy | No | Joy
I’ve been abused enough by the Knicks today. | Sadness | Yes | Sadness
I just can’t believe she had the nerve to call. | Surprise | Yes | Surprise
Table 6: Samples from the GoEmotion Dataset which shows the text, the
annotation on the text, and if it agrees with the majority based on the Agree
and Disagree distribution in Figure 1. In this dataset excerpt, the second
entry is labeled ‘Not Disgust’ by an annotator, aligning with the majority
consensus. However, in entries five and six, the sampled annotators diverge
from the majority, labeling the texts as ‘Not Fear’ and ‘Not Joy’
respectively. Incorporating these differing viewpoints is crucial to ensure
fairness.

##  6 Limitations

Our improved method incorporates a subjectivity parameter, $\psi$, to modulate
the self-guess loss term, impacting the handling of label consensus. Despite
the observable significant effect of $\psi$ on performance, pinpointing the
optimal value is complex, with no precise selection strategy currently in
place. This calls for additional investigation to establish a methodical
approach for $\psi$ determination, which is essential for the method’s
adaptability and accuracy across varying datasets. Additionally, the typical
way to analyze the noise patterns of less subjective datasets is to annotate a
subset of the test set at scale. This gives insight into the noise profile and
can enable the creation of a label noise transition matrix. To fully evaluate
the noise patterns of subjective annotations is much more costly. Annotation
subjectivity can vary through time, and with demographic properties. These
properties make building a noise transition matrix almost impossible due to
the number of factors that can influence the underlying label distribution.
Thus, understanding noise for subjective annotation tasks remains an exciting
challenge we leave for future work.

##  7 Conclusion

In this study, we developed a method for loss-based label correction for a
multitask setting. Specifically, this is applied to subjective datasets from
multiple annotators. We show that our approach preserves disagreeing labels
and that the performance increases under some conditions. We also demonstrated
that it is robust to added noise. Lastly, we demonstrate that adjusting the
degree of loss correction is important and has a large impact on performance.
This is beneficial for future work in collecting subjective labels at scale.

##  8 Acknowledgements

This research was sponsored by the Army Research Laboratory and was
accomplished under Cooperative Agreement Number W911NF-23-2-0224. The views
and conclusions contained in this document are those of the authors and should
not be interpreted as representing the official policies, either expressed or
implied, of the Army Research Laboratory or the U.S. Government. The U.S.
Government is authorized to reproduce and distribute reprints for Government
purposes notwithstanding any copyright notation herein.

## References

  * Arazo et al. (2019) E. Arazo, D. Ortego, P. Albert, N. E., and K. McGuinness. 2019.  Unsupervised label noise modeling and loss correction.  _ArXiv_.
  * Arpit et al. (2017) Devansh Arpit, Stanislaw Jastrzkebski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron C Courville, Yoshua Bengio, and Simon Lacoste-Julien. 2017.  A closer look at memorization in deep networks.  In _Proceedings of the 34th International Conference on Machine Learning-Volume 70_ , pages 233–242. JMLR.org.
  * Cheplygina and Pluim (2018) Veronika Cheplygina and Josien P. W. Pluim. 2018.  Crowd disagreement about medical images is informative.  In _Intravascular Imaging and Computer Assisted Stenting and Large-scale Annotation of Biomedical Data and ExpertLabel Synthesis_ , pages 105–111. Springer.
  * Demszky et al. (2020) D. Demszky, J. Ko, A. Cowen, G. Nemade, and S. Ravi. 2020.  Goemotions: A dataset of fine-grained emotions.  _ArXiv preprint ArXiv:2005.00547_.
  * Ding et al. (2022) Yi Ding, Jacob You, Tonja-Katrin Machulla, Jennifer Jacobs, Pradeep Sen, and Tobias Höllerer. 2022.  Impact of annotator demographics on sentiment dataset labeling.  _Proceedings of the ACM on Human-Computer Interaction_ , 6(CSCW2):1–22.
  * Kennedy et al. (2018) Brendan Kennedy, Mohammad Atari, Aida Mostafazadeh Davani, Leigh Yeh, Ali Omrani, Yehsong Kim, Kris Coombs, Shreya Havaldar, Gwenyth Portillo-Wightman, Elaine Gonzalez, et al. 2018.  The gab hate corpus: A collection of 27k posts annotated for hate speech.  _PsyArXiv. July_ , 18.
  * Krippendorff (2008) Klaus Krippendorff. 2008.  Systematic and random disagreement and the reliability of nominal data.  _Communication Methods and Measure_ , 2(4):323–338.
  * Leonardelli et al. (2021) E. Leonardelli, S. Menini, A. P. Aprosio, M. Guerini, and S. Tonelli. 2021.  Agreeing to disagree: Annotating offensive language datasets with annotators’ disagreement.  _ArXiv_.
  * Mostafazadeh Davani et al. (2022) Aida Mostafazadeh Davani, Mark Díaz, and Vinodkumar Prabhakaran. 2022.  Dealing with disagreements: Looking beyond the majority vote in subjective annotations.  _Transactions of the Association for Computational Linguistics_ , 10:92–110.
  * Pereyra et al. (2017) Gabriel Pereyra, George Tucker, Jan Chorowski, Łukasz Kaiser, and Geoffrey Hinton. 2017.  Regularizing neural networks by penalizing confident output distributions.  _arXiv preprint arXiv:1701.06548_.
  * Reed et al. (2015) Scott Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, and Andrew Rabinovich. 2015.  Training deep neural networks on noisy labels with bootstrapping.  _arXiv preprint_.
  * Snow et al. (2008) Rion Snow, Brendan O’Connor, Daniel Jurafsky, and Andrew Ng. 2008.  Cheap and fast – but is it good? evaluating non-expert annotations for natural language tasks.  In _Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing_ , pages 254–263, Honolulu, Hawaii. Association for Computational Linguistics.
  * Tanaka et al. (2018) Daiki Tanaka, Daiki Ikami, Toshihiko Yamasaki, and Kiyoharu Aizawa. 2018.  Joint optimization framework for learning with noisy labels.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 5552–5560.
  * Waseem and Hovy (2016) Zeerak Waseem and Dirk Hovy. 2016.  Hateful symbols or hateful people? predictive features for hate speech detection on twitter.  In _Proceedings of the NAACL Student Research Workshop_ , pages 88–93.
  * Zhan et al. (2019) Xueying Zhan, Yaowei Wang, Yanghui Rao, and Qing Li. 2019.  Learning from multi-annotator data: A noise-aware classification framework.  _ACM Trans. Inf. Syst._ , 37(2).
  * Zhang et al. (2016) Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2016.  Understanding deep learning requires rethinking generalization.  _arXiv preprint arXiv:1611.03530_.


# Symmetric Strategies for Multi-Access IoT Network Optimization: A Common
Information Approach

Sagar Sudhakara S. Sudhakara is with the Department of Electrical & Computer
Engineering, University of Southern California, Los Angeles, CA 90089 (E-mail:
sagarsud@usc.edu).

###### Abstract

In the context of IoT deployments, a multitude of devices concurrently require
network access to transmit data over a shared communication channel. Employing
symmetric strategies can effectively facilitate the collaborative use of the
communication medium among these devices. By adopting such strategies, devices
collectively optimize their transmission parameters, resulting in minimized
collisions and enhanced overall network throughput.

Our primary focus centers on the formulation of symmetric (i.e., identical)
strategies for the sensors, aiming to optimize a finite horizon team
objective. The imposition of symmetric strategies introduces novel facets and
complexities into the team problem. To address this, we embrace the common
information approach and adapt it to accommodate the use of symmetric
strategies. This adaptation yields a dynamic programming framework grounded in
common information, wherein each step entails the minimization of a single
function mapping from an agent’s private information space to the space of
probability distributions over possible actions.

Our proposed policy/method incurs a reduced cumulative cost compared to other
methods employing symmetric strategies, a point substantiated by our
simulation results.

##  I Introduction

###  I-A Motivation

In today’s rapidly expanding landscape of Internet of Things (IoT)
deployments, the simultaneous operation of a multitude of devices is a
commonplace scenario. These devices, ranging from sensors to actuators, play a
pivotal role in collecting and disseminating data across interconnected IoT
networks. However, as the number of IoT devices continues to grow, so does the
contention for network access to transmit data over shared communication
channels [1]. In this context, addressing data collisions becomes a pressing
concern.

Data collisions occur when two or more IoT devices attempt to transmit data
over the same channel simultaneously, resulting in interference and loss of
data. This not only leads to inefficiency in data transmission but also
jeopardizes the reliability and responsiveness of the entire IoT network. The
consequences of data collisions can be severe, particularly in applications
where timely and accurate data exchange is critical, such as industrial
automation [2], healthcare monitoring, smart cities, and precision
agriculture.

One of the key challenges in mitigating data collisions within the IoT
ecosystem is the need for devices to collaboratively share and manage the
communication medium. Traditional communication protocols, while effective in
point-to-point or small-scale deployments, may become less efficient as the
number of devices grows [3]. To address this challenge, the adoption of
symmetric strategies emerges as a promising solution.

Symmetric strategies involve devices collectively optimizing their
transmission parameters in a coordinated manner. This optimization aims to
reduce the likelihood of data collisions by ensuring that devices share common
rules when accessing the channel. Such strategies, when effectively
implemented, can significantly enhance overall network throughput and reduce
the occurrence of data collisions.

In our paper, our primary focus revolves around the formulation and
implementation of symmetric (i.e., identical) strategies tailored to IoT
sensors. These strategies aim to optimize a finite horizon team objective.
This concentration on symmetric strategies introduces novel dimensions and
complexities into the problem, necessitating innovative approaches for
resolution. To tackle these challenges, we turn to the common information
approach, which enables us to adapt and accommodate the use of symmetric
strategies effectively.

By leveraging the common information approach [4], we construct a dynamic
programming framework rooted in common information. In this framework, each
step involves the minimization of a function that maps from an agent’s private
information space to the space of probability distributions over possible
actions. This approach provides a robust foundation for devising efficient and
collision-avoidance strategies for IoT networks with multiple devices.

The culmination of our research is a proposed policy and methodology that has
demonstrated its efficacy in practice. Through extensive simulation results,
we have validated that our approach incurs a reduced cumulative cost compared
to other methods employing symmetric strategies. This not only underscores the
viability of our approach but also its potential to significantly enhance the
efficiency and reliability of Multi-IoT networks.

Our paper endeavors to contribute meaningfully to the advancement of Multi-IoT
networks by offering innovative solutions for mitigating data collisions and
improving network efficiency. The implications of our work extend to a diverse
array of applications within the IoT ecosystem, ultimately fostering enhanced
data reliability and responsiveness in the era of interconnected devices.

Figure 1: Mitigating Collisions in Multi-IoT Networks.

###  I-B Previous Work

Addressing data collisions in multi-access, single-channel reuse conditions
within Multi-IoT networks involves a diverse array of methods and tools. These
encompass collision avoidance techniques such as CSMA/CA and listen-before-
talk [5], where devices actively monitor channel activity prior to
transmitting. Time Division Multiple Access (TDMA) [6] allots specific time
slots for device communication, reducing overlap. Frequency Hopping Spread
Spectrum (FHSS) [7] permits devices to switch between channels at designated
intervals, minimizing interference. Further enhancements come from randomized
backoff times and collision detection with retries [8]. The central IoT
gateway or coordinator plays a pivotal role, managing channel access,
scheduling transmissions, and prioritizing critical data via Quality of
Service (QoS) mechanisms [9]. These methods extend to interference mitigation,
network synchronization, and machine learning [10], adjusting parameters
adaptively for collision reduction. Additionally, channel sensing and dynamic
channel allocation optimize IoT communication [11]. Moreover, load balancing
distributes devices across multiple channels, mitigating congestion. While
these strategies are applicable in diverse contexts such as smart homes,
industrial IoT, logistics, healthcare, and environmental monitoring, the
unique focus on probabilistic symmetric strategies tailored to IoT sensors is
notably absent in these approaches. These symmetric strategies, aimed at
optimizing finite horizon team objectives, introduce innovative nuances into
the problem. To address these challenges, the common information approach is
employed, enabling the effective adaptation and utilization of symmetric
strategies.

##  II Problem Formulation

We delve with a scenario where a group of $n$ IoT sensors operates in a
slotted multiple access setting. Time is divided into discrete slots, and
within each slot, the sensors can either transmit data or remain idle. In each
time slot, various events occur based on the sensors’ activities. These events
include:

  * •

Successes: If a sensor successfully transmits its data without interference,
it is considered a success event.

  * •

Idle Periods: During some time slots, certain sensors may not have any data to
transmit. In such cases, they remain idle, indicating no transmission
activity.

  * •

Collisions: When multiple sensors attempt to transmit data simultaneously
within the same time slot, collisions may occur.

Every sensor, denoted by agent $i$, possesses a distinct mode $M^{i}$ selected
from the set $\mathcal{M}$, where $i$ spans the range $\\{1,2,3,\dots,n\\}$.
The mode $M^{i}$ attributed to an agent is a time invariant random variable.
The collective set of modes is denoted as $M$, encompassing
$(M^{1},M^{2},\dots,M^{n})$.

Agent mode $M^{i}$ is characterized by three possibilities: Aggressive ($Ag$),
Passive ($Pa$), or a selection made by the designer ($De$). In the Aggressive
mode, an agent exhibits a high transmission probability denoted as $\alpha$.
Conversely, the Passive mode is characterized by a low transmission
probability, represented by $\beta$.

The set of possible actions is formally defined as $\mathcal{U}=\\{0,1\\}$,
wherein the specific control action undertaken by sensor $i$ during time
instance $t$ is symbolized by $U^{i}_{t}\in\mathcal{U}$. The entirety of
control actions for the ensemble of multi IoT sensors is concisely captured as
$U_{t}=(U^{1}_{t},U^{2}_{t},\dots,U^{n}_{t})$.

Information structure, randomization, and symmetric strategies:

At the commencement of time $t$, each IoT sensor $i$, possesses access to its
individual mode, its historical sequence of actions, and the action histories
of all other sensors. Consequently, the entirety of information accessible to
sensor $i$ at time $t$ is concisely represented as:

$I^{i}_{t}=\\{M^{i},U_{1:t-1}\\}.$ (1)

The information at the disposal of sensor $i$ ($i=1,2,\dots,n$) during time
$t$ encompasses two distinct components:

  1. 1.

Common information $C_{t}$ \- This information is shared among all
sensors111It’s important to note that $C_{t}$ doesn’t need to encapsulate the
complete scope of information shared by all sensors; it solely excludes
information that isn’t universally available to all sensors.. $C_{t}$ takes
values in the set $\mathcal{C}_{t}$.

  2. 2.

Private information $P^{i}_{t}$ -Any information exclusively accessible to
sensor $i$ at time $t$ that isn’t encompassed within $C_{t}$ is encapsulated
within $P^{i}_{t}$. The potential values of $P^{i}_{t}$ reside within
$\mathcal{P}_{t}$. (Note that the space of private information is the same for
all the sensors.) We use $P_{t}$ to denote $(P^{1}_{t},P^{2}_{t}\dots
P^{n}_{t})$.

$C_{t}$ should be viewed as an ordered list (or row vector) of some of the
system variables that are known to all the sensors. Similarly, $P^{i}_{t}$
should be viewed as an ordered list (or row vector). The common and private
information available to sensor $i$ at time $t$ is given by

$\displaystyle C_{t}=U_{1:t-1};\leavevmode\nobreak\ P^{i}_{t}=M^{i}.$ (2)

Sensor $i$ harnesses the accessible information to formulate its transmission
decision at time $t$. In the context of multiple sensors, it is a widely
accepted practice to confine agents to deterministic strategies without
compromising optimality [12]. Nonetheless, given that the sensors in our
configuration are constrained to employ symmetric strategies, the introduction
of randomization can be beneficial, as demonstrated in an example provided in
[13]. Sensor $i$ has the flexibility to introduce randomness when making its
transmission choice. Specifically, at each time $t$, sensor $i$ employs its
information $I^{i}_{t}$ to select a probability distribution
$\delta{U}^{i}_{t}$ across the transmission decision space $\mathcal{U}$. This
is mathematically expressed as:

$\delta{U}^{i}_{t}=g^{i}_{t}(I^{i}_{t}),$ (3)

Here, $g^{i}_{t}$ stands as a mapping from $\mathcal{P}_{t}$ to
$\Delta(\mathcal{U})$, where $\Delta(\mathcal{U})$ represents the set of
probability distributions over $\mathcal{U}$. Consequently, the control action
$U^{i}_{t}$ is chosen in a random manner based on the designated distribution,
i.e., $U^{i}_{t}\sim\delta{U}^{i}_{t}$. The primary motivation underlying the
use of randomized strategies is rooted in the constraint imposed on our
problem to adhere to symmetric strategies, as illustrated in reference [2].
The function $g^{i}_{t}$ assumes the role of sensor $i$’s transmission
strategy during time $t$. The collection of these functions,
$g^{i}:=(g^{i}_{1},\ldots,g^{i}_{T})$, collectively defines sensor $i$’s
overarching transmission strategy. The comprehensive set of all possible
strategies for sensor $i$ is denoted as $\mathcal{G}$.

We employ the notation $(g^{1},g^{2},\cdots,g^{n})$ to symbolize the
strategies adopted by sensor 1, sensor 2, …, sensor $n$ correspondingly. Our
primary focus centers on the realm of _symmetric strategies_ , which implies
that all sensors adopt an identical transmission strategy. In the context of
symmetric strategies, the superscript $i$ in $g^{i}$ is omitted, and a
symmetric strategy pair is succinctly represented as $(g,g,\cdots,g)$.

###  II-A Strategy optimization problem for Multi-IoT Sensors

During each time instance $t$, the system incurs a cost
$k_{t}(M^{1},\dots,M^{n},U^{1}_{t}\dots U^{n}_{t})$, which hinges on the modes
and transmission actions of all sensors. The system operates across a time
horizon of duration $T$. The cumulative expected cost associated with a
strategy pair $(g^{1},g^{2},\cdots,g^{n})$ is expressed as:

$J(g^{1},g^{2},\cdots,g^{n}):=\mathbb{E}^{(g^{1},g^{2},\cdots,g^{n})}\left[\sum_{t=1}^{T}k_{t}(M,{U}_{t})\right].$
(4)

In this context, the system cost at time $t$ is represented by the expression:

$k_{t}(M,{U}_{t})=\begin{cases}0,\leavevmode\nobreak\ \text{if only one
of}\leavevmode\nobreak\ u^{i}_{t}=1\\\ 1,\leavevmode\nobreak\
\text{otherwise}.\end{cases}$ (5)

The cost function indicates a successful transmission when solely one sensor
transmits at a given time $t$.

The primary objective is to identify a symmetric strategy pair that minimizes
the overall expected cost among all possible symmetric strategy pairs. In
other words, the goal is to find a strategy $g\in\mathcal{G}$ such that:

$J(g,g,\cdots,g)\leq J(h,h,\cdots,h),\leavevmode\nobreak\ \leavevmode\nobreak\
\forall h\in\mathcal{G}.$ (6)

######  Remark 1.

Our premise assumes that the randomization carried out by each sensor remains
independent over time and uncorrelated with the other sensor’s actions [4].
This is achieved by each sensor having access to independent and identically
distributed (i.i.d.) random variables $K^{i}_{1:T}$ uniformly distributed over
the interval $(0,1]$. These variables, $K^{1}_{1:T}$ and $K^{2}_{1:T}$, are
mutually independent as well as independent from the primitive random
variables. Moreover, each sensor leverages a mechanism $\kappa$ that combines
$K^{i}_{t}$ and the distribution $\delta{U}^{i}_{t}$ over $\mathcal{U}$ to
generate a random action according to the specified distribution. Thus, sensor
$i$’s action at time $t$ is formulated as
$U^{i}_{t}=\kappa(\delta{U}^{i}_{t},K^{i}_{t})$.

######  Remark 2.

It can be shown that the strategy space $\mathcal{G}$ is a compact space and
that $J(g,g,\cdots,g)$ is a continuous function of $g\in\mathcal{G}$. Thus, an
optimal $g$ satisfying (6) exists.

##  III Common information approach

In the context of the Multi IoT Sensor Problem, we embrace the common
information approach introduced in [14]. This methodology reframes the
decision-making challenge from the standpoint of a coordinator endowed with
knowledge of the common information. At each time step, the coordinator
formulates prescriptions that translate each sensor’s private information into
its corresponding transmission action. The behavioral action of each sensor
within this framework materializes as the prescription evaluated using the
current realization of its private information. Notably, since the problem
mandates symmetric strategies for all sensors, we demand that the coordinator
choose _identical prescriptions for every sensor_. To provide a precise
description, let $\mathcal{B}_{t}$ designate the space encompassing all
functions from $\mathcal{P}_{t}$ to $\Delta(\mathcal{U})$. Within this
context, $\Gamma_{t}\in\mathcal{B}_{t}$ signifies the prescription elected by
the coordinator at time $t$. Consequently, the behavioral action of sensor $i$
takes the form: $\delta U^{i}_{t}=\Gamma_{t}(P^{i}_{t})$.

Aligning with the problem’s requirements, sensor $i$’s action $U^{i}_{t}$
manifests through independent randomization governed by the distribution
$\delta U^{i}_{t}$. The coordinator’s prescription selection at time $t$
hinges on the common information available at that juncture and the history of
prior prescriptions. Hence, the prescription is denoted as:

$\Gamma_{t}=d_{t}(C_{t},\Gamma_{1:t-1}),$ (7)

Where $d_{t}$ constitutes a mapping from
$\mathcal{C}_{t}\times\mathcal{B}_{1}\ldots\times\mathcal{B}_{t-1}$ to
$\mathcal{B}_{t}$. The ensemble of mappings $d:=(d_{1},\ldots,d_{T})$ is
acknowledged as the coordination strategy. The coordinator’s objective
revolves around selecting a coordination strategy that effectively minimizes
the total expected cost across the finite time horizon:

$\mathcal{J}(d):=\operatorname{\mathbb{E}}^{d}\left[\sum_{t=1}^{T}k_{t}(M,{U}_{t})\right].$
(8)

The subsequent lemma serves to establish the equivalence between the
coordinator’s reformulation and the original problem. The integration of
identical prescriptions by the coordinator acts as the linchpin for connecting
the coordinator’s strategy to symmetric strategies for the agents in the
original problem.

######  Lemma 1.

The original Multi IoT Sensor Problem and the reformulated coordinator’s
problem are equivalent in the ensuing sense:

(i) For any symmetric strategy pair $(g,g)$, consider the subsequent
coordination strategy:

$d_{t}(C_{t})=g_{t}(\cdot,C_{t}).$

In this scenario, $J(g,g)=\mathcal{J}(d)$.

(ii) Conversely, for any coordination strategy $d$, define the symmetric
strategy pair as follows:

$g_{t}(\cdot,C_{t})=d_{t}(C_{t},\Gamma_{1:t-1}),$

where $\Gamma_{k}=d_{k}(C_{k},\Gamma_{1:k-1})$ for $k=1,\ldots,t-1$.

###### Proof.

The proof hinges on Proposition 3 of [14], coupled with the realization that
the use of identical prescriptions by the coordinator corresponds to the
adoption of symmetric strategies within the original problem.

In this manner, the common information approach bridges the gap between the
original Multi IoT Sensor Problem and the perspective of a coordinator
employing prescriptions based on the shared information to optimize the
sensors’ strategies. ∎

We now proceed with finding a solution for the coordinator’s problem. As shown
in [14], the coordinator’s belief on $P_{t}$ can serve as its information
state (sufficient statistic) for selecting prescriptions. At time $t$, the
coordinator’s belief is given as:

$\displaystyle\Pi_{t}(p)=\operatorname{{\mathds{P}}}(P_{t}=p|C_{t},\Gamma_{1:(t-1)}),$
(9)

for all $p\in\mathcal{P}_{t}\times\mathcal{P}_{t}\dots\times\mathcal{P}_{t}$.
The belief can be sequentially updated by the coordinator as described in
Lemma 2 below. The lemma follows from arguments similar to those in Theorem 1
of [14].

######  Lemma 2.

For any coordination strategy $d$, the coordinator’s belief $\Pi_{t}$ evolves
almost surely as

$\Pi_{t+1}=\eta_{t}(\Pi_{t},\Gamma_{t},U_{t}),$ (10)

where $\eta_{t}$ is a fixed transformation that does not depend on the
coordination strategy.

###### Proof.

We know that the coordinator’s belief at time $t$, is given by:

$\displaystyle\Pi_{t}(m)=\operatorname{{\mathds{P}}}(P_{t}=m|U_{1:t-1},\Gamma_{1:(t-1)}),$
(11)

$\forall\leavevmode\nobreak\ $m$\in\\{Ag,Pa,De\\}$. At time $t+1$,

$\displaystyle\Pi_{t+1}(m):=\operatorname{{\mathds{P}}}(P_{t+1}={m}|U_{1:t},\Gamma_{1:t})$
$\displaystyle=\frac{\operatorname{{\mathds{P}}}(m,u_{t}|u_{1:t-1},\gamma_{1:t})}{\sum_{\tilde{m}}\operatorname{{\mathds{P}}}(\tilde{m},{u}_{t}|u_{1:t-1},\gamma_{1:t})}$
$\displaystyle=\frac{\Pi_{i=1}^{n}\gamma_{t}(m^{i},u_{1:t-1};u^{i}_{t})\Pi_{t}(m)}{\sum_{\tilde{m}}\Pi_{i=1}^{n}\gamma_{t}(\tilde{m}^{i},{u}_{1:t-1};{u}^{i}_{t})\Pi_{t}(\tilde{m})}$
(12)

∎

Using the results in [14], we can write a dynamic program for the
coordinator’s problem. Recall that $\mathcal{B}_{t}$ is the space of all
functions from $\mathcal{P}_{t}$ to $\Delta(\mathcal{U})$. For a
$\gamma\in\mathcal{B}_{t}$ and $p\in\mathcal{P}_{t}$, $\gamma(p)$ is a
probability distribution on $\mathcal{U}$. Let $\gamma(p;u)$ denote the
probability assigned to $u\in\mathcal{U}$ under the probability distribution
$\gamma(p)$.

######  Theorem 1.

The value functions for the coordinator’s dynamic program are as follows:
Define $V_{T+1}(\pi_{T+1})=0$ for every $\pi_{T+1}$. For $t\leq T$ and for any
realization $\pi_{t}$ of $\Pi_{t}$, define

$V_{t}(\pi_{t})=\min_{\gamma_{t}\in\mathcal{B}_{t}}\mathbb{E}[k_{t}(M,U_{t})+\\\
V_{t+1}(\eta_{t}(\pi_{t},\gamma_{t},U_{t}))|\Pi_{t}=\pi_{t},\Gamma_{t}=\gamma_{t}]$
(13)

Expanding on equation (13),

$\begin{split}\min_{\gamma_{t}\in\mathcal{B}_{t}}\Biggl{[}&\sum_{m}\sum_{u_{1:t}}k_{t}(m,u_{t})\pi_{t}(m)\Pi_{i=1}^{n}\gamma_{t}(m^{i},u_{1:t-1};u^{i}_{t})\\\
&+V_{t+1}(\eta_{t}(\pi_{t},\gamma_{t},u_{t}))\pi_{t}(m)\Pi_{i=1}^{n}\gamma_{t}(m^{i},u_{1:t-1};u^{i}_{t})\Biggr{]}\end{split}$
(14)

The coordinator’s optimal strategy is to pick the minimizing prescription for
each time and each $\pi_{t}$.

Let $\Xi_{t}(\pi_{t})$ be a minimizer of the value function in (13). Using
$\Xi_{t}(\pi_{t})$ obtained from the dynamic program in Theorem 1, we can
construct a symmetric strategy pair $({g}^{*},\cdots,{g}^{*})$ as described in
Algorithm 1. This strategy pair minimizes the overall expected cost among all
possible symmetric strategy pairs.

Algorithm 1 Strategy ${g}^{*}$ for IoT Sensor $i$

Input: $\Xi_{t}(\cdot)$ obtained from DP for all $t$

for $t=1$ to $T$ do

Current information: $C_{t},M^{i}$

Update CIB $\Pi_{t+1}=\eta_{t}(\Pi_{t},\Xi_{t}(\Pi_{t}),U_{t})$ {If $t=1$,
Initialize CIB $\Pi_{t}$ using $C_{1}$}

Get prescription $\Gamma_{t}=\Xi_{t}(\Pi_{t})$

Select communication action $U_{t}^{i}=\Gamma_{t}(M^{i})$

end for

We emphasize that the common information based dynamic program is solved
offline. Therefore, the solution $\Xi_{t}$ to this dynamic program is known to
both agents before the agents start operating in their environment. During
their operation, this solution is used in a decentralized manner by the agents
to select their actions as described in Algorithm 1.

##  IV Experiments

For the sake of simplicity and to facilitate a clearer grasp of the design of
symmetric strategies, we have constructed our experiment using only two
sensors. This deliberate reduction in complexity allows us to provide a
clearer insight into strategy designing when sensors are in different modes,
such as Aggressive, Passive, or Designer mode.

Problem Setup Consider a scenario involving two sensors. Each sensor is
capable of operating in one of three modes: Aggressive (Ag), Passive (Pa), or
Designer (De). In Aggressive mode, a sensor has a high transmission
probability ($\alpha$) near 1. In Passive mode, the transmission probability
($\beta$) is low, almost approaching 0. Our objective is to devise symmetric
strategies for the sensors, accounting for the fact that one sensor’s mode is
private to the other and vice versa. The team incurs a cost of 0 for
successful transmissions and a cost of 1 for collisions or idle channel
situations.

Implementation To recap, let’s revisit the concept of $\gamma(p)$, which
denotes a continuous probability distribution on $\mathcal{U}$. In order to
work with this continuous distribution in a discrete manner, we employ
quantization. Specifically, we discretize the probability distribution into 21
distinct probability values, each separated by 0.05. This quantization process
is crucial to enable our analysis within a discrete framework. The time
horizon, labeled as $T$, is set to 10.

The procedure commences with an initial belief matrix B at time $t=1$, wherein
the rows indicate the mode of agent $1$ and columns indicate the mode of agent
$2$. By utilizing equation (21), we progressively update the belief matrix for
each subsequent time step $t$. Following the update of the belief matrices,
our emphasis shifts towards identifying the unique matrices among all these
beliefs. This strategic step prevents redundant recalculations of values,
ultimately boosting computational efficiency and streamlining our analysis
process.

We solve the Coordinator’s Dynamic program given in equation (13) and (14),
store the minimum value for every belief ($\pi_{t}$) that can be achievable at
time $t$ from the updated belief matrices, and back-propagate in time to
evaluate the Value of all the beliefs reachable at time $t-1$.

The value obtained at time $t=0$ is the cumulative expected cost associated
with the symmetric strategy. By executing these steps meticulously, we
successfully transform theoretical concepts into practical computations,
allowing us to uncover valuable insights and deliver conclusive results.

Results 1: We initialize the belief matrix indicated as $\textbf{{B}}_{1}$ in
Figure 2 with specific conditions: agent $1$ is assumed to be in the designer
mode, while agent $2$ can be in either Aggressive or Passive mode, each with a
probability of 0.5. (Here, $De^{i}$ represents agent $i$ being in Designer
mode.)

$\textbf{{B}}_{1}=\begin{array}[]{c|ccc}&\text{$De^{2}$}&\text{$Ag^{2}$}&\text{$Pa^{2}$}\\\
\hline\cr\text{$De^{1}$}&0&0.5&0.5\\\ \text{$Ag^{1}$}&0&0&0\\\
\text{$Pa^{1}$}&0&0&0\end{array}$

We observe scatter plots of value versus belief that depict the introduction
of new observations (actions). As new observations are integrated, updated
belief matrices emerge, each associated with a probability of taking action
$U^{1}_{t}=1$ for every belief, as demonstrated in the Figure 2 below. The
belief matrices indicated in Figure 2 are:

$\textbf{{B}}_{1}$ | $\textbf{{B}}_{2}$ | $\textbf{{B}}_{3}$
---|---|---
$\begin{bmatrix}0&0.5&0.5\\\ 0&0&0\\\ 0&0&0\end{bmatrix}$ | $\begin{bmatrix}0&1&0\\\ 0&0&0\\\ 0&0&0\end{bmatrix}$ | $\begin{bmatrix}0&0&1\\\ 0&0&0\\\ 0&0&0\end{bmatrix}$

Figure 2: Scatter Plot of Value vs. Belief

Beyond time $t=2$, the updated matrices tend to stabilize, enabling sensor $1$
to develop a more refined understanding of sensor 2’s mode. This knowledge
equips agent $1$ to strategically choose actions that minimize the system
cost.

Results 2: We present a comprehensive comparative study between our symmetric
strategic belief approach and alternative belief-based transmission
strategies. This comparison is visually illustrated through a bar plot,
effectively capturing the cumulative expected costs.

The alternative belief-based strategies are characterized by parameters
represented as $(x,y,z)$. Under the premise that sensor 1 operates in designer
mode, it acquires insight into the mode distribution of the other sensor. If
the likelihood of the other sensor being in aggressive mode surpasses the
threshold $x$ where $(\alpha\leq x\leq 1)$, the designer mode triggers action
$(U^{1}_{t})=0$. Similarly, when the probability of the other sensor being in
passive mode exceeds the threshold $y$ where $(1-\beta\leq y\leq 1)$, the
designer mode opts for action $(U^{1}_{t})=1$. In cases where neither of these
conditions holds true, the designer mode of sensor 1 selects action
$(U^{1}_{t})=1$ with a probability of $z$ $(\beta\leq z\leq\alpha)$ recall
where $\alpha$ is high probability when the sensor is in aggressive mode and
$\beta$ is the lowest probability of transmission when the sensor is in
passive mode.

Figure 3: Scatter Plot of Value vs. Belief

As shown in Table 1, our analysis encompasses diverse initial belief matrices,
visually represented as vectors along the x-axis of the bar plot. By
scrutinizing the expected costs associated with various belief-based methods
characterized by the parameters $(x,y,z)$—as indicated in the table below
policy methods—we uncover valuable insights into the effectiveness of our
proposed symmetric strategic belief approach.

Remarkably, the table unequivocally displays the superiority of our method in
achieving the lowest expected cost across a spectrum of initial beliefs
compared to other belief-based methods. The least effective among the belief-
based methods consistently adopt either an aggressive or passive stance, as
exemplified by methods 4 and 5. The simulation code for these analyses is
readily accessible through the following GitHub link .

Initial Belief | Proposed Policy | Policy 2 | Policy 3 | Policy 4 | Policy 5 | Policy 6 | Policy 7 | Policy 8
---|---|---|---|---|---|---|---|---
(0.8,1.1,1) | (1.1,0.8,0) | (1.1,1.1,1) | (1.1,1.1,0) | (0.8,1.1,0.5) | (1.1,0.8,0.5) | (0.9,0.8,0.5)
$[0,\frac{1}{2},\frac{1}{2},0,0,0,0,0,0]$ | 0.5 | 0.5 | 0.5 | 5 | 5 | 2.75 | 2.75 | 0.5
$[0,\frac{1}{3},\frac{1}{3},0,\frac{1}{3},0,0,0,0]$ | 3.457 | 3.667 | 3.667 | 6.667 | 6.667 | 5.167 | 5.082 | 3.667
$[0,\frac{1}{6},\frac{1}{6},0,\frac{1}{6},\frac{1}{6},0,\frac{1}{6},\frac{1}{6}]$ | 3.429 | 3.5 | 4.667 | 5 | 5 | 4.715 | 4.369 | 4.083
$[0,\frac{1}{6},\frac{1}{6},0,0,0,0,\frac{1}{3},\frac{1}{3}]$ | 3.39 | 3.5 | 3.5 | 5 | 5 | 4.205 | 4.295 | 3.5
$[0,\frac{1}{6},\frac{1}{6},0,0,0,0,\frac{2}{3},0]$ | 0.167 | 0.167 | 0.167 | 1.667 | 1.667 | 0.917 | 0.963 | 0.167
$[0,\frac{1}{3},\frac{1}{6},0,0,\frac{1}{6},0,\frac{1}{3},0]$ | 0.167 | 0.333 | 0.167 | 3.333 | 1.667 | 1.042 | 1.835 | 0.25
Table I: Comparison of cumulative Expected cost for different Policies

##  V Conclusion

In conclusion, the optimization of communication within IoT networks,
particularly those with numerous interconnected sensors, is paramount for
ensuring efficient network operation. Collaborative decision-making in
uncertain environments becomes indispensable in these networks, where
simultaneous data transmission poses the risk of collisions and reduced
efficiency.

Symmetric strategies, inspired by cooperative multi-agent scenarios, emerge as
a promising solution. By deploying identical decision-making protocols among
IoT devices, these strategies empower devices to optimize their transmission
patterns. This optimization reduces data collisions, ultimately leading to
improved network throughput.

The transformative potential of symmetric strategies in IoT networks cannot be
understated. These strategies streamline data transmission, enhance network
efficiency, and unlock the full potential of IoT networks. Our paper delves
into the application of symmetric strategies to multi-access IoT sensor
networks, providing valuable insights and empirical evidence on their profound
impact.

Our overarching goal is to pave the way for improved coordination among IoT
devices, ensuring efficient communication and a seamless future for IoT
networks. As the IoT landscape continues to evolve, symmetric strategies stand
as a beacon of promise for achieving these crucial objectives.

## Acknowledgment

The author would like to thank the Annenberg Fellowship and the grants from
the Graduate School research award.

## References

  * [1] R. Khan, S. U. Khan, R. Zaheer, and S. Khan, “Future internet: the internet of things architecture, possible applications and key challenges,” in 2012 10th international conference on frontiers of information technology, pp. 257–260, IEEE, 2012.
  * [2] S. Misra, C. Roy, T. Sauter, A. Mukherjee, and J. Maiti, “Industrial internet of things for safety management applications: A survey,” IEEE Access, vol. 10, pp. 83415–83439, 2022.
  * [3] J. Gubbi, R. Buyya, S. Marusic, and M. Palaniswami, “Internet of things (iot): A vision, architectural elements, and future directions,” Future generation computer systems, vol. 29, no. 7, pp. 1645–1660, 2013.
  * [4] D. Kartik, S. Sudhakara, R. Jain, and A. Nayyar, “Optimal communication and control strategies for a multi-agent system in the presence of an adversary,” in 2022 IEEE 61st Conference on Decision and Control (CDC), pp. 4155–4160, 2022.
  * [5] I. C. S. L. S. Committee et al., “Ieee standard for information technology-telecommunications and information exchange between systems-local and metropolitan area networks-specific requirements part 11: Wireless lan medium access control (mac) and physical layer (phy) specifications,” IEEE Std 802.11^, 2007.
  * [6] W. Stallings, Data and computer communications.  Pearson Education India, 2007.
  * [7] B. Skylar, Digital Communications: Pearson New International Edition uPDF eBook.  Pearson UK, 2015.
  * [8] D. E. Comer, Internetworking con TCP/IP, vol. 1.  Pearson Italia Spa, 2006.
  * [9] S. Li, L. D. Xu, and S. Zhao, “The internet of things: a survey,” Information systems frontiers, vol. 17, pp. 243–259, 2015.
  * [10] M. A. Alsheikh, S. Lin, D. Niyato, and H.-P. Tan, “Machine learning in wireless sensor networks: Algorithms, strategies, and applications,” IEEE Communications Surveys & Tutorials, vol. 16, no. 4, pp. 1996–2018, 2014.
  * [11] I. F. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci, “A survey on sensor networks,” IEEE Communications magazine, vol. 40, no. 8, pp. 102–114, 2002.
  * [12] S. Yüksel and T. Basar, “Stochastic networked control systems: Stabilization and optimization under information constraints.,” Springer Science & Business Media, 2013.
  * [13] S. Sudhakara and A. Nayyar, “Optimal symmetric strategies in multi-agent systems with decentralized information,” arXiv preprint arXiv:2307.07150, 2023.
  * [14] A. Nayyar, A. Mahajan, and D. Teneketzis, “Decentralized stochastic control with partial history sharing: A common information approach,” IEEE Transactions on Automatic Control, vol. 58, no. 7, pp. 1644–1658, 2013.